Advances in Computers, Volume 31

Advances in COMPUTERS VOLUME 31 Contributors to This Volume STEPHEN J. ANDRIOLE MICHAEL CONRAD PIEROCOSI ANTHONY DEB...

Author: Marshall C. Yovits

36 downloads 1126 Views 19MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Advances

in COMPUTERS VOLUME 31

Contributors to This Volume STEPHEN J. ANDRIOLE MICHAEL CONRAD PIEROCOSI ANTHONY DEBONS RENATO DEMORI DAVID I. HEIMANN NITINMITTAL MATHEW J. PALAKAL KISHORS . TRIVEDI

Advances in

COMPUTERS E D I T E D BY

MARSHALL C. YOVITS Purdue School of Science Indiana University-Purdue Indianapolis, Indiana

University at Indianapolis

VOLUME 31

ACADEMIC PRESS, INC. Harcourt Brace Jovanovich, Publishers

Boston San Diego New York London Sydney Tokyo Toronto

THISBOOK IS PRINTED ON

ACID-FREE PAPER.

@

COPYRIGHT @ 1990 BY ACADEMICPRESS,INC.

ALL RIGHTS RESERVED. NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT PERMISSION IN WRITING FROM THE PUBLISHER.

ACADEMIC PRESS, INC. 1250 Sixth Avenue, San Diego, CA 92101

United Kingdom Edition published by ACADEMIC PRESS LIMITED 24-28 Oval Road, London NWI 7DX

LIBRARY OF CONGRESS CATALOG CARDNUMBER: 59-15761 ISBN 0-12-012131-X PRINTED IN THE UNITED STATES OF AMERICA

90919293

9 8 7 6 5 4 3 2 1

Contents

CONTRIBUTORS. . . . . . . . . . . . . . . . PREFACE.. . . . . . . . . . . . . . . . .

. . . . . . . .

vii ix

Command and Control Information Systems Engineering: Progress and Prospects Stephen J. Andriole

1. Introduction . . . . . . . . . . . . . . . . . 2. The Information Systems Engineering Process. . . . . . 3. The Domain of Command and Control . . . . . . . . 4. Command and Control Information and Decision Systems Engineering . . . . . . . . . . . . . . . . . 5. Case Studies in the Design, Development, and Application of C2 Information and Decision Systems . . . . . . . . 6. Next Generation Command and Control Information Systems Engineering . . . . . . . . . . . . . . . . . 7. Summary and Conclusions . . . . . . . . . . . . Appendix A: Group (Army Theater Level) Tactical Planning Substantive and User-Computer Interface Tasks and Requirements. . . . . . . . . . . Appendix B: Storyboards from the Group Planning Prototype References . . . . . . . . . . . . . . . . . .

. . . . . .

2 6 32

.

.

39

.

.

50

. . . .

57 76

. .

77 89 95

. . . .

Perceptual Models for Automatic Speech Recognition Systems Renato DeMori, Mathew J. Palakal and Piero Cosl

1. Introduction . . . . . . . . . . . . . . . . . . . 2. Speech and Speech Knowledge . . . . . . . . . . . . . 3. A Multi-Layer Network Model for ASR Systems. . . . . . . 4. The Ear Model: An Approach Based on Speech Perception . . . 5. The Vocal Tract Model: An Approach Based on Speech Production 6. Conclusions . . . . . . . . . . , . , . . . . . . Acknowledgments . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . , . . . . . .

V

100 101 127 129 150 167 169 169

vi

CONTENTS

Availability and Reliability Modeling for Computer Systems

.

.

David 1 Heimann. Nitln Mittai and Kishor S Trivedl

1 . Introduction . . . . . . . . . . . . . . . . . . . 176 2. Measures of Dependability . . . . . . . . . . . . . . 180 3. Types of Dependability Analyses . . . . . . . . . . . . 200 4. The Modeling of Dependability . . . . . . . . . . . . . 201 5 . A Full-System Example . . . . . . . . . . . . . . . 218 6. Conclusions . . . . . . . . . . . . . . . . . . . 229 Acknowledgments . . . . . . . . . . . . . . . . . 230 References . . . . . . . . . . . . . . . . . . . . 231

Molecular Computing Mlchael Conrad

1. Introduction . . . . . . . . . . . . . . . . . 2. Background . . . . . . . . . . . . . . . . . 3. Theory of Molecular Computing . . . . . . . . . . 4. The Macro-Micro (M-m) Scheme of Molecular Computing . 5. Modes of Molecular Computing . . . . . . . . . . 6. The Molecular Computer Factory . . . . . . . . . . 7. Molecular Computer Architectures . . . . . . . . . 8. Conclusions and Prospects . . . . . . . . . . . . Acknowledgments . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

.

. . . . . . . . .

236 238 246 269 289 303 307 317 318 319

Foundatlons of Information Sclence Anthony Debons

Prologue . . . . . . . . . . . . 1. Introduction . . . . . . . . . . 2. Essences: The Nature of Information . . 3. Structure: The Science of Information . 4. Synthesis: On a Theory of Foundations . 5. Overview . . . . . . . . . . . . Acknowledgments . . . . . . . . References . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . . . . . . . . . .

. . . . .

325 326 327 338 363 369 370 371

AUTHOR INDEX.........................................

379

INDEX ......................................... SUBJECT

387

CONTENTS OF PREVIOUS VOLUMES.........................

397

Contributors Numbers in parentheses refer to the pages on which the authors’ contributions begin.

Stephen J. Andriole (l), Department of Information Systems and Systems Engineering, School of Information Technology and Engineering, George Mason University, 4400 University Drive, Fairfax, Virginia 22030-4444 Michael Conrad (235), Department of Computer Science, Wayne State University, Detroit, Michigan 48202 Piero Cosi (99), Centro di Studio per le Richerche di Fonetica, C N R , Via G. Oberdan. 10.35122 Padova, Italy Anthony Debons (325), Institute for the Development of Expert Application Systems, Robert Morris College, Narrows Run Road, Corapolis, Pennsylvania, 15108-1189; Department of Information Science, University of Pittsburgh, Pittsburgh, Pennsylvania 15260 Renato DeMori (99), McGill University, School of Computer Science, 805 Sherbrook Street West, Montreal, Quebec, Canada H3A 2K6 David I . Heimann (175), Digital Equipment Corporation, 6 Tech Drive, Andover, Massachusetts 01810 Nitin Mittal (1 75), Digital Equipment Corporation, 6 Tech Drive, Andover, Massachusetts 01810 Mathew J. Palakal (99), Purdue University School of Science at Indianapolis, Department of Computer Science, 1201 East 38th Street AD13.5, Indianapolis, Indiana 46205-2868 Kishor S. Trivedi (1 7 9 , Computer Science Department, Duke University, Durham, North Carolina 27706

vii

This Page Intentionally Left Blank

Preface

The serial Advances in Computers provides a medium for the in-depth presentation of subjects of both current and long-range interest to the computer and information community. Within this framework, contributions for appropriate articles have been solicited from widely recognized experts in their fields. The time scale of the invitation is such that it permits a relatively leisurely perspective. Furthermore, the permitted length of the contributions is greater than many other publications. Thus, topics are treated both in depth and breadth. The serial began in 1960 and now continues with Volume 31. These books have played an important role over the years in the development of the computer and information fields. As these fields have continued to expand both in research and resulting applications as well as in their significance -so does the importance of the Advances series. As a consequence, it was decided that Academic Press would again this year publish two volumes, 30 and 31. Volume 30 was published earlier this year. Included in Volume 31 are chapters on command and control information systems, automatic speech recognition, reliability modeling of computer systems, molecular computing, and the foundations of information science. In the first chapter, Professor Andriole presents a multidisciplinary information systems design and development methodology that assumes more complex design challenges than we have faced in the past. The emphasis is on the process by which complex analytical problem-solving requirements are converted into computer-based systems. The author emphasizes the application of the information systems engineering process to command and control information and decision systems. He points out that, without structure, the design and development process will almost always fail. DeMori, Palakal, and Cosi state in their chapter that speaker-independent automatic speech recognition by computers of large or difficult vocabularies is still an unsolved problem, especially if words are pronounced connectedly. Since the early 1970s, there has been substantial progress toward the goal of constructing machines capable of recognizing and/or understanding human speech. One of the key improvements has been the development and application of mathematical methods that permit modeling the speech signal as a complex code with several, coexisting levels of structure. The authors present several past approaches and some current trends in automatic speech recognition research: using models based on speech production, and using models based on speech perception. In the third chapter, Drs. Heimann, Mittal and Trivedi address computer system dependability analysis, which ties together concepts such as reliability, maintainability and availability. It serves, along with cost and performance, as a major system selection criterion. Three classes of dependability measures are ix

X

PREFACE

described: system availability, system reliability, and task completion. The authors point out that the concept of system dependability is being considered with increasing interest as a component of computer system effectiveness and as a criterion used by customers for product selection decisions. They conclude that which of the measures is appropriate depends on the specific application under investigation, the availability of relevant data, and the usage or customer profile. Professor Conrad writes in the fourth chapter about molecular computers that are information processing systems in which individual molecules play a critical role. Natural biological systems fit this definition. Artificial information processing systems fabricated from molecular materials might emulate biology or follow new architectural principles. In either case they would qualify as molecular computers. The term may also apply to simulations of biomolecular systems or to virtual molecular computers implemented in conventional silicon machines. Conrad shows that molecular computing is both a science and a technology. These two factors are highly synergistic. The attempt to synthesize biomimetic or new molecular computing devices is an outgrowth of fundamental research in molecular and cellular biophysics, condensed-matter physics, polymer chemistry, neurophysiology, and computer science. It is likely to lead to new insights into mechanisms and materials that impact these areas as well. In the final chapter, Professor Debons considers the foundations of information science. He states that a perennial question posed by individuals both inside and outside the field of information concerns its nature: What is it? What are its essences, its structures, its boundaries? The study of information can be traced to antiquity, to philosophers and scholars concerned with the nature of knowledge. Contemporary information science arose from the scientific renaissance of the present century spurred by the launching of Sputnik. Advances in electronics, referred to as the “communication revolution,’’ increased the ability to transmit data for processing quickly and for greater distances. Debons considers the nature of the term “information”; he deals with information as a discipline and then synthesizes the various aspects of the science. It is my great pleasure to thank the contributors to this volume. They have given extensively of their time and effort to make this book an important and timely contribution to their profession. Despite the many calls upon their time, they recognized the necessity of writing substantial review and tutorial articles. It has required considerable effort on their part, and their cooperation and assistance is greatly appreciated. Because of their efforts, this volume achieves a high level of excellence and should be of great value for many years to come. It has been a pleasant and rewarding experience for me to edit this volume and to work with these authors. Marshall C. Yovits

Command and Control Information Systems Engineering: Progress and Prospects STEPHEN J . ANDRIOLE Department of Information Systems & Systems Engineering School of Information Technology & Engineering George Mason University Fairfax. Virginia

1. Introduction

2.

3.

4.

5.

6.

7.

. . . . . . . . . . . . . . . . . . . . . . . .

1 . 1 Chapter Overview . . . . . . . . . . . . . . . . . . . . 1.2 Information Systems Engineering Overview . . . . . . . . . . . The Information Systems Engineering Process . . . . . . . . . . . . 2.1 Systems Design in Perspective . . . . . . . . . . . . . . . . 2.2 Conventional Design Methods and Models . . . . . . . . . . . 2.3 The Prototyping Alternative. . . . . . . . . . . . . . . . . 2.4 Requirements Analysis Methods . . . . . . . . . . . . . . . 2.5 Task Requirements Analysis Methods . . . . . . . . . . . . . 2.6 User Profiling Methods . . . . . . . . . . . . . . . . . . 2.7 Organizational/Doctrinal Profiling Methods . . . . . . . . . . 2.8 TheTask/User/Organizational-DoctrinalMatrix . . . . . . . . . 2.9 Some Prototyping Methods . . . . . . . . . . . . . . . . The Domain of Command and Control . . . . . . . . . . . . . . 3.1 The Command and Control Process . . . . . . . . . . . . . 3.2 Command and Control Information and Decision System Requirements . Command and Control Information and Decision Systems Engineering . . . 4.1 C2 Information and Decision Systems Requirements Analysis . . . . . 4.2 C z System Modeling and Prototying . . . . . . . . . . . . . 4.3 Analytical Methods for C z Information and Decision Systems Engineering 4.4 C2 Systems Evaluation . . . . . . . . . . . . . . . . . . Case Studies in the Design, Development, and Application of C2 Information and Decision Systems . . . . . . . . . . . . . . . . . . . . 5.1 The Range of Applications . . . . . . . . . . . . . . . . . 5.2 The Group Planning Prototype . . . . . . . . . . . . . . . Next Generation Command and Control Information Systems Engineering . . 6.1 Emerging Issues and Challenges . . . . . . . . . . . . . . . 6.2 The Range of C 2 Information and Decision Support . . . . . . . . 6.3 Advanced Information Technologies . . . . . . . . . . . . . . 6.4 Integrated C2 Information and Decision Support . . . . . . . . . Summary and Conclusions . . . . . . . . . . . . . . . . . . .

. .

. . . . .

. .

.

. . .

.

. . .

.

. . .

. .

2 2 2 6 7 8 9 10 12 15 20 21 23 32 32 35 39 39 40 42 47 50 50 52 57 51 57 58 74 76

Sections 2.4 through 2.9 draw upon S.J. Andriole, Handbook of Decision Support Systems. published by Petrocelli Books. Inc., Princeton. New Jersey. 1 ADVANCES IN COMPUTERS. VOL. 31

Copyright (0 1990 by Academic Press Inc. All rights of reproduction in any form reserved ISBN 0-12-01213 I - X

2

STEPHEN J. ANDRIOLE

Appendix A: Group (Army Theater Level) Tactical Planning Substantive and User-Computer Interface Tasks and Requirements . -.- . . . . Appendix B: Storyboards from the Group Planning Prototype . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . .

77 89 95

1. Introduction 1.1 Chapter Overview

This chapter attempts several things. It first presents a multidisciplinary information systems design and development methodology that assumes more complex design challenges than we have faced in the past. The emphasis is on the process by which complex analytical problem-solving requirements are converted into computer-based systems. The chapter then turns to the application of the information systems engineering (ISE) process to command and control information and decision systems. Military command and control (as well as its civilian conterpart) presents special problems to the modern systems architect. Users are integral parts of nearly every system, the stakes are high, and the margin for error is small. User and operators come in many shapes and sizes, and robust analytical methods-especially those that deal well with uncertainty and stress-are almost always necessary. This chapter also presents some command and control ISE case studies intended to illustrate the most salient features of the ISE process. It ends with a look at future command and control information and decision systems engineering. 1.2

Information Systems Engineering Overview

Information systems engineering refers to the process by which information systems are designed, developed, tested and maintained. The technical origins of ISE can be traced to conventional information systems design and development and the field of systems engineering. ISE is by nature structured, iterative, multidisciplinary, and applied. The ISE process involves structured requirements analyses, functional modeling, prototyping, software engineering, and system testing, documentation and maintenance. Modern information systems solve a variety of data, information, and knowledge-based problems. Ten years ago most information systems were exclusively data-oriented; their primary purpose was to permit users to store, retrieve, manipulate, and display data. Application domains included inventory control, banking, personnel recordkeeping, and the like. The airline

COMMAND AND CONTROL INFORMATION SYSTEMS ENGINEERING

3

reservation system is representative of the information systems of the 1970s. More recently, expectations about the capabilities of information systems have risen considerably. It is today quite routine to find information systems that provide analytical support to users. Some of these systems help users allocate resources, evaluate personnel, plan, and simulate large events and processes. Systems engineering is a field of inquiry unto itself (Eisner, 1988). There are principles of applied systems engineering and a growing literature that defines a field representing a synthesis of systems analysis, engineering, and economics. Systems engineering involves all the activities that extend over the entire life cycle of a system, including requirements definitions, functional designs, development, testing and evaluation. According to Andrew P. Sage, a prominent systems engineer and contributor to the field, the systems engineer’s perspective is different from that of the product engineer, software designer, or technology developer; while the product engineer deals with detail, the systems engineer takes a “top down” viewpoint. Where the product engineer deals with internals, the systems engineer deals more extensively with the external view of the system., including the system’s interfaces to other systems and its human users, repairers, and managers. Systems engineering is based upon the quantitative skills of the traditional engineer combined with additional quantitative and qualitative skills derived from applied mathematics, psychology, management and other disciplines that support knowledge organization and design. The systems engineering process is a logical sequence of activities and decisions that transform operational needs into a description of system performance parameters and an optimal system configuration (Sage, 1985). The information systems engineering process represents the marriage between the tools, techniques and application domains of information systems and the generic systems engineering process. Figure 1 presents a blueprint for the design, development and testing of information systems. The blueprint calls for the identification of user requirements, the modeling of the requirements, the design, development and testing of working prototypes, the specification of software (programming) requirements, system testing and documentation, and the development of a system maintenance plan. Figure 1 also suggests that there are a variety of methods available to the information systems engineer and several products that ideally emerge from the steps in the ISE process. The sections that follow present more details about the generic ISE process as well as insight into how the process can be applied in the domain of military command and control.

r

R

‘remts

t

METHODS OPTIONS

PRODUCTS

P

I

=

Evolutionary

-

Prototype System stzing

* *bm

cwtpt i

4

-

multi-Attrikttc Utility Cost-Btmfi t -Hybrids...

User-Computcr interaction

bstalKnowledqc Base Specification Analytical Methods Selection SortwEngineering

Specify Software

Data Flow-Onmted

Data Structure-Oriented Object-Oriented... Processor Options Input Device Options Output Device Optiom...

Requirements

+

-

'Identify k d w # r / Software confiauration -I ~

-

Benefit Models Cost-Benefit Models Ootimizsttonr\odets...

-.

I

rdge Structwes

+

Functional DcSU'lptiOtl LherPbrusl

Doamtnt System

+ i -

Training Manual Multi-Attribute Utility Cost-Benefit

Test

Maintain System

FIG. 1. The generic information systems engineering process.

Task Schedules Evaluations

6

STEPHEN J. ANDRIOLE

2.

The Information Systems Englneerlng Process

The overview of the generic information systems engineering process in Section 1 is intended to communicate the many dimensions of the design and development process as well as the sense that the whole process is in fact greater than the sum of its parts. Information systems engineering is a multidisciplinary endeavor anchored very much in the systems approach to problem-solving. But because of the nature of many information systems engineering challenges, there are a number of “watchwords” almost always associated with the design and development of complex systems. Some of these include “multidisciplinary,”“iterative,” and “synthetic.” It is difficult-if not impossible-to design, develop, test, evaluate, or field information systems without insight from the behavioral, mathematical, computer, engineering, and managerial sciences. It is impossible, for example, to design and implement effective user interfaces without insight into the empirical findings from human factors and cognitive science. Similarly,it is impossible to select the right analytical method without an appreciation for the methods that cross-cut the above listed sciences and disciplines. It is also impossible to capture complex analytical requirements after but one attempt. The whole process must be iterative; it always relies heavily upon the synthesis of disparate data, knowledge, experience, and technology. Over the years systems designers have discoveredjust how difficultit can be to capture user requirements. A variety of tools and techniques have been developed to assist systems analysts, but they have often proven inadequateespecially when iequirements are complex and analytical. By and large, systems design and development “life cycle” models fail to recognize the inherent requirements dilemma. Consequently, systems analysts developed a new design perspective, one that assumes that requirements cannot be captured the first time through and that several iterations may be necessary to define requirements accurately. The new perspective is anchored in the value of prototyping. Prototyping “informs” the design process by leveraging increasingly specific and verifiable information in the requirements analysis process. The objective of prototyping is to demonstrate a system concept before expensiveprogramming begins. Successfulprototyping can be traced to iterative requirements analyses, user involvement, and the use of one of several tools for converting requirements hypotheses into a tangible system concept. Prototypers usually build one of two kinds of demonstration systems: “throwaway” and “evolutionary.”Throwaway prototypes are developed when requirements are especially difficult to capture, which may be due to inarticulate users, a complex problem area, or some combination of the two. As the label suggests,they are literally thrown away after each iteration-until


7

one accurately represents requirements. This “final” prototype may then evolve into an evolutionary one, which can be incrementally enhanced over time. The information systems engineering process described here assumes that when requirements are complex and analytical, prototyping will be necessary. The discussion of the larger process that follows is anchored in this assumption. Note also that command and control information systems engineering-discussed in some later sections of this chapter-is nearly always complex and analytical. The assumption thus holds across domains. 2.1

Systems Design in Perspective

Not so many years ago computers were used mostly by scientists and engineers. As the field matured, computing was distributed to a larger set of professionals, including accountants, budgeteers, and managers. The personal computer altered forever the way we think about computing. Initially the appeal of desktop power was mitigated by cost, but as soon as personal computers became affordable, the revolution in personal computing began. Years ago computers were used to perform calculations that were prohibitively expensive via any other means. Early interactive systems were barely so, and engineers had to hack at them until they behaved. When generalpurpose mainframes emerged, large organizations with huge databases expressed the most interest. It is safe to say that most early applications of general-purpose mainframe computers were database oriented. Today there are interactive “decision support systems” that profess to augment the decision-making power of human information processors. There are systems that help users generate options, evaluate options, and interpret the feedback received after they are implemented. There are systems that help users plan, create scenarios, and diagnose diseases. Figure 2 suggests where database-oriented and analytical computing begin and end (Andriole, 1989a). The differences are clear. Analytical problemsolving assumes some degree of cognitive information processing. While all cognitive processing is anchored in “data” and “knowledge” that must be stored and manipulated, there are unique properties of cognitive information processing that call for unique requirements definitions. The difference between the collection and interpretation of diagnostic data illustrates database-oriented versus analytical problem-solving (and, by implication, database-oriented versus analytical computing). As computers become cheaper, smaller and faster, and as expectations about how they can be used rise, more and more instances of “analytical computing” will become necessary and-eventually-commonplace. The

8

STEPHEN J. ANDRIOLE

MA-ORIFNTE D COMPUTING unicatlve

PhysW

COMPUTlNG

Tasks

Instruct Inform 0 Request Query

Search Identify 0 Classify Categorize

0

Medfatlonal

PerceDtual

Tasks

Tasks

File Store Retrieve Sample

ANALYTlCAL

Tasks

Plan Evaluate 0 Prioritize Decide 0

ANALYTICAL COMPLEXITY CONTINUUM FIG.2. Data versus analytical computing.

leverage lies in our ability to identify, define and validate complex requirements. Hence, there is a need for prototyping within the larger structure of multidisciplinary information systems engineering.

2.2

Conventional Design Methods and Models

There are a variety of “conventional” systems design methods available to the software systems analyst and engineer. Dee (1984), Hice et al. (1978), Andriole (1983), Pressman (1987), Leslie (1986), Royce (1970), and Horowitz (1975), among many, many others, all propose some variation of the conventional software systems design and development process anchored in the “waterfall method first introduced by Royce (1970) and Boehm (1976). All of them share some characteristics, such as a sequential nature, a single stage for identifying, defining, and validating user requirements, and an orientation that seduces the designer into treating the process as manageable. What is the problem here? First and foremost is the lack of emphasis upon user requirements. Years ago it was assumed that requirements were easily defined. Since early computing requirements were often database intensive, the assumption was initially valid. But as the need to fulfill analytical requirements grew, conventional life cycle models failed to keep pace. It is possible to conclude that conventional systems design models and methods ignore user requirements-and approaches to their modeling and verification-in favor of emphases that stress the importance of software engineering: program design and structure, coding, testing, and debugging, and the like.


9

This conclusion is supported by the vagueness with which conventional design methodologists treat the whole concept of user requirements. Requirements cannot be defined by simply asking users what they do or by watching them for a while. Worse yet are requirements methods that rely upon “handbooks,” manuals, or other written materials to define and refine user needs. They are worse because they disconnect the systems analyst from the user and presume that requirements can be identified in a vacuum. In a sense the whole issue of “conventional” versus “prototyping” methods and models is a strawman unworthy of serious dispute. Why? Because-as always-the problems the prospective system are intended to solve should determine life cycle assumptions. Designers that begin a priori with a method will often fail, if only because they may end up matching the wrong life cycle assumptions with the wrong problems. Analytical problem-solving requirements cannot be captured via conventional systems design methods or models. Iteration is always the watchword in such cases. On the other hand, problems with an absence of analytical requirements might well be modeled via conventional models. 2.3 The Prototyping Alternative Modern systems design and development directs us to look first at the requirements that the system is intended to satisfy. It then suggests that some kind of modeling of the mock system can redefine our requirements which, in turn, will permit us to develop yet another functional description of the system, and so forth until the model accurately represents and satisfies requirements. Then-and only then-should we turn to software design and engineering, and these steps should in turn determine our hardware configuration. Bringing up the rear are “packaging” tasks, such as the preparation of users’ manuals, and “technology transfer” tasks, such as the introduction of the system into the target environment. There are debates over the way the first iteration of the system itself should be developed. Some hold that a thorough requirements analysis will assure the development of a responsive system, while others feel just as certain about the wisdom of some kind of “prototyping” strategy. Applications prototyping (Boar, 1984), the strategy that assumes that several iterations of an interactive system are necessary and desirable, has become very popular over the past few years. Among other advantages, prototyping supports modular software engineering, permits user participation in the design process, and protects project resources from the jaws of tunnel programming. Most importantly, the applications prototyping strategy permits analysts to keep the requirements analysis process alive during the critical conversion process.

10

STEPHEN J. ANDRIOLE

Prototyping assumes that the first version of the interactive system will be rejected and modified. It assumes that users and designers will have some difficulty identifying and defining critical system functions, and that a limited amount of money should be spent on each prototype until a durable system definition emerges. The money saved should be plowed back into requirements definition, tasks/methods matching, and modeling. Prototyping is as much a state of mind as it is a structured design methodology. Contrary to popular belief, prototyping is highly structured and extremely methodical in application. While some design theorists suggest that prototyping is “loose” and haphazard, successful prototyping requires adherence to a set of reasonably specific principles. The primary assumption that prototypers make is that interactive systems cannot be developed easily, quickly, or without input from prospective users. They assume that the system will have to be developed over and over again, but unlike conventional system developers, prototypers plan for iteration. 2.4

Requirements Analysis Methods

There is no more important yet more neglected step in systems design than requirements analysis. As Meister (1976) and a whole host of others have pointed out, without a clear set of requirements the system will satisfy the needs of the designers and not the intended users. Boar (1984) reports that 2040% of all system problems can be traced to problems in the design process, while 6 0 4 0 % can be traced to inaccurate requirements definitions. The message is clear: Know thy user. The prototyping strategy assumes that requirements cannot all be prespecified, that inherent communications gaps exist among design participants, and that “extensive iteration is necessary, inevitable, desirable, and to be encouraged” (Boar, 1984). The strategy also assumes that requirements do not stop once the tasks the system is supposed to support have been identified and defined. All good requirements definitions consist of user, task, and organizationalldoctrinal requirements. In fact, the best possible requirements definition is a matrix linking all three (user, task, and organizational/doctrinal) dimensions together, as suggested in Fig. 3. Requirements analysis also assumes feasibility. If one were to discover after a significant requirements investment that no one could define requirements, or that the ones that were defined were impossible to satisfy via computerization, or that in order to satisfy the requirements one had to spend ten times what any reasonable person would suggest the system should cost, then the problem can be said to have failed the feasibility set. Feasibility

Filing Retr tev ing w i n g Col lati ng Sortlng

/

form Filling Document Ckecking

/

Telephoning

0ictat ing

/

Conferring Meettng

Data Analysis

Calculation

Planning Decision-Making NAIVE

I

MANAGERIAL

USERS

FIG.3. User/task/organizational requirements matrix.

12

STEPHEN J. ANDRIOLE

assessment is thus one outcome of requirements analysis; the others include task, user, and organizational/doctrinal profiles, and the integrated tasks/ users/organizational doctrinal matrix.

2.5 Task Requirements Analysis Methods Task profiling consists of qualitative and-if possible-quantitative descriptions of the tasks that the system is intended to solve, automate, quasiautomate, or ignore. Task profiles are important because the selection of the right analytical method depends upon how well the tasks have been defined. The tasks themselves should be arranged hierarchically all the way down to the lowest, diagnostic sub-task. While this is not to imply that each and every task and sub-task be elaborately defined, it is to suggest that the task requirements process be highly structured. There are a variety of ways to structure task analyses. It is important to begin with some sense of how tasks differ generally. Over the years the psychological research community has developed a number of “generic” taxonomies that can be used as organizing frameworks for the subsequent development of problem-specific task taxonomies. Fleischman, et al. (1984) present perhaps the most comprehensive review of this literature. They cite several approaches to task classification (and the development of task taxonomies) worth noting: 0

0 0 0

behavior description approaches; behavior requirements approaches; ability requirements approaches; and task characteristics approaches.

Behavior description approaches include those that identify “categories of tasks.. .based upon observations and descriptions of what operators actually do while performing a task.” Behavior requirements approaches emphasize the “cataloguing of behaviors that should be emitted or which are assumed to be required in order to achieve criterion levels of performance.” Ability requirements approaches assume that “tasks are to be described, contrasted, and compared in terms of abilities that a given task requires of the individual performer” or operator; while task characteristics approaches are “predicated upon a definition that treats the task as a set of conditions that elicits performance” (Fleischman, et al.. 1984). With the exception of task characteristics approaches, most approaches try to identify important processes,functions, behaviors, or performance. The ideal task analysis would permit the systems designer to differentiate among the tasks and rank them according to their problem-solving importance.


13

It is important to note that the use of task taxonomies to profile user tasks occurs before, during, and after the task profiling process. Task taxonomies are used initially as organizing frameworks; they are used during the process as substantive and procedural compasses; and they emerge redefined as problemspecific taxonomies after the process. This last use is key: the purpose of applying an existing generic taxonomy (or developing a whole new one) is to help accelerate the development of the required problem-specific taxonomy. As soon as one begins the task requirements analysis one should reach for a generic task taxonomy to guide it. There are a number of methods that have solid track records for developing task profiles. In fact, they all have weaknesses, suggesting that the best approach to task profiling is eclectic, interdisciplinary, and-as alwaysiterative. Our experience with requirements analysis suggests that a single method never really captures the essence of the tasks we are trying to computerize. Our successful task profiles were the result of having applied at least two task analysis methods. The task requirements analysis methods discussed in this section fall into three broad categories (Ramsey and Atwood, 1979): 0 0 0

questionnaire and survey methods; interviews and field observation methods; and simulation and gaming methods.

As suggested, there are at least three ways to identify and define tasks. The first involves asking users what they do and how they do what they do in questionnaires and surveys. The second involves asking them in person (in a variety of different settings), while the third suggests that the best way to profile tasks is through a simulation or gaming exercise. Inherent in all of these methods is the belief that-given enough time and money-tasks can always be identified and defined. Nothing is farther from the truth. There are many tasks that defy precise description; it is also naive to assume that all users are articulate. Hence there is a need for the iterative prototyping strategy, which assumes that users are often unable to define their tasks and that some tasks are much more resistant to definition than others. There are at least five ways to profile requirements via questionnaires and surveys (Ramsey and Atwood, 1979): 0 0 0

0 0

importance ratings questionnaires; time estimate questionnaires; repertory grid techniques; Delphi techniques; and policy capture techniques.

14

STEPHEN J. ANDRIOLE

The key to the successful use of questionnaire and survey methods lies in one’s ability to select users with unusually good diagnostic understandings of what they do; users unable to introspect may only feed back perceptions of what they think they do, not accurate descriptions of what they actually do. There are also obvious situations where questionnaires and/or surveys would be inappropriate. If a system is intended to serve a small but elite group of military analysts, then it is unlikely that any real insight could be gained from the results of a questionnaire (which would probably be ignored anyway). On the other hand, if the system is intended for use throughout the military or throughout a particular subset of the military (for example, throughout the strategic intelligence community), and the user population is geographically dispersed, then mailed questionnaires may be the only way to go. Interview and field observation methods include the following (Ramsey and Atwood, 1979): 0

0 0 0

unstructured and structured interviews; ad hoc working-group-based methods; critical incident techniques; and formal job analysis techniques.

If the truth be told, the overwhelming majority of task requirements analyses consist of unstructured interviews and possibly an ad hoc working group session or two. A series of questions are usually posed to one or more interviewees who tend to perceive requirements as anecdotes of their (usually limited) experiences. While these anecdotes are useful, they too often take the place of a structured requirements database. The participatory approach to interactive systems design should be stressed again here. A few hours of a user’s time is really quite worthless. If the system is to be responsive to real needs, then a users’ strike force must be established. As suggested earlier, users should be made members of the design team and given important design responsibilities. It is also important to note that techniques such as formal job analysis are best suited for defining non-cognitive tasks, and that other methods, such as structured interviews, working groups and protocol analyses, are more likely to yield useful cognitive task definitions. The application of simulation and gaming methods essentially calls for a scenario, some experts, and some techniques for documenting what happens when the experts are asked to address the scenario. Ramsey and Atwood (1979) and others (Andriole, 1983; Carlisle, 1973) suggest at least three kinds


15

of simulations and games: 0 0 0

paper simulations; protocol analysis; and interactive simulation or “gaming.”

Nearly all of the above requirements analysis methods can trace their origins to disciplines other than computer science or information systems. In fact, most of them can be traced to psychology and management science. This alone attests to the interdisciplinary aspect of requirements analysis, and the need to involve specialists from the behavioral, computer and management sciences in the requirements definition and validation process. There are also aspects of the process that defy precise structuring. It is important to remember that requirements analysis is as much an art as it is a science. There is a “feel” to requirements analysis that comes after an analyst acquires a great deal of experience. Good requirements analysts also tend to learn a great deal about the target applications area; some of them become almost expert at the very tasks they are trying to define. The use of generic task taxonomies is intended to guide the requirements analysis process, but the tasks in the taxonomies are not intended to replace those identified during the actual collection phase of the process. Figure 4 suggests how the taxonomies can be used to (a) guide the initial process, and then (b) yield specific tasks and sub-tasks in a resource allocation scenario. Alternative requirements analysis methods are presented in the figure as intervening and iterative variables. 2.6

User Profiling Methods

Who will be using the system? Will the system be used by those relatively unsophisticated in the use of interactive systems, or is the user group experienced? These and similar questions are related to user profiling, the second critical dimension of the requirements definition. Users come in many shapes and sizes. There are a number of ways to classify them as well. They can be classified by job function, by their level of experience with interactive computing, by their role in a larger problem-solving process, or by some combination of these and additional criteria. Ramsey and Atwood (1979) mix some criteria to produce at least three classes of users: 0 0 0

naive; managerial; and scientific-technical.

GENERIC TASK TAXONOHIES

Behavior Description

REQUlREMENTS ANALYSIS METHODS

Questionnaire & Survey Requirements Interview & Observation

Requirements

u

\ 1 Simulation & Oaming wethods

SPEC1FIC TASKS

RESOURCE ALLOCATION: 0 Gather Project Data 0 Prioritize Projects (By Benefit) 0 Prioritize By Cost 0 Conduct CostBenefit Analyses Rank-Order Project 'Investments" 0 vary costs & Benefits ( "What i f..." )...

Character istl

t I

FIG.4. The task requirements analysis process.

USER/

+ ORBANIZATIONAL

PRorEs

REQUIREMENTS MODELINW PROTOTYPINW STORYB M R DING


17

This classification of users according to their experience and job function can tell us a great deal about how an information or decision system should be designed, but it also leaves out some important information. Will the system be used by “frequent” or “infrequent” users? Will it be used by users under “situational” pressure? Will the users be part of a larger organizational hierarchy, such as always occurs in the military? User profiling, like task profiling, should begin with a look at some of the existing user taxonomies. But the profiler should make sure the taxonomies reflect the application, that they are based upon criteria meaningful to the community the profiler is trying to help. As a general rule of thumb, the following questions should be posed before, during, and after the user profiling process: 0 0

0

0 0 0

How experienced with interactive computing are the users? How experienced with analytical methodology are the users; are they inclined to think analytically or are they more “passive” users? How frequently will they use the decision support system? What cognitive “styles” will they bring to the system? To what extent is their behavior determined by their role or rank? How high are their expectations about what the system can do for them?

The answers to these questions (and others-see below) will yield a user profile that will inform the systems design and development process; without the answers the designers will speculate about-or ignore altogether-the kind(s) of problem-solvers that will operate the system. Unfortunately, user requirements analysis methodology is not nearly as well developed as task requirements analysis methodology. Methods for developing taxonomies based on experience and cognitive “styles” and requirements are thus not altogether different from task requirements methods, though the focus is very different. There are several ways to gather information about how experienced with interactive computing and analytical methodology the users are. Note that the experience that should be measured includes experience with computing and analytical methods, and analytical computing. These distinctions are important because many systems tend to be model-oriented while much user experience with computing is data-oriented. Users who feel very comfortable with a sophisticated database management system cringe at the thought of interacting with a trivial analytical program. Conversely, users familiar with modeling software often find data retrieval and display programs completely useless.

18

STEPHEN J. ANDRIOLE

Conventional requirements data collection methods, like interview and field observation methods, can yield a good deal of insight into the users experience with analytical computing. But for these methods to be effective a great deal of front-end work must be done. The following questions can be used to structure an interview or interpret field observation: 0 0 0

0

0

What is the nature of your prior experience with computing?; has it been primarily data or model oriented? Are you a frequent (more than ten times a month) or infrequent (less than ten times a month) user?; do you avoid computers whenever possible? Do you have any formal training in analytical methodology?; if so, in what methods? What analytical programs have you used? What are your expectations about decision support?

These and similar questions can be used to profile users according to their general computing experience and their experience with analytical computing specifically. Scales can be developed to measure this experience, though they need only be very crude. It is also possible to observe users in an analytical computing scenario where they must interact with a system that makes certain demands on their problem-solving skill. There are two kinds of methods for profiling users’ cognitive styles and capabilities. The first involves applying one or more generic descriptive cognitive taxonomies, while the second assumes that insight can be gained by applying a generic ability requirements taxonomy. These taxonomies can be used to organize field- and scenario-based observation exercises. If there is time, and the circumstances are right, questionnaires can be administered to profile cognitive preferences and problem-solving styles. There are a variety of questionnaires available that purport to measure cognitive capabilities and styles, though few of them have been scientifically validated. Cognitive profiling seeks to identify user perceptual and mediational processes. It is important to define these processes, because they tell us a great deal about how the system should be designed and how it should interact with its users. Cognitive profiles can suggest, for example, that graphic output is inappropriate, that the interaction pace should be slow, and that the analytical method in the system should be highly visible. By and large, cognitive profiling informs the design of the man-machine interface and the system’s behavioral characteristics. Figure 5 suggests how the user profiling process works.

REQUIREMENTS ANALYSIS HETHODS

GENERIC USER TAXONOMIES

[

Questionnaire & Survey Methods

Experience Taxonomies

€22 Taxonomies

Cognitive Taxonomies

-+

& Observation

1

SPECIFIC USER PROFILES

-b

Simulation &Gaming

FIG.5. The user requirements analysis process.

4-

TASKS/ ORBANIZATIONAL PROF1LE

1

REQUlREMENTS HODELI NO/ PROTOTYP I N 6 1 STORYBOARD1N6

20

STEPHEN J. ANDRIOLE

2.7

Organizational/Doctrinal Profiling Methods

Until recently, very little attention was given to the impact that an analytical problem-solving system might have on an organization or bureaucracy. After countless systems were thrown out due largely to their incompatibility with established eficient problem-solving procedures, designers began to take note of the environment in which their systems were expected to perform. First, unless the mission explicitly calls for it, designers should try to avoid creating the impression that the system will change the way things are now done. The most appropriate support image suggests that the system can help organize and expedite problems that are otherwise tedious and time-consuming. Many early support systems were enthusiastically accepted when they helped reduce information overload, filter and route information, and structure decision option selection problems. But resistance grew when they moved into the prescriptive provinces previously the exclusive preserve of humans. Worse yet are the decision aids and support systems that only try to change an organization’s structure but try to do it with exotic analytical methodologies that require six months of “interactive training” before the system can be used. Systems will fail if they are incompatible with the organizations they are intended to support, regardless of how well designed they are, just as mediocre systems will excel within organizations with which they are perfectly compatible. Designers must also understand doctrine and the requirements that it generates. If the focus here were on basic systems research, then the issue would not be as important, but since the focus is on applied systems design and development, the issue is unavoidable. The least developed requirements methodology is that available for organizational-doctrinal profiling. As suggested above, the interactive systems design community has only recently recognized the importance of organizational context. Consequently, there are relatively few methods for profiling the organization and its doctrine that must be served by the system. The two general methods discussed here include critical activity profiling and compatibility testing methods. It is essential that an organization’s “mission” be fully understood before the system is functionally modeled. Here the reference is not to the individual tasks that comprise the mission (which are the focus of task requirements analysis), but rather to the higher-level function the organization is supposed to perform. The relationships that the organization has with other organizations are also critical. Critical activity profiling methods are primarily observation oriented. They are also fed by voluminous mission descriptions (also know as “policies and procedures” manuals). It is important to identify and define an organization’s


21

critical activities, because while a system may well help individuals solve specific low-level problems, the solutions themselves may be incompatible with the organization’s mission and “modus operandi.” Compatibility testing methods provide insight into an organization’s modus operandi and provide the means for averting major inconsistencies and incompatibilities between the system, the organization, and its “doctrine.” What are the organization’s policies, procedures, and “protocols”? How are problems solved within the organization? What is the hierarchical structure? Can the flow of information in the organization be modeled? Is it clear where the system will fit within this flow? The methodology for profiling organizations comes from the organizational development community. In one study (Srinivasan and Kaiser, 1987),an attempt was made to measure relationships between organizational factors and systems development. It was determined that the characteristics of an organization can (positively or negatively) affect systems design and development progress. Another study (Baroudi et al. 1986) suggested that user involvement in the systems design process predicted to levels of user satisfaction and usage. These and other studies suggest the likely relationship between organizational profiles and the extent to which systems support or hinder organizational performance. It is safe to say that there is by no means an abundance of generic (or specific) organizational/doctrinal taxonomies targeted at interactive computer-based systems design and development. There are, however, a number of taxonomies that recognize organizational personalities and pathologies. Unfortunately, this literature is of limited use. The best way to proceed is to develop a set of questions, identify a set of issues, and analyze organizational manuals that shed light upon the organization’s mission and modus operandi. One should then gather some data via direct observation, supplement it with codified doctrine, and then develop a crude organizational profile as it pertains to the system under development. Figure 6 suggests how organizations can be profiled. 2.8

The Task/User/Organizational-Doctrinal Matrix

A good requirements analysis will enable one to construct a problemspecific three-dimensional matrix, as suggested in Fig. 3; it will also permit the development of a prototype. But why go through all the trouble? The reason is that numerous design issues can only be solved through the matrix. For example, user type@)will determine the type of interactive dialogue one should use. Tasks will determine the analytical method selected to drive the system, while organizational-doctrinal considerations will determine the system’s interface, input requirements, output, physical size, and security

QENERIC ORGANIUT IONAL TAXONOHIES

Structural Taxonomies

N

OR6ANIUT IONS

"Cultural" Taxonomies

REQUIREMENTS ANALYSIS HETHODS

& Survey

&Observation

SPECIFIC ORGANIZATIONAL PROFILES 0

MIssion

0

Policies & Procedures

0

"Personality "...

"Strategic & Tactici31"

FIG.6. The organization requirements analysis process.

TASKS/

i- USER

pRTLE

REQUIREMENTS NODELINW PROTOTIPINO/ STORYBMRDINB


23

characteristics. I t is essential that user, task, and organizational-doctrinal definitions be deueloped and integrated before the design process proceeds any farther. As Fig. 7 suggests, the requirements matrix leads directly to the prototype. Requirements define the hypothetical system concept embodied in the prototype. The importance of requirements analysis cannot be overstated in the overall design and development process. The whole point of prototyping is to validate requirements via some tangible representation of the system concept. The extent to which requirements are accurately identified initially will determine the number of prototypes necessary to validate requirements.

2.9 Some Prototyping Methods There are several ways to capture the essence of the system to be built. As soon as the requirements analysis is completed, the prototyping strategy requires the development of some kind of representation or model of how the system will operate. Remember that this prototype will be temporary; it is intended to introduce the system concept to the users. They will no doubt find it flawed, and it will be adjusted (again and again) until they are “satisfied,” knowing full well that they might never really be happy with the design (even as members of the design team!). Such is the fate of the systems designer. A good prototype, or requirements model, serves many purposes simultaneously. As suggested, it fosters discussion about what the system should and should not do. But it also verifies the results of the requirements analysis. As members of the design team, users can inspect the integrated model and recommend changes. Finally, the model permits the design team to display something to its users early on in the design process, something that stimulates the design team, pleases management, and convinces users that the team is dedicated to solving their problems. There is very little agreement about which prototyping methods work best. Some believe that conventional flowcharting is sufficient, while others demand a “live” demonstration of the system-to-be. There are at least four viable prototyping methods, including the development of narratives, the development of flowcharts, methods based upon other information theories and methods, and those that yield “storyboards.” 2.9.1

Narrative Methods

Narratives remain powerful communication tools. When well done, they can accelerate the design process. Ideally, a narrative should describe what the system will do, indicate its input requirements, describe and illustrate its

THREE-DIMENSIONAL REQUIREMENTS NATRIX

a l

HODELINB BETHODS

Narrative Methods

-b h) P

i i l Cherting

Storybogrding

4I

T

FIG.7. The requirements/modeling/prototyping process.

I

PROTOTYPING OPT IONS

Evolutionary

9 Hybrid

I


25

output, and suggest a softwarelhardware configuration. At the same time, it should not be so long or verbose as to discourage study. Its prose should be terse and to the point; it should also be illustrated with simulated screen displays. Narratives should be used only when the system is relatively uncomplicated, when the tasks to be performed are less than cognitive. They should also be used only when users will find them appropriate. Many military users, for example, would find narratives too tedious for serious study. 2.9.2 Flowcharting Methods We are all familiar with conventional (logic) flowcharts. In the hands of an experienced systems analyst, logic flowcharts are rich with information, but in the hands of a novice they are meaningless. But there are other flowcharts that can serve larger groups. Van Duyn (1982),for example, suggests that there are a variety of flowcharts that can be used to develop prototype system models, which include 0

0 0

0

0

0

Conceptual Flowcharts-pictorial presentations of the flow of information; General System Flowcharts- top-level visual presentations of the system intended for management inspection; Functional Flowcharts-visual presentations of the system, subsystem, or program showing the functions of data with no decision factors or other variables to distract the viewer; Logic Flowcharts-visual presentations of the flow of data through a subsystem and/or program, the location of decision processes, and the control of logic through switching or complicated decision processes. Logic flowcharts are the conventional ones intended to reduce coding and debugging time; Job Step Flowcharts-Visual presentations of a computer processing operation which often consists of one or more programs that process all input and generate all output; and Work Flowcharts-Visual presentations of the flow of paper or manual work.

2.9.3

Generic Model-Based Methods

Figure 8 presents some “off-the-shelf” modeling techniques that can be used to represent a particular system. So long as the problem area “fits” the model (and vice versa) one or more of the models may work, but one must be very careful to match the right model with the right requirements definition.

APPRDACH

DccisionTheory

Hodels

tlodcls o f Human Information Processing

DESCRl PT ION

COMHENTS

These models concern the decision-making behavlor o f the user. They require the Specification of: ( 1 1 a set o f possible states o f the world, with their estimated probabilities, and ( 2 ) a set of possible declslons, or courses o f action, which might be taken, together with their expected values and cost in the various possible states o f the world. Considering the values and costs, together with the evidence o f particular world states, a decision-theoretlc model can select courses of action.

Decision-theoretic models can be used to suggest 'optimal- decisions or to describe the observed decision-making behavior o f users. in both modes, these models are frequently used in decision aids. If it is reasonable to describe user behavior in terms o f such a model, these models can also be useful to the system designer, as by suggesting Information required by the user.

In general, these models Involve a characterization of: ( 1 ) the task environment,including the Problem and means o f solution available. (2) the problem space employed by the subject to represent the problem and Its evolving solution. and (3)the procedure developed to achieve a SOlUtiOn. The method used to develop such models involves intensive analysis of the problem to be solved and of protocols obtained from problem solvers during solution.

Ideally, such efforts might lead to an integrative model o f human information procexing useable in a variety o f design applicatlons. However, existing models are either too task-specific for thls use o r are insufficiently detailed. Futhermore,relstlonships between task requirements and human performance capabilities and limitations are inadequatetly understood for human Information processing tasks. There are many good models applicable to very specific tasks. I

Computer

System Hodels

These models attempt to describekhe behavior of t h e computer component of an interactive system. but do not attempt to model user performance in detail. Some o f the models do characterize user behavio- in terms o f the statistical properties of user commands f o r a particular application. The models usually attempt to predict such as system response time. CPU and memory loads, and I/O requirements.

1

,

These models tend to be relatively crude.but can be useful In determining whether or not user requirements with respect to response time and other gross system performance measures can be satlsfied by a proposed system. They are of little assistance in determining what the user requirements are.

FIG.8. Some modeling techniques.

APPROACH

DESCRl PT ION

COH Pl ENTS ~~~

Network tlodels

~

~

~

~~

These models treat user and system as equivalent elements in the over-all process. The individual task performed b y both the user and the system are described in terms of expected performance and in terms o f logical predecessor-successor relationships The relationships define a network of tasks which i s used as a performance model o f the user-computer system. Such models are usually used to predict either the probability o f failure o r -success', or the completion time, o f an aggregate set o f tasks.

Network models allow performance data about users and computer systems t o be integrated i n a single model even though original data came from a variety of sources. However, performance data must be provided f o r each task.as must rules for combining performance data from each individual task t o obtain aggregated performance predictions. This i s oefln difficult because of questionable o r lacking empirical data, and because performance interactions among tasks (especially cognitive tasks o r tasks performed in parallel) may be v e r y complex. Performance distributions are often assumed without data. In spite o f these difficulties, the process o f constructing such models i s often a valuable source o f understanding.

These models are based on control theory,statistical estimatlon, and decision theory. The user i s regarded as an element i n a feedback control loop. Such models are usually used to predict over-all performance o f the user-computer system in continuous control and monitoring tasks.

Control-theoretic models are more quantitative than other performance models. They may address user-computer communication broadly.but they ordinarily do not deal w i t h details o f the interface, such as display design. Therefore, their utility as an aid to the interface system designer may be limited. Not much work has y e t been done in applying these modeling techniques to situations in which the main user activities are monitoring and decisionmaking. w i t h infrequent control actions.

~~

ControlTheory tlodels

FIG. 8 (Cont'd). Some modeling techniques.

28

STEPHEN J. ANDRIOLE

Figure 8-from Ramsey and Atwood (1979)-describes following models: 0 0 0 0 0

and discusses the

Decision-theory models; Human information processing models; and Computer system models. Network models; Control-theory models;

2.9.4 Screen Display and Storyboarding Methods Perhaps the most useful prototype is one that displays to users precisely what they can expect the system to do-at least hypothetically. Paper copies of screen displays are extremely useful, since they permit users to inspect each part of the interactive sequence. Boar (1984) regards screen displays as acceptable “hybrid prototypes.”

FIG.9. Illustrative storyboard (I). This display suggests how the work space within the menu structure can be filled with accessible data and information. In this case, the data is map-based, suggesting to the user that the primary “object” of analysis and manipulation will be the tactical map.. . the display also suggests that the planner can access any of the “elements” of tactical planning which reside along the left side of the display . . .


29

While useful, paper screen displays pale against the impact of computergenerated (and animated) screen displays. Dubbed “storyboards,” computergenerated displays simulate man-computer interaction. With new animation packages, it is now possible to animate the storyboard and thereby mimic sophisticated interactive graphics capabilities. The animated storyboard and its paper equivalent provide users with the best of both worlds. The computer-generated storyboard permits them actually to experience the system, while the paper copy enables them to record their comments and suggestions. Each “run” through the storyboard becomes a documented experiment filled with information for the design team. The paper copies also comprise a permanent record of the iterative modeling process, an invaluable contribution to corporate or military institutional memories. Figures 9-13 suggest what storyboards look and “feel” like. A typical storyboard will have over a hundred such displays intended to communicate to users what the system will do, how it will d o it, and how users will be expected to work with it. When strung together, these storyboards will communicate an interactive system concept to users, who are then free to comment upon and criticize the concept, thus triggering the development of

FIG. 10. Illustrative storyboard (11). This display suggests how a user can execute planning tasks; in this case, the user is interested in the Mission and has selected information regarding the Mission’s Objectives. . .

FIG.It. Illustrativestoryboard(III).Thisdisplaydisplays theMission’sObjectivesto theuser via integrated text and graphics . . . the Corps area of interest/operations/influenceis displayedto the planner; the Objectives are also described in abbreviated text.. . the integration of text and graphicssupports important cognitive functions . . .

FIG.12. Illustrative storyboard(IV). The planner selects Blue COAs . . . 30


31

FIG. 13. Illustrative storyboard (V). Blue COA #Z is displayed . . .

the next prototype (or the enhancement of an evolutionary one)-all suggested by Fig. 1.

as

2.9.5 Software Specification and Engineering

Only after a credible system concept has emerged should the design team turn toward the specification and development of software. There are a variety of tools available for software specification, including the now-popular computer-aided software engineering (CASE) tools, which permit the implementation of a variety of diagramming techniques, such as data flow diagramming and other forms of structured modeling. Some of the more sophisticated CASE tools provide software engineering environments that support actual programming. 2.9.6 Hardware RequirementslHardware-Software Configura lion

Figure 1 suggests that the information systems engineer also needs to consider hardware requirements and the overall hardware/software configuration. In more than a few instances hardware will be pre-selected prior to

32

STEPHEN J. ANDRIOLE

the requirements analysis. Unfortunately (for designers), the target system will have to be developed on existing hardware (and often in a pre-specified, highlevel language). On those occasions when there is hardware flexibility,then the hardware configuration should match user requirements and what the software engineers believe is the best way to implement the system concept. 2.9.7

Testing and Evaluation

There are at least two ways to think about testing and evaluation. On the one hand, software should be tested to make sure it is doing what it is supposed to do. The emphasis here is on algorithmic testing. There are a number of tools and techniques available to the software tester, including quality assurance, fault tolerance and redundancy testing, among others (Fairley, 1985). But on the other hand is the kind of evaluation conceived at a much higher level of abstraction. Here evaluation is focused on measuring the extent to which the system satisfies user requirements. If a system’s algorithms fire beautifully but the system itself fails to support its users, then the enterprise has failed. Methods for system evaluation include multi-attribute utility (MAU) assessment (Adelman, 1990), cost-benefit and hybrid methods. 2.9.8 Documentation and Maintenance

Systems are not complete until they are documented and a feasible maintenance plan is developed and implemented. Documentation should include (at a minimum) a users’ manual, a functional description of the system, and the software specifications. A “manager’s guide” is also useful. Documentation can be embedded in the system and/or paper-based. Good documentation bridges the gap between a system’s description and training. A good maintenance plan is realistic and field-tested before it is installed. It is essential that users not be left on their own and that the design and development team be ready, willing and able to support their system. This has budgetary implications that must be appreciated throughout the design and development life cycle.

3.

The Domain of Command and Control

3.1 The Command and Control Process

Command and control (C’) is part of the force effectiveness process, as Fig. 14 suggests. C2 is an element of force effectiveness, as well as a means for


33

OVERALL FORCE EFFECTIVENESS

&

PERFORMANCE

/ I

W E A P 0 N

S

/

Deve 1opment DEPLOYMENT

&

IMPLEMENTATION

’ . . ,

PROCUREMENT

TEST I NG

4

4

WEAPONS SYSTEMS DEVELOPMENT

PERSONNEL SYSTEMS DEVELOPMENT

P E R S 0

N N E L -

FIG.14. The force effectiveness process.

the enhancement of human performance. Figure 15 suggests that computerbased decision aids and larger information systems can support a number of activities, including C2. Command and control (C2)is the process by which military a,nd civilian “commanders” exercise authority and direction over their human and material resources to accomplish tactical and strategic objectives (JCS, 1976). C2 is accomplished via the orchestrated implementation of a set of facilities, communications, personnel, equipment, and procedures for monitoring, forecasting, planning, directing, allocating resources, and generating options to achieve general and specific objectives. In industry, managers and corporate leaders identify market objectives and then mobilize resources to achieve them; in the military, commanders plan and execute complicated, phased operations to fulfill their missions. Commanders in industry mobilize factories, aggressive managers, line workers, and their natural and synthesized resources to produce superior products. Commanders in the military mobilize weapons, troops, and sophisticated communications apparatus to defend and acquire territory and associated military and political objectives.

PLATFORMS SENSORS

7

WEAPONS C2, INTELLIGENCE

INFORMATION & DECISION SYSTEMS DOD PERSONNEL

-b SELECTIOND -1 ORGANIZATION & TRAINING

FIG. 15. The range of information and decision systems applications.

READINESS & EFFECTIVENESS


35

Decision-making lies at the heart of C2. While commanders will always need data, information, and knowledge to inform their decisions, the decisionmaking process itself can be supported by C2 decision support systems. Such systems support the “cognitive” functions of the commander. Some of these include the nature of threats, assessments of his or her organizational capabilities, and the identification of operational opportunities. C2 decision and information systems also recognize decision-making constraints, such as limited time and incomplete and ambiguous information. Figure 16 suggests the range of Cz information and decision systems opportunities (Andriole, 1987a-e). There are currently a variety of decision and information systems that support decision-making in the cells in the matrix. There are systems that support decision-making at the National Military Command System level, for the Unified and Specified Commands, the Services and in the field. Note also that Fig. 16 indicates that there are strategic, theater, allied and tactical levels, and that decision-making is presumed to be very different at various points along the war-peace continuum. Figure 17 suggests the range and complexity of the C2 decisions that a Tactical Flag Commander must make. Commanders at all levels and in all branches of the military must solve similar problems and make the same kinds of decisions. 3.2 Command and Control Information and Decision System Requirements Perhaps the best way to understand where C2decision aids and information systems can help the most is to identify the special problems that commanders routinely face. Some of these problems include 0

0

Sub-optimal information management information “overload” difficulty finding key information poor information presentation incorrect information ambiguous information incomplete information Limited option generation and implementation Limited alternative generation sub-optimal option evaluation limited scenario generation capabilities limited real-time simulation capabilities.

36

PLAN REVIEW/

COMMANDER

PREPARATION

FIG. 17. Tactical flag command decision-making.

37

38

STEPHEN J. ANDRIOLE

r

DIFFICULT DATA BASE SEARCHING

& & POOR INFORMATION PRESENTATION

I

INFORMAT’ION OVERLOAD A

t

INEFFECTIVE INDIVIDUAL h GROUP DECISION MAKING FIG.18. C2 information processing and decision-making problems.

These and additional problems are summarized in Fig. 18. Figure 18 also suggests where C2 information and decision systems can yield the greatest payoff. Since approximately 1979, a date that marks the beginning of the widespread proliferation of microcomputers throughout the defense establishment, decision aids and larger information systems have been designed, developed and applied to a variety of C2 decision-making problems. C2 information and decision support systems help commanders discriminate among alternatives, simulate the implementation of options, and evaluate the impact of decisions made by commanders in various situations. They help commanders test assumptions, perform “what-if” analyses, and conduct decision-making “post mortems.” C2 information and decision systems support the C2 process in a variety of ways. Figure 19 suggests the range of C2 information and decision systems applications. It also suggests the varied nature of C2requirements. The C2 information and decision systems engineering process is correspondingly broad and complex, as suggested in Section 4.


39

FIG. 19. C2 requirements and aiding applications areas.

4.

Command and Control Information and Decision Systems Engineering 4.1

C2Information and Decision Systems Requ irements Ana Iys is

The design and development of information and decision systems intended for use by relatively inexperienced computer users to solve analytical problems is fundamentally different from the design and development of systems intended to provide inventory control support to frequent users. Those that design and develop C2 information and decision systems have, accordingly, perfected their requirements analysis techniques. Many of these techniques rely upon the structured interviewing of commanders to determine their decision-making needs. C information and decision support systems designers have also endorsed the “rapid prototyping” approach to systems design, since it is so difficult to capture C2 decision-making requirements the first time. C2 systems designers thus build several working prototypes of their systems to help validate requirements before proceeding with full-scale system

40

STEPHEN J. ANDRIOLE

development. Finally, the C2 ISE community has devoted a great deal of attention to how systems can be evaluated. C2 systems designers identify and refine decision-making requirements by employing a number of methods. These include survey and questionnaire methods, methods based upon interviews and (direct and indirect) observation, and simulation and gaming-based methods. The key to C2 requirements analysis lies in the identification of the essential decision-making tasks, tasks that when well performed can significantly enhance C2 decision-making performance. The requirements analysis methods are employed to identify not only critical C2 decision-making tasks, but profiles of the users and organization in which the system will reside as well. User profiles, tasks profiles and profiles of the organization comprise the requirements equation, as suggested in Fig. 20. Figure 20 presents a three-dimensional requirements matrix that illustrates the intersection of tasks, users and organizational characteristics. Each cell in the matrix represents a requirements challenge. The tasks in the matrix are generic; in practice, a C2 requirements analyst would convert those generic tasks into very specific tasks (pertaining to, for example, resource allocation problems, tactical planning, and target value analysis). Perhaps the same requirements analyst would specify users in greater detail than simply inexperienced/experienced/infrequent user/frequent user. Organizationaldoctrinal characteristics might also be specified in greater detail. Regardless of the level of detail (and the methods used to achieve it) the requirements matrix suggests that prudent designers of C2 information and decision systems deal with all three dimensions of the requirements challenge. 4.2

C2System Modeling and Prototyping

Prototyping is sanctioned by the C2 design community because it is so difficult to identify and refine C2 requirements (especially decision-making requirements) the first time through the requirements process. The prototyping premise calls for the design and development of a working model of the information or decision support system under development, the solicitation of reactions to the model from prospective users, and the refinement of the model when requirements can be validated. Prototyping calls for iteration. It also calls for the development of two kinds of prototype systems: “throwaway” and “evolutionary” systems. Throwaway systems are used when requirements are especially difficult to capture; evolutionary ones are used when C2 requirements are less elusive. A great many C2 information and decision systems tackle problems for the very first time. Many Cz functions and tasks have been manual for years; because information technology has evolved so quickly, requirements that

o

F

/

Activate Adjust Syncronize...

/

Advise

,f

Request...

@

Information Processing Problem-Solving and Decision-Making ...

1$

Search For / Receive Information Identify Objects, Actions, Events...

/

INEXPERIENCED (1)

EXPERIENCED (€1

I/E INFREQUENT

FIG.20. C2 user/task/organizational-doctrinalrequirements matrix.

42

STEPHEN J. ANDRIOLE

were once believed too difficult to satisfy are now yielding to creative systems designs. While the results are often impressive, “evolutionary development” is almost always necessary.

4.3 Analytical Methods for C2Information and Decision Systems Engineering Those who design, develop, and evaluate C2systems call upon the social, behavioral, engineering, mathematical, computer and management sciences. C2 information and decision systems design and development is multidisciplinary by nature and necessity. A variety of analytical methods and other tools and techniques are available to the designer of advanced C2 systems. Figure 21 identifies the major decision-aiding technologies. Czinformation and decision systems designers have a variety of analytical methods at their disposal. The key lies in the correct matching of analytical methods to problems. There are several primary methods classes worth discussing here. They include decision analytic methods, operations research methods, methods derived from computer science and artificial intelligence, and methods derived from the field of human factors engineering. 4.3.1

Decision Analytic Methods

Some of the methods, tools and techniques used to drive Cz systems include utility/value models, probability models, and mixed value-probability models. Utility/value models come in a variety of forms. Some are based upon conventional cost-benefit models and assumptions. Some are based upon the treatment of value as “regret,” that is, the “flip side” of utility, since many Cz commanders perceive costs more vividly than benefits. Others are based upon multi-attribute utility assessment (MAUA) models. MAUA models are powerful tools for assessing the relative value of alternative courses of action. The methodology is generic. It can be used to assess the relative value of courses of action, personnel, or objects or processes of any kind. In the civilian sector, MAUA models are used to assess the value of alternative sites for new factories, alternative business plans, and corporate jets. In the military, they are used to assess alternative tactical plans, the performance of competing weapons systems, and the value of alternative investments in high technology. Probability models, including probability trees, influence diagrams, and Bayesian hierarchical inference models, identify data, indicators, events, and activities that when taken together predict the likelihood of single or multiple events. Figure 22, from the Handbook for Decision Analysis (Barclay, et al., 1977), presents a Bayesian hierarchical inference structure intended to determine the likelihood of a country developing a nuclear weapons capability.

Berations Research 0 Deterministic Models/ Optimization 0 Stochastic Models Pattern Recoanition 0 Discrimination Technique 0 Classification Techniques

->ANALYSES

Decision Analvsis Utility/Value Methods Probability Models Mixed (ValueProbabillty) Models Knorledaa-Based lechniaues Expert Systems Planning b Problem Solving Pattern Directed Inference

HAWAGEHENT

Database Management Document Retrieval Message Processing

INTERFACES

Advanced Interface Techniaues 0 MapITerrain Display Systems Natural Language Interfaces New Input Techniques (Voice Recognition) Hum811F8ctors Enaineerinq Iechni aues Man/Machine Interfaces Embedded Training

FIG.21. Major information processing and decision-aiding technologies.

44

H1 H2

STEPHEN J. ANDRIOLE

- Country A Intends t o develop a nuclear weapons capability within 5 years - Country A does not intend to develop a nuclear weapons capability within 5 years

DATUH 1

ACTIVITY I

DATUM 2

COUNTRY A THREATENS COUNTRY B WITH USE OF "DRASTIC" WEAPONS IF TERRORISM CONTINUES

NUCLEAR RhD PROGRAfl

R&D PROGRAM DIRECTORS MEET IN WEEK-LCNG CONFERENCE

INDICATOR I

INDICATOR 2

ENRICHMENT PLANT EXPANSION

INCREASED USE OF NUCLEAR

PHOTO-RECON. ADDITIONAL COOLING TOWERS

20% INCREASE HEAVY WATER

DATUM 3 NO OBSERVED CHANGE IN

PLUTONIUM WATER TO OTHER COUNTRIES CANCELLED

ACTIVITY 2 HIGH-EXPLOSIVE R h D PROGRAM

INCREASED SCIENTIFIC ACTIVITY

DECREASE IN PUBLICATION ON HIGH-EXPLOSIVE RESEARCH

FIG.22. Hierarchical inference structure for nuclear weapons production.

In its computer-based form, the model permits analysts to determine how new evidence affects the likelihood of a given country's intention to develop nuclear weapons. The model works via assessments of the relationships among data, indicators, and activities that chain-react up the model to determine the probability of the hypotheses that sit at the top of the structure. Mixed value-probability models often drive C2systems. The most common form of the mixed model is the probability tree, which generates values for outcomes given the likelihood of events and the value of their occurrence. 4.3.2

Operations Research Methods

There are a number of tools and techniques that comprise the range of operations research methods (Thierauf, 1978). Several that deserve special


45

mention include linear programming, dynamic programming, integer programming, queuing theory, aspects of systems analysis, and even the classic quantitative-empirical inferential statistical methods. Linear programming is representative of operations research methods that seek optimization. Linear programming methods can be applied to complicated resource allocation and optimization problems when the following conditions exist (Thierauf, 1978): 0 0 0 0 0

the parameters of the problem constitute a linear function; alternative resource mixes are possible; the linear functions (and constraints) can be expressed nathematically; the mathematical relationships among variables can be mapped; and resources are finite (and quantifiable).

Linear programming enables a problem-solver to optimize the allocation of resources according to a specific goal. There are two primary linear programming methods: the graphic method and the powerful and popular simplex method. The graphic method involves the plotting of the linear function and constraints in a multidimensional space and then solving the simultaneous equations of the plotted lines. The simplex method involves the implementation of an iterative mathematical process until the best solution is found. Linear programming methods are flexible because they permit asset, constraint, and goal manipulation. Dynamic programming methods also account for time intervals. These and related optimization methods can be used to solve a variety of C2 problems, including especially route planning, resource allocation, weapons assignment, equipment reliability assessments, production planning, and the numerous “assignment” problems that surround so many C2 decisions.

4.3.3 Computer Science and Artificial Intelligence Methods Computer science is a discipline with roots in information theory and mathematics that links electronic data processing with data and models for data storage and retrieval. The tools and techniques of computer science make it possible to implement a variety of analytical methods that are more accurately located within one or more of the above categories. Pattern recognition, queuing, networking, inventory modeling, and simulation, while quite frequently considered computer science methods, really belong to the operations research community. Database management methods really belong to the management science community, while document retrieval methods belong to librarv and information science. The key to understanding

46

STEPHEN J. ANDRIOLE

the range of methods (from any of the classes) lies not in strict definitions of disciplines or fields of inquiry, but in the development of comprehensive, nonredundant taxonomies of methods. Ideally, such taxonomies will be anchored in the social, behavioral, engineering, computer, mathematical and management sciences. “Conventional” algorithmic methods refer to those used to collect, refine, store, route, process and create data and information for specific problemsolving purposes. In many cases, this amounts to writing algorithms to implement decision analytic, operations research, or management science methods. On other occasions it reduces to the development of tabular and graphic displays, while on still others conventional computer science methods are applied to database housecleaning chores. Artificial intelligence (AI) methods seek to identify, codify and process knowledge. A1 systems differ from conventional ones in a number of important ways. First, conventional systems store and manipulate data within some very specific processing boundaries. A1 systems store and apply knowledge to a variety of unspecified problems within selected problem domains. A1 systems can make inferences, implement rules of thumb, and solve problems in certain areas in much the same way humans solve problems. The representation of knowledge is the mainstay of AI. There are a number of options available to the “knowledge engineer,” the A1 systems analyst with responsibility for converting problem-solving processes into executable software. The most popular knowledge representation technique is the rule, an “if-then” formalism that permits knowledge engineers to develop inferential strategies via some relatively simple expressions of knowledge. For example, if a tank will not start, it is possible to write a series of rules that represent the steps a diagnostician might take to solve the problem: if if if if

the engine will not start, then check the battery; the battery is OK, then check the solenoid; the solenoid is OK, then check the fuel tank; and the fuel tank is full, then check the starter.. .

These simple rules might be expanded and re-ordered. Hundreds of rules can be used to perform complicated diagnostic, maintenance, and planning tasks. Some other knowledge representation techniques include frames, inference networks, and object-attribute-value triplets (Andriole, 1986). All knowledge representation techniques strive to represent the processes by which inferences are made from generic knowledge structures. Once knowledge is represented it can be used to drive expert systems, natural language processing systems, robotic systems and vision systems. Expert systems


47

embody expert knowledge about problem-solving; natural language systems permit free-form interaction with analytical algorithms; robotic systems use knowledge to identify objects and manipulate them through complex environments; and vision systems intelligently process images and “see,” recognize, and respond to their micro environments (Andriole and Hopple, 1988). 4.3.4

Human Factors Engineering Methods

In addition to the analytical methods discussed above, there are several classes of methods that the C2 information and decision systems designer must understand. These include all those pertinent to the actual means by which the system is connected to its user. Here the reference is to the usercomputer interaction routines, the use of appropriate displays, error handling, response time, and all those issues relevant to how easy the system is to use and how productive it makes its user. All of these issues, tools, techniques and methods fall under the general auspices of human factors engineering (Norman and Draper, 1986). 4.3.5

The C2TaskslMethods Matching Process

Analytical methods are best exploited when they “match” a specific C2 requirement. Figure 23 suggests that the selection of an analytical method cannot be made independent of the requirements the system under development is intended to satisfy. The tasks/methods matching step in the C2 systems design and development process is critically important to the application of successful systems (Andriole, 1989b). 4.4

C2Systems Evaluation

C2 information and decision systems are evaluated somewhat more comprehensively than conventional software systems. The reason why is simple. C2 information and decision systems are inherently user-oriented. Evaluations of their performance must therefore attempt to measure the extent to which the system supports requirements, interfaces well with its users, supports the organizational mission it is intended to serve, contains efficient algorithms, can be maintained, and is (well or badly) documented. In other words, systems evaluation deals with all of the conventional software performance issues as well as those that pertain to how well or badly the system serves its users and organizations. One of the very best approaches to the evaluation of decision aids and support systems belongs to Adelman (1990). Figure 24 presents his

I I

I

C 2 REWIREHENTS

C 2 PROBLEM ASSESSHENT

I

ANALYSIS

c IDENTIFICATION OF CANDIDATE ANALYTICAL HETHODS I

8

I

I

I

I

DECISION ANALYTIC METHODS

Al HETHODS

OPERATIONS RESEARCH HETHODS

C2 TASKIMETHODS HATCHING

+

FUNCTIONAL SYSTEH HODELIN6 & PROTOTYPING

h SOFTWARE

FIG.23. C2 information systems engineering and analytical methods selection process.

1

0.0 I .C A i d N s e r Interface 1 .I flatch w i t h personnel 1.1.1 Training h technical background

1.1.2 Work style, workload and interest 1 . I .3 Operational needs 1.2 Aid characteristics 1.2.1 General 1.2.1 . I Ease of use 1.2.1.2 Understanding Aid's processes 1.2.1.3 Ease of training 1.2. I .4 Response time 1.2.2 Specific 1 2.2.1 User interface 1.2.2.2 Completeness o f data files 1.2.2.3 Accuracy o f expert judgements 1.2.2.4 Ability to modify judgements 1.2.2.5 Understanding o f Aids algorithms 1.2.2.6 Utility of graphs 1.2.2.7 Utility o f print-outs 1.2.2.8 Understanding o f text

Overall Utility

2.0 User/Aid Organization 2.1 Efficiency factors 2.1 .l Acceptability o f time for 2.1.1.1 Task accomplishment 2.1.1.2 Data management 2.1.1.3 Set-up requirements 2.1 .2 Perceived reliability under average battle conditions 2.1.2.1 Skill availability 2.1.2.2 Hardware availability 2.2 flatch w i t h organizational factors 2.2 1 Effect on organizational procedures and structure 2.2.2 Effect on other people's position in the organization 2.2.2.1 Political acceptability 2.2.2.2 Other people's workload 2.2.3 Effect on information flow 2.2.4 Side effects 2.2.4.1 Value in performing other tasks 2.2.4.2 Value to related organizations 2.2.4.3 Training value

3.0 Organization/Environment

FIG.24. Adelman's multi-attribute utility evaluation model.

3.1 Decision accuracy 3.2 Match between Aid's technical approach and problem's requirements 3.3 Decision process quality 3.3.1 Ouallty o f framework f o r incorporating judgement 3.3.2 Range o f alternatives 3.3.3 Range o f objectives 3.3.4 Weighting o f consequences o f alternatives 3.3.5 Assessment o f consequences o f alternatives 3.3.6 Re-examination o f decision= making process 3.3.7 Use o f information 3.3.8 Consideration o f implementation and contingency plans 3.3.9 Effect on group discussions 3.3.10 Effect on decisionmaker's confidence

50

STEPHEN J. ANDRIOLE

multiattribute utility assessment structure for the evaluation of information and decision systems. Note the orientation to users, organizations, and accuracy. In a full-blown evaluation this structure would be used in conjunction with a conventional software quality assurance model to determine how well the system operates and supports users.

5.

Case Studies in the Deslgn, Development, and Applicatlon of C2Information and Decision Systems 5.1 The Range of Applications

Cz information and decision systems have been applied to a range of Cz problems. Many of these systems have remained as prototypes, since they were intended only to demonstrate or prove a concept. Many others, however, have enjoyed operational success. 5.1.1

Some Working Prototypes

CONSCREEN is a prototype system designed to assist tactical planners in the identification and evaluation of alternative courses of action (Martin, et al., 1983). The method that it uses is multi-attribute utility assessment. The system calls for the user to evaluate a number of courses of action vis-a-vis a set of criteria designed to measure each course of action’s desirabilitycriterion by criterion. Figure 25 presents the criteria in the multi-attribute utility structure used by planners to evaluate alternative courses of action. After planners assess the alternative courses of action, the system calculates the plan’s overall utility, or value, to the planner. The system generates a rankordering of plans according to the extent to which they satisfy the criteria. OBlKB is an expert system that helps tacticians manage the order of battle (Weiss, 1986). OBlKB uses rules to help commanders make real-time battle management decisions. The system uses a graphic (map-based) interface intended to widen the communications bandwidth between the system’s knowledge base and its users. The specific tasks that the prototype performs include tracking enemy movements, identifying enemy location and disposition, and generating estimates of the battlefield situation. KNOBS (Knowledge-BasedSystem)assists tactical Air Force commanders in mission planning, specifically offensive counter-air mission planning (Tachmindji and Lafferty, 1986). The system assists commanders in the allocation of their resources, selecting the weapons that should be matched with known targets, prioritizing the air tactics that should be used, and the assessment of the impact of constraints, such as adversary behavior and ’

MISSION ACCOtlPLISHtlENT

I

THE~TER CONSIDERATIONS

GLOBAL CONSIDERATIONS I

I

READINESS

DOMESTIC

OBJECTIVE OFFENSIVE

1

ECONOMY OF FORCE

UNITY OF COMMAND

I

SURPRISE

I

I

INTERNATIONAL

52

STEPHEN J. ANDRIOLE

weather. KNOBS is knowledge-based, that is, it uses a knowledge base consisting of frames and rules about mission planning. 5.7.2 Operational Systems

Information and decision systems are used by commanders in a number of environments. They are used to assess the value of alternative targets. They are used for complex route planning. They are used to match weapons to targets. Many of these systems are “embedded” in much larger weapons systems, while others are “stand-alone” adjuncts to larger systems. Several prototypes have led to the issuance of “required operational capability” memoranda which have, in turn, produced support for the development of full-scale decision-making and larger support systems. In the late 1970s, for example, the U.S. European Command (EUCOM) required the capability to perform real-time decision analyses of complicated tactical option selection problems. This requirement led to the development of several systems for EUCOM use. The U.S. Air Force is embedding decision aids in cockpits as well as in the Tactical Air Commands. The Navy is using information systems for the placement of sonar buoys, for battle management planning, and for distributed decision-making. The Worldwide Military Command and Control System (WWMCCS) is perhaps the military’s largest information system that embeds a variety of decision aids. Information and decision systems are used in the intelligence community to estimate adversary intentions, forecast international crises, and assess the likelihood of new weapons programs in selected countries.

5.2 The Group Planning Prototype The tactical group planning systems engineering process represents how generic ISE principles are often applied to design and develop Cz information and decision systems. This section describes how requirements were converted into a working prototype, This section describes progress made by George Mason University’s Department of Information Systems and Systems Engineering (GMU/ISSE) on a project funded by TRWs Internal Research and Development (IR&D) program. The research was clearly directed: design and develop some prototype decision support systems predicated upon the importance of group problem-solving and the arrival of high-resolution large-screen displays. It is described here because it represents an example of structured information systems engineering. Requirements were identified and modeled, and an interactive prototype was developed.


53

The research areas of interest during the project included the need for interactive systems to support group problem-solving, the need to design and develop systems that could be used in actual operational environments, and how large-screen display technology can be exploited for enhanced human problem-solving. The substantive area of interest was (Army) tactical planning at echelons above Corps.

5.2.7 Information Systems Engineering Backdrop The project assumed from the outset that a structured systems design and development would yield useful results. We adopted the classic systems engineering life cycle (DSMC, 1986) and then modified it for information systems engineering purposes. A substantial amount of project resources were thus devoted to requirements analysis and functional modeling (see Section 5.2.2). 5.2.2 Requirements Analyses We began the research with requirements analyses in both domains. The first analysis identified the tasks and sub-tasks that tactical planners (at echelons above Corps) must perform to develop Concepts of Operations. A “requirements hierarchy” was developed that identified and described each task and sub-task. We then developed a second hierarchy that focused exclusively upon user-computer interaction (UCI) requirements (Andriole, 1987a, 1987b). Both of these hierarchies-with narrative equivalentsappear in Appendix A. We then identified a set of group requirements as an overlay to the substantive and UCI requirements (Andriole, 1988). Some of these include the (system’s) capability to share data and knowledge, share and challenge perspectives, defuse biases, focus discussion, and present data, inferences, options, explanations and recommendations in ways compatible with the requirements, the characteristics of users, and the organization structures the systems might eventually support. Our requirements analyses also developed some profiles of planners and crisis managers, which by and large characterized these users as inexperienced with advanced computing and likely to make intermittent use of interactive group decision support systems (Ehrhart, 1988).

5.2.2.7 Substantive Requirements The substantive requirements that we identified-as suggested in the hierarchies-were varied, reflecting the variety of tasks and sub-tasks that decision-makers perform as they generate courses of action. Theater planners require information about terrain, adversary capabilities, their own combat capabilities, and other aspects of

54

STEPHEN J. ANDRIOLE

tactical planning. The substantive requirements hierarchies identify all such tasks and sub-tasks, arranging them in way that permitted us to convert them into system (interface) characteristics. 5.2.2.2 UCl Requirements In addition to substantive requirements hierarchies, hierarchies were developed that identified the unique user computer interface (UCI) requirements that the users and the substantive requirements would necessitate. The UCI requirements were decidedly visual, suggesting the need for interactive and animated graphic displays. This requirements finding was not surprising: planners and crisis managers are trained in the visual and graphic via the use of maps and the communication of complex concepts such as risk, opportunity, and constraints (Andriole, 1987e). All sorts of display requirements emerged. Some were anchored in the need for graphic equivalence (Andriole, 1986) of complicated data sets. Some required animation, while others required graphics for direct manipulation of battlefield data and processes. The system concept that emerged reflected these requirements and our response to them. 5.2.2.3 Group Problem-Solving Requirements We identified a set of group problem-solving requirements as well (Andriole, 1988). These requirements were screened with reference to the substantive and UCI requirements. The intention was to restrict group requirements to the domains, users and unique interface requirements therein; the system concepts that emerged reflected this screening. Group decision support systems design and development has received some increased attention over the past few years (Stefik, et al., 1987; Sage, 1988; DeSanctis and Gallupe, 1987; Hart, et al., 1985; Schweiger, et al., 1985; and Gladstein and Reilly, 1985). Much of this literature is abstract and theoretical, while much less is applied. The storyboard prototypes developed during the project were problem-driven, not theoretically inspired, except where theoretical findings mapped clearly onto the application area. (At the same time, it is safe to say that theoretical work in human factors, cognitive science, and UCI technology played a larger role in the design and development of the prototype.) 5.2.3 Storyboard Prototypes

The ISE process suggests that before programming can begin, requirements must be identified and modeled. Storyboarding is a technique that converts requirements into system concept, a model of what the system will do when it is programmed (Andriole, 1989b). The working model of the system is a


55

prototype designed to accelerate the requirements validation process. The purpose of the prototype is to enhance communication between systems analysts and users. We designed and developed several interactive storyboard prototypes for the project and we converted the substantive and UCI requirements into interactive system concepts that represented how the system might operate when actually programmed. 5.2.3.1 Master Menu Structures The master menu structures were designed with inexperienced users in mind, with substantive and UCI requirements in mind, and with the intention of eventually deploying the system via a large-screen display system. We thus gave up substantial portions of the displays-for stationary on-screen menu options-since the impact would be less keenly felt when projected on a larger screen. The master menu structure-along with a representative set of screen displays-appears in Appendix B. 5.2.3.2 Prototype Capabilities The storyboard prototypes were developed on an Apple Macintosh 11. Color was utilized to communicate a variety of concepts, options, data, and information. The displays in Appendix B suggest how the storyboard can be operated, its interface capabilities, and what the displays look like to users. The menu structure permits users to execute commands by clicking on just a few icons which represent the elements of tactical planning and counterterrorism crisis management and available options. Another way to describe the interface and operation of the prototype is to liken the elements to “objects” and system commands as “functions.” It is possible to convert narrative commands such as “show us the courses of action that G2 expects Red to consider” by clicking on two icons, “Show” and “Enemy COAs.” It is possible to overlay weather onto the COAs by clicking on “Overlay,” “Area Characteristics,” and “Weather.” It is possible to “mix and match” commands to create great flexibility, and thereby permit users to access information in the system in non-sequential random ways. This flexibility is important to the overall system concept. Many systems permit users to interact with the system’s contents in rigid ways; we tried to design a system concept and interface that would permit users relatively unrestricted access to the system’s data and knowledge bases. 5.2.3.3 System Sizing We undertook a “sizing” effort to determine how difficult it would be to convert the working prototype into a working system. It was determined that programming would indeed be feasible on the Apple Macintosh I1 (or on a variety of other systems); it would also be possible to

56

STEPHEN J. ANDRIOLE

develop the necessary data and knowledge bases for selected domains (Andriole, 1987e; Andriole, et al., 1988). 5.2.4 IS€ Results

Modern computing technology permits us to design and develop interfaces and interaction routines that a few years ago would have been prohibitively expensive. The storyboard prototypes demonstrated some new approaches to UCI design, group decision support, and the use of large-screen display technology. The prototypes suggested that large-screen-based decision support systems can be used to help solve analytical problems via option generation and inference-making. Previous large-screen display-based systems have excelled in conventional database management tasks but have failed to provide analytical support to users. We believe it is possible to go far beyond conventional display capabilities. The prototypes also suggested how requirements can be converted into system concepts that exploit advanced display technology. The UCI requirements hierarchies developed during the course of the project identified a variety of simple and complex displays that could be implemented only with powerful graphics capabilities. We learned how to leverage color, graphics, animation, and the like during the course of the project. We also designed an interface that is very flexible, thus relieving users from any significant learning burden, It is possible to use the prototypes immediately. If the actual system existed, it is difficult to imagine the need for a users’ manual longer than a few pages. The prototypes also illustrated the power of direct manipulation interfaces (Potter, 1988). Next-generation personal’ and group-oriented workstations will employ such interfaces, since they link users directly with system capabilities, permit easy interaction, and provide systems designers with modular flexibility for enhanced evolutionary development. Many of the requirements identified during the requirements analysis phase of the project could not be satisfied with the project’s hardware/software configuration. Here the reference is to visual requirements such as actual maps, photographs, video information and sound, among other “media.” Next-generation systems will incorporate multimedia technology directly into their designs (see Section 6.9). Planners will, for example, be able to drive down highways, peer over bridges, and assess key terrain via film footage of the area of operations. Groups will be able to move from conventional alphanumeric data and information to multimedia with ease. Information systems engineers will be able to satisfy user requirements with a larger arsenal, providing users with presentation technologies that will narrow the communications bandwidth between users and system capabilities.


6.

57

Next Generation Command and Control Information Systems Engineering

6.1

Emerging Issues and Challenges

The design, development and use of information and decision supportswithin and beyond the domain of command and control-will change dramatically as we approach the 21st century. Our expectations for what these systems should do are rising as rapidly as the requisite technology is evolving. By the year 2000, problem-solvers will use advanced systems to deal with all sorts of simple and complex problems. They will also benefit from systems capable of providing much more than database support and low-level inference-making. Just as important, the distribution of computing power will be expanded beyond specialized professionals; information and decision systems will be available to us all on and off the job. This section examines the new trends, and attempts to describe how C 2 information and decision systems will be designed, developed and used a decade or so from now. It is thus speculative, though anchored in trends definable today. The section addresses the following questions: 0

0

0

How will definitions of information and decision support evolve over time? How will emerging methods, models and technologies affect the design, development and use of next-generation C2 information and decision systems? What role will future systems play in the aggregate information systems world? 6.2

The Range of

C’ Information and Decision Support

It is safe to say that many information and decision systems support command and control decision-making indirectly. There are systems that manage projects, provide easy access to operational data, and otherwise deal with relatively structured problems. This distinction between “structured” and “unstructured” targets of opportunity is important to understanding the range of today’s systems and the promise of tomorrow’s. Early proponents of information technology hoped that their systems would help decisionmakers generate, compare, and implement decision options, but most systems indirectly support these activities. Real-time option generation and evaluation has evaded designers-except in some rare instances where a special (single) purpose system was developed to address a very well-bounded problem (Andriole, 1989a).

58

STEPHEN J. ANDRIOLE

The range of next-generation C2 information and decision support systems applications will grow considerably. Next-generation systems will be capable of addressing operational, tactical and (some) strategic structured and unstructured problems via the application of data, knowledge, and models that exploit their integration. Figure 26 suggests where the action will be. The technology in applied analytical methodology, user-computer interface (UCI) techniques, and display technology, will grow considerably as the nature and depth of application domains expands. The new, applications perspective on Cz information and decision systems support will be extremely broad, reflecting the capabilities of new systems that will be embedded and functional on many levels. Future systems will permit decision-makers and information managers, resource allocators and administrators, and strategic planners and inventory controllers to improve their efficiency. The broad perspective will be permitted by the new technology (see Section 6.3) that will emerge over the next several years and by centralization in the military workplace, a centralization that will not be “ideological” but rather driven by the same technology that will permit the design and development of more powerful systems. In all likelihood, the movement toward technological networking and system integration will translate into new imperatives for the management of information and decision-making. In short, command centers of the future will transform themselves because new technology will permit the change and, significantly, because it will demand it. What will be driving what? Will new C2 requirements suggest new system requirements, or will new systems suggest new requirements? Would nextgeneration systems look the way we expect them to look if they were conceived in an applications vacuum, or are the interpretation and anticipation of applications driving the “form” that future systems will take? While definitions of decision support will grow, so too will our understanding of computer-based problem-solving. Decision support, while very broad in concept and application, will nevertheless be subsumed under the general rubric of computer-based problem-solving which, over time, will also experience radical change. Expectations about what computers can do for users will continue to rise. Consistent with the evolution in expectations about the power of next-generation systems, computer-based problem-solving systems of all kinds must satisfy analytical requirements. 6.3

Advanced Information Technologies

There are a variety of tools, methods, techniques, devices and architectures available to the information systems engineer; many more will emerge as we

1980s C 2 INFORNATION h DECISION PROBL ENS

\

I

ANALY 1ICAL MODELS & NETHODS

L

NEXT GENERATION PROCESS1NWDISPLAY TECHNOLOGY

W ul

ADVANCED USER- CONP UTER INTERFACE (UCI) TECHNOLOGY

L7+ C2 INFORMATION DECl SlON

PROBLEMS FIG.26. Technology opportunities for next generation Cz information systems engineering.

1

0 Probability

0

0

Assessment Anomalous Event flatrix 0 Brainstorming

0

0

0

QUALITATIVE

Case Study Panels structured Opinion Polling) 0 Simulated Opinion Polling

Influence Diagramming Hierarchical Inference 0 Decision Analysis 0 llultiattribute Utility

Cost-Benefit Analysis Change Signals monitoring

0 0

-

0 GE W I

2. STRUCTURED

-

3. TIME SERIES/

0

-

BOOTSTRAPPING

0 m

0

Growth Curves, Trends, h

0

Descriptive Profiling Correlation

0

Leading Indicators

5. STATISTICAL/ OPERATIONS

.

0 0

Cycles 0 Smoothing Hethods

0

QUANTITATIVE

meory

Relevance Trees

llarkw flodels Bayesian

0 Qiantai

Choice

Sampling Pattern Recognition 0 Linear Programming 0 Dynamic Programming 0 Qieuing Theory

0

0

0 Econometric Hodels 0

System Dynamics plodels (Slmulatlon)

Conventional Algorithmic Nethods 0 Hassage Processing 0 Scheduling flethods

0

INFORnATlON Expert Systems Natural Language Processing 0 Others 0

0

FIG.27. A taxonomy of methods and models (Hopple, 1986).


61

move toward the 21st century. The challenge-as always-lies in the extent to which designers can match the right tool or method with the appropriate problem. This section looks at a number of technology options now available to the designer, options that will evolve quite dramatically during the next five to ten years. 6.3.1 Models and Methods

Figure 27 suggests the range of methods and models available to the designer today (Hopple, 1986).The taxonomy-which expands the notion of methodological support introduced in Section 4.3-is by no means complete, though it is representative of the way methods, tools and techniques can be categorized and assessed. Figures 28,29, and 30 from Sage and Rouse (1986) suggest how several of the major methods classes can be described and assessed. “Assessment,” of course, is the key. Information systems engineers must know precisely what method (or methods) to apply to which requirements. Figure 31 from Andriole (1989a) suggests how methods can be rankordered against a set of requirements.

Cognitive Science

Artificial Intelligence

Objectives

I

Methods

Oprratlons Research and Control Theory

Software Performance

Explanat ion Of Cognition

Aiding Decisions

Prediction and Optimization

Symbolic ; Powerful Software Tools

Symbolic; ComputationOriented

Numerical ; Axiom at ic Dec. Rules

Numerical; Predictive Models

Decislon Prescript ions; Software/Tools

Opti ma1 Solutions

WellFounded and Structured

Breadth and Rigor

Biased Toward Choice; Not Judgement i n Context

Avoids Context; Assurnpt ion Laden

~~

Software and Tools

Products

Decislon Analysis

~~~

~~

Thew ies ; Training and ~~

Strength

I

ExDloitation of context; Software Tools

Pr mess Over Product of Behavior

I

weaknesses

I

Often Ad Hoc; Avoids Engineering Methodology

Often Ad Hm; Avoids Psychological Methodology

FIG.28. Methods/models descriptions (Sage and Rouse, 1986).

A l L Expert Systems Objectives/ Expectations Typical Analytical Concern or Ouestion to Subject Products

I

output Weaknesses: Input Process

Output

I

Human/System Interaction

-

Effective Interface (Adaptive, intelligent?) Between Humans and Decision Tools Transparent Interface

Design o f "Intelligent" Systems (KBS) Human-like Capacities Heuristics

Normative Modeling. Aids t o Option Generation, choice & selection

"Models" o f Human Processes, Search 8. Representation, Symbolic Processing

Decomposition, structure elicitation Prototyping; Performance Modelins o f Subjective Judgements, Evaluation o f Cost/Benefit MAU, Bayes Policy Capturing

LISP Machines, Shells

Process Aids (Computer. Other) Structure

Explicit Model. Ad Hoc

Normative Models, Face Validity o f Models

Explanation Facility

Audit Trail Sensitivity Analysis

I

Primitive

-

-

Strengths: Input Process

Decision Analysts h Decision Support

G e n i a i lnlensive

Comprehensibility, May Use Unrealistic Belief Structures

Normative Models

Comprehensibili ty

Sensitivity Analysis FIG.29. Multi-criteria methods assessment (Sage and Rouse, 1986).

I10 Devices, Configurations

OR h Control Engineering

Products w m

Strengths : Input Process output Weaknesses: Input Process output

-

Facilitate Storage, Retrieval, Manipulation o f Data (Information)

Understanding & Describing Human Cognition

Linear Programs, Math Modeling, Optimization, Dynamic Systems Analysis, Statistics

Relational, Hierarchical, Spatial Structuring, Retrieval Strategies

Modeling Theories Lens Model Attribution Theory Empirical/Experimental

System Model "Autopilot," etc. (Control Systems) Aids Modeling/Simulation Languages

DBMS Software, Database Designs

Human Models/Decision Heuristics (Validated? 1 Capabi lit y and/or Limitation Assessments

Objectives/ Provide Results of Quantitative Expectations Analysis t o Clients Typical Analytical Concern or Question t o Subject

Cognitive Science/ Psychology

Data Base Management

Can Use Massive Real Data Rigorous; Engineering Methods Quantification Requires Volumes o f Data

I I

Experimental /Empiri cat

Inflexible

"Artificial" Lack o f Face Validity (GIGO)

Mind Reading, laboratory Bound

Explanation Facility

Narrow Domains

FIG.30. Multi-criteria methods/models assessment (Sage and Rouse, 1986).

-

Activate

a

Close.

f

Adjust Syncronize...

3

5%

88

sI s ax +9

AA

O(

IV

Advise Inform Instruct Request...

ccs

ccs

ccs

Al

Al

Al

Al

dR

oh

I

&

MS DA

MS DA

MS DA

MS

css

MS

DA A1 OR DA At OR

MS OR A1 DA OR A1 DA

A

MS DA CCS

ccs

MS

ccs

MS

MS

OR Al DA INEXPERIENCED (I)

A1 OR DA EXPERIENCED (El

A1 OR DA

MS

ccs ccs

:

I

I

CCS MS At Al DA OR OR DA DA Al Al OR MS DA OR MS

ccs

Information Processing Problem-solving And Decision-making.. . Search For/ Receive Information Identify Objects, Actions, Events...

mnm

ccs

MS

ccsi ccs

ccs

Al OR CCS MS DA

I/E INFREQUENT

USERS FIG.3 1. Some models/methods ranking.

I

DA OR CCS

---

Legend

Decision Analysis Operations Research Conventional Computer Science A1 = Artifical Intelligence HS = Management Science


65

Over the past few years the ISE community has seen the preeminence of knowledge-based tools and techniques, though the range of problems to which heuristic solutions apply is much narrower than first assumed. It is now generally recognized that artificial intelligence (AI) can provide knowledgebased support to well-bounded problems where deductive inference is required (Andriole, 1990; Andriole and Hopple, 1988). We now know that A1 performs less impressively in situations with characteristics (expressed in software as stimuli) that are unpredictable. Unpredictable stimuli prevent designers from identifying sets of responses, and therefore limit the applicability of “if-then’’ solutions. We now know, for example, that expert systems can solve low-level diagnostic problems, but we cannot predict Soviet intentions toward Poland in 1995. While there were many who felt from the outset that such problems were beyond the applied potential of AI, there were just as many who were sanguine about the possibility of complex inductive pr oblem-solving. The latest methodology to attract attention is neural-network-based models of inference-making and problem-solving. As Fig. 32 suggests, neural networks are applicable to problems with characteristics that are quite different from those best suited to AI. Neural networks are-according to Hecht-Nielsen (as quoted in North, 1988)-“computing systems made up of a number of simple, highly interconnected processing elements which process information by their dynamic state response to external inputs.” Neural nets are non-sequential, non-deterministic processing systems with no separate memory arrays. Neural networks, as stated by Hecht-Nielsen, comprise many simple processors that take a weighted sum of all inputs. Neural nets do not execute a series of instructions, but rather respond to sensed inputs. “Knowledge” is stored in connections of processing elements and in the importance (or weight) of each input to the processing elements. Neural networks are allegedly non-deterministic, non-algorithmic, adaptive, selforganizing, naturally parallel, and naturally fault-tolerant. They are expected to be powerful additions to the DSS methodology arsenal, especially for datarich, computationally intensive problems. The “intelligence” in conventional expert systems is pre-programmed from human expertise, while neural networks receive their “intelligence” via training. Expert systems can respond to finite sets of event stimuli (with finite sets of responses), while neural networks are expected to adapt to infinite sets of stimuli (with infinite sets of responses). It is alleged that conventional expert systems can never learn, while neural networks “learn” via processing. Proponents of neural network research and development have identified the kinds of problems to which their technology is best suited: computationally intensive; non-deterministic; nonlinear; abductive; intuitive; real-time; unstructured or imprecise; and nonnumeric (DARPA/MIT, 1988).

66

STEPHEN J. ANDRIOLE

FIG.32. The applicability of artificial intelligence and neural network-based models and methods.

It remains to be seen if neural networks constitute the problem-solving panacea that many believe they represent. The jury is still out on many aspects of the technology. But like AI, it is likely that neural nets will make a measured contribution to our inventory of models and methods. What does the future hold? Where will the methodological leverage lie? In spite of the over-selling of AI, the field still holds great promise for the design and development of C2 information and decision systems. Natural language processing systems-systems that permit free-form English interaction-will enhance the efficiency of information and decision systems support and will contribute to the wide distribution of information and decision systems. The Artificial Intelligence Corporation’s INTELLECT natural language processing system, for example, permits users to interact freely with a variety of database management systems. The BROKER system, developed by Cognitive Systems Inc., permits much the same kind of interaction with the Dow Jones databases. These systems are suggestiveof how natural language interfaces will evolve over time and of how


67

users will be able to communicate with databases and knowledge bases in ways that are compatible with the way they address human and paper data, information and knowledge bases. When users are able to query their systems in much the same way they converse with human colleagues, then the way problem-solving systems will be used will be changed forever. Of particular interest in the disproportionate attention that natural language interfaces have received vis-a-vis expert systems. This imbalance will be redressed by the year 2000. Expert systems will also render many decision-making processes routine. Rules of tactical planning, resource allocation, and target-weapons matching will be embedded in expert information and decision systems. Problems that now have to be re-solved whenever a slight variation appears will be autonomously solved. Smart database managers wilI develop necessary databases long before decision support problems are identified. Next-generation systems will be capable of adapting from their interaction with specific users. They will be able to anticipate problem-solving “style” and the problem-solving process most preferred by the user. They will be adaptive in real-time and capable of responding to changes in the environment, such as a shortage of time. The kinds of problems that will benefit the most from A1 will be wellbounded, deductive inference problems about which a great deal of accessible and articulate problem-solving expertise exists. The community will abandon its goals of endowing computer programs with true inductive or abductive capabilities in the 1990s, and the dollars saved will be plowed back into socalled “low-level’’ AI. Future information systems engineers will also benefit from a growing understanding of how humans make inferences and decisions. The cognitive sciences are amassing evidence about perception, biasing, option generation, and a variety of additional phenomena directly related to modeling and problem-solving. The world of technology will be informed by new findings; resultant systems will be “cognitively compatible” with their users. Next-generation systems will also respond to the situational and psychophysiological environment. They will alter their behavior if their user is making a lot of mistakes, taking too long to respond to queries, and the like. They will slow down or accelerate the pace, depending on this input and behavior. The field of cognitive engineering-which will inform situational and psychophysiological system design strategies-will become increasingly credible as we approach the 2 1st century. The traditional engineering developmental paradigm will give way to a broader perspective that will define the decision-making process more from the vantage point of requirements and users than computer chips and algorithms. Principles of cognitive engineering will also inform the design and human-computer interfaces (see Section 6.3.2).

68

STEPHEN J. ANDRIOLE

Some future software will be generic and some will be problem-specific. Vendors will design and market generic accounting, inventory control, and option selection software. These models will be converted into templates that can be inserted directly into general-purpose systems, The template market will grow dramatically over the next five to ten years. It is extremely important to note the appearance of system dkvelopment tools. Already there are packages that permit the development of rule-based expert systems. There are now fourth-generation tools that are surprisingly powerful and affordable. These so-called “end-user’’ systems will permit onsite design and development of systems that may only be used for a while by a few people. As the cost of developing such systems falls, more and more throwaway C2 systems will be developed. This will change the way we now view the role of decision support in any organization, not unlike the way the notion of rapid application prototyping has changed the way application programs should be developed. Hybrid models and methods drawn from many disciplines and fields will emerge as preferable to single-model-based solutions, largely because developers will finally accept diverse requirements specifications. Methods and tools drawn from the social, behavioral, mathematical, managerial, engineering, and computer sciences will be combined into solutions driven by requirements and not by methodological preferences or biases. This prediction is based in large part upon the maturation of the larger design process, which today is far too vulnerable to methodological fads. Hybrid modeling for information and decision systems design and development also presumes the rise of multidisciplinary education and traini’ng, which is only now beginning to receive serious attention in academia and industry. 6.3.2 User-Computer Interface (UCI) Technology

Twenty years ago no one paid much attention to user interface technology. This is understandable given the history of computing, but no longer excusable. Since the revolution in microcomputing-and the emerging one in workstation-based computing-software designers have had to devote more attention to the process by which data, information and knowledge are exchanged between the system and its operator. There are now millions of users who have absolutely no sense of how a computer actually works, but rely upon its capabilities for their professional survival, A community of “thirdparty” software vendors is sensitive to both the size of this market and its relatively new need for unambiguous, self-paced, flexible computing. It is safe to trace the evolution of well-designed human-computer interfaces to some early work in places such as the University of Illinois, the Massachusetts


69

Institute of Technology, (in what was then the Architecture Machine Group, now the Media Lab), Xerox’s Palo Alto Research Center (Xerox/PARC), and, of course, Apple Computer, Inc. The “desk-top” metaphor, icon-based navigational aids, direct manipulation interfaces, and user guided/controlled interactive graphics-among other innovations-can all be traced to these and other organizations. Where did all these ideas come from? The field of cognitive science and now “cognitive engineering” is now-justifiably-taking credit for the progress in UCI technology, since its proponents were the (only) ones asking why the usercomputer interaction process could not be modeled after some validated cognitive information processing processes. UCI models were built and tested, and concepts like “spatial database management” (from MIT’s Architecture Machine Group (Bolt, 1984)),hierarchical data storage, and hypertext were developed. It is no accident that much UCI progress can be traced to findings in behavioral psychology and cognitive science; it is indeed amazing that the cross-fertilization took so long. UCI progress has had a profound impact upon the design, development and use of C2 information and decision systems. Because many of the newer tools and techniques are now affordable (because computing costs have dramatically declined generally), it is now possible to satisfy complex UCI requirements even on personal-computer-based systems. Early data-oriented systems displayed rows and rows (and columns and columns) of numbers to users; modern systems now project graphic relationships among data in highresolution color. Information systems engineers are now capable of satisfying many more substantive and interface requirements because of what we have learned about cognitive information processing and the affordability of modern computing technology. The most recent progress in UCI technology is multimedia, or the ability to store, display, manipulate and integrate sound, graphics, video and good-oldfashioned alphanumeric data (Ragland, 1989; Ambron and Hooper, 1987; Aiken, 1989). It is now possible to display photographic, textual, numerical, and video data on the same screen, as Fig. 33-from Aiken (1989)-suggests. It is possible to permit users to select (and de-select) different displays of the same data. It is possible to animate and simulate in real-time-and costeffectively.Many of these capabilities were just too expensive a decade ago and much too computationally intensive for the hardware architectures of the 1970s and early 1980s. Progress has been made in the design and execution of applications software and in the use of storage devices (such as videodisks and compact disks (CDs)). Apple Computer’s Hypercard software actually provides drivers for C D players through a common UCI (the now famous “stack”). Designers can exploit this progress to fabricate systems that are consistent with the way their users think about problems. There is no question

Linkages

.-.---.-.--

I * . ‘ . -

I

-J 0

Code FIG.33. Multimedia technology (Aiken, 1989).

u


71

that multimedia technology will affectthe way future systems are designed and used. The gap between the way humans “see” and structure problems will be narrowed considerably via the application of multimedia technology. Direct manipulation interfaces (DMIs) such as trackballs, mice and touch screens have also matured in recent years and show every likelihood of playing important roles in next-generation information and decision systems UCI design and development.While there is some growing evidence that use of the mouse can actually degrade human performance in certain situations, there are countless other situations where the payoff is empirically clear (Ramsey and Atwood, 1979; Ledgard et al., 1981; Bice and Lewis, 1989). Touch screens are growing in popularity when keyboard entry is inappropriate and for rapid template-based problem-solving (Smith and Mosier, 1984). The use of graphical displays of all kinds will dominate future UCI applications. Growing evidence in visual cognition research (Pinker, 1985) suggests how powerful the visual mind is. It is interesting that many problemsolvers-professionals who might otherwise use information or decision systems-are trained graphically, not alphanumerically. Military planners receive map-based training; corporate strategies use graphical trend data to extrapolate and devise graphic scenarios; and a variety of educators have taken to using case studies laden with pictures, icons, and graphics of all kinds. Complicated concepts are often easily communicated graphically, and it is possible to convert complex problems from alphanumeric to graphic form. There is no question that future C2 systems will exploit hypermedia, multimedia, and interactive graphics of all kinds. Speech input and output should also emerge over the next five to ten years as a viable UCI technology. While predictions about the arrival of “voice activated text processors” have been optimistic to date, progress toward even, continuous speech input and output should be steady. Once the technology is perfected there are a number of special-purpose applications that will benefit greatly from keyboard- and mouse-less interaction. The use of advanced UCI technology will foster a wider distribution of information technology. Early information and decision were used most productively by those familiarwith the method or model driving the system as well as interactive computing itself. In other words, in order to exploit information technology one had to have considerable computing expertise. Advanced UCI technology reduces the level of necessary computing expertise. Evidence suggests that training costs on the Apple Macintosh, for example, are lower because of the common user interface. Pull-down and pop-up menus, windows, icons, and direct manipulation via a mouse or trackball are all standard interface equipment regardless of the application program (and vendor). If you know how to use one Macintosh program, chances are you can use them all to some extent. Such interface uniformity is unheard of in other

72

STEPHEN J. ANDRIOLE

than Macintosh-based software systems, yet illustrates the enormous leverage that lies with the creative application of advanced UCI technology. UCI technology will also permit the use of more methods and models, especially those driven by complex-yet often inexplicable-analytical procedures. For example, the concept of optimization as manifest in a simplex program is difficult to communicate to the typical user. Advanced UCI technology can be used to illustrate the optimization calculus graphically and permit users to understand the relationships among variables in an optimization equation. Similarly, probabilistic forecasting methods and models anchored in Bayes’ Theorem of conditional probabilities, while computationally quite simple, are conceptually convoluted to the average user. Log odds and other graphic charts can be used to illustrate how new evidence affects prior probabilities. In fact, a creative cognitive engineer might use any number of impact metaphors (such as thermometers and graphical weights) to present the impact of new evidence on the likelihood of events. Finally, advanced UCI technology will also permit the range of information and decision support to expand. Any time the communications bandwidth between system and user is increased, the range of applied opportunities grows. UCI technology permits designers to attempt more complex system designs due to the natural transparency of complexity that good UCI design fosters. Some argue that the interface may actually become “the system” for many users. The innards of the system-like the innards of the internal combustion engine-will become irrelevant to the operator. The UCI will orchestrate the process, organize system contents and capabilities, and otherwise shield users from unfriendly interaction with complex data, knowledge, and algorithmic structures.

6.3.3 Hardware The hardware that supports the application of information technology and the information systems engineering process today is “conventional.” There are turnkey systems as well as generic hardware configurations that support the use of numerous information and decision systems. CPUs, disk drives, keyboards, light pens, touch screens, and the like can be found in a variety of DSSs. There are also microcomputer systems, as well as systems that require larger (minicomputer) hardware configurations. Next-generation C2 information and decision systems will be smaller and cheaper, and therefore more widely distributed. They will be networked, and capable of up-loading and down-loading to larger and smaller systems. Input devices will vary from application to application as well as the preferences of


73

the user. As suggested above, voice input will dramatically change the way a small set of systems are used in the future; voice activated text processing will expand system capabilities by linkages to word processing and report preparation in a “natural” unobtrusive way, though it is likely that robust voice activated systems will not appear until the late 1990s. Many systems will have embedded communications links to databases and knowledge bases, other systems on networks, and the outside world via conventional communication systems. IBM’s acquisition of Rolm suggests that the merger between computing and voice systems is well underway. Future systems will have (some selected) voice input capabilities, conventional headset communications, deep database linkages, and a “place” on a much larger information and decision support system network. Briefcase- and smaller-sized computers will become widespread. The embedding of spreadsheets in popular portable microcomputers suggests that information and decision support chips will be developed and embedded in future hardware configurations. In fact, not unlike some of the more powerful calculators of the 1970s, future systems will permit users to mix and match chips within a single processor. Future C2 information and decision systems will also be integrated with video display systems of several genres. There will be video-disk-based systems as well as packaged systems that integrate powerful computergenerated imagery capabilities. The cost of both video options is falling rapidly, and the military consumer of the future will be able to select the one that best serves his or her needs. It is safe to say that video will become integral to future information and decision support. Behavioral scientists have just about convinced system architects-via the amassing of tons of evidence-that information, concepts, and many ideas can be communicated much more effectively via graphic, symbolic, and iconic displays (Smith and Mosier, 1984; Schneiderman, 1987). Systems that do not have these and related capabilities will fail. The revolution in high-resolution display technology will exert a profound impact upon next-generation systems design and use. Many UCI technologies will exploit high-resolution displays, thereby accelerating the movement toward graphic computing. Processor technology is also evolving rapidly. Just a decade ago, most of us computed on Intel 8088 microprocessors, while today everyone is waiting for the 486. Processors such as the Motorola 68030 (and next-generation 68040) have placed enormous power not only in the hands of users, but-perhaps more important-system designers as well. It is safe to say that applications software is today lagging the capabilities of such chips; at the same time, even

74

STEPHEN J. ANDRIOLE

assuming a consistent lag, systems in the 1990s and beyond will benefit from applications software that exploits the revolution in microprocessor design. The issue of power, however, does beg the question of larger requirements. In other words, it is safe to assume that raw computing power will be ready for next-generation system concepts. The challenge-as always-will lie in the application of the power to validated user requirements. If the truth be told, there are many successfulsystems that today require less than 20% of available computational power; many future systems may well find themselves with abundant power-and nowhere to go! Regardless of available computing power, information and decision systems engineers will have to adhere to sound information systems engineering principles well into the 1990s and through the foreseeable future (Andriole, 1990). We are witnessing the demise of the distinction among mainframe, miniand microcomputers. Tomorrow there will be “workstations.” Some will be more powerful than others, but nearly all will be available to individuals at reasonable prices. The balance between capability and price will continue to perplex vendors, since users will demand more and more capabilities for less and less money. Pricing strategies will determine how much power becomes “affordable.” Future systems design and use will work within this changing marketplace and, because of some new usage strategies (see Section 3.4), will remain largely unaffected by the instability of the workstation marketplace. 6.4

Integrated C2Information and Decision Support

Information and decision systems will be used very differently in the future than they are today. They may well function as clearinghousesfor professional problems. They may prioritize problems for military commanders, and they may automatically go ahead and solve some of them. They will become problem-solving partners, helping us in much the same way colleagues now do. The notions of systems as software or hardware, and users as operators, will give way to a cooperative sense of function which will direct the design, development, and application of the best C2 information and decision systems. They will also be deployed at all levels in the military organization. The distribution of DSSs will permit decision support networking, the sharing of decision support data, and the propagation of decision support problemsolving experience (through the development of a computer-based institutional memory of useful “cases” that might be called upon to help structure especially recalcitrant problems). Efficient organizations will actually develop an inventory of problem/solution combinations that will be plugged into their larger computer-based problem-solving systems architectures.


75

Next-generation systems will also communicate with systems in other organizations in other parts of the world. Falling costs of satellite communications will permit global linkages and contact with databases, expert systems, inventories, and the like, thereby multiplying the capabilities of in-house systems by orders of magnitude. This global networking is not decades away, but only five to ten years away. The military’s Worldwide Military Command and Control System (WWMCCS) and its WWMCCS Information System (WIS) represent the most ambitious attempts to coordinate and network information and decision systems. Unfortunately, WWMCCS and WIS do not support users in ways that approach technological capabilities or operator expectations. Next-generation information and decision systems engineering will solve many of the most serious problems with networks like WWMCCS and WIS. Advanced technology will permit linkages and coordination that were not possible ten years ago. The most important change will occur in the way next-generation information and decision systems interface with other information systems. Most contemporary systems are “disembodied,” that is, distinct from larger corporate, government or military information systems. Actual use of many systems involves leaving one system to activate another. It is common in the military for users to work alternately with mini- and microcomputers, manually feeding the output from one system into the other. A good deal of this can be explained by acquisition and procurement craziness, but just as much can be traced to obsolete concepts of how computer-based problemsolving systems should be used. As the range of target problems and capabilities increases, fewer and fewer systems will be disembodied; on the contrary, the most successful systems will be embedded in larger organizational and executive information systems. Future Cz information and decision systems will provide “portals” for users to explore. It will be possible to perform all sorts of tasks via myriad application programs (that ideally will have common user-computer interfaces). The whole concept of “decision support” will evolve to accommodate changes in the larger corporate, governmental, and military information systems structure. Networking and advanced communications technology will permit linkages to databases and knowledge bases-and the routines to exercise them. Not only will distinctions among mainframe, mini- and microcomputing fade, but distinctions among management information, executive information, and decision support systems will also cloud. Ironically, the concept of centralization may reappear, not with reference to central computing facilities but with regard to enormous systems conceived functionally as hierarchies of capabilities. Users may well find themselves within huge computing spaces capable of supporting all kinds of problemsolving. Advanced communications technology will make all this possible;

76

STEPHEN J. ANDRIOLE

users will be able to travel within what will feel like the world's largest mainframe, which conceptually is precisely what a global network of data, knowledge, and algorithms is. The same users will be able to disengage the network and go off-line to solve specific problems. This freedom will expand the realm of analytical computing in much the same way microcomputing expanded the general user community. Finally, all this technology will permit designers to fulfill user requirements in some new and creative ways. Until quite recently, technology was incapable of satisfying a variety of user requirements simply because it was too immature or too expensive. We have crossed the capability/cost threshhold; now designers can dig into a growing toolbag for just the right methods, models, and interfaces. By the year 2000, this toolbag will have grown considerably. Talented C2 information systems engineers should be able to match the right tools with the right requirements to produce systems that are user-oriented and cost-effective. The future of C2 information and decision systems design, development and use is bright. While some major changes in technology and application concepts are in the wind, next-generation systems will provide enormous analytical support to their users. We can expect the range of decision support to grow in concert with advances in information technology.

7.

Summary and Conclusions

This chapter has covered a lot of ground. Its goal has been the description and analysis of the generic information systems engineering (ISE) process, the domain of military command and control (C'), and the application of the principles of multidisciplinary information systems engineering to C2 information and decision systems engineering. Several key arguments have been made. One suggests that the range of tractable problems is growing as our information technology (and design strategies) grow. We are now in a position to satisfy more user and system requirements than we were able to approach just five years ago. New opportunities for the application of advanced information technology are rising dramatically. Next-generation C2information and decision systems will look and feel very different to users; they will be far more powerful, much easier to use, and able to communicate with problem-solving cousins distributed across large, secure and reconstitutable networks. The generic ISE process will also grow over time. Its multidisciplinary flavor will expand to embrace more and more disciplines and fields of inquiry. The need for cross-fertilization will become self-evident as our understanding of


77

substantive and user interface requirements deepens. It is likely that the need for multidisciplinary ISE will be addressed by the industrial, military, larger governmental, and academic communities via the development of thoroughly integrated research and development, and educational and training programs. ISE represents a relatively new way to think about systems design and development; C2 represents an expanding applications domain; the marriage between ISE and C2 is likely to yield some creative system solutions to “old” and “new” requirements. But perhaps most important, the central theme of this chapter-and its essential argument-is that without structure the design and development process will almost always fail. The ISE state of mind calls for the consistent application of a set of tools and techniques that together constitute a structured design methodology. The chapter also recognizes the importance of perennial information technology assessment. System solutions are not found only in structured design methodology; there is considerable leverage in the application of advanced and emerging technologies. ISE is structured, yet flexible enough to exploit new technological opportunities. Finally, there is an educational and training challenge assumed by ISE, a challenge that calls for multidisciplinary education and training. If we are unable to produce competent information and decision systems engineers, then our design philosophy and methodology will barely affect the systems landscape.

Appendix A Group (Army Theater Level) Tactical Planning Substantive and User-Computer Interface Tasks and Requirements This appendix contains the lists of substantive and user-computer interface (UCI) requirements that were distilled from interview and simulation data. The substantive requirements list the functions and tasks that planners must perform to generate optimal courses of action, while the UCI requirements reflect the kinds of displays and interaction routines necessary to support relatively inexperienced computer users. The lists are in two forms. They are organized as graphical hierarchies and in narrative form. The narratives provide some detail about precisely what the (substantive or UCI) requirement actually calls for. The requirements were, in turn, converted into “storyboards” of how the group decision support system might actually operate. These storyboards were organized into a working prototype; several storyboards from the prototype appear in Appendix B.

78

STEPHEN J. ANDRIOLE

Substantive Planning Requirements R

Planning Requirements 1

Mission Statement 2

Military Objectives

3 1

Objective Rank-Ordering

Area Charmteristics

2

3

Topographic

3

CIimatic/Weather

Telecommunications

1 h b a t Capabilities 2

2

Red Capabilities

3

Location/Disposition

3

Time/Space Factors

3

"Efficiency"

Blue Capabilities


13I

StrengthlReinforwments

I

3 Composition

I 3 I Location/Disposition I 3 1 Time/SpaceFactors I 3 I “Efficiencf“ I

I I

I

‘

2 Relative Assessments

3 Strengths

I 4 1 Redstrengths I 4 IBlueStrengths

I I

3 Vulnerabilities

I 4 1 RedVulnerabilities I

I

4 lBlue Vulnerabilities

1 I I Operational Concepts

I

2m

I 3 1 AreaAssumptions 4

I

Suitability

I 4 I Acceptability 4

2

I

Success Probability

Pertinent Red Capabilities

I 3 I Red Military Objectives

I

79

80

STEPHEN J. ANDRIOLE

3 RedCDAs 3 RedVulnerabilities 1 Operations Concept

2 RedCapabilities

3

Operational Capabilities

3

Distilled RedCapabilities

2 BlueCOAs 3

ArhmtagWDiWantqes

3

COA Vulnerabilities

2 COASelection 3

3 3

AlternativeCOAs Relative COA Comparisons

COA Rank-Ordering

3

ForceAllocation &Training

3

Supporting Operations

3

4

Logistics Operations

4

OtherOperations

Command RelatiMS

3 Deployment Summary 3 Employment Summary


Substantive Planning Requirements Description R> Planning Requirements

,

List of functional plannlng requirements

1 MiSsion statement

Requirement to undwstandmlsslon

2, Military Oblectlw

Requlrement to understand mllitwy objectives

3, Spedfic Objectives Ob]ective Rank- OrderIng

Requirement to understendspecific objectives Requirement to understendrank-ordering of objectives

I > Area Characterlstics

Requirement to understandarea

2, w a p h i c

Need to understandgeoOraphicfeatures

3> T w e p h i c

Topographic Information requirements Wographic information requirements Climatlc/wealher information requirements

Hydrwaphic Climatic/Weather 2, Transportation Telmmunlcatlons

Transportation information requirements Telecommunicationsinformation regulrements

1, Combat Cepabilities

Relative combat capabilities information requirements

2, RedCapabilltles

Need to understandRed m b a t cap&ilities

3, Strength/Reinforcements

Overall and reinforcements strength information requirements Need to understandRed composition Need to Identify location and understanddisposition Need to understandRed time/space factors Need to assess "efficiency"

Composition Locatlon/Disposition Factors Time/"Efficiency"

2, Bluecapabilities

Need to understandBlue wmbat capabilities

3, Strength/Reinforwments

Need to assess Blue strength and reinforcements Need to understandBlue composition Need to identify Blue location mi understanddisposition Need to mess tlme/spw factws Need to BSSBSS Blue "efficiency"

Composition Location/Disposttion Time/Spsce Factors "Efflciency"

2, RelativeAssessments

Need to infer "net"effects

3, Strengths

Need to emss relative strengths

4, RedStrengths

Need to dtitermlne relatlve Red s t r w h s

Blue Strengths

3, vulnerabiiities 4,

Red Vulnerabllities Blue Vulnerabilities

Need to determine relative Blue strengths Need to assess relative vulnerabilities Need IodetermlneRedvulnerabllitles Needto &itermine Bluevulnerabllities

81

STEPHEN J. ANDRIOLE

82 1> Operational Concepts

Need to formulate initial COAS

2, CQQS

Need to develop strawman courses of action

3, Objectives Area Assumptions Strawman COAs

Need to re-assw military objectives Need to identify wea assumptionsvis-a-vis CMS Need to develop strawman CfMs F68sibility vis-a-vis "sultebility" Feaslbilityas to "acceptability" Need todetermine SUCC~SSprobability

4, Suitability Acceptability Success Probability 2, Pertinent RedCapabilitles

~ e e dto determine pertinent Redoapabllitiesvls-a-vis CfMs

3, Red Military Objectives

Need to revisit Red military objectlves EIeed to revisit likely Red COAs Need to revlsit Redvulnerabilities

1> Operationsbnwpt

~ e e dto develop concept of operations

2, Redcapabilities

Need to revisit Red capabilities

3, Operational Capabilities Distilled Red Capabilities

Re-determination of Red capabilities Distillation of Redcapabilities vis-a-vis Blue CW

2, BlueCW

Re-analysis of clue murses of action

3, Wantapes/DisadvantaOes "Sensitivity" Analysis COA Vulnerablllties

Determine advantm and disadvant#&s of each COA Naed for Sensitivity analysis via variation of assumptions Determlnevulnerabilitiesof each BlueCOA

2, COAselection

Need to analyze ad select mongalternative CMS

3, AlternativeCOAs

Revbitation of alternative of Blue CW Need tompareandcontrast alternatlveCfMs Final rank-ordering of Blue CoAS

Red COAs Red Vulnerabilities

RelatlveCM Comparlson CM Rank-Ordering 2, COA-bCDo

Need to translate Blue COA into concept of aperations

3, Forw Allocation & Timing

Ned to determineforce allocationsand timing Need to identify and & m i b e supporting operations

SupportingOperations 4, Logistics Operations Other Operations

Need to determine logistics operations information requirements Naed to identify o m Supportingoperations

3, Command Relations Deployment Summary Employment Summary

Need to determinem m a n d relations Naed to develop @loyment summary Need to develop employment summary (operation concept)


User-Computer Interface (UCI 1 Requirements

1 Area Dtsplay Requlrernents

2

2

Moblltty D i s p l w

3

OPFOR Moblllh/

3

Blue

Key Terrain Dtsplays

3 Major Obstacle Displays

4 Mountalns 4

Cltles

4

SwampArees

4 Other

4

Contours/Rel tef/Topography

4 Major Elevatlon Displays 4 Man-Made Objects Displays 2

Plannlng Dtsplays

3

OPFOR

4

Avenues of A p p r m h

4 AssemblyAress/Attack Positlons

83

84

STEPHEN J. ANDRIOLE

3

Blue 4 Avenues of Approach

1

4

AssemblyAreas/Attack Positions

4

Major Supply Points

2

Weather Displays

2

Other Displays

OPFOR Displays

2

Disposition Displays

2

Condition/Strength

3 Conventional Form

3

NuclearForces

2

Air Support Dlsplays

2

Major Logistics Displays

2 COAsDisplays 1

Blue Displays

2

Disposition Displeys

2

Condition/Strength

3

Conventional Form

3 NuclearForcss 2

Air Support Displeys

2

Major Logistics Displays

2

CMsDisplays

I


1 Interpretive Displays 2

"Qualitative" Displays

3 Risk Displays 3 Constraints Displays 3 Vulnerability Displays 3 Opportunity Displays 3 Other Qualitative Displays 2

2

"Quantitative" Displays

3

Relative OPFOR Capabilities

3

Relative Blue Capabilities

"Cognitive" Displays

3

Cognitive Consistency

4

Conceptual Equivalence

4

Transition Displays

4

Analogical Displgrs

4

Doctrinal Displays

5 Doctrinal Options 1 Interaction Displays

2 Navigational Displays

85

86

STEPHEN J. ANDRIOLE

3 "Fly-Around" Capabilities 3 "Hold& Wait" Capabilities 3 Process Madel Displays

2

4

PrimaryProcesses

4

"Active" Help Displays

4

"Passive" Help Displays

4

"Active" Training

Manipulation Displays

3 Graphic Equivalence Displays 4 Summary Data Displays

4 Explanations 3 Map-Based Displays 4 Overlays 4

Explanations

2 Dialogue Displays

4

Iconic

4

Other


User-Computer interface (UCI) Requirements Descriptions R, UCI Requirements 1> Arsn Display Requirements

Disply requirements for general area of interest

2, Mobillty Displays

Dlsplaysof Red end Blue mobllity m r i d o r s

3, OPFDR Mobility

Requirementsfor OPFOR mobility options displays Requirementsfor Blue mobillty optionsdbplys

Blue

2, Key Terrain Displays

Requirementsfor key terrain dlsplays

3, Major Obstacle D i s p l v

Major obstacles displays

4, River Displays Mountains cities Swamp Arm Other

Dlsplays of river obstacles Displays of mwntaln obstacles Displays of urban obstacles Displays of major swamp ares Other displays of major obstacles

3, "Feature" Displa/s

Key terrain "features" d i s p l m

4, Contours/Relief/Topography Major Elevation Displays Mm-Made O b j W Displays

Contours/relief/topcgraphy displays Major elevatlan displays Displays of man-made objects/features

2, Planning Displays

Displays for general planning

3, OPFOR

Diplays for OPFOR planning

4, Avenues of Approach Assembly Ares/Attack Positions Malor Communication Lines Major Supply Points

Displays of possible 8venw of apprwh (Red) Displaysof assemblyweasatidattack positions(Red) Dlsplays of malor mmunlcation lines (Red) Displays of major supply points (Red)

3, Blue

Displays for Blue plennlng

4, Avenues of Apprcah Assembly AreasIAttack Positions Major Communication Llnes Major Supply Points

Displays of possiblellvmues of approach (Blue) Displaysof bssemblyarmandattack positions(Blw) Displays of major ammunication lines (Blue) Displays of major supply points (Blue)

2, Weather Displays other Dlsplays

Displays of seasonallcurrent weather Displays of other weacharacteristics

1, OPFPR Displays

Displays of OPFOR characteristics and capabilities

2, Disposition/Displays Condition/Strength

Displays of Red disposition D l s p l m of mndition and strength of Red

87

88

STEPHEN J. ANDRIOLE

3, ConventimlForas Nuclear Forces

D i s p l a y s o f m @ t l l ~faasand l readiness Displays of nuclear forces end readiness

2, Air Support Displays Major Logistics Displays cQl\s Displays

Displaysof Red air support Displays of malor logisticalcapabilities Displays of likely Red coursBs Of action (0%)

1> Blue Displays

Displaysof Bluecharacteristicsandcapabillties

2, Dispositian Displays Condition/Strenpth

D i s p l ~ oBlueloostionwddisposition f Displays of Blue mdltion fxdstrength

3, Conventional Forces

Displays of conventional forces and readiness Displays of nuclear capabilities and readiness

Nuclear Forces

2, Air Support Displays Major Logistics Displays W Displays

Displaysof Blueair support capabilities Displays of major logisticscapabilities Displaysof feasibleBlueCOAs

1>

Displays that support interpretation of substance

interpretlve Displays

2, "Qualitative" Displays

Displays of "qualitatlve" phenomena

3, Risk Displays

Displays that convey risk Displays that communicate operational constraints Displays that communicatevulnerabilities (Redand Blue) Displays that communlcate opportunities (Red and Blue) Displays of other qualitative aspects of situation

ConstraintsDisplays Vulnerability Displays Opportunity Displays Other Qualitative Displays

2, "Quantitatlve" Displays

Displays of "quantitative" information

3, Relative WFOR Capabilities

Displaysof relativeOPFOR combat capabilities Displays of relative Blue m b a t capabilities

2, "Cognitive" Displays

Displays that support spacific aqnitive functions

3, Cqnitive Consistency

Displays that support doctrinal models of planning

4, Conceptual Equivalence Transition Displays

Displays that support mceptual equivalence Displays that support easyaqnitive transition

3, Option Qaneration

Displays that support option generation

4, Analogical Dlspiays

Displays that present analogical information

5, Current Analog Displays "Old Analog Displays

Displays of current relevant analogs (cases) Displays that present "old but pertinent cases

4, Doctrinal Displays

Displays that present information on doctrine

5, Definitional Displays Doctrinal Options

Displays of current doctrinal explanations Displays that present doctrinal planning options

1 > interaction Displays

Displays that support Smmth user interaction

2, Navigational D i s p l w

Displays that support efficient Wtem naviption

Relative Blue Capabilities


3, "Fly-Around" Capabilities "Hold h Wait" capabilities P r o m s ~ o d eDisplays l

Capability to "fly-around system options and data Capability to "hold" system or have system "wait"

Displays that present the problem-solving process

4, Primary Proasses Sub-Process Displays

Displays of primary (overall) problem-solving process Displays that present sub-process problem-solving models

3, Adaptive Help Displays

Displays that present help

4, "Active" Help Displays "Passive" Help Displays

D i s p l w that p r m n t system-controlled help Displays that respond to user queries for help

3, Adaptive Training

Displgs that support adaptive trainlng

41 "Active" Training "Passive" Training

Displays that support system-man@ training Displws that support training by user request

2, Nanipulation Displays

Displays for data/process manipulations

31 Graphic Equivalence Displqs

Graphichlphanumeric equivalence displays

4, Summary Data Displays Explanat ions

Displays of all data and information Explanation displays of system-generated options

3, Map-Based Displays

Displqs that support map manipulations

4, Overlays Explanations

Displqs that permit "mix and match" overlays Displays that support graphic/map- bawl explanatlons

2, Dialogue Displqs

Displqs that support appropriate dialogue

3, Alphanumeric Dialogue

Displays that support alphanumeric dialogue options Displays that support graphic interaction

Graphic Dialogue Displays 4, lmnic Other

Displays that support the u s of on- line imns Other displgs that support graphic dialogue

Appendix B Storyboards from the Group Planning Prototype

This appendix contains a number of storyboards (screen displays)extracted from the group decision support system prototype. These storyboards describe the overall "master menu" structure of the system, the sub-menu structure, and some of the actual functional displays and routines in the prototype. Storyboards represent integrated collections of screens that suggest to users what the system will do (as well as what it will not do). The menu options are "active" in storyboard prototypes. Users can select menu options, and the prototype will respond immediately to the query.

ENEMY CAPABILITIES

AREA CHARACTER!STI CS

ALL I ED CAPABlLlTl ES PAUSE

EXPLAIN SIMULATE

I

SEND

TH EATER- LEVE L PLANNING

SHARE

(NATO vs Warsaw Fact)

COMPARE

UPDATE OVERLAY

7 1

ENTER ~~

STOP ENEMY

ALLIED COAS

COAS

OPERATIONAL CONCEPTS

MISSION

Infantry Strength

PAUSE

Air Strength

I I

I SIMULATE

I

Armored

Strength

A r t i l l e r y Strength Nuclear Strength Location I D1sposlt ion

SHARE

COMPARE UPDATE

I OVERLAY I

I I

ENTER

I ZOOM

I

1

I

I

ENEMY

COAs

ALLIED COAs

OPERATIONAL CONCEPTS 90

STOP

MISSION ENEMY CAPABlLIT1ES

ALLIED L P A B I LIT1ES I

Telecommunications

1

SIMULATE

Social

E

I

Economic Poli t lcal

1

1 PAUSE

Transportation

EXPLAIN

COMPARE

f

Terrain Weather

SHOW

SHARE

I

UPDATE OVERLAY ENTER

I

ENEMY

COAs

I

ALLIED COAs

I

MISSION

Infantry StrETIgth

PAUSE

Air Strength Armored

EXPLAIN

A r t i l l e r y Strength Nuclear Strength

I ~

I 1

-

Strength

Location / Disposlt ion

S'MULATE

I

SHARE

COMPARE UPDATE OVERLAY ENTER FLY

ENEMY

COAs

ALLIED COAs

91

MISSION ENEflY CAPAEl LIT1ES

AREA CHARACTER1STI Cs

ALL IED CAPABILITIES PAUSE

SHOW

t EXPLAIN

SEND

SIYUlATE

SHARE

COMPARE

UPDATE

OVERLAY ENTER Likely ~~

Possible

ZOOM

I

STOP

Unlikely

OPERATIONAL CONCEPTS

~~

ENEflY CAPABlLlTl ES

AREA CHARACTER1STI CS

ALL IED CAPAElLIT1ES

SHOW

EXPLAIN SIMULATE SHARE COMPARE UPDATE

1

OVERLAY

ENTER

Likely

FLY

ZOOM

1

Possible Unlikely

OPERATIONAL CONCEPTS 92

I

STOP

93

94


95

REFERENCES Adelman, L. (1990). “Evaluating Decision Support Systems.” QED Information Sciences, Inc., Wellesley, Massachusetts. Adelman, L., and Donnell, M. L. (1986). Evaluating Decision Support Systems: A General Framework and Case Study. I n “Microcomputer Decision Support Systems: Design, Implementation and Evaluation” (S. J. Andriole, ed.). QED Information Sciences, Inc., Wellesley, Massachusetts. Aiken, P. H. (1988a). “An Evaluation of the Capabilities of a DSS Based on Hypermedia Technologies.” George Mason University, Fairfax, Virginia. Aiken, P. H. (1988b). “A Demonstration of the Contribution of Hypermedia Technologies to Decision Support.” George Mason University, Fairfax, Virginia. Aiken, P. H. (1989). “A Hypermedia Workstation for Software Engineering.” George Mason University, Fairfax, Virginia. Ambron, R., and Hooper, C. (1987). “Interactive Multimedia: Visions of Multimedia for Developers, Educators and Information Providers.” Microsoft Publishers, Inc., Redmond, Washington. Andriole, S. J. (1983).“Interactive Computer-Based Systems Design and Development.” Petrocelli Books, Inc., Princeton, New Jersey. Andriole, S. J. (1985). “Applications in Artificial Intelligence.” Petrocelli Books, Inc., Princeton, New Jersey. Andriole, S. J. (1986). Graphic Equivalence, Graphic Explanations and Embedded Process Modeling for Enhanced User-Computer Interaction. IEEE Trans. Systems. Man and Cybernetics SMC-16 (6), 919-926.

96

STEPHEN J. ANDRIOLE

Andriole, S. J. (1987a).“Interactive Decision Aids and Support Systems for Strategic and Tactical Command and Control.” International Information Systems, Inc., Marshall, Virginia. Andriole, S. J. (1987b).“Functional Modeling for Theater-Level Planning and Decision-Making.” Department of Information Systems and Systems Engineering, George Mason University, Fairfax, Virginia. Andriole, S. J. (1987~).“User-Computer Interaction Requirements for Theater-Level Tactical Planning.” Department of Information Systems and Systems Engineering, George Mason University, Fairfax, Virginia. Andriole, S. J. (1987d).“User-Computer Interaction Requirements for Counter-Terrorism Crisis Management.” Department of Information Systems and Systems Engineering, George Mason University, Fairfax, Virginia. Andriole, S. J. (1987e). “The Design and Development of an Intelligent Planning Aid: The TACPLAN Prototype.” International Information Systems, Inc., Marshall, Virginia. Andriole, S. J. (1988). “User-Computer Interaction (UCI) Requirements for Group Problem Solving.” George Mason University, Fairfax, Virginia. Andriole, S. J. (1989a).“Decision Support Systems: A Handbook for Design and Development.” Petrocelli Books, Inc., Princeton, New Jersey. Andriole, S. J. (1989b). “Storyboard Prototyping: A New Approach to User Requirements Analysis.” QED Information Sciences, Inc., Wellesley, Massachusetts. Andriole, S. J. (1990). “Information System Design Principles for the 90s: Getting It Right Through Multidisciplinary Information Systems Engineering.” AFCEA International Press, Fairfax, Virginia. Andriole, S. J., and Hopple, G. W. (1984).They’re Only Human: Decision-Makers in Command and Control. Signal, October, 61-66. Andriole, S. J., and Hopple, G. W. (1988). “Defense Applications of Artificial Intelligence.” Lexington Books, Inc., Lexington, Massachusetts. Andriole, S. J., Ehrhart, L. S., Aiken, P. H., and Matyskiela, W. W. (1988). “Storyboarding Prototypes for Group Planning and Decision-Making.’’ Department of Information Systems and Systems Engineering, George Mason University, Fairfax, Virginia. Barclay, S., Brown, R. V., Kelly, C. W., 111, Peterson, C. R., Phillips, L. D., and Selvidge, J. (1977). “Handbook for Decision Analysis.” Decisions and Designs, Inc., McLean, Virginia. Baroudi, J. J., Olson, M. H.,and Ives, B. (1986). An Empirical Study of the Impact of User Involvement on System Usage and Information Satisfaction. Comm. ACM 29 (3). Bernstein, A. (1985). Shortcut to Systems Design. Business Computer Systems, June. Bertcher, H. J. (1979).“Group Participation.” Sage Publications, Newbury, California. Bice, K., and Lewis, C. (1989).“Wings for the Mind: Conference Proceedings: Computer Human Interaction.” Addison-Wesley Publishing Co., Reading, Massachusetts. Boar, B. (1984).“Application Prototyping: A Requirements Definition Strategy for the 80s.” Wiley Interscience, New York. Boehm, B. W. (1976).Software Engineering. IEEE Trans. Computers C-25, December. Bolt, R. A. (1984). “The Human Interface: Where People and Computers Meet.” Lifetime Learning Publications, Belmont, California. Brooks, F. P. (1987). No Silver Bullet: Essence and Accidents of Software Engineering. IEEE Computer, April. Carlisle, J. H. (1973).“Comparing Behavior at Various Computer Display Consoles.” The RAND Corporation, Santa Monica, California. DARPA/MIT (1988).“DARPA Neural Network Study.” AFCEA International Press, Fairfax, Virginia. Dee, D. (1984). Developing PC Applications. Datamation, April. DeSanctis, G., and Gallupe, R. B. (1987).A Foundation for the Study of Group Decision Support Systems. Management Science 33 (5), 589-609.


97

Ehrhart, L. S. (1988). “Storyboard Architectures for Group Problem-Solving.’’ Department of Information Systems and Systems Engineering, George Mason Universiiy, Fairfax, Virginia. Eisner, H. (1988). “Computer-Aided Systems Engineering.” Prentice-Hall, Englewood Cliffs, New Jersey. Fairley, R. (1985). “Software Engineering Concepts.” McGraw-Hill, New York. Fleischman, E. A., Quaintance, M. K., and Broedling, L. A. (1984). “Taxonomies of Human Performance.” Academic Press, New York. Galitz, W. 0. (1984). “Humanizing Office Automation.” QED Information Sciences, Inc., Wellesley, Massachusetts. Gladstein, D. L., and Reilly, N. P. (1985).Group Decision Making Under Threat: The TYCOON Game.” Academy of Management J . 28 (3), 613-627. Gomaa, H., and Scott, D. (1981). Prototyping as a Tool in the Specification of User Requirements. Fifth Intl. Conf. Software Engineering. Grief, I., and Sarin, S. (1986). Data Sharing in Group Work. Con$ Computer-Supported Cooperative Work, Proc. 1986, pp. 175-183. Hare, A. P. (1982). “Creativity in Small Groups.” Sage Publications, Newbury Park, California. Hart, S., Boroush, M., Enk, G., and Hornick, W. (1986). Managing Complexity Through Consensus Mapping: Technology for Structuring Group Decisions. Academy of Management Rev. 10 (3),587-600. Hendrick, C. (1987). “Group Processes.” Sage Publications, Newbury Park, California. Hice, G. F., Turner, W. S., and Cashwell, L. F. (1978). “System Development Methodology.” North-Holland, New York. Hopple, G. W. (1986). Decision Aiding Dangers: The Law of the Hammer and Other Maxims. IEEE Trans. Systems, Man and Cybernetics SMC-16 (6). Horowitz, E. (1975). “Practical Strategies for Developing Large Scale Software Systems.” Addison-Wesley, New York. JCS (Joint Chiefs of Staff) (1976). “Dictionary of Military Terms.” US. Government Printing Office, Washington, D.C. Lakin, F. (1986).A Performing Medium for Working Group Graphics. ConJ Computer-Supported Cooperative Work, Proc. 1986, pp. 255-266. Ledgard, H., Singer, A., and Whiteside, J. (1981).“Directions in Human Factors for Interactive Systems.” Springer-Verlag, New York. Lehner, P. E. (1986). “Decision Aid Design.” PAR Technology Corporation, Reston, Virginia. Leslie, R. E. (1986). “Systems Analysis and Design: Method and Invention.” Prentice-Hall, Englewood Cliffs, New Jersey. Martin, A. W., Esoda, R. M., and Gulick, R. M. (1983). “CONSCREEN-A Contingency Planning Aid.” Decisions and Designs, Inc., McLean, Virginia. Meister, D. (1976).“Behavioral Foundations of System Development.” John Wiley & Sons, Inc., New York. Norman, D. A,, and Draper, S. W., eds. (1986). “User Centered System Design.” Lawrence Erlbaum Associates, Hillsdale, New Jersey. North, R. L. (1988). Neurocomputing: Its Impact on the Future of Defense Systems. Defense Computing, January-February. Pinker, S., ed. (1985). “Visual Cognition.” MIT/Bradford Books, Cambridge, Massachusetts. Potter, A. (1988). Direct Manipulation Interfaces. Al Expert, October, 28-41. Pressman, R. S. (1987). “Software Engineering: A Practitioner’s Approach.” McGraw-Hill, New York. Ragland, C. (1989). Hypermedia: The Multiple Message. MacTech Quarterly, Spring. Ramsey, H. R., and Atwood, M. E. (1979).“Human Factors in Computer Systems: A Review of the Literature.” Science Applications, Inc., Englewood, Colorado.

98

STEPHEN J. ANDRIOLE

Royce, W. W. (1970). Managing the Development of Large Software Systems: Concepts and Techniques. In “TRW Software Series.” TRW, Inc., Redondo Beach, California. Sage, A. P. (1985). “Systems Engineering.” School of Information Technology and Engineering, George Mason University, Fairfax, Virginia. Sage, A. P. (1988).Group Decision Making: Can Information Technology Help the Go-Ahead Company? Decision Technologies, 39-50. Sage, A. P., and Rouse, W. B. (1986).Aiding the Decision-Maker Through the Knowledge-Based Sciences. IEEE Trans. Systems, Man and Cybernetics SMC-16(4). Schneiderman, B. (1987). “Designing the User Interface.” Addison-Wesley, New York. Schweiger, D. M., Sandberg, W. R., and Ragan, J. W. (1985). Group Approaches for Improving Strategic Decision-Making: A Comparative Analysis of Dialectical Inquiry, Devil’s Advocacy, and Consensus. Academy of Management J . 29 (l), 51-71. Smith, S. L., and Mosier, D. (1984).“Design Guidelines for the User Interface to Computer-Based Information Systems.” the Mitre Corporation, Bedford, Massachusetts. Srinivasan, A., and Kaiser, K. M. (1987). Relationships Between Selected Organizational Factors and Systems Development. Comm. ACM 30 (6). Stefik, M., Bobrow, D. G., Lanning, S., and Tater, D. (1986). WYSIWYG Revised: Early Experiences with Multi-User Interfaces. Proc. CSC W’86 Conf. Computer-Supported Cooperative Work, pp..276-290. Stefik, M., Foster, G., Bobrow, D. G., Kahn, K., Lanning, S., and Suchman, L. (1987). Beyond the Chalkboard: Computer Support for Collaboration and Problem-Solving in Meetings. Comm. ACM 30 (l), 32-47. Tachmindji, A. J., and LalTerty, E. L. (1986).Artificial Intelligence for Air Force Tactical Planning. Signal, June, 110- 114. Thierauf, R. J. (1978).“An Introductory Approach to Operations Research.” John Wiley & Sons, New York. Van Duyn, J. (1982).“DP Professional’s Guide to Writing Effective Technical Documentation.” John Wiley & Sons, New York. Weiss, A. H. (1986).An Order of Battle Adviser. Signal, November, 91-95.

Perceptual Models for Automatic Speech Recognition Systems RENATO DEMORI McGill University School of Computer Science Montreal, Quebec, Canada

MATHEW J. PALAKAL Purdue University School of Science at Indianapolis Department of Computer Science Indianapolis, Indiana

PIER0 COSl Centro di Studio per le Richerche di Fonetica C.N.R. Padova, Italy

1. Introduction. . . . . . . . . . . . . . . . . . . . . . . 2. Speech and Speech Knowledge . . . . . . . . . . . . . . . 2.1 Acoustic Characteristics of Phonemes . . . . . . . . . . . 2.2 Speech Recognition Systems . . . . . . . . . . . . . . 3. A Multi-Layer Network Model for ASR Systems . . . . . . . . . 4. The Ear Model: An Approach Based on Speech Perception. . . . . . 4.1 Speaker-Independent Recognition of Ten Vowels In Fixed Contexts . 4.2 The Recognition of Phonetic Features . . . . . . . . . . . 4.3 Recognition of New Vowels and Diphthongs . . . . . . . . . 4.4 WordModels . . . . . . . . . . . . . . . . . . . . 5. The Vocal Tract Model: An Approach Based on Speech Production. . . 5.1 The Skeletonization Algorithm. . . . . . . . . . . . . . 5.2 Line Description . . . . . . . . . . . . . . . . . . . 5.3 Description of Frequency Relations Among Spectral Lines . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

.

100 101 103 11I 127 129 136 141 146 148 150 154 158 159 167 169 169

99 ADVANCES IN COMPUTERS, VOL. 31

Copyright 01990 by Academic Press. Inc. All rights of reproduction in any form reserved ISBN 0-12-012131-X

100

RENATO DEMORI e t a / .

1.

Introduction

Speaker-independent Automatic Speech Recognition (ASR) by computers of large or difficult vocabularies is still an unsolved task, especially if words are pronounced connectedly. Efforts and progress toward the solution of this problem are reported in the recent literature (Bahl et al., 1983; Levinson, 1985; Kopec and Bush, 1985; Kimball et al., 1987). During the past two decades, there has been substantial progress toward the goal of constructing machines capable of understanding and/or recognizing human speech. One of the key improvements has been the development and application of mathematical methods that permit modeling the speech signal as a complex code with several coexisting levels of structure. The ultimate goal of research on automatic speech recognition is to give the machine capabilities similar to humans to communicate in natural spoken languages. Such research is of great interest from both the application point of view and the research point of view. Since speech is our most natural mode of communication, we should have the potential of machines that more fully accomodate to the human user, rather than perpetuating the trend of our mechanical slaves actually enslaving us in unwanted diversions, such as learning keypunching, typewriting, and complex programming methods (Lea, 1979). From the application point of view, there are several advantages of voice input to machines: voice input leaves eyes and hands free, it needs little or no user training, and it permits fast, multimodal communication, freedom of movement and communication. Speech recognizing machines have been possible application in many areas, such as office automation, assembly line inspection, airline reservation and aids for the handicapped. From the research point of view, automatic speech recognition is a difficult problem extending over the past four decades. Even though significant progress was made in the past, the ultimate goal, a perfect listening machine, is yet to be achieved. Several areas of human perception of voice have yet to be explored, and the findings of such research must be exploited for building listening machines. What has been done so far is based mostly on analytical methods, and only very recently have researchers incorporated detailed speech knowledge in their recognition models. Another key improvement has been the development of various recognition models for ASR systems, such as syntactic models, probabilistic parsing models, expert system models, and network models. Under network models, the most successful ones are stochastic models, procedure network models, and artificial neural network models. In Section 2 of this chapter we will discuss the fundamentals of speech production and speech knowledge, various techniques that are used in

PERCEPTUAL MODELS FOR AUTOMATIC SPEECH RECOGNITION SYSTEMS

101

speech recognition systems, and, briefly, some of the most successful speech recognition systems. In Section 3, we will discuss some of the recent advances in speech recognition research, such as the use of artificial neural network models and a special case of Hidden Markov models. In this section we will also compare and contrast speech-perception-based ASR systems (the ear model) and the conventional speech-production-based ASR systems (the vocal tract model). Some preliminary results are also presented for the earmodel-based system that uses multi-layer networks and also for the more conventional system that uses Markov models with line parameters based on the fast fourier transform (FFT).

2.

Speech and Speech Knowledge

In this section we present a review of the various components in the human speech production system. A brief discussion on the characteristic features of various sound classes is also presented. Finally, we review some of the basic and advanced methods for automatic speech recognition and describe some of the popular ASR systems. Figure 1 shows the organs of our speech production system. Speech sound is produced when air flows through and resonates in the vocal tract. Different sounds are produced because of different vocal tract configurations. For a class of speech sounds, such as vowels, there is a set of resonant frequencies characterizing each sound in the class. Also, different sounds are produced depending upon the source of excitation. During speech production, the articulators move continuously, rather than discretely, resulting in a continuum of acoustic characteristics. There are two basic types of sound sources in speech: periodic vibration of vocal folds, and turbulent noise. When the speaker exhales, air passes through the larynx. If the glottis is partially closed, then the air passing through the constriction causes the vocal cords to open and close quasi-periodically, producing the voiced sound. The rate of vibration, which is controlled by vocal cord tension, is called the fundamental frequency or pitch. When excitation is at the glottis, the vocal folds remain open and cause a weak turbulence, to produce aspiration sounds. A constriction in the vocal tract causes turbulent noise, which has a flat spectrum, and is called a frication sound. The peaks in the spectrum of voiced sound are called formants and are labeled as F1, F2, . . . , Fi, where F1 is called the Jirst formant and so on. The lips, tongue, jaw, and velum can be moved to change the shape of the vocal tract. The resultant vocal tract acts as a cascade of resonators, which filter the

102


1. Lips 2. Teeth 3. Teeth-ridge 4. Hard plate 5. Velum

6. Uvula 7. Blade of tongue 8. Front of tongue 9. Back of tongue

10. Pharynx 11. Epiglottis 12. Vocal cords 13. Tip of tongue 14. Glottis

FIG.1. Organs of speech production.

source. The poles of the vocal tract transfer function generate spectral peaks called the formants. In the case of nasals, sound passes through the nasal cavity, but the mouth cavity, which is closed, acts as a side branch and introduces zeros in the spectrum. The interaction of the poles and zeros can change the frequency of the formants (Schwartz, 1982); Early speech scientists described speech sounds in terms of particular characteristics of speech, such as voiced, unvoiced, front, back, etc. (Oppenheim and Schafer, 1968; Jakobson et al., 1952). During speech production, the articulators move relatively slowly from one position to another. The articulators often do not reach their “target” positions due to contextual effect of neighboring phones: this is called coarticulation (Heffner, 1950). Therefore, the spectral sequence associated with a particular phone can vary widely depending on the adjacent phones. Different speakers’ vocal apparatus can vary in terms of the source spectrum, the length of the vocal tract, and the relative shape of the vocal tract. For this reason, the speech of adult males and females differs; typically, the


103

pitch period of female speech is about 20% shorter, causing an average 20% increase in the formant frequencies (Fant, 1966). In addition to the differences due to speaker, dialect, and phonetic context, there is also random variation in the pronounciation of speech sounds. Even for the speech of a single speaker, the spectral properties present cannot be converted back to a phonetic string without the use of higher-level knowledge. The articulatory movements vary for different speech sounds: for some the vocal tract configurations are stable, while for others, they are not. For example, the configuration for the sound /miis more stable than for an / r / . In the velum is lowered, while the sound / r / the vowel-to-nasal transition for /m/, is produced by retroflexing the tongue. The tongue cannot move as fast as the velum, and this causes the difference in the configuration dynamics. 2.1

Acoustic Characteristics of Phonemes

A brief discussion of the acoustic nature of each phonetic group is considered now. More details on these materials can be found in (Schwartz, 1982; Zue and Schwartz, 1979; Rabiner and Schafer, 1978; Skinner, 1977).

Vowels Vowels are produced by exciting the open vocal tract with a periodic (voiced) source. Vowels are often characterized by substantial energy in the low- and mid-frequency regions. The energy in the high frequencies above 3500Hz are less important for vowel characterization. The characteristics of different vowels depend on the location of the tongue, position of the jaw, and the degree of lip rounding. The resulting shape of the vocal tract determines the formant frequencies. The three classes of vowels- back, central, and front-occur as a result of the tongue position. In general, when the tongue moves forward, the second formant rises; as the tongue moves higher or the jaw rises, the first formant decreases. Lip rounding lowers all of the first three formants. Figure 2 shows acoustic waveforms (Rabiner and Schafer, 1978), and Fig. 3 shows the resonance characteristics of vowels (Wakita, 1977). Many vowel recognition methods measure the first three formants in the middle portion of the vowel and compare those values against stored targets. These values are called vowel loci. Variances of formant frequency distributions for each vowel around vowel loci are speaker independent. Consonants

The consonants are divided into several groups depending on the manner in which they are articulated. The five such groups in English are the plosives, fricatives, nasals, glides, and affricates. Consonants from different “manner-of-articulation” groups often have different acoustic correlates.

I

I

/

FIG.2. Acoustic waveforms of vowels (Rabiner and Schafer, 1978).


105

U

d

“f W 0

a

J

n I 4

FREOUENCY

(kHz 1

FIG.3. Resonance characteristics of vowels. Reprinted with permission from “Normalization of Vowels by Vocal Tract Lens and Its Application to Vowel Identification,” H. Wakita, IEEE Transactions on Acoustic Speech and Signal Processing, April 1977. 01977 IEEE.

Consonants within a “manner-of-articulation” group differ in their voicing characteristics and the position of constriction. The acoustic properties of consonants differ both within the consonants and in the adjacent vowels in the form of formant transitions. This problem must be considered in order to recognize consonants. Nasals The nasals (/m/, /n/,/ng/)are always adjacent to a vowel, and are marked by a sharp change in intensity and spectrum, corresponding to the closing of the oral cavity and opening of the velum. Nasal sound is produced by a glottal excitation and vocal tract constriction at some point. By lowering the velum, the air flow is forced through the nasal cavity. Nasals are very difficult to recognize, since nasal murmur differs significantly from speaker to speaker, because of differences in the size and shape of the nasal and sinus cavities. Nasal murmur is also heavily affected by phonetic environment. Some of the nasal characteristics are the prominence of a low-frequency spectral peak at around 300 Hz, little energy present above 3k Hz, and sharp spectral discontinuity between the nasal murmur and the adjacent vowel. Figure 4 shows acoustic waveforms of the nasal sounds /m/and /n/.

---1

1

1

1

I l l I ' l l

" I l l

I I l (

(-7

FIG.4. Acoustic waveforms of nasal sounds (Rabiner and Schafer, 1978).


107

Liquids and Glides This group is also sometimes known as semi-vowels. Such consonants often appear next to vowels as in the case of nasals. These sounds are produced by a constriction in the vocal tract that is smaller than that of vowels but still large enough so that no turbulence is generated. Each phoneme in this group has a close association with certain vowels, such as

l w l => Iul, i r i => 131,

IYl * lil, III * 101.

These consonants are distinguished from other consonant groups in that the rate of articulatory movement is considerably slower, which implies slower formant transitions and the formants of these sounds have the following qualitative relation with respect to the formants of adjacent vowels: / w / has lower F1 and F 2

/ I / has lower F1 and F2 with higher F 3 frf has lower F 3 / y / has very low F1 with higher F 2 . The formant patterns within the phonemes are similar to some vowels, and their distinguishing characteristics are often detected by comparison with those of the adjacent vowels. Plosives Plosives are also known as the “stop” consonants. Plosive sounds are classified into two groups: voiced or lax ( / b / ,Id/, / g / ) and unvoiced or tense ( f p f , ftJ, flcf). Voiced plosive sounds are produced by building up pressure behind a total constriction somewhere in the oral tract. During the total constriction no sound is radiated through the lips. However, a small amount of low-frequency energy is radiated through the walls of the throat called the voice bar. Unvoiced plosive sounds are produced in the same way as voiced sounds, except that during total constriction, vocal cords do not vibrate. Plosive consonants are considered to be the most difficult consonants to recognize, for the following reasons:

The production of a stop is dynamic, involving a closure and release period. The complicated nature of this production results in many diverse acoustic cues. The acoustic events during the production of the sound can be omitted or severely distorted.

/UH-P-A/

/UH-B-A/

FIG.5. Acoustic waveforms of plosive sounds (Rabiner and Schafer, 1978).


109

Some of the characteristics of voiced and voiceless stops are the following: (a) The plosives are characterized acoustically by a period of prolonged silence, followed by an abrupt increase in amplitude at the consonantal release. The release is accompanied by a burst of frication. (b) For voiceless stops, the aspiration noise is generated at the glottis. (c) The voice onset time (VOT), which is the duration between the release and the onset of normal voicing for the following vowel, is longer for unvoiced (30 to 60 ms) than for voiced (10 to 30 ms) stops. (d) The voiced stops are often prevoiced, creating the voice bar in the lowfrequency region. (e) The amplitude of the burst is significantly different between voiced and voiceless plosive sounds. Figure 5 shows acoustic signals for samples of voiced and voiceless plosives. Fricatives Like plosives, there is a voiced fricative group ( / v / , / T H / ,/z/, / z h / ) and a voiceless fricative group consisting of (If/, 101, Is/, / s h / ) . Unvoiced fricatives are produced by exciting the vocal tract by a steady air flow that becomes turbulent in the region of a constriction in the vocal tract. Unlike the unvoiced fricatives, voiced fricatives are produced by vocal cord vibration and excitation at the glottis. Since the vocal tract is constricted at some point, the air flow becomes turbulent. Voiced fricatives often have simultaneous noise and periodic excitations which cause great amount of low-frequency energy in the beginning of frication. Voiced fricatives are also shorter than unvoiced fricatives. Acoustic signals for some of the fricative sounds are shown in Fig. 6. Affricates The affricates ( / c / ,/ j / ) are often considered as a plosive followed by a fricative. These sounds are often modeled as a sequence of two phonemes (/c/ as /t-s/ and / j / as /d-z/). The duration of frication is often very short compared to other fricatives.

The properties of each class of phonemes can be considered as acoustic knowledge about the speech sounds. However, these properties are not independent, and they may be distorted or omitted. Because the cues for a phoneme are so redundant, the human speaker tends to be rather careless about producing these prototypical features for a given phoneme. Distortions may vary from speaker to speaker, or even over time for the same speaker. Despite the above mentioned problems regarding human speakers, the human listener somehow has no trouble discarding the “bad” features and

/UH-V-A/

/UH-ZH-A/

I

r

w -oo- . t

FIG.

mrr

6. Acoustic waveforms of fricative sounds (Rabiner and Schafer, 1978).

c


111

accepting only the “good” ones. This is possible because “higher-level context” is available, and also because humans use phonotatic constraints in decoding distorted syllables. This is a clear indication that enough information to decode the phonemes is present in the acoustic signal. Therefore, phonetic recognition algorithms must consider several features jointly, rather than a particular feature. Given several features that each contribute toward making phonetic distinctions, the Acoustic Phonetic Recognizer must enlist the aid of a multi-dimensional feature selection and pattern recognition algorithm to design the optimum classifier (Schwartz, 1982). Many approaches have been used to incorporate various features that are present in speech sound. Some of those important techniques are discussed in the next section. 2.2

Speech Recognition Systems

During the past two decades, there has been substantial progress toward the goal of constructing machines capable of understanding and/or recognizing human speech. One of the key improvements has been the development and application of mathematical methods that permit modeling the speech signal as a complex code with several coexisting levels of structure. For any speech recognition system, the spectrum is usually represented by Fourier coefficients, zero crossing rate, or the parameters of some local model of signal such as linear prediction coefficients. Temporal information can be directly obtained as in the case of voice onset time. Prosodic information is often extracted by estimating fundamental frequency to represent pitch and the logarithm of energy integrated over 45-ms intervals to measure intensity (Levinson, 1985). Presently, features obtained this way are neither robust nor invariant with respect to speaker. As a result of some psychophysical experiments, (Miller et al., 1951), there is an assertion that speech is a composite signal, hierarchically organized so that simpler patterns at one level are combined in a well-defined manner to form more complex patterns at the succeeding level. Such an organization strategy is easily explained in terms of information-theoretic principles. The structures at each level of the hierarchy serve to constrain the ways in which the individual patterns associated with that level can be combined. The constraints build redundancy into the code, thereby making it robust to errors or variations caused by a speaker. This way relatively few primitive patterns can be combined in a multilevel hierarchy according to a complex code to form a rich, robust information-bearing structure (Levinson, 1985). Spectra and prosodics, the primitive patterns according to linguistic theories, can be combined in several ways to form phonemes (Cohen and

112

RENATO DEMORI et a / .

Mercier, 1975; Nakatsu and Kohda, 1978; Woods, 1975), broad phonetic catagories (Shipman and Zue, 1982; Chen, 1980), diphones (Scaglioda, 1983), demisyllables (Rosenberg et al., 1983; Ruske and Schotola, 1982), syllables (DeMori et al., 1985),supra-segmental phrases (Lea et al., 1975; Mercier et al., 1979), and sentences (Perennou, 1982; Walker, 1975).For the implementation of theories, data structures such as templates (Bourlard et al., 1984; Haton and Pierrel, 1980; Aldefeld et al., 1980), formal grammars (Myers and Levinson, 1982;Bahl et al., 1979; Levinson, 1979), Markov chains (Baker, 1975b; Jelinek and Mercier, 1980; Rabiner et al., 1983), fuzzy sets (DeMori, 1973), and hash tables (Kohonen et al., 1980) have been used. In nonparametric methods the primitive measurements of the speech signal can be compared without regard for their temporal location. However, sequences of these measurements, such as the one required to represent speech signals of greater temporal extent, must, due to the nonstationarity of the speech signals, take account of time to be meaningfully compared (Levinson, 1985). A comparison approach between the two methods, according to Levinson, is shown in Fig. 7. It has been argued that nonparametric methods are easy to train, but as a classifier the parametric methods perform just the opposite in terms of com-

Training Phase Vector Quantization &

Parameter Estimation

I

Models

0

z

.CI

E

&'

c

Time Aligned Distance Computation

4

Probability Estimation

4 Vector Quantizer

FIG. 7. Comparison of parametric and nonparametric methods (Levinson, 1985).


113

plexity. Template matching, stochastic modeling, and probabilitic parsing were the most successful models. Some of the benchmark systems developed using the above approach are described briefly in the following sections. Most of the following discussions are summarized from Moore (1984), DeMori and Probst (1986) and Haton (1984). 2.2.1

Template Matching

Template matching is based upon principles of nonparametric estimation of likelihoods by means of invariant metrics. In template matching, each recognition unit (a unit could be a word or a phoneme, etc.) is represented by at least one template, created from a set of training utterances. Each template contains a sequence of patterns extracted in time. The patterns are spectral and/or prosodic features. During matching processes, the target pattern is matched against stored templates, and the matched template with a minimum distance is the selected candidate. The whole success of the pattern matching approach lies in the comparison process. Absolute Pattern Match The most basic comparison process is simply to correlate the timefrequency word patterns produced by a pre-processor in order to determine the distance between an unknown word and each template. This may not be possible, because words are often of different duration and their corresponding patterns are of different sizes. A potential solution to this problem can be obtained by aligning the beginnings of all the patterns, and by correlating only over the areas of overlap. This simple technique, experimented with by White and Fong (1975), requires N vector comparisons per pattern match, where N is the number of vectors in the smallest pattern. Best Absolute Time Alignment An alternative to aligning the beginnings of words in order to perform an absolute comparison is to adjust their relative timing to maximize the correlation of the overlap. That is, starting with the beginnings aligned, the patterns are shifted with respect to each other until the ends align. The similarity of the pattern overlap is calculated at each shift, and the highest similarity is the result of the comparison. Computationally this scheme is expensive. Recognition experiments carried out using this method by Moore (1984) found no significant improvements in recognition. Linear Time-Normalization The previous techniques do not consider the fact that the same word is very rarely the same duration on different occasions. In order to handle this

114

RENATO DEMORI eta/.

problem, the patterns are uniformly time-normalized to make them the same size. This is known as linear time-normalization. For practical applications, either the template patterns are time-normalized to the unknown pattern, or all patterns are time-normalized to a pre-set duration. If a very small vocabulary (10 to 30 words) is used, such techniques perform well. A commercial system made available by Interstate Electronics called VRM used a linear time-normalization approach. Several other commercial systems used such techniques. The performance also depends on the inherent confusability of the words, consistency of speakers, type of features used, and the number of training samples allowed. Nonlinear Time-Normalization Linear time-normalization does not perform well for larger vocabularies. The reason is that making the pattern of fixed length is not an adequate model of what actually happens when people make words longer or shorter. A model of time-scale distortion that allows different sounds to be distorted differentially would align the pattern more meaningfully. One approach to computer recognition of speech requires that we compare two sequences of elements and compute the distance between them by finding an optimal alignment or correspondence between the elements of one sequence and those of the other. In speech research, these sequence comparison methods are capable of using dynamic programming to perform nonuniform dynamic time warping (DTW). The name refers to allowing nonlinear distortions of time scales in computing the acoustic similarity between a reference prototype and features extracted from the input utterance, thereby taking into account speaking-rate variability as well as sustitution, insertion, and deletion errors. Dynamic programming is an efficient algorithm by which both optimal alignments and the resulting distances are computed at the same time. Data and prototypes to be matched are represented by discrete sequences produced after either synchronous or asynchronous sampling of the continuous speech signal; due to normal speech variability, two sequences arising from two utterances of the same word may exhibit a number of local differences. These local differences may be that one element has been substituted for another, that an element has been inserted, or that an element has been deleted. Other local differencemodels are conceivable, such as one that allows expansion of a single element into several elements or compression of several elements into a single element, as independent types of local difference. Given two sequences and costs (or weights) of the local differences, an alignment is assigned a cost equal to the sum of the costs of the local differences in it; the distance between the two sequences is the least cost of any alignment. DTW is essentially a two-stage process. Figure 8 illustrates the first stage. Two abstract speech-like patterns are shown, one vertically and one hori-


115

zontally. Each pattern has time frames consisting of three-element vectors; the vertical pattern has four frames, and the horizontal has five. The matrix in the center is known as the distance matrix. It contains numbers that correspond to the distances between the frames in one pattern and the frames in the other pattern. For example, the number 20 in the top right hand corner indicates that the first frame of the vertical pattern is quite different from the last frame of the horizontal pattern. Similarly, the 1 indicates that the second frames of each pattern are very similar. The distance is actually calculated by taking the sum of the squares of the differences between each pair of frames. The second stage is to find the path through the distance matrix, from the top left hand corner to the bottom right hand corner, that has the minimum accumulated sum of distances along its length. This path is the required nonlinear relationship between the timescales of these two patterns, and it is found by dynamic programming. Dynamic programming involves the regular application of a local optimization procedure which ultimately leads to an overall global solution. In this case a local decision function is used, together with the distance matrix, to construct a second matrix called the cumulative distance matrix. Figure 9 illustrates the process. The local decision function is shown in Fig. 9a. It defines that a path may arrive at any particular point either vertically, horizontally or diagonally, and is applied as follows.

FIG.8. Distance matrix obtained after comparing two abstract patterns. Reprinted with permission from “Systems for Isolated and Connected Word Recognition,” R. K. Moore, New Sysfems and Architectures for Automatic Speech Recognition and Synthesis, Springer-Verlag, 1985.

116

RENATO DEMORI et al.

(a)

A

?

t

FIG.9. Demonstrationof Dynamic Time Warping (Moore, 1984).(a) Local decision function. (b) Partially completed cumulative distance matrix. (c) Completed cumulative distance matrix. (d) Record of local decisions. Reprinted with permission from “Systems for Isolated and Connected Word Recognition,”R. K. Moore, New Systems and Architectures for Automatic Speech Recognition and Synthesis, Springer-Verlag. 1985.

For each point in the cumulative distance matrix, add the cheapest cost of getting to that point to the cost of being at that point, and enter it in the matrix. The cheapest cost of getting to a point is the smallest of the values in the previous entries (as defined by the local decision function), and the cost of being at a point is simply the value taken from the corresponding position in the distance matrix. Hence, if this process is applied iteratively, starting at the top left hand corner of the matrix, it is possible to complete all the entries in the cumulative distance matrix. Figure 9b shows the cumulative distance matrix in the process of being filled in. The “?” indicates the point being considered, and the three previous points are highlighted. The cost of getting to the point is the minimum of 19,13 and 21, and the cost of being at that point is 12 (from the distance matrix in Fig. 8). Hence the cumulative distance entered at that point is 25 (13 + 12).


117

Figure 9c shows the cumulative distance matrix completely filled in. The number in the bottom right hand corner is highlighted because this is the overall distance between the two patterns; it is the sum of distances along the least-cost path through the distance matrix. To find the path it is necessary to remember at each point in the calculation exactly which local decisions were made (horizontal, vertical or diagonal). Figure 9d shows all of these decisions. It can be seen that they form a tree radiating from the top left hand corner (where the calculation started). The actual minimum-cost path is found by tracing back along the local decisions, starting at the bottom right hand corner (where the calculation ended). Referring back to the distance matrix (Fig. 8), the calculation shows that the least-cost path takes the route 7 + 1 + 5 + 12 + 2; no other path has a cumulative sum less than 27. The formulation for this dynamic programming is the following recursive expression: D(i, j ) = d(i, j )

+ min [ D ( i

-

1, j ) , D ( i - 1, j - 11, D(i, j - 111,

where 1 I i I I and 1 I j I J and ( I and J are the numbers of frames in the two patterns being compared), d is a distance measure between two frames, and the initial condition is D(0, 0) = 0. The overall distance between the two patterns is D(1, J ) . Dynamic programming techniques originally developed for isolated word recognition have also been applied to the problem of recognizing connected words. Here, the spoken input is a sequence of words from a specified vocabulary, and matching occurs against isolated word reference templates. We are given an input pattern with some number of time frames. We also possess a set of reference templates, where each template has a length equal to the number of frames in that template. The goal is to find the sequence of templates that best matches the input pattern for a particular match criterion. The concatenation of templates is referred to as a super reference pattern. Two proposed solutions to this problem can be found in the two-level algorithm of Sakoe (1979) and the level-building algorithm of Myers and Rabiner (1981). Also worth mentioning in this context is the one-stage dynamic programming algorithm of Ney (1984). A brief description of Myers and Rabiner’s level-building dynamic timewarping (DTW) algorithm for connected word recognition is as follows. The underlying idea is that the matching of all possible word sequences can be performed by successiveconcatenation of reference patterns. At the beginning, the time registration of the test pattern against a given super reference pattern is considered; it is observed that the algorithm can be implemented in levels, that is, one reference (of the super reference pattern) at a time. The computation matches test frames only against frames within a particular reference; the set of accumulated distances between different segments of the

118

RENATO DEMORI et a/.

test pattern and that reference is saved and used as a set of initial distances for the next level. This idea is then extended to a level-building algorithm with multiple reference patterns, that is, when each reference of the super reference pattern is one of a set of reference patterns. The recognition performance of isolated word recognizers based on DTW techniques is significantly better than that obtainable from linear timenormalization. This is because DTW provides a far more realistic timescale compensation process; greater variability can be accommodated, hence larger vocabularies may be used. Also, by using relaxed endpoint constraints (the position where the timescale registration path is allowed to start and end), DTW does not suffer from the same dependency on endpoint detection as linear time-normalization. Hence the segmentor can be much simplier, and it is left to the DTW process to decide precisely where the words begin and end.

2.2.2 Network-Based Systems In the previous section we studied dynamic time-warping systems originally developed for isolated word recognition and later extended to recognition of strings of connected words. In this section we look at two representative network-based systems, Carnegie- Melon University’s Harpy system and IBM’s Markov modeling system, which are directed toward the more difficult problem of continuous speech recognition. In the general form of this problem we are interested in large-vocabulary, speaker-independent recognition; the two systems under consideration restrict the problem considerably by introducing grammatical and/or task constraints so that a simple finite-state model may be built of the entire language to be recognized. Both systems compile knowledge at different levels of the language model into an integrated network. In the Harpy system, phonetic, phonological, lexical, and syntactic constraints have been combined into a single model which generates all acceptable pronunciations of all recognizable sentences;in the IBM system, each word of the top-level language model is replaced by a phonetic subsource, and then each phone is replaced by an acoustic subsource, yielding a model of all acoustical realizations of sentences in the language. An important difference between the two networks is the fact that, in the IBM system, all sources and subscources are Markov models, while in Harpy, Markov networks have given way to transition networks with no a priori probabilities associated to symbols that label transitions; as already mentioned, in both cases the integrated language models are finite-state models. Another important difference is that Harpy uses segmentation while the IBM system does not. In Harpy, the acoustic signal is divided into variablelength segments that represent “stable” portions of the acoustic signal;


119

spectral characteristics of each segment are then determined for use in phone template matching. The assumption here is that, given enough allophone templates, it is reasonable to attempt labeling of segments using pattern matching techniques. Asynchronous segmentation is performed top-down and then the network is used to select prototypes to be matched with the data. In the IBM system, no attempt is made to segment the speech into phonemelike units: instead, a time-synchronous acoustic processor produces parameter vectors computed from successive fixed-length intervals of the speech waveform. A parameter vector coming from a 10-ms frame is matched against a set of prototypes: the parameter vector is then labeled by giving it the name of the prototype to which it is closest. Another possibility is that of using the input vector for retrieving a priori probabilities of different labels.

2.2.3 The Harpy Sysrern The Harpy system is an attempt to combine the best features of the Hearsay I system and the Dragon system (Baker, 1975a).The most significant aspects of the system design are an integrated network language model (knowledge representation) and the use of beam search through the network during recognition. Segmentation is attempted, phonetic classification depends on unique templates, and word juncture knowledge is an integral part of the network. A word network exists such that any path through the network gives an acceptable sentence. Each word is replaced by a pronunciation network which represents expected pronunciations of the word. After words have been replaced by their subnetworks, word juncture rules are applied to the network to model phone string variations due to influences of neighboring words. During compilation into the composite network, various optimization heuristics are applied to yield an efficient phone network, that is, a network of acceptable pronunciations. During the recognition process, Harpy attempts to find an optimal sequence of phones satisfying two criteria: a) the sequence must represent a legal path through the network, and b) the sequence should consist of phones with high acoustic match probabilities. It is possible that the best fit to a particular segment in the left-to-right search does not correspond to the correct interpretation; to compensate for this, beam-search strategy is used in which a group of near-miss alternatives around the best path is examined. When the end of the sentence is reached, the phone sequence with the lowest total distance is selected; backtracing through the globally best sequence obtained at the end of forward searching yields the desired phone and word assignments. A pronunciation dictionary and phone characteristics allow us to replace words with their subnetworks. A simplified subnetwork for the word “was” is

120


shown in Fig. 10. As before, redundant paths are removed; phonetic symbols are taken from the ARPAbet (Lesser et al., 1975). So far we have seen illustrations of syntactic knowledge and lexical knowledge, although information about phone duration has been deliberately omitted from the latter. The phonetic network attempts to capture intraword phonological phenomena; word boundary phonological phenomena, on the other hand, are represented by word juncture rules, which contain examples of insertion, deletion, and substitution of phones at word boundaries. The word juncture rules are then applied to the network. Finally as before, redundant paths are removed. Harpy’s front end performs rough segmentation based on zero-crossing rates and peaks in smoothed and differenced waveform parameters called the zapdash parameters. Quasi-stationary segments derived from the zapdash parameters are matched against phone templates. These phone templates are linear prediction spectral templates, and comparison is based on Itakura’s minimum prediction residual error measure, which computes similarity in spectral shape. The spectral templates are talker specific but new templates may be learned automatically, for example, by adapting speaker-independent templates. The beam-search strategy for searching the finite-state graph prunes from further consideration paths scoring less than a variable threshold, rather than using a priori probabilities to find the most likely path through the network. In systems like Harpy and Hearsay I1 (Erman et al., 1976), segments are detected asynchronously and then labeled. Labeling a variable-length segment consists of recording, for each allophonic template, the probability that the segment represents an occurrence of that particular template. In contrast, synchronous nonsegmenting systems consider successive fixed-length frames of the speech signal. For each frame, we obtain a vector x of parameters representing the spectrum of that frame of speech. In vector quantization, our problem, for each such x, is to find that codeword x iin a codebook of stored prototypes whose spectral distance from x is a minimum. In this speech coding technique, we have the collection of possible reproduction vectors

FIG. 10. Nonredundant phonetic network for the word “was” (DeMori and Probst, 1986).


Find X-

121

9 with

minimal d(x,? ),

FIG.11. Vector quantizer encoder (DeMori and Probst, 1986).

xlr x 2 , . . . , x,, which is stored in the reproduction codebook or simply the codebook of the quantizer; the reproduction vectors are called codewords (or templates). Moreover, we have a distance classifier which allows us to compare vectors according to a spectral distance. The encoding is illustrated in Fig. 11. Problems in constructing a good vector quantizer include choosing a good set of spectral representatives in the codebook, usually through training. More details about vector quantization can be found in Gray (1984).

2.2.4 The ISM System

IBM has developed two benchmark systems. The first one is a speakertrained continuous speech recognition system with a recognition of 91% on words contained in sentences in the 1000-word vocabulary set (Jelinek et al., 1975;Jelinek, 1976).The second system is an isolated wordrecognition system with an accuracy of 95% on an 8000-word office correspondence vocabulary (Bahl et a!., 1983). This system has recently been enhanced by expanding the vocabulary to 20,000 words (Averbuch et al., 1987). The system is based on Markov models of language and has been implemented using two control strategies: a) a Viterbi algorithm, and b) a leftto-right stack decoder algorithm that estimates the probability that a given partial hypothesis can be extended to yield the actual sentence. Important aspects of the system design include the presence of a priori transition probabilities in the finite-state language model and the formulation of speech

122


recognition as a problem of maximum-likelihood decoding. As such, statistical models of the speech production process are required. The choice among these two control strategies and the decoding methods mentioned earlier is a function of degree of task constraint, the size of the state space. In the IBM approach, the allowed sentences are either described a priori by an artificial grammar or else limited by a vocabulary and a task domain in which models may be constructed from observed data. The distinctive feature of the IBM approach is that speech recognition is formulated as a problem in communication theory. The speaker and the acoustic processor are conceptually combined into a single unit, the acoustic channel. Figure 12 shows the relation between the text generator, the acoustic channel, and the linguistic decoder. In Fig. 12, w is a string of words generated by the text generator, y is a string of acoustic processor output symbols (more specifically, a string of prototype identifiers, one for each 10 ms of speech),and w’ is the word string produced by the linguistic decoder as an estimate of the word string w. The acoustic channel provides the linguistic decoder with a noisy string from which it must attempt to recover the original message. The linguistic decoder searches for a word string w that maximizes the probability P(w,y) of the joint observation of (w,y) at the two ends of the channel. A stochastic model of the acoustic channel will account for both the speaker’s phonological and acousticphonetic variations and for the unvarying performance of the acoustic processor. Given models that specify both P(w)and P ( y 1 w), the linguistic decoder may determine w using some algorithm that is appropriate to the size of the language. The model of text generation is a language model. Both the language model and the acoustic channels are Markov sources consisting of states connected by transitions; with each transition there is an associated output word. A probability is attached to each transition. In the IBM system, the language model assigns probabilities to strings of words. For the acoustic channel model for single words, a phonetic Markov subsource is associated to each word. The possible output strings, drawn from an alphabet of phones, are all the different phonetic pronounciations of the word. An example is shown in Fig. 13. For each word there is a set of phonetic subsources, and for each phone there is a set of acoustic subsources. An acoustic subsource-for a phone is a Markov source whose output alphabet

Text

Generator

W

b Acoustic Channel

A

0 Linguistic decoder 4

FIG.12. Speech recognition as a communication problem (DeMori and Probst, 1986).

c


T,2/3

IH,1/3

DX,1/3

P,2/3

123

FIG. 13. Phonetic subsource for the word “two” (DeMori and Probst, 1986).

contains the output symbols of the acoustic processor and which specifies both possible acoustic processor outputs for each phone and their probabilities. More details on stochastic decoding and its performances can be found in Bahl et al. (1983) and Schafer and Rabiner (1975). Results obtained on stochastic-model-based network systems show that there is no significant difference in recognition accuracy from that of the DTW approach. However, from a computational point of view the Markov models require an order of magnitude less storage and execution time; where the DTW based techniques have a very simple training phase (only data collection) and a very complicated recognition phase, Markov models are just the reverse. It has been overwhelmingly agreed that Markov models provide the correct balance for any practical system (Moore, 1984). 2.2.5 Knowledge-Based Systems The purpose of a “good” recognition model is to take knowledge and generalize it appropriately to assess new events. This is possible only by a proper understanding of the variabilities involved. The conclusion drawn after the ARPA project on Speech Understanding Research (SUR) was that there still exists a great need for integrating more speech knowledge into ASR models in order to solve difficult tasks, as well as to achieve better recognition results. The performance of an automatic speech recognizer ultimately depends on the amount and quality of the training material. However, if the dimensionality of the representation is raised, then recognizers are always going to be undertrained. It is therefore vital to know how the knowledge embedded in the training material can best be structured and hence utilized. In theory, it ought to be possible to extract a great deal of structural information from the speech signal itself since humans can do it. So the question is how to obtain more knowledge from speech data. The main characteristic of knowledge is that it is highly domain-dependent. Abstractly speaking, knowledge is made up of descriptions, relationships, and

124

RENATO DEMORI et el.

procedures corresponding to a given domain of activity. In practice, knowledge can take many diverse forms. It roughly consists of “objects” and their relationships in a domain, together with the procedures and heuristics for manipulating the relationships. It is obvious that the error-prone nature of speech data makes it necessary to have an efficient cooperation between highly diversified knowledge sources: knowledge concerning phonetics, phonology, prosody, lexicon, syntax, semantics, and pragmatics. The choice of adequate structures for representing the available knowledge sources is a crucial problem in speech understanding, as well as in any A1 system. Several approaches were taken in the past which include several interesting ideas. One possible solution is to use a single structure in which all the diverse knowledge sources are integrated. This was the solution chosen in the Harpy system. Harpy integrates knowledge of all levels in a precompiled network which contains the various phonetic transcriptions of all syntactically legal sentences. The only disadvantage of this approach is that the size of the network becomes too large and storing all possible sentences causes the system to be too rigid. A second solution is at the other extreme from that of Harpy, a total independence of the various knowledge sources. A hierarchical method for implementing such a scheme, called the blackboard model, was used in the Hearsay I1 system (Lesser et al., 1975). Figure 14a shows an example of a blackboard organization. In this approach the knowledge sources are independent processes which, in principle, are not aware of each other and which asynchronously post hypotheses at various levels (phoneme, syllable, word, etc.) to a global data base called the blackboard. This way a sentence is described at different levels. Invoking a given knowledge source is datadirected in the sense that specific preconditions must be fulfilled to access the blackboard. Blackboard schemes have been successfully applied to various other A1 areas such as vision (Prager et ul.,) and signal interpretation. A third solution, which is intermediate between the network model and the blackboard model, is the hierarchical model as shown in Fig. 14b. In this approach the processing is controlled by some kind of control structure or supervisor. In contrast with the data-driven, asynchronous activation of Hearsay 11, the knowledge sources in this model are activated by the supervisor. This way control strategies can be tested by modifying the supervisor. The Hwim system of BBN was based upon such an approach. Rule-Based Expert Systems Recent results obtained in A1 are largely due to sophisticated problemsolving systems called expert systems. For a well defined and restricted


Lexical Access

125

1

BOARD

FIG. 14. (a) Example of a blackboard model. (b) Example of a hierarchical system.

domain, expert systems are able to reach the level of expertise of a human expert. Instead of the classical two-tiered organization of data and program, expert systems introduce a more flexible, three-level structure of data, knowledge base, and control (Haton, 1980). Figure 15 shows the overall organization of an expert system.

126


MAN MACHINE INTERFACE

I

Language Processing .Data and Knowledge Acquisition

CONTROL STRATEGY Inference Explations about Reasoning

*

KNOWLEDGE BASE Knowledge Sources Rules Data and Facts

The knowledge base is used by the system for analyzing the problem deductively. It typically incorporates some kind of data-activated operators whenever specific preconditions are met during the problem-solving process. Expert systems make it possible to split a complex expertise into a large number of relatively simple rules. A human expert often seems to use a production rule scheme while reasoning. Therefore, these systems can be successfully applied to various aspects of speech recognition that require solving specific and limited problems. Some of the attempts made so far which use expert systems approach are in speech spectrogram interpretation, multiexpert structure for accessing large lexicon. In the multi-expert system, different experts in the society execute in parallel various algorithms derived from a task-decomposition of the speech recognition algorithm. The major difference between Hearsay 11’sblackboard model and the expert system lies at the level of the control structure. In Hearsay I1 the only way the various knowledge sources (the experts) can communicate is by asynchronously posting hypotheses in a data base. The knowledge sources are triggered when specific preconditions in the data base are satisfied. In the expert society each expert is provided with a specific control strategy and communicates directly with other experts. This strategy makes use of planning algorithms and is related to the A1 concept of frames which provides an interesting framework for the predictive use of knowledge. An example of this can be found in (DeMori et al., 1987a). We have seen several approaches to speech recognition each using varying amounts of speech knowledge and different ways of knowledge representation. Template matching techniques use constraints to define a manageable task and are easy to develop, but they use very little speech knowledge. The


127

Harpy system used more speech knowledge integrated into its network model. The IBM system showed techniques for combining speech knowledge with well defined mathematical models. Such techniques did manage to take into consideration speaker variations and coarticulation effects to a certain extent. The Hearsay I1 and Hwim systems showed the importance of using independent knowledge sources even though both used different types of knowledge representation. Expert systems were shown to be promising for knowledge representation in a natural way, and they are especially attractive if the domain is specific, small, and well defined. Most of the past work on ASR clearly demonstrates that the solutions of difficult speech recognition tasks involving speaker-independence, connected speech or large vocabularies need much more speech knowledge, knowledge from all levels, in their models. Faster and larger computing systems may help in solving this problem. It is also understood that speech signals contain sufficient knowledge, since humans seem to process it very easily. In order to integrate knowledge from different sources that are of different nature and that are available at different levels, one should adopt different types of knowledge integration techniques, by applying the most appropriate ones at each level, rather than adopting just one specific model, in a recognition model. In the next section of this chapter we will discuss the use of artificial neural network models and a special case of Hidden Markov models that can be used in ASR systems. We will also compare and contrast speechperception-based systems (the ear model) and more conventional speech recognition systems. The ear-model-based speech recognition system uses multi-layer networks, whereas the more conventional system uses Markov models with FFT-based line parameters.

3. A Multi-Layer Network Model for ASR Systems Coding speech for automatic speech recognition (ASR) can be performed with multi-layer networks (MLN). This approach is interesting because it allows one to capture relevant speech properties useful for ASR at the stage of coding. MLNs are networks with an input layer of nodes, one or more hidden layers and an output layer whose nodes represent a coded version of the input. Nodes are connected by links. Weights are associated to links. All the links bringing a signal into a node contribute to the calculation of the excitation of that node. The excitation is the sum of the product of the weights of each link and the value of the output coming from the node the link carries its signal from. The output of a node is a function of the node excitation. By choosing the link weights a large variety of coders can be designed having specific properties. Link weights can be obtained by a learning process. Learning can

128


be supervised or unsupervised. When learning is supervised, the network input is fed by sets of patterns. Each set corresponds to a class of patterns that have to be coded with the same values appearing at the output nodes. The output nodes are clamped with the desired values, and algorithms exist for computing the values of the link weights in such a way that the network codes the sets of input patterns as desired. These learning algorithms have a relevant generalization capability. Many scientists are currently investigating and applying learning systems based on MLNs. Definitions of MLNs, motivations and algorithms for their use can be found in Rumelhart et al. (1986), Plout and Hinton (1987), and Hinton and Sejnowski (1986). Theoretical results have shown that MLNs can perform a variety of complex functions (Plout and Hinton, 1987). Applications have also shown that MLNs have interesting generalization performances capable of capturing information related to pattern structures as well as characterization of parameter variation (Bourlard and Wellekens, 1987; Watrous and Shastri, 1987). Algorithms exist for MLNs with proven mathematical properties that allow learning to be discriminant and to focus on the properties that make different patterns belonging to different classes. Furthermore, in MLNs the knowledge about a set of competing classes(in our case Speech Units or phonemes) is distributed in the weights associated to the links between nodes. If we interpret each output of the coder as representing a phonetic property, then an output value can be seen as a degree of evidence with which that property has been observed in the data. Two important research problems can be studied with such an approach. The first problem investigates the possibility of learning the features of each phoneme only in some phonetic contexts and relying on the generalization capability of a network for generating correct hypotheses about phonemes in contexts that have not been used for learning. The second problem is similar to the first, but deals with the possibility of learning all the required features and using them for correctly hypothesizing phonemes that have not been used for learning. In order to study the second problem mentioned above, it is necessary to code the output with some features in order to learn features and to represent each class (phoneme or speech unit) as a combination of features. We have chosen as main features the place of articulation and the manner of articulation related to tongue position. The reason is that these features are well characterized by physical parameters that can be measured or estimated. Phoneticians have characterized vowels and other sounds by discretizing place of articulation and manner of articulation related to tongue position, which in nature are continuous acoustic parameters. We have inferred an MLN for each feature, and we have discretized each feature with five


129

qualitative values, namely P L 1 , . . . , P L i , . . . , PL5 for the place and MN1, ..., M N j ,..., M N 5 for the manner. We have used 10 vowels pronounced by many speakers in a fixed context for training the two networks, each vowel being represented by one of the PLi and one of the M N j . In order to describe all the vowels of American English with enough redundancy, we have introduced another network with two outputs, namely T = tense and L = lax. We have also inferred the weights of a network with 10 outputs, one for each vowel. The performances of this network have shown that it is possible to obtain an excellent generalization of the parameters when training is performed on a limited number of male and female speakers using data that make evident acoustic properties having little variance across speakers when the same vocalic sound is pronounced. The performances of this network have also been used as reference. Tests have always been performed with new speakers. The first test consists of pronouncing the same vowels in the same context as in the data used for learning. This test is useful for comparing the results obtained from a mathematical model of the ear with those obtained from the more popular Fast Fourier Transformation (FFT).This test is also useful for assessing the capabilities of the network learning method in generalizing knowledge about acoustic properties of speakers pronouncing vowels. The second test has the objective of recognizing vowels through features. This test has been useful for investigating the power of the networks with respect to possible confusions with vowels not used for learning. The third experiment consists of attempting to recognize new vowels pronounced by new speakers in order to investigate the capability of the networks to detect the same features used for learning but integrated into sounds that were not used for learning. This generalization capability was verified with eight new sounds pronounced by 20 new speakers. Without any learning on the new sounds but just using expectations based on phonetic knowledge on the composing features and their time evolutions, an error rate of 7.5% was found. In the next section we describe in detail the mathematical model of the ear and a multi-layer network model that is used in speaker-independent recognition of 10 vowels in fixed contexts. 4.

The Ear Model: An Approach Based on Speech Perception

Cochlear transformations of speech signals result in an auditory neural firing pattern significantly different from the spectrogram, a popular timefrequency-energy representation of speech. In recent years basilar membrane, inner cell and nerve fiber behaviour have been extensively studied by auditory physiologists and neurophysiologists,

130


and knowledge about the human auditory pathway has become more accurate. A considerable amount of data has been gathered in order to characterize the responses of nerve fibers in the eighth nerve of the mammalian auditory system using tone, tone complexes and synthetic speech stimuli (Seneff, 1984, 1985, 1986, 1988; Delgutte, 1980; Delgutte and Kiang, 1984a-d). Phonetic features probably correspond in a rather straightforward manner to the neural discharge pattern with which speech is coded by the auditory nerve. For these reasons, even an ear model that is just an approximation of physical reality appears to be a suitable system for identifying those aspects of the speech signal that are relevant for recognition. The computational scheme proposed in this chapter for modeling the human auditory system is derived from the one proposed by Seneff (1984).The overall system structure, which is illustrated in Fig. 16, includes three blocks.

INPUT SIGNAL

1

40

- channels

Critical Band Filter Bank BASILAR MEMBRANE RESl'ONSE

Hair Cell Synapse Model FIRING PROBABILITY

Synchrony Detector

SYNCHRONY SPECTRUM

FIG.16. Block diagram of the ear model.


131

The first two of them deal with peripheral transformations occurring in the early stages of the hearing process, while the third one attempts to extract information relevant to perception. The first two blocks represent the periphery of the hearing system. They are designed using knowledge of the rather well known responses of the corresponding human auditory stages (Sinex and Geisler, 1983). The third unit attempts to apply a useful processing strategy for the extraction of important speech properties, such as spectral lines related to formants. The speech signal, band-limited and sampled at 16 kHz, is first pre-filtered through a set of four complex zero pairs to eliminate the very high and very low frequency components. The signal is then analyzed by the first block, a 40-channel critical-band linear filter bank. The transfer functions of the filters are depicted in Fig. 17. Filters were designed to optimally fit physiological data (Sinex and Geisler, 1983) and are implemented as a cascade of complex high frequency zero pairs with taps after each zero pair to individual tuned resonators. Figure 18 shows the block diagram of the filter bank. The transfer functions of the filters are Hi(z) = PF(z)SC,(z)

for i = 1,. . . , 40,

(1)

with PF(z) = (1 - ZlZ+)(l

-

ZTz-')(l

-

Z2ZP)2(1 - z,z-')2(1 - Z4z-')2,

and SCi(Z) =

[(l - z q z - ' ) ( l - ZP:z-')-y [(I - Ppiz-'(I - PP:z-')12

fl

j = 4 0 , ...,i

(1 - zsjz-')(l - Z s y ) (3)

where i = 1, . . . ,40 and ZS are the zeroes of the serial branch, ZP the zeros of the parallel branch and PP are the poles of the parallel branch. Filter resonators consist of a double complex pole pair corresponding to the filter center frequency ( C F )and a double complex zero at half its CF. Frequencies and bandwidths for zeros and poles were designed almost automatically by an interactive technique developed by S. Seneff and described in her thesis (Seneff, 1985). The second block of the model, whose block diagram is shown in Fig. 19, is called the hair cell synapse model. It is nonlinear and is intended to capture prominent features of the transformation from basilar membrane vibration, represented by the outputs of the filter bank, to probabilistic response properties of auditory nerve fibers. The outputs of this stage, in accordance with Seneff (1985), represent the probability of firing as a function of time for a set of similar fibers acting as a group. Four different neural mechanisms are

132


Input Signal

I

f

I

Filter Bank

Basilar Membrane Response FIG. 17. Frequency responses of filters introduced to simulate the basilar membrane.

modeled in this nonlinear stage. A transduction module which half-wave rectifies its input has the transfer function y(n) shown in Fig. 20, corresponding to the following relation: y(n)

=

GHw{l + A tan-' [ B x ( n ) ] } G eABx(n) HW

for x(n) > 0 for x(n) I 0.

(4)


133

FIG. 18. Structure of the filter bank.

The rectifier is applied to the signal to stimulate the high-level distinct directional sensitivity present in the inner hair cell current response. The shortterm adaptation, which seems due to the neurotransmitter release in the synaptic region between the inner hair cell and its connected nerve fibers, is simulated by the so-called “membrane model.” The mathematical equations

134


FIG. 19.

Block diagram for obtaining the firing probability


135

Transduction Module ( Half-Wave Rectifying Model)

FIG.20. Transfer function of the transduction module.

describing the mechanism that influences the evolution of the neurotransmitter concentration inside the cell membrane are given by the following:

The meaning of the signals in Equation ( 5 ) is defined in Fig. 21. The third unit in Fig. 19 represents the observed gradual loss of synchrony in nerve fiber behaviour as stimulus frequency is increased. It is implemented by a simple low-pass filter with the following transfer function: H(z) =

[

- %R

1 - GLSRZ-l

],

N

= 4.

The last unit is called “Rapid Adaptation.” It performs “Automatic Gain Control” and implements a model of the refractory phenomenon of nerve fibers. It is based on the following equation:

where (x(n)) is the expected value of x(n) obtained by sending x(n) through a first-order low-pass filter having the transfer function

136

RENATO DEMORI eta/.

Adaptatlon Module ( based on ‘Membrane Model’ )

Membrane s(t) source concentration

_j, >‘

target-region concentration

$ 1

> > ’‘ > ‘ > > ‘‘

> 1,

and to 1 when s(n

Advances in Computers, Volume 31

Advances in Computers, Volume 49

Advances in Computers, Volume 55

Advances in Computers, Volume 56

Advances in Computers, Volume 19

Advances in Computers, Volume 12

Advances in Computers, Volume 43

Advances in Computers, Volume 28

Advances in Computers, Volume 9

Advances in Computers, Volume 11

Advances in Computers, Volume 7