Advances in Computers, Volume 22

Advances in COMPUTERS VOLUME 22 Contributors to This Volume ALFST. BERZTISS BRUCEG. BUCHANAN B. CHANDRASEKARAN 0. D...

Author: Gemignani | Michael C.; Lakshmivarahan | S.; Wasserman | Anthony I.

46 downloads 1228 Views 19MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Advances

in COMPUTERS VOLUME 22

Contributors to This Volume

ALFST. BERZTISS BRUCEG. BUCHANAN B. CHANDRASEKARAN 0. DUDA RICHARD MICHAEL C. GEMIGNANI S. LAKSHMIVARAHAN SANJAY MITTAL S A T l S H THATTE I . WASSERMAN ANTHONY

Advances in

COMPUTERS EDITED BY

MARSHALL C . YOVITS Purdue School of Science Indiana University-Purdue University at Indianapolis Indianapolis. Indiana

VOLUME 22

1983

ACADEMIC PRESS A Sub\ldi.irv of Hdrcourr B r x e Jovdnovich. Puhli\herc

New York Paris

London San Diego San Francisco

Si30 Paulo

Sydney

Tokyo

Toronto

COPYRIGHT @ 1983, BY ACADEMIC PRESS, INC. ALL RIGHTS RESERVED. NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC O R MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT PERMISSION IN WRITING FROM THE PUBLISHER.

ACADEMIC PRESS,INC.

11 1 Fifth Avenue, New York, New York 10003

United Kingdom Edition published b y ACADEMIC PRESS, INC. (LONDON) LTD. 24/28 Oval Road, London NWI IDX

LIBRARY OF

CONGRESS CATALOG CARD

NUMBER:59-15761

ISBN 0-12-012122-0 PRINTED IN THE UNITED STATES OF \MERICA

83 84 85 86

9 8 7 6 5 4 3 2 1

Contents

CONTRIBUTORS TO VOLUME 22 . . . . . . . . . . . . . . . . . PREFACE. . . . . . . . . . . . . . . . . . . . . . . . . .

ix xi

Legal Protection of Software: A Survey Michael C. Gemignani

1 . Introduction

2. 3. 4. 5. 6. 7.

. . . . . . . . . . . . . . . . . . . . . . .

Copyright . . . . . . . . . . . . . . . Patents . . . . . . . . . . . . . . . . Trade Secrecy . . . . . . . . . . . . . Some Additional Open Questions . . . Conclusion . . . . . . . . . . . . . . Appendix . . . . . . . . . . . . . . . Selected Bibliography . . . . . . . . Index of Cases Cited . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

3 10 24 26 30 32 43 43

Algorithms for Public Key Cryptosystems: Theory and Applications

.

S Lakshmivarahan

1. 2. 3. 4. 5.

Introduction . . . . . . . . . . . . . . . . Mathematical Preliminaries . . . . . . . . Examples of Public Key Cryptosystems . . Applications . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . Note Added in Proof . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45 64 82 94 101 102 107

Software Engineering Environments Anthony 1. Wasserman

1 . Introduction . . . . . . . . . . . . . . . . . . . . . . . 110 2 . The Software Life Cycle . . . . . . . . . . . . . . . . . 1 1 1 3 . Management Procedures . . . . . . . . . . . . . . . . . 115

vi

CONTENTS

4 . Software Development Methodology

. . . . . . . . . . . .

6. 7. 8. 9.

An Example: The User Software Engineering Methodology . . The Software Development Environment . . . . . . . . . . 138 The Physical Environment . . . . . . . . . . . . . . . . 141 Toward Improved Software Engineering Environments . . . . 149 10. Conclusion . . . . . . . . . . . . . . . . . . . . . . . 158 References . . . . . . . . . . . . . . . . . . . . . . . 159

Principles of Rule-Based Expert Systems

.

.

Bruce G Buchanan and Richard 0 Duda

1. 2. 3. 4. 5. 6.

Introduction: What Is an Expert System? . . . . . . . . . . 164 Representation of Knowledge . . . . . . . . . . . . . . . 173 Inference Methods in Expert Systems . . . . . . . . . . . 184 Reasoning with Uncertainty . . . . . . . . . . . . . . . . 190 198 Key Concepts . . . . . . . . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . 205 Appendix . Answers to Questions about MYCIN’s Consultation in Section 1.1 . . . . . . . . . . . 207 General References . . . . . . . . . . . . . . . . . . . 210 References . . . . . . . . . . . . . . . . . . . . . . . 210 Conceptual Representation of Medical Knowledge for Diagnosis by Computer: MDX and Related Systems

.

B Chandrasekaran and Sanjay Mittal

1. Introduction

2. 3. 4. 5. 6. 7. 8.

. . . . . . . . . . . . . . . . . . . . . . .

Overview of the Conceptual Structure Methodology . . . . Diagnostic Problem Solving . . . . . . . . . . . . . . . Auxiliary Systems: Intelligent Access to Medical Data . . . Evaluation of Diagnostic Performance . . . . . . . . . . Extensions t o Diagnostic Problem Solving . . . . . . . . Comparative Remarks . . . . . . . . . . . . . . . . . Concluding Remarks . . . . . . . . . . . . . . . . . . Appendix A . Performance of MDX on an Example Case . . Appendix B . Detailed Example of Query Evaluation in PATREC . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . .

218

. 221

. . . . . . .

230 240 266 271 274 275 278

. 289 292

vii

CONTENTS

Specification and Implementation of Abstract Data Types Alfs

1. 2. 3. 4. 5. 6. 7. 8.

T. Bentiss and Satish Thatte

Introduction . . . . . . . . . . . . . . . . Axiomatic Specifications for ADT . . . . . The Meaning of Algebraic Specifications . . Consistency and Completeness . . . . . . Implementation and Verification . . . . . . Problems Associated with Data Abstraction . A Practical Approach to Data Abstraction . Conclusions and Future Trends . . . . . . References . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

296 302 309 323 329 . . . . . . . . 335 . . . . . . . . 339 . . . . . . . . 348 . . . . . . . 350

AUTHOR INDEX. . . . . . . . . . . . . . . . . . . . . . . . 355 SUBJECT INDEX. . . . . . . . . . . . . . . . . . . . . . . . 362 CONTENTS OF PREVIOUS VOLUMES . . . . . . . . . . . . . . . . 371

This Page Intentionally Left Blank

Contributors to Volume 22 Numbers in parentheses indicate the pages on which the authors’ contributions begin

ALFST. BERZTISS. Department of Computer Science, University of Pittsburgh, Pittsburgh, Pennsylvania 15260 (295)

BRUCEG. BUCHANAN. Department of Computer Science, Stanford University, Stanford, California 94305 (163)

B. CHANDRASEKARAN, Department of’Computer and information Science, The Ohio State University, Columbus, Ohio 43210 (217)

RICHARD 0. DUDA,Laboratory for Artificial intelligence Research, Fairchild Camera and instrument Corporation, Palo Alto, California 94304 (163)

MICHAEL c. GEMIGNANI, College of Sciences and Humanities, Ball State University, Muncie, Indiana 47306 ( I ) S. LAKSHMIVARAHAN, School of Electrical Engineering and Computer Science, The University of Oklahoma, Norman, Oklahoma 73019 (45)

SANJAY MITTAL,*Department of Computer and Information Science, The Ohio State University, Columbus, Ohio 43210 (217) SATlSH THATTE,? Department of Computer Science, University of Pitts-

burgh, Pittsburgh, Pennsylvania 15260 (295)

ANTHONY I . WASSERMAN, Medical Information Science, University of California, Sun Francisco, Sun Francisco, California 94143 (109)

*Present address: Xerox Palo Alto Research Center, 3333 Coyote Hill Road, Palo Alto. California 94304. ?Present address: Computer and Communication Sciences, University of Michigan, Ann Arbor, Michigan 48109. ix


Preface With the publication of Volume 22 of Advances in Computers we continue the presentation of subjects which are not only timely but also of long-range interest to the computer and information science community. Contributions have been solicited from well-known experts who recognize the importance of providing review or tutorial articles covering their areas of expertise. Advunces in Computers permits the publication of survey-type articles which are written from a relatively leisurely perspective and which may be somewhat lengthier than other publication media are willing to accept. It is thus possible to treat topics both in depth and in breadth. Volume 22 continues a serial publication which began in 1960. In these 22 volumes many articles have had a significant impact on the development of computers and their applications. That this serial has continued for these many years is a tribute to its reputation and the regard in which it is held. Included in this volume are contributions on legal protection, cryptography, software engineering, expert systems, computer-aided medical diagnosis, and data abstraction. In the first article Michael Gemignani states that by any standards, software is big business. Tens of billions of dollars each year are spent on software development. He points out that by far the largest volume of software development is done in-house. Hardware vendors provide substantially more software than do custom- and packaged-software houses. He predicts that however large the software industry is now, it is likely to grow even more rapidly than the hardware side. The capabilities of hardware have outstripped software technology. This contribution focuses on three important means of protecting intellectual property as it applies to software: copyright, patent, and trade secrecy. Gemignani concludes that there remains a serious question concerning the true importance of legal protection for software for the software industry. Although many developers are affixing copyright symbols as a matter of form, it is not clear what percentage of these copyrights are being registered. Furthermore, he points out that the software industry has now thrived for many years in the absence of a clear law of protection, and it seems unlikely that the legal confusion will be cleared up in the near future. There is little evidence that the software industry has suffered significant harm due to the xi

xii

PREFACE

legal confusion in the past, nor are there signs that this industry will cease growing owing to continuing legal doubts. Dr. Lakshmivarahan considers public key cryptosystems. Historically, the use of cryptography was exclusively confined to military and diplomatic communities to obtain secrecy of data in communication. In recent days, however, cryptography has been going public; both private companies and corporate sectors alike have started using cryptography to protect the secrecy of sensitive data. Some of the recent advancements in computer and communication technologies and extensive computerization of information storage and transmission are primarily responsible for this explosive interest of the private sector in cryptography. The very technology which made many of the marvels of the computerized society possible could well be used for compromising the security of the cryptosystems. All these possibilities have evoked the interest of private citizens and commercial organizations in acquiring far more secure cryptosystems. Recent advancements in cryptographic techniques, known as public key cryptography, have provided very elegant solutions not only to secrecy and authentication, but also to protection against forgery. In his article, Lakshmivarahan surveys recent advances, considering both theory and possible applications. He concludes that there is no known public key system to date that is unbreakable against an attack with unlimited computing resources. In the third article, Anthony Wasserman discusses software engineering. He states that every organization has a software development methodology and a software development environment. Vast segments of our entire postindustrial civilization are highly dependent upon the proper functioning of this software and the machines on which it executes. With this tremendous growth in the volume and criticality of software has come a growing need to improve the quality of software and the process by which it is produced. The problems of developing new software systems are accompanied by the problems of maintaining and enhancing existing software systems. Accordingly, it is necessary to create and to use techniques that improve the productivity of both individual software developers and the organizations to which they belong. Furthermore, it is necessary to assure the quality of such systems, both in terms of their conformance to user requirements and their reliable operation. Throughout history, advances in tools have led to higher quality products, improved productivity of workers, and occasionally to social and cultural changes. Wasserman expects the same thing to happen with tools for system development as newer, more sophisticated tools are built and integrated into methodologies and environments for system design and development.

PREFACE

xiii

According to Bruce Buchanan and Richard Duda, an expert system is a computer program that provides expert-level solutions to important problems and is heuristic, transparent, and flexible. The key ideas have been developed within artificial intelligence over the past 15 years, but in the past few years more and more applications of these ideas have been made. In this contribution the authors familiarize readers with the architecture and construction of one important class of expert systems, called rule-based systems. They provide a framework for understanding this advancing frontier of computer science. They conclude that expert systems represent an important set of applications of artificial intelligence to problems of commercial as well as scientific importance. Furthermore, they predict that technological innovations will be incorporated into expert systems as the conceptual difficulties of representation and inference in complex domains yield to standardized techniques; that systems will use much larger knowledge bases than the few hundred to few thousand rules now used and they will be linked electronically to large data bases to facilitate inference and avoid asking questions whose answers are matters of record; and that more powerful system-building frameworks will be developed to reduce the time it takes to iterate on the build-test-refine cycle and to increase the range of applications. In their contribution, B. Chandrasekaran and S. Mittal describe and discuss an approach to the design of medical decision-making systems based on the notion of conceptual structures for knowledge representation. A collection of related systems that has been under development exemplifies this approach, but the ideas are more general than the particular systems to be described. The authors point out that within a decade since the beginning of the modern electronic computer age, many attempts to use the power of the computer in the difficult task of medical decision making were begun. Early attempts combined elementary statistical and logical techniques, but soon a large proportion of work in computer-aided medical decision making began to be devoted to the application of Bayesian or related statistical classification techniques. A major line of work began in the mid-1970s when researchers in artificial intelligence began to consider problems of medical reasoning. The development in artificial intelligence of a subarea called knowledge-based systems has given a great deal of impetus to the design of computer programs that solve a variety of clinical problems: diagnosis, therapy selection, explanation of a medical state of affairs, etc. They conclude that the central determinant of effective use of knowledge is how it is organized; that in a given domain of expertise, there are different types of problem solving that can go on: that for each type of problem solving, there exists a separate knowledge structure, with the associated problem-solving mechanism

xiv

PREFACE

embedded in it; that in expert problem solving, the knowledge structure available for problem solving of a given type can in principle be decoupled from commonsense knowledge; and that the totality of reasoning and problem solving by a clinician is decomposable into a number of problemsolving regimes. Chandrasekaran and Mittal believe that the future of computer-based medical consultation research is very bright, both in view of its educational contributions in uncovering the structure of thought processes in medical reasoning, as well as in view of its potential impact on health-care delivery. In the final article, abstract data types are discussed by Alfs Berztiss and Satish Thatte. They state that a radical change is taking place in the interpretation of the nature of data structures. The description of a data structure is no longer to be a recipe for arranging a section of computer memory in some special way. Instead, a data structure is to be identified with the operations that are applicable to it, and the only way to generate an instance of the data structure is to be by means of a sequence of applications of the operation. This is what they call data abstraction. They regard this as an essential component of an ongoing revolution in computer programming, but much work remains to be done before it becomes a truly practical programming tool. They conclude that formal specifications are here to stay. They believe that algebraic specifications have advantages over operational specification, but they are unable to predict whether one or the other of the two will ultimately win out. Very likely both approaches will coexist. They believe that there is much to be gained by taking the algebraic rather than the operational approach outside a functional setting. I am pleased to thank the contributors to this volume who have given extensively of their time and their energy in order to provide this significant contribution to their profession. They are busy people and yet they were willing to write outstanding review and tutorial articles in areas in which they have expertise. Their cooperation and assistance have been greatly appreciated. It is because of their efforts that this volume continues to display the high quality typical of this serial. The volume should be of interest and value for many years to come. It has been a rewarding experience for me to edit this volume and work with these authors.

MARSHALL C. YOVITS

Advances

in COMPUTERS VOLUME 22


Legal Protection of Software: A Survey MICHAEL C. GEMlGNANl College of Sciences and Humanities Ball State University Muncie, Indiana 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Copyright.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 A Primer of Copyright Law. . . . . . . . . . . . . . . . . . . . . 2.2 History and Current State of the Law . . . . . . . . . . . . . . . 3. P a t e n t s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 A Primer of Patent Law . . . . . . . . . . . . . . . . . . . . . . 3.2 History and the Law Prior to Diehr and Brudley . . . . . . . . . . 3.3 Diehr and Bradley and the Current State of the Law . . . . . . . . 4. Trade Secrecy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Primer of Trade Secrecy Law . . . . . . . . . . . . . . . . . . . . . 5. Some Additional Open Questions . . . . . . . . . . . . . . . . . . . . 5.1 Does the New Section 117 of the 1976 Copyright Act Preempt Trade Secrecy? . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 To What Degree Is Software Patentable?. . . . . . . . . . . . . . 5.3 Is a Program in Object Code Only Copyrightable?. . . . . . . . . . 5.4 Can ROMs Be Patented and Copyrighted? . . . . . . . . . . . . . 6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. A p p e n d i x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Diamondv. Diehr,49U.S.L.W.4194(1981) . . . . . . . . . . . . 1.2 In re Bradley, 600 F.2d 807 (C.C.P.A. 1979). affd, 49 U.S.L.W.4250(1981) . . . . . . . . . . . . . . . . . . . . . . . Selected Bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . IndexofCasesCited. . . . . . . . . . . . . . . . . . . . . . . . . .

.

I 3 3 6 10

. .

. .

.

10 12 15 24 25 26

26 28 29 29 30 32 32 37 43 43

1. Introduction

By any standards, software is big business. Tens of billions of dollars each year are spent on software development. In 1980, IBM alone sold almost $2 billion in software and software services; the largest custom software supplier, Computer Sciences Corporation, sold $33 1 million; and the largest packaged-software house, Management Science America, sold $48 million worth. 1 ADVANCES IN COMPUTERS. VOL 22

Copynghl 0 1983 by Academic Press, Inc. All nghts of reproduction In any form reserved. ISBN 0-12-012122-0

2

MICHAEL C. GEMlGNANl

Several conclusions can be drawn from these figures about the economics of the software side of computing. First, by far the largest volume of software development is done “in-house.” About 80% of the programs that business and government need, they write themselves. Second, hardware vendors provide substantially more software than d o custom- and packaged-software houses. However large the software industry is now, it is likely to grow even more rapidly than the hardware side. The capabilities of hardware have outstripped software technology. Machines with vast memories, parallel processors and vector operations, and superfast switching can be limited by the efficiency of the software they utilize. In the early days of computing, the primary expense was the hardware, which consumed some 80% of data processing budgets. Now hardware accounts for only some 40% of data processing budgets, while software and maintenance accounts for the rest. Projections indicate that in the near future hardware will be but 20% of computing costs. Given the critical nature of software to computing and its increasing economic and technological importance, protection of proprietary interests in software is a growing concern. The investment in developing a complex program can run into the millions of dollars; quite naturally, the developers want to make sure that no one takes free advantage of their investment. Many are the programmers too who, having come up with what they believe is an original and commercially valuable idea, wonder how they can protect their intellectual “property” against infringement. This article will survey one of the most active areas in the rapidly evolving field of computer-related law: the legal protection of software. Although the article was current at the time it was written, the reader should be aware that new court decisions and legislative actions could make some of the statements herein obsolete even before the book is off the presses. Likewise, this article is not intended to give legal advice concerning specific situations, but is rather a survey of and an introduction to an area that is of significance to every programmer. Individual problems requiring legal assistance should be brought to a competent attorney. Not every means of legal protection will be touched on here. Unfair competition, antitrust, and contractual means are not discussed. Nor does the article discuss technological means of protection, even though such means are in widespread use throughout the software industry. We will discuss patents, copyrights, and trade secrets. We will outline the hasic legal concepts and rules and provide some of the historical background needed to understand the current state of the law. We will summarize the current state of the law and speculate on what the future might or should

LEGAL PROTECTION OF SOFTWARE

3

hold. Finally, we will touch on some of the as yet unanswered questions concerning software protection.

2. Copyright 2.1 A Primer of Copyright Law

What can be copyrighted? The 1976 Copyright Act states the following: Copyright protection subsists ... in original works of authorship fixed in any tangible medium of expression, now known or later developed, from which they can be perceived. reproduced, or otherwise communicated, either directly or with the aid of a machine or device.

The meaning of works ofauthorship is very broad in its scope. The category under which programs are classified by the Copyright Office is “literary works,” which includes “works, other than audiovisual works, expressed in words, numbers, or other verbal or numerical symbols or indicia, regardless of the nature of the material objects, such as books, periodicals, manuscripts, phonorecords, film, tapes, disks, or cards, in which they are embodied” (17 U.S.C. P l O l ) . A “work of authorship” must nevertheless be “original.” However, the degree of originality needed for copyright is minimal. “All that is needed to satisfy both the Constitution and the statute is that the ‘author’ contributed something more than a ‘merely trivial’ variation, something recognizably ‘his own.’ Originality in this context ‘means little more than a prohibition of actual copying’ [Alfred Bell & Co. v. Catalda Fine Arts, Znc., 191 F.2d 99, 102-103 (2d Cir. 1951)l. The condition of being “fixed in a tangible medium of expression” posed serious problems for programs under the 1909 Copyright Act, the forerunner of the current 1976 act. These problems are evidenced by Data Cash Systems, Znc. v. JS & A Group Znc., 480 F. Supp. 1063 (N.D. Ill. 1979), a case litigated in the Northern District of Illinois and decided in October of 1979. Even though the case was decided after the passage of the new copyright act, it involved a dispute which arose prior to the new act’s taking effect. Data Cash marketed a computer chess game in which the program for each set was contained only in “read-only memory” (ROM). When Data Cash sued a competitor for infringing upon this program, the judge ruled that the defendant’s ROM could not be a “copy” of the plaintiffs ROM because a ROM was not something which could be “seen and read” with the naked eye. Under the 1909 act, a “copy must be in a form which others can see and read.” It obviously matters little ”

4


whether a particular program is copyrighted if one does not infringe upon the copyright by loading the program into a computer and running it without authorization; but the conclusion one must draw from Data Cash is that a program in memory only, or even on tape or on the disk, is not something which is subject to meaningful copyright protection. The situation, however, is quite different under the 1976 Copyright Act. Memory, whether primary or secondary, is a tangible medium of expression. The program stored in memory can surely be perceived and even reproduced with the aid of a machine, the computer itself. Thus the Data Cash case would have been decided quite differently had the 1976 Copyright Act been applicable. Indeed, there is evidence to believe the case may have been wrongly decided even under the old copyright act. In affirming the decision of trial court, the Court of Appeals for the Seventh Circuit gave as the sole reason for affirmance the fact that the ROM in question had no copyright notice affixed to it [628 F.2d 1038 (7th Cir. 1980)l; the question of whether ROMs could be copyrighted was entirely avoided. There seems, then, little reason why software should not be copyrightable under the terms of the 1976 act. In fact, the Register of Copyrights has been accepting programs for copyright registration since 1964. The legislative history of the 1976 Copyright Act clearly demonstrates that Congress expected programs to be copyrightable, and Congress has codified this view in a recent amendment to the 1976 act which will be discussed later. The matter, then, should be well settled, but there remain two problems, one constitutional and the other practical. The mere fact that Congress believes something is copyrightable, or even legislates that it is so, does not necessarily make it so. There are certain classes of things which the Supreme Court has declared are simply not proper subject matter of copyright, even though, on their face and in fact they may be original writings of an author. “In no case does copyright protection for an original work of authorship extend to any idea, procedure, process, system, method of operation, concept, principle, or discovery, regardless of the form in which it is described, explained, illustrated, or embodied in such work.’’ [17 U.S.C. 0102(b)] The Supreme Court in Gottschalk v. Benson reaffirmed the rule that algorithms and mathematical formulas are not copyrightable, although some specific representation of a particular formula or algorithm might be. Thus one could not copyright the quadratic formula, but one could copyright a textbook explaining the derivation and use of the quadratic formula. If one considers a program as merely an algorithm, then it would not be copyrightable at all. If, as is more reasonable, one considers a program to be a concrete representation of some algorithm, the algorithm which underlies


5

the program is neither protected nor protectable by any copyright whatsoever. If someone does not infringe upon a copyright by simply using an algorithm contained in the copyrighted work, how does one infringe on a copyright? First of all, there must be copying if there is to be infringement. If the defendant did not use the plaintiff’s work to produce his own, then there is no infringement. The existence of striking similarities between the two works can be prima facie evidence of copying, but the defendant may reply that the similarities came about through coincidence, by use of some common source, or because of the limited number of ways in which the particular subject matter might be expressed. Inasmuch as copying is a crucial element in establishing infringement, the plaintiff must also show that the defendant had some reasonable opportunity to see his work. Even if copying can be established, there still may be no infringement. The court must determine what part of the copyrighted work is entitled to protection. Once this is done, a further determination can be made if the appropriation of that protected portion was improper. For example, given a copyright on the source code for a complex routine, we already have seen that the underlying algorithm or algorithms are not protected by copyright; thus their use by another would not be an infringement. However, the FORTRAN code for implementing the algorithms may be protected inasmuch as coding represents an original contribution by the programmer. If someone then copies a subroutine from that program into another program, a court might determine that the subroutine was such a significant part of the program that the appropriation was improper, or it may find that the subroutine was so minimal or commonplace that its use did not infringe upon the copyright, that is, that the use in issue was fair. The line between improper appropriation and fair use is far from clear. A court will consider such factors as the extent to which the defendant relied on the plaintiffs work, as well as the quantity and quality of the defendant’s own contribution to the copy. The test for improper appropriation is not what an expert might believe, but the “response of the ordinary lay person” [see, e.g., Universal Athletic Sales Co. v. Salkeld, 51 1 F.2d 904, 907 (3d Cir. 1979, cert. denied, 423 U S . 863 (19791. Subject to certain provisos, the copyright owner has certain exclusive rights. These are (1) to make copies of the copyrighted work, (2) to prepare derivative works from the copyrighted work, (3) to distribute copies, and (4) to display or perform the work publicly. A major concern with regard to program copyrights, a concern which was alleviated only through special legislation, is that certain standard practices in computing would constitute infringements even though they are necessary in order to

6


use a legitimately obtained copy of a copyrighted program, and the copyright holder almost certainly has no objection to the infringing practices in the first place. Merely loading a program into a computer or creating a backup file constitutes the creation of copies, a right reserved exclusively to the copyright owner. Thus, theoretically, someone could purchase a copy of a program but then not be able to run it without infringement. Similarly, when someone modifies a program, he is making a derivative work, again a right reserved to the copyright owner. Thus, someone buying a canned program and changing it to meet his specific needs would technically be infringing. Obviously, owners of program copyrights could hardly object to reasonable computing practices if they expect anyone to buy copies of the program, and no one using the program in good faith should be subject to potential liability for infringement either, simply because of the letter of law. 2.2 History and Current State of the Law

The Register of Copyrights has accepted computer programs for copyright registration since 1964. Despite this, by 1977 less than 1300 programs had been registered, and the vast bulk of these were from IBM and Burroughs. Hardware manufacturers have traditionally favored copyrights over other forms of protection, because copyright protection is so weak that it is conducive to a relatively free exchange of software, which enhances the possibilities of machine sales. It is estimated that some 1 million programs are developed each year; hence there is every reason to believe that the availability of program copyrights has not generated wide enthusiasm within the software industry. Also indicative of this apathy toward copyrights has been the minimal number of court cases involving alleged copyright infringement regarding software. One of the more interesting such cases is Synercom Technology, Znc. v. Uniuersify Computing Co., 462 F.Supp. 1003 (N.D. Tex. 1978). A competitor of the copyright holder wrote a preprocessor program that allowed users of the plaintiff’s software service to convert to the competitor’s service without changing the input format. The competitor constructed his preprocessor program from descriptions which were contained in the plaintiffs instruction manual, which was also copyrighted. The court held that the “order and sequence” of software formats are “ideas expressed,” which are not copyrightable, rather than the “expression of an idea,” which would be. Even so, the court was able to protect the plaintiff by finding that the competitor relied too heavily upon the instruction manual and thus infringed upon that copyright, this not in preparation of the preprocessor program, but in writing his own instruction manual.


7

The 1909 Copyright Act was passed in a much simpler age as far as the dessemination of ideas is concerned. As technology became more sophisticated and new media of communication were introduced-for example, photocopying and cable television-the 1909 act posed ever more complex problems for the courts. It has been apparent for many years that a thorough revision of the copyright statute was critically needed, but, for a variety of reasons, not the least of which was the intricate nature of the new technologies and the novel legal issues they posed, Congress was not able to enact a new statute until 1976. Even then, Congress did not feel it could yet deal satisfactorily with the question of software copyrights, even though Congress did believe that software should be subject to copyright protection. Consequently, Congress established the Commission on New Technological Uses of Copyrighted Works (CONTU), which was charged with making recommendations concerning appropriate legislation for the protection of software. In the meantime, Congress incorporated a section (1 17) in the 1976 act which essentially froze program copyrights at their legal status quo prior to the passage of the Act. The CONTU issued its final report in 1978, recommending that a new Section 117 be inserted in the 1976 Copyright Act to clear up some of the uncertainties concerning software copyrights. The CONTU recommendations were signed into law at the end of 1980 as a rider to a patent revision act (P. L. 96-517). This legislation reads in full as follows: Sec. 10 (a) Section 101 of title 17 of the United States Code is amended to add at the end thereof the following new language: “A ‘computer program‘ is a set of statements or instructions to be used directly or indirectly in a computer to bring about a certain result.” (b) Section I17 of title 17 of the United States Code is amended to read as follows:

“5 117. Limitations on exclusive rights: Computer programs “Notwithstanding the provisions of section 106, it is not an infringement for the owner of a copy of a computer program to make or authorize the making of another copy or adaptation of that computer program provided

“ ( I ) that such new copy or adaptation is created as an essential step in the utilization of the computer program in conjunction with a machine and that it is used in no other manner, or “(2) that such new copy or adaptation is for archival purposes only and that all archival copies are destroyed in the event that continued possession of the computer program should cease to be rightful. “Any exact copies prepared in accordance with the provisions of this section may be leased, sold, or otherwise transferred, along with the copy from which such copies were prepared, only as part of the lease, sale, or other transfer of all rights in the program. Adaptations so prepared may be transferred only with the authorization of the copyright owner.’’

8

MICHAEL C. GEMIGNANI

The intent of this amendment to the 1976 Copyright Act was to bring software squarely under the purview of the act, but, at the same time, remedy some of the difficulties that the actual use of such software caused in terms of traditional copyright law. This, then, is where program copyright law now stands. From a practical point of view, however, program copyrights may be of dubious value, which may explain the intense lack of interest in them by the software industry. At the outset, it is difficult to even detect that a possible infringement has occurred. In many instances, a file might be copied without a trace and without the knowledge of its rightful owner. Inasmuch as one does not need to sell a purloined program to another to gain great commercial benefit from its possession, an infringer can use an illicitly gained copy in his own operation without anyone outside being the wiser. Even if a possible infringement is detected, it would be hard to show copying. The most important part of the program, the algorithm, is not protectable, and the defendant may well argue that he developed the program on his own or had his own staff code the algorithm. Programs which do exactly the same thing can be so totally different in appearance that it might be quite d f i c u l t to convince a judge or jury of striking similarities or improper appropriation. There is in any situation involving possible infringement, or any other legal controversy for that matter, the practical question of whether what may be gained from winning in court will outweigh the cost and risk of losing. A victorious plaintiff in an infringement action may also recover attorney’s fees, but if the plaintiff loses, he may find himself liable for the defendant’s attorney’s fees, and these alone can be substantial, not to mention his own legal fees, which he would have to pay himself. Copyright confers rather weak protection, there are serious problems in taking advantage of what little protection it does afford, and few software developers seem very interested in it. So why be concerned with it at all? One reason is that copyright may, in certain instances at least, be an excellent form of protection. Traditional copyright of traditional writings such as instruction manuals, program documentation, warranties, and the life may work very well, as it did in the Synercom Technology case. A second reason is that it is still unsettled as to whether the recent softwarerelated amendment to the 1976 Copyright Act will preempt the use of state trade secrecy law. Preemption occurs when Congress legislates in an area in which the Constitution affords it jurisdiction and in such a way that it intends to make the federal statute the law to the exclusion of any state law which deals with the same matter, even if the state law does not actually contra-


9

vene the federal law. Thus if Congress passes one set of air quality standards and a state passes another more stringent set, then, if Congress intended to preempt air quality standards with its legislation, a defendant accused of violating the stricter state standards but not the federal standards will be able to successfully defend himself on the grounds that the federal law preempts the state law and therefore the state law cannot stand. In the copyright context, if Congress intended that copyright should be the sole means of legal protection accorded to software, then state trade secrecy law would be preempted and software suppliers would have to depend on copyrights whether they wanted to or not. The issue of preemption still seems to be an open one. One case which ruled for preemption (Auco Corp. v. Precision Air Parts, Znc., CA 79255-N, M.D. Ala. 1980) is not conclusive inasmuch as the case on which the trial judge relied was decided prior to the 1976 Copyright Act; but the Auco court did hold that, at least in Alabama, copyright does preempt trade secrecy. The facts and law of the Alabama case are somewhat unusual, and the passage of the 1976 Copyright Act may completely alter the legal climate anyway; even if the court of appeals sustains the trial court on the issue of preemption, the outcome and precedent will be binding only in Alabama. Moreover, the majority of cases decided to date-those of which this writer is aware, excepting Auco-hold against preemption. In a recent case, Warrington Associates, Inc. v. Real-Time Engineering Systems, Inc., Judge James Moran of the Federal District Court in Chicago reasoned that preemption does not apply because copyright and trade secrecy cover different aspects of a work. Copyright protection extends only to the particular expression of a work, but trade secrecy protection “extends to the very ideas of the author, subject, of course, to the requirement that the idea have some originality” (decided August 26, 1981). Judge Moran added that disclosure does not end a copyright owner’s rights, but it could well end the benefits of trade secrecy. Several other cases have also held against preemption and most commentators feel that Judge Moran’s views will prevail. A strong contrary argument was made by Peter Luccarelli, Jr., in his article “The Supremacy of Federal Copyright Law over State Trade Secrecy Law for Copyrightable Computer Programs Marked with a Copyright Notice,” in the Fall 1981 issue of Computer Law Journal. Nevertheless, preemption is an issue to watch. If the 1976 Copyright Act as amended to include software is held nationally to preempt trade secrecy law, the impact on the software industry will be enormous; but at this moment copyrights play but a marginal role in the legal protection of software.

10


3. Patents 3.1 A Primer of Patent Law In order to be patentable, an invention or discovery must have utility, novelty, and unobviousness. Almost any program would have utility inasmuch as all it would have to do to possess this attribute is to be able to perform at least one of the beneficial functions that its inventor claims for it. Novelty is a bit more difficult to establish; an invention is novel if all of the elements of the invention or their equivalents are not found in a single prior art structure or device where they do substantially the same work in substantially the same way. In simplistic terms, the basic issue as far as novelty is concerned is this: Has the invention or something essentially equivalent to it been in use prior to the patent application? Obviousness is perhaps the most problematic attribute. The test for obviousness was set forth in Graham v. John Deere Co., 383 U.S. 1 (1966): [Tlhe scope and the content of the prior art are to be determined; differences between the prior art and the claims at issue are to be ascertained; and the level of ordinary skill in the pertinent art resolved. Against this background, the obviousness or nonobviousness of the subject matter is determined.

The basic question being asked here is, could someone familiar with the subject matter of the invention have recreated the invention at the time it was invented if he had only bothered to turn his mind to it? Thus a programmer cannot necessarily patent a program (assuming programs are patentable) just because he is the first to write that particular program. If the program is such that any competent programmer could have written it at that time if he had wanted to, then the program would lack the unobvious character required for patentability. Unlike the modest protection conferred by copyright, patent protection is strong indeed: A patent is a grant of the right to exclude others from making, using or selling one's invention, and includes the right to license others to make, use or sell it. It is a legitimate monopoly . . [Valmont Industries, Inc. v. Yuma Manufacturing Co., 296 F. Supp. 1291, 1294 (D.Colo. 1969).]

.

Even if someone discovers a patented invention totally independently without any knowledge whatsoever of that invention, not only will he not get a patent, but use of that invention, even in all innocence, will constitute an infringement upon the patent.


11

Such strong protection comes only at the end of a tedious and lengthy process if the patent is ever issued at all. Copyright registration is a simple and inexpensive process requiring merely completing a form and sending it, a small fee, and a copy of the work to be registered to the Copyright Office. No search need be made by either the author or the Copyright Office to determine if the work is truly original. An inventor seeking a patent is “chargeable with full knowledge of all prior art, although he may be utterly ignorant of it” Alfred Bell v. CarafduFine Arts, Znc., 191 F.2d 99, 103 (2d Cir. 1951). For this reason, the Patent Office requires a cornplete search of all prior patents submitted to it to determine if the invention in question is truly novel. It normally takes 2 or 3 years and many thousands of dollars to obtain a patent. Moreover, the services of a skillful patent attorney are all but indispensable, because the smallest slip in preparing the elaborate set of claims that define the invention can be fatal to the application itself or to the protection the inventor expected to be accorded later. Just as “writings” are divided into different categories for purposes of copyright, so too must patents be sought under specific classifications. A patent sought under the wrong classification may be held invalid, even though it would have been valid had it been sought under another classification. Claims for program patents-remember that claims define the scope of the invention-are necessarily concerned with either machines or processes. Machine patents cover devices which must be used in connection with an actual mechanism. A process, on the other hand, is described by the United States Supreme Court as [a] mode of treatment of certain materials to produce a given result. It is an act, or series of acts, performed upon the subject matter to be transformed and reduced to a different state or thing. [Cochrune v . Deener, 94 U.S. 780, 788 (1877).]

Just as not every original writing of an author may be copyrighted, there are also inventions which cannot be patented even if they are useful, novel, and unobvious. The Supreme Court in Gottschalk v. Benson, 409 U.S. 63 (1972), stated, or rather restated, a principle which it has articulated a number of times before: Phenomena of nature, abstract intellectual concepts, mental processes, and algorithms are not patentable because they are the basic tools of scientific and technological work. As will be seen later, the court’s attitude toward algorithms may be changing, but, for the moment, no mathematical algorithm or equation as such is patentable. One is therefore faced again with the question of the relationship of a program to its underlying algorithm. A major practical issue concerning the value of program patents is that patent protection and secrecy are incompatible. Once a patent has actu-

12


ally been obtained, the program must be made available for public inspection at the Patent Office, and anyone who wants one may legally obtain a copy of it. Moreover, the way the patent law reads, it is even probable that the copy available for inspection would have to be well documented. The reason for this is that anyone making a patent search must be able to clearly determine if his program infringes upon a patented program, and he can only do this if he understands how the program operates. When does infringement occur? Generally, the allegedly infringing invention must have substantial identity of function, means, and results. Therefore, with regard to machines, an infringing device must perform substantially the same function by substantially the same means, and the principle or mode of operation must be the same as that of the machine infringed upon. An infringing process must operate in substantially the same way and under the same physical laws as the infringed process to produce the same result. It is not at all clear what these tests imply with respect to software. 3.2 History and the Law Prior to Diehr and Bradley

As early as 1964, the Patent Office had expressed a belief that programs were not patentable inasmuch as they were “creations in the area of thought.” In 1965, the President’s Commission on the Patent System was established to suggest revisions of patent law; this commission delivered its report at the end of 1966. As part of its recommendations, the commission stated the following: A series of instructions which control or condition the operation of a data processing machine, generally referred to as a “program,” shall not be considered patentable

regardless of whether the program is claimed as: (a) an article, (b) a process described in terms of the operations performed by a machine pursuant to the program, or (c) one or more machine configurations established by a program.

Although legislation was introduced in both houses of Congress to implement the commission’s recommendation, this legislation did not pass, so the recommendation never gained the force of law. Nevertheless, in 1968, the Patent Office issued a statement that “computer programming per se [sic]... shall not be patentable.” In 1969, the Court of Customs and Patent Appeals-that appellate court to which patent seekers can bring adverse rulings of the Board of Patent Appealsdisagreed with this new policy of the Patent Office in In re Prater, 415 F.2d 1393. The Court of Customs and Patent Appeals (C.C.P.A.) explicitly stated that it saw no reason, constitutional or otherwise, for denying program patentability under either machine or process claims. The Patent Office had also used the so-called Mental Steps Doctrine in rejecting


13

Prater’s application. The process which Prater described in his application could be carried out entirely with pencil and paper or, theoretically, even in one’s head, even though Prater had designed a machine which actually did the steps mechanically. The Patent Office held that when the novelty of an invention lies in a process which can be carried out as “mental steps,” then the invention cannot be patented. The C.C.P.A. rejected the Mental Steps Doctrine in In re Musgraue, 431 F.2d 882, 888-889 (C.C.P.A. 1970). The court declared that the doctrine might have validity only in those instances where the steps in question had to be performed mentally and could not be performed by a machine. In 1969, in In re Bernhart, 417 F.2d 1395, the C.C.P.A. also stated that a computer which was programmed in a new and unobvious way was physically different from a computer without that program. Even if the programming did not produce a completely new machine, it still produced a “new and useful improvement” and was therefore statutory subject matter for a patent. Bernhart’s invention was a method of illustrating a three-dimensional object using an already existing computer and plotter, and there was little disagreement that the real novelty of the invention lay in a set of mathematical equations. In 1971, the C.C.P.A. reaffirmed its position that a program which does nothing more than enhance the operation of a computer is appropriate subject matter for a patent. [In re Mctlroy, 442 F.2d 1397; In re Benson, 441 F.2d 682 (C.C.P.A. 1971), rev’d sub nom. Gottschalk v. Benson, 409 U.S. 63 (1972)l. It was in 1972 that the Supreme Court decided its first case involving a software patent. Gottschalk v. Benson involved an application for a patent for an algorithm to convert binary-coded decimal notation into the pure binary notation for the same number. The Supreme Court held that inasmuch as the process in question was an algorithm, it was unpatentable. The Court believed that programs were “specific applications” of algorithms, which, in turn, were procedures “for solving a given type of mathematical problem.” The Court also believed that Benson’s claims were so broad that they encompassed not just one particular use of the algorithm, but all possible uses, and hence sought to preempt the algorithm itself. No one could obtain a monopoly on a method of solving a class of mathematical problems, as well as all uses to which the method of solution might be put. The Court did note that the method for which a patent was sought could be camed out on existing computers, or even with pencil and paper or in one’s head, but it made no determination whether the Mental Steps Doctrine had any validity. The Court also explicitly declared that it was not saying that programs are not patentable, and it requested Congress to act to settle the matter.

14


The next Supreme Court decision involving program patents did not arise until 1976. In the meantime, the Court of Customs and Patent Appeals continued its policy of finding statutory subject matter in inventions which included programs, provided that the claims combined both machine and process. The C.C.P.A. was precluded by Benson from permitting a patent on any algorithm as such. In Dann v. Johnston, 425 U.S. 219 (19761, the Supreme Court dealt with a patent application for a “record-keeping machine for financial accounts.” The Patent Office had rejected Johnston’s application in part because it felt it would give him a monopoly on a method of banking. The C.C.P.A. reversed the determination of the Patent Office saying that a bank could keep its records in any way that it wanted to, provided that it did not use Johnston’s machine. The Supreme Court did not resolve any real issues with its opinion because it held that Johnston’s invention was unpatentable, as it lacked unobviousness. The Court restated its position that it had not said that programs were not patentable. If it was trying to allow more time for Congress to settle the question of program patentability, it must have felt frustrated, for Congress failed to act, then or now. Congress has consistently refused to take any stand on the issue of program patents. The next opinion of the Supreme Court regarding program patents was not to come until 1978. In the meantime, the C.C.P.A. continued to reverse the Patent Office, which, likewise, continued its policy of rejecting applications involving computer programs. However, because it could not contradict the Supreme Court, the C.C.P.A. rejected applications which seemed too close to being mere use of a computer to solve mathematical problems. It did, however, find the following to be patentable: a process using a computer to control and optimize the operation of a system of multiplant units, a system for typesetting using a computer-based control system in conjunction with a conventional typesetter, and a method of using a computer to translate from one language into another. The C.C.P.A. suggested a two-step analysis of computer claims. It was first to be determined if the claim recited an algorithm in the Benson sense. If no algorithm was found, then none was present to preempt. If there was an algorithm, then the claim had to be analyzed to determine if it was preempted. It did not seem to matter much to the Court whether there was activity which followed the solution of the mathematical equations used in the invention, provided that there was no attempt to gain a monopoly on the method of solution itself. If the solution was to be used for one special purpose, it still left the algorithm available for whatever other uses to which one might wish to put it. The C.C.P.A. also criticized the Supreme Court’s definition of algorithm, pointing out that most pro-


15

cesses are algorithms according to the dictionary definition [In re Freeman, 573 F.2d 1237, 1246 (1978)l. In In re Toma, 575 F.2d 872 (C.C.P.A. 19781, the Court of Customs and Patent Appeals stated that even if an algorithm is the only novel aspect of an invention, the invention may still be statutory subject matter. If this position had been sustained, then it would have left little doubt that computer programs were, in fact, patentable subject matter, provided only that the patent sought covered some specific application of the program and did not attempt to gain exclusive control over the underlying algorithm. However, in Parker v. Flook, 437 U.S. 584 (1978), the Supreme Court held against this position. Dale Flook’s invention involved a method of updating “alarm limits,” used to detect the presence of abnormal conditions in a catalytic conversion process. The method involved three steps: The first step was a collection of the values of the parameters needed to compute an alarm limit, the second was the actual computation of an updated alarm limit, and the third was a substitution of the updated alarm limit for the old value. The only novelty in the process rested in the algorithm used to compute the alarm limit. The Court stated that the mere fact that an algorithm was used in the process did not preclude patentability, but that the algorithm had to be considered as part of the prior art in determining whether the process was patentable. Once the algorithm was considered part of the prior art, there was nothing novel in the process, hence it was not patentable. Even though Flook did not specify any device which would gather the values of the parameters needed, or exactly what would be done if an updated alarm limit exceeded a predetermined value, neither did he attempt to preempt all uses of his algorithm. His use of the algorithm was narrowly focused on catalytic conversion, and the entire world could have used the algorithm freely for any other purpose. The Court also held that “an improved method of calculation, even when tied to a specific end use, is unpatentable.” Once again the Supreme Court repeated that computer programs might yet be patented, and once again it asked Congress to settle the question. Fluuk remained the latest word on program patentability until two recent cases. 3.3 Diehr and Sradiey and the Current State of the Law

James Diehr and Theodore Lutton developed a process for curing rubber into precision products. Their application for a patent on this process was filed in August 1975, some 54 years before the Supreme Court finally decided their case. The legal road to a patent can be long and expensive.

16


Although the precision curing of rubber depends on several variables, the most difficult to deal with was temperature, in that the temperature inside the molding press had to be estimated. Once the temperature inside the mold was known, however, the exact moment to open the mold for a perfect cure could be calculated using an equation which was already well known in the industry. Diehr and Lutton placed a thermocouple inside the mold which fed realtime temperature readings to a computer. The computer then used the temperature readings to determine if the mold should be opened. When the computer calculations produced a result that equalled the value which theoretically produced the optimal cure, a signal was sent to automatically open the mold and stop the curing process. The Patent Office rejected Diehr’s application on the sole ground that his claims represented nonstatutory subject matter. More specifically, the computations which the computer camed out were nonstatutory according to the way the Patent Office interpreted Gottschalk v. Benson (discussed earlier), and the rest of the process was so ordinary as to be unpatentable. The Court of Customs and Patent Appeals reversed the Board of Patent Appeals, stating, not unreasonably, that an invention should not become unpatentable just because it happens to incorporate fhe use of a computer. The C.C.P.A. noted that Diehr’s claims were not directed to a mathematical algorithm or even to an improved method of calculation inasmuch as the method of calculation was already very well known when the application was filed. The claims, rather, recited an improved process for molding rubber by doing something which had never been successfully done before, that is, measuring the temperature inside the rubber mold. The Supreme Court itself held, by a bare majority of one Justice, Because we do not view respondents’ claims as an attempt to patent a mathematical formula, but rather to be drawn to an industrial process for the molding of rubber products, we aflirm the judgment of the Court of Customs and Patent Appeals. (Diamond v. Diehr, No. 79-1112, slip opinion at 17.)

A short time after the Supreme Court decided Diehr, it affirmed by an equally divided court (4-4) an opinion of the Court of Customs and Patent Appeals which involved patentability of firmware, In re Bradley, 600 F.2d 807 (C.C.P.A. 1979). According to the C.C.P.A. opinion, the invention did not relate to any specific application of a computer, but only to the internal operation of the machine and “its ability to manage efficiently its operation in a multiprogrammed format.” The invention consisted of a firmware module (“hardware elements permanently programmed with microcode”) which directed data transfers between scratch-pad registers and the system base located in main memory.


17

The Patent Office had rejected Bradley’s application on the grounds that the invention resides in a “data structure” or an algorithm designed to control the multiprogramming computer to solve the particular problem indicated.

Bradley appealed the denial to the Board of Patent Appeals, arguing that no algorithm was involved in his invention, at least no mathematical algorithm. His claims, he asserted, were directed at data structures in hardware and met all the criteria needed for patentability. The board, however, upheld the rejection of the application, apparently on the basis that it felt that even though the claims were not expressed in mathematical language, they were still in the class of algorithmic and mathematical entities, which Benson had said could not be patented. In support of its position, the board made the following astonishing statement: Since digital computers normally operate in some numerical radix, binary coded decimal, or the like, we consider the operation of apellants’ claimed invention to be mathematical. Every operation performed in appellants’ invention as claimed involves the accommodation of data and instructions to the size of the registers in memory, and to the positional assignment to the registers in memory by the use of some numerical measure or quantity effected by way of electrical signals. In whatever form the instructions employed in appellants’ invention are characterized, numerical or otherwise, we think it accurate to say that the operation of appellants’ structure is mathematical and that the instructions constitute a procedure which is algorithmic in character, to the same degree as that of the Waldbaum structure [another invention whose application the board had rejected] and the satisfactory operation of the apparatus claimed represents the successful solution of a mathematical problem falling within the definition of algorithm supplied in Benson and reiterated in Flook.

It is very difficult to conceive of any computer-related invention of any kind that might be patentable if one accepts the logic of this statement. Use of data in numerically coded form, whatever the nature of the data, even if it were names or a picture of Mars, would preclude patentability per se. The Court of Customs and Patent Appeals, which has always been a bit more evenhanded in these matters than the Patent Office, rejected this reasoning entirely. As the C.C.P.A. said, “The board’s analysis confuses WHAT the computer does with HOW it is done.” The C.C.P.A. pointed out that Bradley had not attempted to claim the information which the module manipulated; instead, the court felt that the device was similar to a strictly mechanical adding machine that does not attempt to embrace within its patent any particular calculation which might be made upon it. Nor was Bradley attempting to claim the information actually embodied in the firmware. For all this, the C.C.P.A. might

18


have rejected the application had it found it embodied an algorithm. The C.C.P.A. held to its two-step test-first, is there a Benson-type algorithm, and, if so, do the claims preempt it-but found no mathematical algorithm whatsoever in the claims. The presence of calculations does not transform the invention into a method of calculation. The important question now is this, What is the state of the law concerning software patents? The question is not an easy one, and answers have been varied. When Diehr was first announced, some felt that the decision opened the way to software patents; others said the decision might provide the basis for software patents, but perhaps not; and still others were certain that Diehr made it more clear than ever that software as such was quite unpatentable. It would be more pleasing if the law were an exact science, but it is not, and it never will be. Until we have more case law to work from, we can only speculate on what the Supreme Court meant to say in Diehr and Bradley, and, even then, we will never be certain. When, and if, the Supreme Court rules on another case involving software patents, the composition of the Court will have changed, as may have some of the views of the continuing Justices; hence, in this situation, the future does not necessarily provide much of a guide to the past. We cannot, however, dismiss such an important issue without making some attempt at setting forth the alternatives. We therefore now turn our attention to looking at what Diehr and Bradley definitely say and what they might say concerning the patentability of software. One interpretation one might give to Diehr is that the Supreme Court is frustrated with dealing with program patent cases and wants to get out of the business. Congress was ignoring the Court’s pleas to settle the basic policy issues legislatively, despite the Court’s care in avoiding taking any position that would seem to be inconsistent with software patents. The Patent Office was showing a continuing hostility not only to software patents, but to any process that even involves a computer at any stage. The Court of Customs and Patent Appeals was continuing to be as liberal as it felt it could be concerning such patents, given such constraints as previous Supreme Court decisions had placed on it. The Supreme Court, given this state of confusion (which to some extent it had helped bring about), might simply be saying, “Enough! The Patent Office may no longer deny a patent solely because software is involved. The invention considered as a whole must be examined, and only then can the decision as to patentability be made.” If the Supreme Court is trying to get out of the business of deciding software patent cases-and they can do so because they have the discretionary power to decline to hear future cases in this area-that still leaves the question of how one is to consider software in viewing an invention as

LEGAL PROTECTION OF SOflWARE

19

a whole. Inasmuch as the Court made a special point in stating that it was not overruling Flook with Diehr, Diehr must be interpreted in the light of Flook. Flook stated that any algorithm contained in a process for which a patent is sought must be considered as part of the prior art. If software and algorithm are interchangeable in the mind of the Court, then the answer is clear: Software will have to be considered as part of the prior art; but it is not clear that program and algorithm are indeed synonymous, and the legal relation between the two is one of the most important outstanding questions in this area. It also remains true that no method of calculation as such can be patented; thus the process for which a patent is sought must almost certainly be directed at something other than the solution of a mathematical equation. The centrality of the notion of algorithm relative to software patentability cannot be overestimated. Often it is the algorithm itself which is of prime commercial importance and which the inventor would patent if he could. Some believe that Diehr implied some prospect that the Supreme Court will review its operational definition of algorithm and arrive at a less restrictive one that might open the way toward some form of patentability of algorithms themselves. This somewhat optimistic view is based on the text of Footnote 9 in Diehr, which reads as follows: The term “algorithm” is subject to a variety of definitions. The Government defines the term to mean: “1. A fixed step-by-step procedure for accomplishing a given result: usually a simplified procedure for solving a complex problem, also, a full statement of a finite number of steps. 2. A defined process or set of rules that leads [sic] and assures development of a desired output from a given input. A sequence of formulas and/or algebraic/logical steps to calculate or determine a given task; processing rules.”

This definition is significantly broader than the definition this Court employed in Benson and Nook. Our previous decisions regarding the patentability of “algorithms”

are necessarily limited to the more narrow definition employed by the Court and we d o not pass judgment on whether processes falling outside the definition previously used by this Court, but within the definition offered by the Government would be patentable subject matter.

If one had a “fixed step-by-step procedure” which solves a nonmathematical problem, then that might be patentable. Thus original merge-sort

routines or programs for manipulating large data bases might be patentable, even though one might have to suppose that a matrix inversion algorithm or an improved method for solving a partial differential equation would not be, because that involves mathematics. Needless to say, the distinction between the mathematical and the nonmathematical as far as the patentability of algorithms seems artificial and, in terms of law, unjust, assuming that one can meaningfully make this

20


distinction to begin with. This author, for one, would hate to see a case go to court on the sole issue of whether or not a particular algorithm was, or was not, mathematical. Any court attempting to make such a distinction would be dabbling in the philosophy of science, or perhaps just plain witchcraft, rather than the law. In fairness to the courts, they are, of course, faced with new technologies with regard to which old legal concepts begin to come apart, yet they cannot shirk their duty to decide the cases which come before them simply because they are hard or complex, or even because they involve making new law. The recently decided landmark case of Diamond v. Chakrabarty forced the Supreme Court to decide whether new man-made life forms were patentable subject matter. Just a few years earlier, if someone had asked whether Dr. Frankenstein could hold a patent on his famous monster, the matter would have been treated as a joke. Now no one dares laugh. However, in Chakrabarty, the Supreme Court seemed to be saying, “We intend to interpret the patent laws as broadly as possible. Where there are questions of high policy, such as whether science should be permitted to produce new forms of life, we will leave the answers to those whom the Constitution charges with framing policy, such as Congress. We will merely attempt to say what the legislation means.” Nevertheless, it is evident that not all forms of life are patentable. If the Court is willing to distinguish between patentable life forms and nonpatentable life, then is it too much to ask that it make, or allow others to make, the distinction between patentable algorithm and nonpatentable algorithm, or between nonpatentable law of nature and a patentable creation of the human mind which may look to the uninitiated like a law of nature? Despite the fact that the Supreme Court claimed that it was not overruling Flook, there is no inconsiderable problem in reconciling Flook with Diehr. There is little seeming difference between Diehr’s process for molding rubber and Flook’s process for updating an alarm limit. There is, however, one quite significant difference, namely, Diehr’s process is clearly geared to a “transformation of matter,” changing rubber from an uncured state to a cured state, whereas Flook’s process, on the other hand, takes one set of numbers and gives but another number. Matter is not transformed, only numbers. We might therefore interpret Diehr as simply stating that unless a process actually changes matter, as opposed to data or numbers, it cannot be patented; but if the process does transform matter, then it may be patentable even though it employs a computer program as part of the process. Even so, one would still have to determine how to view the program in determining the patentability of the entire process. Again, according to Flook, the algorithm must be considered to be part of the prior art, and the


21

process as a whole must be viewed in this light. In Diehr, the method of calculating the time to open the mold was already well known when Diehr invented his process. If it had not been well known, the Supreme Court tells us that for patent purposes, it would have to be assumed to be so. In Flook, however, it is the method of calculation itself which is the improvement, that which makes the invention new and worthwhile. Thus it seems we must conclude that even when a transformation of matter takes place, if the transformation takes place solely because of an improved method of calculation, it would seem that the process involved cannot be patented. Still, the matter cannot be left to rest at that point. For example, where does a method of calculation begin or end if it is inextricably wed t o a process? If, say, two metals are to be mixed to produce an alloy of precise tensile strength, should a computer program which determines the exact amount of each metal to use be patentable subject matter? Would it or should it make any difference if the program-controlled hoppers released quantities of the metals according to the computer’s instructions? Is such a program an “improved method of calculation” in the sense that the Court meant in Flook? Why, indeed, should an improved method of calculation be unpatentable if it satisfies all the other criteria for patentability? Oddly enough, one may also interpret Diehr as having nothing to do with software patentability. The Supreme Court itself said that the patent at issue was really nothing other than for “an industrial process for the molding of rubber products.” It is only the dissent which makes much of the fact that software is involved, but the dissent is not what lawyers quote when they wish to win a case. It is true that the process in question uses a computer to solve an already known mathematical equation in order to determine the exact moment at which to open the mold, but what technological process does not use mathematical equations or laws of nature in their design, even though such laws and equations cannot themselves be patented? The equation which the computer solves was already used extensively in rubber curing, a process which the Court decided some years ago was patentable subject matter. It would therefore be wholly inconsistent for the Court to now decide that a new and improved method of that same patentable process was unpatentable merely because it made use of new and improved technology. What then is the state of the law concerning patents for software? The dissenting Supreme Court Justices in Diehr may have phrased it best: [The] cases considering the patentability of program-related inventions do not establish rules that enable a conscientious patent lawyer to determine with a fair degree of accuracy which, if any, program-related inventions will be patentable.

22


The matter is even more clouded because of the Supreme Court’s decision in Diamond v. Bradley. An equally divided Supreme Court affirmed without opinion the decision of the Court of Customs and Patent Appeals (C.C.P.A.) rendered under the title In re Bradley, 600 F.2d 807 (C.C.P.A. 1979). Chief Justice Warren Burger recused himself in Bradley because of a potential conflict of interest. It is, of course, true that an affirmance by a tie vote does not carry the same value as a precedent as would an opinion which represents the clear majority of the Justices, but Bradley remains the Court’s latest word on “firmware” patentability. Moreover, if the Justices are really “fed up” with the question of software patentability, and hence will be reluctant to consider future cases in this area, Bradley may represent a precedent of much greater strength than the tie vote might otherwise entitle it to. What, then, does Bradley tell us? First, the C.C.P.A. at least still believes that a “program or programrelated inventions” can be statutory subject matter for patents; the C.C.P.A. believes it is possible to write a patentable program. In view of the rulings of the Supreme Court, the C.C.P.A.’s view would seem tenable only if a program can be separated from the notion of algorithm, i.e., program and algorithm are not synonymous, and if a program can be interpreted as something other than an “improved method of calculation.” One might analogize the difference between a program and its underlying methodology to the difference between an idea and its expression, a distinction that is of importance in copyright law. If calculation includes just calculation with numbers or information expressed numerically, then a program is decidedly a method of calculation. The information stored in a computer and processed by the program must be in numerical form, even if the data involved is not actually numeric in nature. This is the position the Patent Office took in turning down Bradley’s application to begin with. But digitizing information does not truly make the information numerical, of course. Very many, if not most, modern technological processes would be disqualified for patents if such disqualification could come about merely because they involve the manipulation of information in numerical form. Bradley also stated that the C.C.P.A. will apply a two-part test to determine if some computer-related invention is patentable subject matter: Even though the claimed invention is a machine, we must nevertheless determine whether the claim recites a mathematical algorithm, and, if so, whether it preempts the use of the algorithm.

Thus the first question is whether the invention depends on an algorithm. If it does, then it must not preempt that algorithm. The algorithm itself


23

cannot be monopolized, but some specific use of the algorithm can be. This sounds fine in theory, but it is not clear how fully it squares with the position of the Supreme Court. Flook’s method for updating an alarm limit in a catalytic conversion process involved an algorithm, but no attempt was made to preempt that algorithm; it was clearly tied to one specific end use and to no other. It may be, however, that Flook was not specific enough. He did not specify how the initial parameters used in his algorithm were to be obtained or precisely what was to be done once the computation had produced a new alarm limit. Diehr’s process described how to obtain the initial parameters and what to do with the computation. The distinction between Flook’s and Diehr’s inventions is subtle, but apparently critical. Bradley also tells us that the C.C.P.A. will not permit a patent on a true algorithm, i.e., a mathematical method of calculation as such. This is a direct corollary of the holding that no patent may preempt every use of an algorithm, which would certainly be the case if the algorithm per se could be patented. Several interesting questions are presented by Bradley. For example, does the C.C.P.A. adhere to the Transformation of Matter Doctrine? It would appear that it does not, because the device in Bradley or the process it embodies does not involve the transformation of matter. The doctrine-which states that, in order t o be patentable, a process must transform matter-may be obviated in Bradley, because the firmware itself was treated as a machine rather than a process; but, in another sense, this distinction is artificial because firmware is the hardware embodiment of software. Hardware and software are basically interchangable. Should the fact that a program is realized as firmware substantively affect its patentability? It would seem that it should not for the following reason. If firmware containing a program is patentable, but that program itself is not, then a case could be made that whenever anyone actually entered the program into a computer, by creating a machine embodiment of it by such action, there must be an infringement; thus the distinction between software and firmware patentability would be illustory. Nevertheless, the C.C.P.A. does seem to rely on the fact that machine claims are made: Appellants have characterized their combination of hardware elements as a mechanism which enables the computer to alter information in its system in a manner not previously possible. They are in no way claiming the altered information; in fact, the particular information acted upon by the appellants’ invention is irrelevant to the operation of the invention itself. We see no difference in this regard, with respect to being within 5 101, between appellants’ claimed invention and a strictly mechanical adding machine, which is certainly statutory if claimed in a manner which does not embrace any particular calculation that the machine is capable of making.

24


To consider firmware of the same nature as an adding machine is not as ingenuous as maintaining, as did the Patent Office, that no data processing invention could be patentable subject matter because data processing manipulates numbers and is, therefore, a computational algorithm precluded from patentability by Benson, but it is a suspicious assertion nonetheless: Finally, there is the critical question as to whether the information which the firmware contains is patentable: If appellants were claiming the information embodied in the firmware or the firmware itself, per se, a different case would be presented. We express no opinion on the statutory nature of such an invention, a question not before us. Appellants are claiming a combination of hardware elements, one of which happens to be a portion of the computer’s control storage mjcroprogrammedin a particular manner. It is this subject matter with which we must deal.

The C.C.P.A. relies heavily on the form of Bradley’s patent claims; thus. “data structure” is treated as a configuration of hardware. This is consistent with the Court’s previous opinions, but it, in a real sense, begs the issue whether the firmware can be separated from the information it contains, or whether it is equivalent to it. In what ways, that make sense as far as patent law is concerned, do a program, a circuit diagram obtained directly from the program, and firmware which is designed from the circuit diagram differ from one another? The question is now a central one in patentability of software, and the answer is far from clear.

4.

Trade Secrecy

Observation of standard practices in the software area and at least one survey indicate that trade secrecy is the legal means most preferred by software suppliers for the protection of their product. This is not to say that trade secrecy is an ideal means of protection. Indeed, technological means of protection such as supplying only object code or encrypting sensitive programs seem to be relied on far more heavily than any form of protection which the law might provide. Many software suppliers will place a copyright notice on their product, but this is almost always in conjunction with a notice that the software is also a proprietary secret as well as a contract which attempts to prevent unauthorized disclosure of the software. In addition to this, the supplier will also try to prevent source code from being available to the purchaser, or, if the source code is required as part of the sale or license, the supplier will place “fingerprints” in the code to identify it as his as well as to enable tracing if an unauthorized copy appears elsewhere; or he may write in a “logic bomb”


25

to cause the program to self-destruct once the license for its use expires or some attempt is made to modify it. A Primer of Trade Secrecy Law

Unlike patents and copyrights, which are creatures of Federal law, trade secrecy law varies from state to state. Although suggestions have been made for a Federal trade secret statute, none has every been passed. Because the law of trade secrecy is dependent on the jurisdiction, problems can arise for firms which have dealings in a great many states. Also, because trade secrecy is a creature of state law, it can be preempted by Federal law. Should there be a Federal trade secret statute, it is more than likely that it will replace all state legislation in this regard, even if the state statutes are stricter than it is. What is a trade secret? The Restatement (first) of Torts provides the most widely accepted definition: A trade secret is “any formula, pattern, device or compilation of information which is used in one’s business, and which gives one a competitive advantage over those who do not know it.” The same Restatement also gives the grounds upon which someone may be liable to another for damages upon use of the other’s trade secret: One who discloses or uses another’s trade secret, without privilege to do so, is liable to the other if (a) he discovered the secret by improper means, or (b) his disclosure or use constitutes a breach of confidence reposed in him by the other in disclosing the trade secret to him, or (c) he learned the secret from a third person with notice of the fact that it was secret and that the third person discovered it by improper means or that the third person’s disclosure of it was otherwise a breach of his duty to the other, or (d) he learned the secret with notice of the fact that it was a secret and that its disclosure was made to him by mistake.

These definitions and principles are widely accepted, although not necessarily interpreted in exactly the same way in every state. There is, of course, no question that a computer program can qualify as a trade secret, but merely calling a program a trade secret does not make it one in the eye’s of the law. A trade secret must, first of all, be a secret. Something which is general knowledge in the industry is never a trade secret; secrecy implies that knowledge of the fact is limited in its scope. Moreover, the owner of the trade secret must treat it like a secret. Publication of an algorithm in a professional journal is inconsistent with a declaration that it is a proprietary secret of the author. The owner must take reasonable steps to preserve the secrecy. Few courts will permit recovery of a trade secret lost through its owner’s carelessness. Courts generally require as well that the secret be used in its owner’s legitimate business activities. A computer program for the office betting

26


pool, although secret, would not qualify as a trade secret. There must usually also be some element of novelty in the secret, although not to the degree required for patentability, and the owner is customarily expected to have some investment or economic stake in the secret as well. If a business has spent hundreds of thousands of dollars developing a quality general ledger program, the theft of that program will be regarded more seriously by a court than the copying or unauthorized use of a routine sort program that any competent programmer could have written in a couple of hours. 5.

Some Additional Open Questlons

Having now had an overview of the three basic means of legally protecting software, we are ready to consider some of the still unanswered questions in this area. 5.1 Does the New Section 117 of the 1976 Copyright Act Preempt Trade Secrecy?

This question was touched on earlier, and the answer, in Alabama at least, seemed to be yes. There is a good chance that the answer elsewhere, however, will turn out to be no. If software suppliers had to depend solely upon copyright protection to the exclusion of trade secrecy, this would have profound implications. Suppliers could still attempt to keep their wares secret through technological means and could still write clauses into license and sales agreements contracting against unauthorized disclosure. What might be more difficult to deal with are situations where employees run off to start or join competitive firms and take the trade secrets of their former employer with them. Although actual copying of a copyrighted program in such circumstances would be an infringement actionable in Federal court, the underlying logic or algorithm would be an idea to which copyright protection would not attach, hence it would, in the opinion of this writer, be more difficult for the injured former employer to recover damages. There was at one time a fear that patent protection precluded trade secrecy, and, in one sense, patents and trade secrets are incompatible. Once a patent is granted, there must be full and open disclosure of the invention; one cannot have both a patent and maintain secrecy as well. But it is now well settled that one can maintain trade secrecy under state law if one is willing to forgo patentability. Once one loses a trade secret


27

through loss of the secrecy, there is no going back to obtain a patent. The situation is far more complex, however, with regard to copyrights. First, a copyright attaches to an “original writing of an author” as soon as it is created; this is true whether the author wishes the protection or not. The copyright protection may be lost, however, if the work is disseminated without notice of the copyright. The best form for a copyright notice is 0

year of publication

name of owner

Various other legends can be used in place of the familiar “0,” but the latter is internationally recognized whereas the others are not. A problem arises with the year of publication required in the copyright notice. If the work is kept secret or its distribution is restricted to a special group, it is not published, i.e., made public, in the technical sense. If it is published, then on may question whether it is secret; that is, if one encounters a work with a copyright notice and a year of publication, a case can be made that the person can assume the work is not a trade secret. There is thus some inherent contradiction created by the 1976 Copyright Act between placing a proper copyright notice upon the work and at the same time declaring that the work is also a trade secret. The software developer who wishes to take advantage of copyright protection as well as the law of trade secrecy is faced with a dilemma: To omit the copyright notice risks losing the copyright protection then and later, but including it risks the loss of trade secrecy status. The question has not yet been settled in the courts, but it is probable that courts will find trade secrecy and copyright compatible, particularly if the owner of the software marks it clearly as both a copyrighted work and a trade secret. The owner has no choice as to the copyright, and it is doubtful that a court will penalize the owner for something over which he has no control, unless it is found that copyright totally preempts trade secrecy, in which case the owner will be forced to rely solely on copyright. Even if copyright should be found to preempt trade secrecy law, this will not mean the end of trade secrecy. First, there are commercially valuable properties which cannot be the subject matter of either copyright or patent protection, for example, ideas and mathematical equations. There is other intellectual property, and even physical inventions, which may be of questionable patentability or copyrightability, yet still of substantial importance to a trade or business. If one is willing to totally forgo patent or copyright protection and rely solely on trade secrecy, then there is simply no question of preemption. The preemption question with regard to copyright arises when someone attempts to place a “writing” under

28

MICHAEL C.GEMIGNANI

both trade secrecy and copyright protection by affixing the copyright notice while still maintaining that the work in question is secret. Nothing precludes the sole use of trade secrecy in the first place if one is willing to accept the risks involved; nevertheless, as has already been pointed out, such trend as there is thus far in court rulings is leading away from preemption. 5.2 To What Degree Is Software Patentable?

The situation regarding software patentability is nothing short of chaotic. From a purely practical view, however, few software developers have sought patents. As we have seen, it is an expensive, time-consuming, and risky venture at best, requiring full disclosure of the software if the patent application is successful; the probability that a patent, once granted, will not survive a challenge in the courts is high. The question, therefore, may be more of academic interest than one of burning concern in the real world. Although the situation is so murky as to defy any definition, there do seem to be certain “rules of thumb” which are probably valid: (1) A mathematical algorithm as such is probably not patentable, at least according to the view of algorithm used by the Supreme Court in Benson. There is some evidence that the Court might at some future time be willing to consider a broader view of algorithm which might permit some algorithms to be patentable. (2) An invention which involves an algorithm, however, is not unpatentable per se; rather, the invention must be looked at as a whole. However, it would also seem that the algorithm must be considered as part of the prior art in this consideration; thus if the algorithm is the only novel feature of the invention, the invention is probably not patentable. The Supreme Court has also declared that “an improved method of calculation, even when tied to a specific end use, is unpatentable subject matter.” (3) If software, or firmware, is found not to contain an algorithm, or at least a mathematical alogrithm, then it may be found to be patentable. The Court of Customs and Patent Appeals, for example, held that Bradley’s firmware for improving the operation of a central processing unit did not contain an algorithm. It also held that there was no algorithm present in software which translated one language into another. (4) The Court of Customs and Patent Appeals still seems willing to apply a pre-Hook test in which the court will first examine the claims for the invention to see if a Benson-type algorithm is present and, if one is found, whether there is an attempt to preempt all uses of that algorithm.


29

The narrower the use to which the algorithm is put, the greater the chance that the patent application will survive judicial scrutiny. The definition of algorithm which the C.C.P.A. is using seems to be broader than that which the Supreme Court used in Benson and Flook. There may thus still be hope that some algorithms at least are patentable if the claims are properly drawn.

It is not clear what effect, if any, the patentability of certain types of algorithms would have. Such patentability might spur greater interest in research into more efficient and powerful software, but such research is proceeding apace at the moment anyway; more probably, it would be greeted with much the same indifference that software patentability has received in the past. 5.3 Is a Program in Object Code Only Copyrightable?

Under the 1909 Copyright Act, in order for a work to be copyrightable it had to be humanly comprehensible. Under the 1976 Copyright Act all that is required is that the work can be “perceived, reproduced, or otherwise communicated, either directly or with the aid of a machine or device.” The Copyright Office has been reluctant to register object code because it cannot be understood as readily as source code, although object code has been accepted for registration. Needless to say, software developers feel a bit safer registering only the object code because a potential infringer is then far less likely to gain useful information in examining the material on file at the Copyright Office. Despite the discomfort of the Copyright Office, there seems to be no legal reason why object code cannot serve as the vehicle by which a program is registered. In the event of alleged infringement, there may be a heavier burden on the plaintiff to show that the defendant’s source code was copied from the plaintiff‘s object code, or even that one object module is an infringing copy of another. Object code is comprehensible to a technician familiar with the instruction set of the computer on which the object code runs. It can, of course, also be “reverse compiled” to produce at least one source program. The legal issue of whether the object code suffices for registration is not as difficult as proving infringement if only the object code has been registered. 5.4 Can ROMs Be Patented or Copyrighted?

The J S & A case discussed earlier was upheld on appeal on the narrow grounds that the ROM in question had no copyright notice; inasmuch as

30


the case was decided under the 1909 Copyright Act, the lack of notice was fatal to the plaintiff’s claims. But can a ROM which has a copyright notice be the proper subject matter for a copyright? Bradley tells us that at least under certain conditions ROMs might be patentable. One condition of patentability is utility, and utility generally precludes copyrightability. Nevertheless, it also seems that ROMs which embody a mathematical algorithm may not be patentable because a mathematical algorithm is not patentable subject matter. Then is an ROM a “copy” of an object or source program if such a program was used in its design? Perhaps, but one could as well argue that the program is but the blueprint or “circuit diagram” and not equivalent to the object designed from it any more, say, than a house is equivalent to the copyrighted blueprints the builder used in its construction. The question is not of purely academic interest, particularly as more and more software is embodied in firmware. For an extensive discussion of this question from the viewpoint of two attorneys flatly opposed to the copyrightability of ROMs and object code, the reader is referred to “Can We Stop: A Memorandum Submitted to the US International Trade Commission, In the Matter of Certain Coin-Operated Audiovisual Games and Components Thereof” by Richard Stern and Jeffrey Squires, IEEE Micro, Vol. 2, No. 1 (February 1982), p. 12. A very recent case, Tandy Corporation v. Personal Micro Computers, Znc.,decided August 31, 1981 by the Federal district court for the Northem District of California held that ROMs can be copies of any copyrighted program which is imprinted in their circuitry. The court explicitly rejected the old criteria that the contents of the chip would have to be humanly comprehensible to be copyrightable; it was adequate that the computer could read the program embodied by the chip and carry it out. This case also provides strong support for the position that object code alone is a copy of the source program from which it is derived, and the object code itself is copyrightable subject matter.

6. Conclusion

This article has focused on three important means of protecting intellectual property as it applies to software: copyright, patent, and trade secrecy. Deliberately omitted were technological means of protection and several less important legal means such as unfair competition and antitrust, or restraint of trade, a weapon which can be wielded both offensively and defensively with regard to trade secrets.


31

This area of the law is one which is still evolving rapidly, and it is not unlikely that portions of this chapter will be obsolete by the time it appears in print. Nevertheless, there remains a serious question concerning the true importance of legal protection for software for the software industry. Although many developers are affixing copyright symbols as a matter of form, it is not clear what percentage of these copyrights are being registered. Until rather recently, in fact, only some 1200 programs had been registered, most of these by IBM and Burroughs. The Patent Office receives about 400 applications a year for inventions involving software, but it is estimated that some 1 million or more programs are written each year in the United States. There has been no stampede to either the Copyright Office or the Patent Office to seek protection for software. Trade secrecy has been the means of protection of choice if any legal means of protection has been predominant, particularly coupled with nondisclosure clauses in licenses. Nevertheless, statistics reveal that technological means of protection, such as supplying only object code, are relied on far more frequently than any legal means or even all legal means put together. One may also reasonably surmise that most people who license or buy software, as opposed to writing it themselves, do so because either it is more economical to buy it outside or they have no in-house expertise available to write it for them. Since expertise can be hired, even the latter reason is really an economic one. As long as software licenses are priced low enough to make it unreasonable to steal the software, or the maintenance, updates, and other services that are supplied by a legitimate vendor are so important that stealing would be counterproductive because the services would not be available, stealing is not likely to be a major problem. Realistically, some piracy is likely to take place with any packaged software, just as shoplifting is a “normal” part of the day-to-day operations of supermarkets and recording songs from a radio is often done in private homes. The price of any commodity is based on the demand. One can often make a fair profit in spite of those who illicitly misappropriate property. Another critical economic consideration with regard to legal means of protection is the cost of litigation to enforce one’s rights. Attorneys’ fees may be higher than the total provable damages. If damages are awarded, they must still be collected; it is a hollow victory to gain a $1 million judgment against a bankrupt corporation. Statutory damages and recovery of attorneys’ fees are possibilities in actions for infringement of a patent or copyright, but attorneys’ fees can also be awarded to the de-

32


fendant if he prevails, so this is a two-edged sword, and there remains the real threat that the court may find a patent or even a copyright to have been invalidly granted. The software industry has now thrived for many years in the absence of a clear law of protection, and it seems unlikely that the legal confusion will be cleared up in the near future. There is little evidence that the software industry has suffered significant harm due to the legal confusion in the past, nor are there signs that this industry will cease growing owing to continuing legal doubts. 7. Appendix

What follows are edited versions of two very recent and important court opinions regarding program patents: The first, Diamond v. Diehr, is an opinion of the Supreme Court; the second, In re Bradley, is actually an opinion from the Court of Customs and Patent Appeals which was affirmed without an opinion by an equally divided supreme Court. 7.1 Diamond v. Diehr, 49 U.S.L.W. 4194 (1981)

Mr. Justice Rehnquist delivered the opinion of the Court.

I The patent application at issue was filed by the respondents on August 6, 1975. The claimed invention was a process for molding raw, uncured synthetic rubber into cured precision products. The respondents claim that their process ensures the production of molded articles which are properly cured. Achieving the perfect cure depends upon several factors, including the thickness of the article to be molded, the temperature of the molding process, and the amount of time that the article is allowed to remain in the press. It is possible, using wellknown time, temperature, and cure relationships, to calculate by means of the Arrhenius equation when to open the press and remove the cured product ... . Because the temperature inside the press has heretofore been viewed as an uncontrolled variable, the conventional industry practice has been to calculate the cure time as the shortest time in which all parts of the product will definitely be cured ... . The respondents characterize their contribution to the art to reside in the process of constantly measuring the actual temperature inside the mold. These temperature measurements are then automatically fed into a computer which recalculates the cure time by use of the Arrhenius equation. When the recalculated time equals the actual time that has elapsed


33

since the press was closed, the computer signals a device to open the press. According to the respondents, the continuous measuring of the temperature inside the mold cavity, the feeding of this information to a digital computer which constantly recalculates the cure time, and the signaling by the computer to open the press are all new in the art. The patent examiner rejected the respondents’ claims on the sole ground that they were drawn to nonstatutory subject matter ... . H e determined that those steps in the respondents’ claims that are carried out by a computer under the control of a stored program constituted nonstatutory subject matter under this court’s decision in Gottschalk v. Benson, 409 U.S. 63 (1972). The remaining steps-installing the rubber in the press and the subsequent closing of the press-were “conventional in nature and cannot be the basis of patentability.” The examiner concluded that the respondents’ claims defined and sought protection of a computer program for operating a rubber molding press. The Patent and Trademark Office Board of Appeals agreed with the examiner, but the Court of Customs and Patent Appeals reversed. The court noted that a claim drawn to subject matter that is otherwise statutory does not become nonstatutory because a computer is involved. The respondents’ claims were not directed to a mathematical algorithm or an improved method of calculation but, rather, recited an improved process for molding rubber articles by solving a practical problem which had arisen in the molding of rubber products.

I1 Last term in Diamond v. Chakrabarty, 447 U.S. 303 (1980), this Court discussed the historical purposes of the patent laws ... . A s in Chakraburry, we must here construe 35 U.S.C. 0101, which provides the following: Whoever invents or discovers any new and useful process, machine, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this Title.

In cases of statutory construction, we begin with the language of the statute. Unless otherwise defined, “words will be interpreted as taking their ordinary, contemporary, common meaning,” and, in dealing with the patent laws, we have more than once cautioned that “courts ‘should not read into the patent laws limitations and conditions which a legislature has not expressed.’ The Patent Act of 1793 defined statutory subject matter as “any new and useful art, machine, manufacture or composition of matter, or any new or useful improvement [thereof].” Not until the patent laws were recodified in 1952 did Congress replace the word art with the word pro”

34


cess. It is that latter word which we confront today, and in order to determine its meaning we may not be unmindful of the committee reports accompanying the 1952 act which inform us that Congress intended statutory subject matter to “include anything under the sun that is made by man.” Although the term process was not added to 35 U.S.C. §lo1 until 1952, a process has historically enjoyed patent protection because it was considered a form of “art” as that term was used in the 1793 act. In defining the nature of a patentable process, the Court stated the following: That a process may be patentable, irrespective of the particular form of the instrumentalities used, cannot be disputed... . A process is a mode of treatment of certain materials to produce a given result. It is an act, or series of acts, performed upon the subject matter to be transformed and reduced to a different state or thing.

Analysis of the eligibility of a claim of patent protection for a “process” did not change with the addition of that term to 0101. Recently, in Gottschalk v. Benson ..., we repeated the above definition recited in Cochrane v. Deener, adding, “Transformation and reduction of an article ‘to a different state or thing’ is the clue to the patentability of a process claim that does not include particular machines.” Analyzing the respondents’ claims according to the above statement, we think that a physical and chemical process for molding precision synthetic rubber products falls within the 0101 categories of possibly patentable subject matter.

I11 Our conclusion regarding the respondents’ claims is not altered by the fact that in several steps of the process a mathematical equation and a programmed digital computer are used. This court has undoubtedly recognized limits to 0101 and every discovery is not embraced within its statutory terms. Excluded from such patent protection are laws of nature, physical phenomena, and abstract ideas: “An idea of itself is not patentable.” “A principle, in the abstract, is a fundamental truth; an original cause; a motive; these cannot be patented, as no one can claim in either of them an exclusive right.” Only last term, we explained the following: A new mineral discovered in the earth or a new plant found in the wild is not patentable subject matter. Likewise, Einstein could not patent his celebrated law that E = mc* ;

nor could Newton have patented the law of gravity. Such discoveries are ‘manifestations of nature, free to all men and reserved exclusively to none.’

Our recent holdings in Gottschalk v. Benson and Parker v. Flook, both of which are computer related, stand for no more than these long established principles. In Benson, we held unpatentable claims for an algorithm


35

used to convert binary coded decimal numbers to equivalent pure binary numbers. The sole application of the algorithm was in connection with the programming of a general-purpose digital computer. We defined algorithm as a “procedure for solving a given type of mathematical problem,” and we concluded that such an algorithm, or mathematical formula, is like a law of nature, which cannot be the subject of a patent. Parker v. Flook presented a similar situation. The claims were drawn to a method for computing an “alarm limit.” An “alarm limit” is simply a number and the court concluded that the application sought to protect a formula for computing this number. Using this formula, the updated alarm limit could be calculated if several other variables were known. The application, however, did not purport to explain how these other variables were to be determined, nor did it purport “to contain any disclosure relating to the chemical processes at work, the monitoring of the process variables, or the means of setting off an alarm system. All that is provided is a formula for computing an updated alarm limit” 437 U S . at 586. In contract, the respondents here do not seek to patent a mathematical formula; instead, they seek patent protection for a process of curing synthetic rubber. Their process admittedly employs a well-known mathematical equation, but they do not seek to preempt use of that equation; rather, they seek only to foreclose from others the use of that equation in conjuction with all the other steps in their claimed process. ... Obviously, one does not need a “computer” to cure natural or synthetic rubber, but if the computer use incorporated in the patent process significantly lessens the possibility of “overcuring” or “undercuring,” the process as a whole does not thereby become unpatentable subject matter. Our earlier opinions lend support to our present conclusion that a claim drawn to subject matter otherwise statutory does not become nonstatutory simply because it uses a mathematical formula, computer program, or digital computer. In Gotrschafkv. Benson, we noted that “it is said that the decision precludes a patent for any program servicing a computer. We do not so hold.” Similarly, in Parker v. Flook, we stated that “a process is not unpatentable simply because it contains a law of nature or a mathematical algorithm.” It is now commonplace that an application of a law of nature or mathematical formula to a known structure or process may well be deserving of patent protection. IV We have before us today only the question of whether the respondents’ claims fall within the §lo1 categories of possibly patentable subject matter. We view the respondents’ claims as nothing more than a process of molding rubber products and not as an attempt to patent a mathematical

36

MICHAEL C.GEMlGNANl

formula. We recognize, of course, that when a claim recites a mathematical formula (or scientific principle or phenomenon of nature), an inquiry must be made into whether the claim is seeking patent protection for that formula in the abstract. A mathematical formula as such is not accorded the protection of our patent laws, and this principle cannot be circumvented by attempting to limit the use of the formula to a particular technological environment. Similarly, insignificant postsolution activity will not transform an unpatentable principle into a patentable process. To hold otherwise would allow a competent draftsman to evade the recognized limitations on the type of subject matter eligible for patent protection. On the other hand, when a claim containing a mathematical formula implements or applies that formula in a structure or process which, when considered as a whole, is performing a function which the patent laws were designed to protect ..., then the claim satisfied the requirements of $101. Because we do not view the respondents’ claims as an attempt to patent a mathematical formula but, rather, to be drawn to an industrial process for the molding of rubber products, we affirm the judgment of the Court of Customs and Patent Appeals. The citations and footnotes have generally been omitted. Footnote 14, however, is sufficiently intriguing to warrant providing an edited version: 14. Arguably, the claims in Flook did more than present a mathematical formula. The claims also solved the calculation in order to produce a new number or “alarm limit” and then replaced the old number with the number newly produced. The claims covered all uses of the formula in processes “comprising the catalytic chemical conversion of hydrocarbons.” ... The claims, however, did not cover every conceivable application of the formula. We rejected in Flook the argument that because all possible uses of the mathematical formula were not preempted, the claim should be eligible for patent ‘protection. Our reasoning in Flook is in no way inconsistent with our reasoning here. A mathematical formula does not suddenly become patentable subject matter simply by having the applicant acquiesce to limiting the reach of the patent for a formula to a particular technological use. A mathematical formula in the abstract is nonstatutory subject matter regardless of whether the patent is intended to cover all uses of the formula or only limited uses. Similarly, a mathematical formula does not become patentable subject matter merely by including in the claim for the formula token postsolution activity such as the type claimed in Nook. We were careful to note in Flook that the patent application did not purport to explain how the variables used in the formula were to be selected, nor did the application contain any disclosure relating to chemical processes at work or the means of setting off an


37

alarm or adjusting the alarm limit. All the application provided was a “formula for computing an updated alarm limit.’’ The dissenting opinion, well worth reading, is omitted.

7.2 In re Bradley, 600 F.2d 807 (C.C.P.A. 1979), aff’d, 49 U.S.L.W. 4250 (1981) Rich, Judge. This appeal is from the decision of the Patent and Trademark Office (PTO) Board of Appeals (board) affirming the rejection of claims 1-6, all of the claims in the appellants’ application serial No. 570,331, filed April 21, 1975, for “Switch System Base Mechanism,” as being drawn to subject matter which is nonstatutory under 35 U.S.C. 0101. We reverse. The Invention

The appellants’ invention is in the field of computer technology. It does not relate to computer applications, i.e., any specific task that a computer is asked to perform, but, rather, to the internal operation of the computer and its ability to manage efficiently its operation in a multiprogrammed format. Specifically, the invention relates to altering or repositioning information in the computer’s system base. They accomplish their result by employing a “firmware” module, consisting of hardware elements permanently programmed with a microcode, which directs the data transfers, between the scratch-pad registers and the system base located in main memory, which are necessary to effect the alteration. The Rejection The examiner rejected the appealed claims on the authority of Gottschalk v. Benson, 409 U.S. 63, 93 S.Ct. 253, 34 L.Ed.2d 273, 175 USPQ 673 (1972) (hereinafter Benson) before the Supreme Court’s decision in Parker v. Flook, 437 U .S. 584,98 S.Ct. 2522,57 L.Ed.2d 45 I , 198 USPQ 193 (1978) (hereinafter Flook). In his final rejection, dated October 27, 1976, the examiner stated that the subject matter “deemed as the invention” is “a data structure” and then made the following analysis: The invention resides in a “data structure” or an algorithm designed to control the multiprogramming computer to solve the particular problem indicated.

Under the ruling in Gottschafk v. Benson (409 U.S. 63,93 S.Ct. 253,34 L.Ed.2d 273), 175 USPQ 673, the instant claims, depending upon a pro-

38


gram-implemented algorithm for patentability, are deemed nonstatutory subject matter. The appellants requested reconsideration and agreed that their claims are directed to data structures in hardware which are “specific new, novel and unobvious (in) arrangement.” They asserted that by stating that the invention resided in a “technique,” the examiner was clearly disregarding the claims and interpreting the invention strictly on the basis of what is found in the specification, because no “technique” is claimed. They stated that even if a technique (i.e., process) were claimed, Benson does not render all such inventions nonstatutory, and that their invention does not involve a mathematical algorithm.’ In his answer before the board, the examiner noted that all of the limitations found in claim 1 were old in the art and that the “claim is thus reciting prior art coupled with subject matter which the U.S. Supreme Court has found to be non-statutory in Benson.” The appellants responded in their reply brief that it makes no difference whether individual elements are old in the art and that it is the elements in combination which define the invention as a whole. The Board The board rendered its decision on September 20, 1978, after the Supreme Court’s decision in Flook. After incorrectly stating that none of the claims recited the term firmware,* the board analyzed the appellants’ claims element by element, concluding that the only novel arrangement of the recited structures resided in the microprogramming, “which together with its attendant memory hardware appears to constitute firmware.” I In Benson, the Supreme Court was clearly limiting its discussion to mathematical algorithms:

The patent sought is on a method of programming a general purpose digital computer to convert signals from binary-coded decimal form into pure binary form. A procedure for solving a given rype of mathematical problem is known as an “algorithm.” The procedures set forth in the present claims are of that kind; that is to say, rhey are a generalized formulation for programs to solve mathematical problems of converting one form of numerical representation to anoiher. [409 U S . at 5,93 S.Ct. at 254, 175 USPQ at 674 (emphasis ours).] It is not clear whether the examiner regarded the appellants’ invention as a mathematical algorithm, but, as will become evident, the board did so regard the invention.

* Claim 3, as well as dependent claims 4-6, explicitly recites firmware. Firmware is a term of art in the computer field and refers to microinstructions permanently embodied in hardware elements. For a further discussion, see Ross, The Patentabiliry of Computer “Firmware.” 59 JPOS 731 (1977). We need not and do not decide at this time whether firmware per se is statutory under 35 U.S.C. 8101, because the invention as a whole is not directed thereto.


39

Apparently on the basis of Flook, the board affirmed the rejection because it was of the opinion that the appealed claims are directed to a method of calculation or mathematical algorithm. The board found the claims similar to the claims at issue in In re Waldbaum, 559 F.2d 61 1, 194 USPQ 465 (Cust. 8z Pat. App. 1977), which included language characterized by the board as “obviously related to calculating and mathematical problem solving.”’ Although the claims here at issue do not contain similar mathematical language, the board said that this “does not make the functions attendant the ‘means’ of appellants’ claims any less mathematical or less related to an algorithm within the meaning assigned that term by the USSC in Benson.” To support its conclusion that the appealed claims are mathematical in nature, the board relied on a statement in the specification to the effect that all of the data in the computer are in binary form, but may be interpreted as binary-coded decimal, decimal, or alphanumeric. We reproduce the board’s reasoning in full: Since digital computers normally operate in some numerical radix, binary, binary coded decimal, or the like, we consider the operation of appellants’ claimed invention to be mathematical. Every operation performed in appellants’ invention as claimed involves the accommodation of data and instructions to the size of the registers in memory, and to the positional assignment to the registers in memory by the use of some numerical measure or quantity effected by way of electrical signals. In whatever form the instructions employed in appellants’ invention are characterized, numerical or otherwise, we think it is accurate to say that the operation of appellants’ structure is mathematical and that the instructions constitute a procedure which is algorithmic in character, to the same degree as that of the Waldbaum structure and that satisfactory operation of the apparatus claimed represents the successful solution of a mathematical problem falling within the definition of algorithm supplied in Benson and reiterated in Flook.

In summary, the board stated that the claims are drawn to apparatus in form only, and couple the apparatus (which it asserts is old in the art) “with subject matter, namely, programming, which is nonstatutory under ,~ and Flook cases.. . .” the Benson, C h r i ~ t e n s e nWaldbaum Waldbaum claim 1 reads in pertinent part,

a method ... to count the number of busy lines ... comparing means ... to derive the number ofl’s in said data word ... [Emphasis ours.]

It is clear that this claim recites a mathematical algorithm. It solves a mathematical problem, to wit, counting a number of busy lines in a telephone system.

In re Christensen, 478 F.2d 1392, 178 USPQ 35 (Cust. & Pat. App. 1973).

40


Opinion 1. The examiner’s basis for the rejection is grounded on the erroneous interpretation of the Supreme Court’s decision and opinion in Benson, namely, that all computer program or program-related inventions are nonstatutory under $101. Both the Supreme Court’ and this court6 have thoroughly repudiated this view. Our decision, therefore, is based solely on the analysis made by the board. 2. The board said that the claims do not directly recite a mathematical formula, algorithm, or method of calculation, but, nevertheless, held the claims to be mathematical in nature. As appears from the quoted portion of the board opinion, the board regarded the fact that digital computers operate in some number radix as conclusive on the issue of whether the appealed claims recite a mathematical algorithm in the Benson and Ffook sense. The board did not, however, direct attention to any specific formula it thought is utilized, or to what, if anything, the mathematical calculations alleged to be present in the claims are directed.

We do not agree with the board. We are constrained to reject its reasoning. Such reasoning leads to the conclusion that any computer-related invention must be regarded as mathematical in nature, a conclusion which is not compelled by either Benson or Flook. The board’s analysis confuses what the computer does with how it is done. It is, of course, true that a modern digital computer manipulates data, usually in binary form, by performing mathematical operations, such as addition, subtraction, multiplication, division, or bit shifting, on the data. But this is only how the computer does what it does. Of importance are the significance of the data and their manipulation in the real world, i.e., what the computer is doing. It may represent the solution of the Pythagorean theorem or a complex vector equation describing the behavior of a rocket in flight, in which case the computer is performing a mathematical algorithm and In Benson, the Court stated the following: Very simply, our holding today is that a claim for an improved method of calculation, even when tied to a specific end use, is unpatentable subject matter under 5101. [409 U.S.at 59511.18, 98 S.Ct. at 252811.18, 198 USPQ at 19911.18.1 See In re Gelnovatch, 595 F.2d 32,36-37,201 USPQ 132, 141 (Cust. & Pat. App. 1979); In re Johnson, 589 F.2d 1070, 1075, 200 USPQ 199. 205 (Cust. & Pat. App. 1978); In re Sarkar, 588 F.2d 1330, 1333, 200 USPQ 132, 137 (Cust. & Pat. App. 1978);In re Freeman, 573 F.2d 1237, 1244, 197 USPQ 464,470 (Cust. & Pat. App. 1978); In re Castelet, 562 F.2d 1236, 1240, 195 USPQ 439,443 (Cust. & Pat. App. 1977); In re Chatfield, 545 F.2d 152, 155156, 191 USPQ 730,733-734 (Cust. & Pat. App. 19761, cert. denied, 434 U.S.875,98 S.Ct. 226, 54 L.Ed.2d 155, 195 USPQ 465 (1977).


41

solving an equation. This is what was involved in Benson and Flook. On the other hand, it may be that the data and the manipulations performed thereon by the computer, when viewed on the human level, represent the contents of a page of the Milwaukee telephone directory, or the text of a court opinion retrieved by a computerized law service. Such information is utterly devoid of mathematical significance. Thus the board’s analysis does nothing but provide a quick and automatic negative answer to the $101 question simply because a computer program is involved. The appellants have continuously insisted that they are claiming a new and unobvious combination of hardware elements, i.e., a new machine or apparatus.’ The issues of novelty and unobviousness8 are not before us, but we agree with the appellants insofar as they characterize their invention as a machine or apparatus. The board, likewise, seems to agree on this point. In its opinion, it identifies all of the “means” of the appellants’ claim I as structural hardware elements, such as registers, portions of main memory and control store,9 and other computer components. Thus the claim falls literally within the boundaries of § l O l . The appellants have characterized their combination of hardware elements as a mechanism which enables the computer to alter information in its system base in a manner not previously possible. They are in no way claiming the altered information; in fact, the particular information acted upon by the appellants’ invention is irrelevant to the operation of the invention itself. We see no difference in this regard, with respect to being within § l O l , between the appellants’ claimed invention and a strictly mechanical adding machine, which is certainly statutory if claimed in a manner which does not embrace any particular calculation that the machine is capable of making. The PTO regards as significant the fact that firmware is involved in the present invention. In a sense, firmware may be likened to software (computer programs); it is information which has been embodied into hardware by, for example, destroying fusible links in a read-only memory (ROM) array. In the appellants’ invention, the information contained within the

’ 35 U.S.C. 8101 states the following: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement rhereof, may obtain a patent therefor, subject to the conditions requirements of this title. [Emphasis ours.] 8 3 5 U.S.C. 8102, 103.

The control store of a computer is a plurality of multibit storage locations (memory) containing microinstructions, which, when decoded by appropriate circuitry, provide control signals that cause specific operations to take place in the computer’s processing unit.

42


firmware, which is located in the control store of the computer, directs the desired information transfers within the system base. If the appellants were claiming the information embodied in the firmware or the firmware itself per se, a different case would be presented. We express no opinion on the statutory nature of such an invention, a question not before us. The appellants are claiming a combination of hardware elements, one of which happens to be a portion of the computer’s control store microprogrammed in a particular manner. It is this subject matter with which we must deal. From our reading of the appellants’ specification, their claimed “data structure” is merely that which results from the arrangement of the recited hardware elements in the claimed manner. It is the result of certain structural “means” performing certain recited functions as explicitly sanctioned by 35 U.S.C. 0112, sixth paragraph. We disapprove the board’s distillation of the appellant’s claim down to the information contained in the firmware in order to hold it nonstatutory. The invention is not claimed in that manner, and in this case we seen no reason to view the claim format as a subterfuge for masking the presence of an essentially nonstatutory invention. Even though the claimed invention is a machine, we must nevertheless determine whether the claim recites a mathematical algorithm and, if so, whether it preempts the use of the algorithm. In re Noll, 545 F.2d 141, 148, 191 USPQ 721, 726 (Cust. & Pat. App. 1976)]. In doing so we apply the two-step test in In re Freeman, supra note 6. When we examine the appellants’ invention as a whole under the first step of this test, including the information microprogrammed into the firmware element as depicted in Figs. 14(a-i) and 15(b-c), we fail to detect the presence of any mathematical algorithm. In altering information in the system base as desired, certain “calculations” are made, such as determining whether a given quantity is equal to 0 or, as noted by the solicitor, multiplying an address in memory by 16 to arrive at another address. However, it certainly cannot be said that comparing with 0 or multiplying by 16 is preempted by the appellants’ claims. Furthermore, the presence of these calculations does not transform the invention as a whole into a method of calculation (cf. In re Gelnouarch, supra note 6 ) . There is no solution of an equation, such as the new alarm limit in Flook, or the equivalent pure binary number, as in Benson, present in the computer after the task has been completed. (see I n re Chatfield, supra note 6). In summary, we have examined the claims thoroughly and we do not find any mathematical formula or mathematical method of calculation, improved or otherwise, which is either claimed as such or find that the


43

invention is a combination of tangible hardware elements-a machineincluding some hardware elements which contain microprogrammed information termed “firmware.” We do not find the invention to be nonstatutory under the authority of Benson o r Flook, or under the authority of our own cases, such as those cited at note 6 ante. Therefore the decision of the board is reversed. Reversed. SELECTED BIBLIOGRAPHY Brooks, D. (ed.) (1980). “Computer Law: Purchasing, Leasing, and Licensing Hardware, Software, and Services.” Practising Law Institute, New York. Brooks, D. (ed.) (1981). “Computer Law 1981: Acquiring Computer Goods and Services.“ Practising Law Institute, New York. Commission on New Technological Uses of Copyrighted Works (1978). “Final Report.” Copyright Office, Washington, D.C. Gasaway, L., and Murphy, M. (1980). “Legal Protection for Computer Programs.” CAUSE Publications, Boulder. Gernignani, M . (1981). “Law and the Computer.” CBI, Boston. Gemignani, M. (1980). Legal protection for computer software: The view from ’79. Rutgers J . Computers, Technol. & the Law 7, 269-312. Luccarelli, P. (1981). The supremacy of federal copyright law over state trade secret law for copyrightable computer programs marked with a copyright notice. ComputerlLaw J . 111, 19-52. Novick, M . , and Wallenstein, H. (1980). The algorithm and computer software patentability: A scientific view of a legal problem. Rutgers J . Computers, Technol. & rhe Law! 7, 313342. Potenza, J. (1982). Copyright protection in the object code of a computer program. Bull. L a w , Sci. & Technol. (ABA) 38, 2-4. Pressman, C. (ed.) (l98l), “Registrant Workbook: The Second National Software Protection Conference.” University of Chicago Center for Continuing Education, Chicago. Rose, G. (ed.) (1981). “Protecting Trade Secrets.” Practising Law Institute, New York. Schmidt, W. (1981). Legal proprietary interests in computer programs: The American experience. Jurimetrics J. 21, 345-404. Stem, R.,and Squires, J. (1982). Can we stop’?IEEE Micro 2, 13-24. Stem, R. (1981). Another look at copyright protection of software: Did the 1980 act do anything for object code? ComputerlLaw J . 111, 1-18. INDEXOF CASESCITED Format: Case caption; reference to reporter series, if any. Alfred Bell & Co. v . Catalda Fine Arts, Inc., 191 F.2d 99 (2d Cir. 1951). Avco C o p . v. Precision Air Parts, CA 79-255-N (M.D. Ala. 1980). Cochrane v. Deener, 94 U.S. 780 (1877). Dann v. Johnston, 425 U.S. 219 (1976). Data Cash Systems, Inc. v. JS & A, Inc., 480 F. Supp. 1063 (N.D. 111. 1979). Diamond v. Bradley, 49 U.S.L.W. 4250 (1981).

44


Diamond v. Chakrabarty, 48 U.S.L.W. 4714 (1980). Diamond v. Diehr, 49 U.S.L.W. 4194 (1981). Gottschalk v. Benson, 409 U.S. 63 (1972). Graham v. John Deere, 383 U S . 1 (1%6). In re Bradley, 600 F.2d 807 (C.C.P.A. 1979). In re Chatfield, 545 F.2d 1236 (C.C.P.A. 1976). In re Christensen, 478 F.2d 1392 (C.C.P.A. 1973). In re Freeman, 573 F.2d 1237 (C.C.P.A. 1978). In re Gelnovatch, 595 F.2d 32 (C.C.P.A. 1979). In re McIlroy, 442 F.2d 1397 (C.C.P.A. 1971). In re Musgrave, 431 F.2d 882 (C.C.P.A. 1970). In re Prater, 415 F.2d 1393 (C.C.P.A. 1%9). In re Noll, 545 F.2d 141 (C.C.P.A. 1976). In re Toma, 575 F.2d 872 (C.C.P.A. 1978). In re Waldbaum, 559 F.2d 611 (C.C.P.A. 1977). Parker v. Flook, 437 U.S. 584 (1978). Synercom Technology v. University Computing Co., 462 F. Supp. 1003 (N.D. Tex. 1978). Tandy Corp. v. Personal Micro Computers, Inc., 524 F. Supp. 171 (N.D. Cal. 1981). Universal Athletic Sales v. Salkeld, 511 F.2d 904 (3d Cir. 1975). Valmont Industries v. Yuma Manufacturing Co., 296 F. Supp. 1291 (D. Colo. 1969). Warrington Associates, Inc. v. Real-time Engineering Systems, Inc., CA80-CI349 (N.D. Ill. 1981).

Algorithms for Public Key Cryptosystems: Theory and Application S . LAKSHMIVARAHAN School of Electrical Engineering and Computer Science The University of Oklahoma Norman. Oklahoma

1 . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 . 1 Cryptography: Nature and Scope . . . . . . . . . . . . . . . . . . 1.2 Cryptanalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Classical Cryptosystems: Examples . . . . . . . . . . . . . . . . . 1.4 Public Key Encryption System: Basic Concepts . . . . . . . . . . . 1.5 Mathematical Formalism for Public Key Cryptosystem . . . . . . . . 2 . Mathematical Preliminaries . . . . . . . . . . . . . . . . . . . . . . . 2.1 Divisibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Congruences . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Primitive Roots and Discrete Logarithms . . . . . . . . . . . . . . . 2.4 Quadratic Residues . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Prime Factorization and Primality Testing . . . . . . . . . . . . . . 2.6 Finite Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . 3. Examples of Public Key Cryptosystems . . . . . . . . . . . . . . . . . 3.1 Algorithms Based on Exponentiation and Discrete Logarithms . . . . . 3.2 Algorithms Based on Exponentiation and Prime Factorization . . . . . 3.3 Algorithms Based on the Knapsack Problem . . . . . . . . . . . . . 3.4 Algorithms Based on Algebraic Coding Theory . . . . . . . . . . . . 4 . Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Digital Signatures . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Read-only Secure Communications . . . . . . . . . . . . . . . . . 4.4 Conference Key Distribution Systems . . . . . . . . . . . . . . . . 4.5 Data-Base Security . . . . . . . . . . . . . . . . . . . . . . . . 5 . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Note Added in Proof . . . . . . . . . . . . . . . . . . . . . . . . . .

.

1

45 45 50 53 60 62

64 64 68 71 75 79 80 82 82 85 88 93 94 94 96 98 99 100

101 102 107

Introduction

1.1 Cryptography: Nature and Scope

Historically. the use of cryptography was exclusively confined to military and diplomatic communities to obtain secrecy of data in communica45 AOVANCES IN COMPUTERS. VOL. 22

Copyright 8 1983 by Academic Press. Inc . All rights of reproduction in any form reserved. ISBN 0-12-012122-0

46

S. LAKSHMIVARAHAN

tion (Kahn, 1966, 1967), but in recent days cryptography is going public; both private companies and corporate sectors alike have started using cryptography to protect the secrecy of sensitive data. Some of the recent advancements in computer and communication technologies and extensive computerization of information storage and transission are primarily responsible for this rather explosive interest of the private sector in cryptography (Kahn, 1980, 1981). In fact, with the widespread proliferation of computer networks, electronic fund transfer networks, instant electronic mail, point-of-sale terminals, banking from home (and the anticipated emergence of a checkless society), conferencing through computers, etc., have very much increased the vulnerability to wiretapping (or spying) and accessibility of data (intentional or unintentional), both in stored form and during communication. The very technology which made many of the marvels of the computerized society possible (Martin, 1978) could well be used for compromising the security of the cryptosystems. All these possibilities have justifiably evoked the interest of private citizens and commercial organizations in acquiring cryptosystems which are far more secure, as well as methods for ensuring protection against forgery. Recent advancements in cryptographic techniques, known as public key cryptography (Diffie and Hellman, 1976a,b), have provided very elegant solutions not only to secrecy and authentication, but also to protection against forgery through a provision called digital signatures. Our aim in this article is to provide a survey of these advancements, considering both the underlying theory and the varied applications. We begin by introducing a few useful concepts, definitions, and notations. The message to be transmitted or stored is called the plain text; the process of transforming it and thereby locking the contents of the plain text from being known to others is called encryption or enciphering. The transformed plain text is called encrypted or ciphered text or a cryptogram. Often the encrypting transformation itself is called a cipher. Decryption (or deciphering), the opposite of encryption, consists in unlocking the ciphered text to get back the original plain text. The locking and unlocking is done by a key which is known only to the legitimate senderreceiver and the encryption-decryption pair constitutes a cryptosystem. Cryptography deals with the analysis and design of cryptosystems. In the conventional or classical cryptosystem the encryption and decryption processes use the same key (see Fig. 1). This in turn necessitates that the key be distributed in secret (by registered mail or armed courier). Classical cryptosystems are also known as private key or symmetric cryptosystems (Simmons, 1979). The modern public key cryptosystem, on the other hand, uses different keys for the encryption and decryption pro-

ALGORITHMS FOR PUBLIC KEY CRYPTOSYSTEMS

M

47

C = Ek(MI

cesses (see Fig. 2). The encryption key is generally made public in the form of a directory, but the decryption key is kept secret. In view of this, a public key cryptosystem is also known as an asymmetric cryptosystem (Simmons, 1979). Thus, every cryptosystem, classical or public key type, consists of ( I ) a fixed (encryption and decryption) algorithm (or a set of rules or transformations) and (2) the relevant key(s). It should be emphasized that secrecy of data using cryptography is obtained not by keeping the details of the algorithms for encryption and decryption secret, but by keeping only the relevant key secret. Cryptography is not to be confused with (source) coding, such as the Huffman code, Shannon-Fano code, or with the (channel) encoding and decoding of messages for communication through noisy channels. In the coding schemes, the alphabet and/or words in the plain text message are replaced by certain prespecified code words. For example, in the wellknown Huffman code (for plain text messages in English), each letter of an alphabet is coded as a binary string (see Table I). An important feature of this code is that the length of the code word for a letter depends on the

FIG. 2. Public key cryptosystem: k , , encryption key (public); k d , decryption key (secret).

48

S. LAKSHMIVARAHAN

TABLE I

Letter

E T A 0 N

Huffman code 100

M

001

U G

1111 1110 1 100 101I 1010 0110 0101 11011

R I S H D L F

C

Letter

Huffman code 00011 00010

oooO1

Y

00000

P W B

110101 011101 01 1100 1101001

V K X J

01111 01001

Q

Olooo

Z

110100011 1101m1

I10100000 I101000101 1101000100

relative frequency of that letter in normal plain text: A greater frequency implies a shorter code word. It is well known that in English the letter E occurs most frequency and 2 often occurs least frequently (see Table 11). Furthermore, in Huffman code, no smaller code word is a prefix of any longer one and hence there is a unique decoding algorithm (see Fig. 3).

Letter

Frequency

E T N

260 I 86 156 154 148 I48

R I 0 A

S D H

146

c

126 88 I0 70 60

F

56

L

Letter

Frequency

P

54 54 50

U

M Y G

W V B X K Q

38 34 31 26 18 10

I

J

5 4

Z

2


49

FIG.3. Huffman decoding tree.

Thus Huffman coding, in contrast to other coding schemes such as ASCII code, where each character is represented as a bit string of constant length (seven for ASCII code), is always associated with message compression. While it is true that message compression can enhance data security, it cannot totally replace encryption; for in this coding (or compression) scheme, there are no (secret) keys and knowledge of the coding scheme or algorithm alone will compromise the system. The channel encoding-decoding operations, on the other hand, are used to provide noise immunity in message transmission. The encoding operation often introduces redundancy in the message for the purposes of error detection and correction. There is a plethora of channel encoding-decoding algorithms known in the literature. For many years it was felt that no encoding scheme could provide data security. However, very recently it was shown (Berlekamp et al., 1978) that the problem of decoding a general linear code is computationally difficult or infeasible. Based on this theme, MeEliece (1978) demonstrated a public key cryptosystem using the now well-known Goppa codes. While fast algorithms for decoding Goppa codes are known (Patterson, 1979, by suitably permuting the columns of the generator matrix and keeping this information secret, one can make the decoding problem computationally very difficult. For a detailed discussion of the use of encoding schemes in public key cryptography, the reader is referred to Section 3. In general, it is conceivable that one could use source coding, encryption, and channel encoding in that order. In this article, we confine our attention primarily to cryptography. For more information on (source) coding and (channel) encoding-decoding, we refer the reader to a number of fine textbooks by Ash (1965), Berlekamp (19681, Hyvarinen (1968), Peterson (1961), and McEliece (1977), to mention only a few.

50

S. LAKSHMIVARAHAN

1.2 Cryptanalysis

Wiretapping o r spying is the act of intercepting and collecting messages from a communication channel while a transmission is in progress. Encryption goes a long way in providing secrecy and protection against immediate exposure of plain text and makes the job of extracting the information from the communication channel not obvious. The act of analyzing the encrypted text with a view to obtain the contents of the plain text without the knowledge of the (secret) key is called crypranalysis. Cryptanalysis is a highly specialized form of applied mathematics and requires training in a variety of disciplines such as probability theory, statistics, algebra, and number theory, to mention a few, and the ability to combine it with intuition. Cryptanalysts often have a variety of side information regarding the cryptosystem: the nature of the algorithms, the language of communication, the overall (general) context of the messages, the properties of the language, such as the frequency of single letters (see Table 11), diagrams (how frequently ae occurs compared to qy, for example), and trigrams, etc. Furthermore, all the natural languages (for example, English, German, French) are (statistically) redundant in the sense that even if in some words certain letters are missing, by using the frequency distribution one can predict those missing symbols with a very high probability. In fact, the (statistical) redundancy is a “blessing in disguise” for the cryptanalyst. The primary motivation for this activity often arises from the fact that the cost of cryptanalysis in many cases will only be a small fraction of the gains that are attainable if the plain text messages (being communicated or stored) were to be known. Furthermore, the advances in computer technology-increasing speed and reducing the cost of the hardware-have made possible the development of multimicrocomputer systems which can carry out many tasks in parallel; and using these sophisticated computing systems, cryptanalysis that used to take weeks and months could be done in a very small fraction of time. From what has been said, it is clear that the basic requirement in the design of a cryptosystem is that the enciphering and deciphering operations be inexpensive but immune or impenetrable to cryptanalysis. Is there a cryptosystem which is known to be totally immune to cryptanalysis? The answer is yes, and it is called the one-time tape or pad system (Shannon, 1949). However, in spite of such an assured performance, this one-time pad system has a number of disadvantages: It needs keys which are random and at least as long as the message. Operationally, this involves the transfer of large amounts of key, which is very time-consuming and expensive. This is a major handicap in using this system in a computer communication network, where there could be potentially n(n - 1)


51

pairs of keys between all possible pairs of users, where n is the number of users. In view of this, the one-time pad system is used only in extremely sensitive areas, such as the Washington, D.C.-Moscow Embassy hot line and military command control systems, for only in these cases can one even try to justify the overhead and cost. Having thus ruled out the use of the highly secure but expensive one-time pad system for regular commercial and business applications, the question remains as to how one should go about choosing a cryptosystem to ensure a certain level of security. Unfortunately there is no general method for measuring the security of cryptosystems, and manufacturers frequently rely on a certification process which is often based on certain ad hoc or heuristic procedures. We hasten to add that history is replete with instances wherein systems that were believed to be secure were soon compromised (Kahn, 1980). We wish to remind the reader that a good portion of the literature on cryptography is classified. All of our answers and observations relate only to that body of knowledge that is disseminated through the open literature. Any acceptable certification process should take into account the hostility of the environment in which a cryptosystem will be operating. Often the hostility of an environment is judged by the nature of the cryptanalytic attack. There are at least three well-known forms of attack and they are (1) cipher text only attack, ( 2 ) known plain text attack, and ( 3 ) the chosen plain text attack. The cipher text only attack is a very weak form of attack in which the cryptanalyst compromises the security of a system just based on the side information he or she has about the system and using (perhaps an unlimited amount of) ciphered text collected through wiretapping. Any system that succumbs to this rather weak form of attack is totally useless. In this form of attack the amount of ciphered text (expressed as its length) needed to uniquely solve for the key is called the unicity point or distance (Shannon, 1949). The unicity distance is a function of the cipher (such as the number of distinct keys) and the redundancy of the language of the plain text. For the computation of the unicity distance for various ciphers the reader is referred to Deavours (1977). If the cryptanalyst possesses a good number of corresponding plain text-ciphered text pairs, then he o r she can use the known plain text attack. There are a number of ways in which the cryptanalyst can obtain a copy of plain text-ciphered text pairs. A company (or a main diplomatic office), for example, may encrypt and send information regarding a certain product (or some foreign policy matter) for later press release. It is conceivable that an opponent can obtain a copy of this press release in both the plain text and ciphered text forms. Also, standard formats for addressing persons, such as dear sir/madam or sincerely yours, may often reveal portions of plain text. Immunity to this attack is often taken as a

52

S. LAKSHMIVARAHAN

satisfactory measure of performance in practice and is used for the certification process. The third and perhaps most vicious form of attack is the chosen plain text attack, wherein the opponent has the ability to implant a plain text of his choice and observe the corresponding enciphered text. This is a very difficult form of attack and is rare. While one might agree to use the known plain text attack for certification purposes, the question of how to quantify the security of a cryptosystem needs to be addressed. There are at least two different ways in which the security of a cryptosystem has been quantified in recent years. The first and perhaps most well-known measure is total, unconditional, or perfect security. A system is said to be totally secure if it cannot be compromised even under the assumption of unlimited resources at the disposal of the cryptanalyst. The one-time pad system referred to earlier belongs to this category. In these systems, even if the cryptanalyst chooses to exhaust all possible keys, he or she will end up having an equally large number (one for each key) of plain texts to choose from. For example, consider a message, say, 1000 bits long. In the one-time pad system the key is also 1000 bits long and is randomly chosen from the set of all 2’Oo02 possible keys. To exhaust all these keys in a fixed time implies that the cryptanalyst must possess unlimited computing resources that can be used in parallel. Even assuming that this equipment is available, in the end the cryptanalyst will only be left with a glut of all possible decryptions, many of which could be meaningful, and a lot of uncertainty regarding the correct message. A little reflection, however, reveals that the assumption of unlimited resources at the disposal of the cryptanalyst is as unrealistic, at least in the context of many commercial and business applications, as the ability of the cryptographer to use the one-time pad system for security purposes. The unrealism essentially arises from the cost effectiveness of these assumptions. At the other extreme, if the length of the key is too small, the system could be compromised even with very limited resources. hence a more practical approach to security is to make the key sufficiently long in such a way that a cryptanalyst possessing reasonably good (but not unlimited) computing power cannot possibly decipher the plain text being communicated (or stored) within a valid time frame. If messages are very long, a same key will be used again and therefore one would like to pick the key long enough so that (under the assumption of finite resources at the disposal of the cryptanalyst) it should be infeasible to exhaust all possible keys. Any system that is secure from this point of view is called a computationally secure system. To get a feel for this measure, consider a key 128 bits long. An exhaustive analysis would necessitate decryption with each of the 2128> distinct keys. Excluding leap years, there are only 3.15 X lo7 sec in a


53

year. If we assume that deciphering with one key can be done in 1 x sec, then in 1 year we can exhaust only 3.15 x 10l6 keys. To exhaust all the keys, then, would require at least 3.17 x lo2’years! In this sense analysis is computationally infeasible. We hasten to add that a computationally secure system could succumb to analysis under unlimited computating resources. All the modern public key cryptosystems are designed from this angle of computational security. An inherent flexibility of this approach is that should any future breakthrough in hardware technology make deciphering possible at extremely high speeds, we simply need to increase the key length which will, in turn, make exhaustive analysis still infeasible. The theory of unconditionally secure cryptosystems was developed by Shannon (1949) based on concepts of redundancy, uncertainty, equivocation, etc., which he earlier developed in the context of information theory. The one-time pad cryptosystem, the only system known to be unconditionally secure, was invented (Kahn, 1967) and used by the U.S. Army Corps in the 1920s. The concept of a computationally secure cryptosystem is of more recent origin and is due to Diffie and Hellman (1976b). While this latter approach is more practical, it does not, however, provide a complete solution to the basic problem of the design of provably secure systems; for, using this approach, we could only show that exhaustive analysis, which is, in fact the worst case, is computationally infeasible. In other words, this approach provides only an upper bound on the time for cryptanal y sis. Computational complexity theory, a young and important branch of computer science, deals primarily with the analysis and design of algorithms, especially with the theory of lower and upper bounds. In their article “New Directions in Cryptography,” Diffie and Hellman (1976b) brought into focus the importance of computational complexity theory in the design of secure cryptosystems. We shall discuss these complexity issues later. 1.3 Classical Cryptosystems: Examples

For the purposes of later comparison and for the sake of completeness in this section, we illustrate through a number of examples the basic ideas of the design of classical cryptosystems. Let C be a finite alphabet set and .A the set of all plain texts or messages. Clearly, M C Z*, the set of all strings or sentences of finite length over the alphabet set Z. For example, if X is the English alphabet set, then M corresponds to the set of all (meaningful) sentences in English. In computer communications, the character set is often represented by binary strings of fixed length (seven

54

S.LAKSHMIVARAHAN

A

B

c D E F G H I

0

0 01

02 03 04 05

06 07 08

J K L M N 0 P Q R

W 10 I1 12 13 14

S T

u

V W X Y Z ba

15

16 17

18

19 20 21 22 23 24 25 26

in ASCII code.) In this case, we can take Z = (0, 1) and At will constitute the set of binary strings obtained by any acceptable representation scheme. Yet another useful method of representing messages is to represent each alphabet by a fixed integer (see Table 111). All these examples show that there is great flexibility in the choice of a basic alphabet set and what constitutes the set all plain text messages. Without loss of generality, we shall assume that (1) the ciphered text is also over the same alphabet set Z and the set of all ciphered texts V C Z * and (2) that the encryption does not increase (or decrease) the length of the plain texts; that is, the encryption is length invariant. Let K refer to the set of possible keys. A cryptosystem is a pair (& , Dk) of a single-parameter family (parameterized by the key k E K ) of transformations Ek:x*-+ Z*

and

D k : x * + Z*

where C1. Ek(M) = C c2. &(MI) # Ek(M2)

and if

Dk(C) = M MI f M2

Condition C1 implies that Dk is the inverse of Ek and condition C2 demands that Ek be one to one, in the sense that distinct input strings are encrypted into distinct strings. Implicit in this definition is the fact that both encrypting and decrypting transformations use the same key. Also recall that ]MI = IC), where 1x1 refers to the length of the string x. We do not, however, require that Ek, # Ek2 if kl # k2. Furthermore, (e = {ylEk(x)= y and x E A};in other words, the system will encrypt any string

55


over C irrespective of whether or not the string in question is a meaningful sentence. The encrypting transformations (and hence the cryptosystems) are divided into two groups, depending on how Ex is defined: ( I ) stream encryption (or cipher) and (2) block encryption (or cipher). In the stream encryption, Ek is actually defined as a natural extension of a one-to-one mapping E k :C + C, where we have the following:

( I ) &(A) = A, where A is the null string; that is, IAl (2) Ek(XU) = Ek(X)Ek(U),where X E C* and E c.

=

0.

Since x A = Ax = x for any string x E C*, and Ek and E k coincide on C, we shall not distinguish between Ek and Ek and represent both by Ex . In a block encryption, however, messages are often blocked into strings of fixed length, for example, messages of length 64 bits, and Ex is defined block by block. Thus if the block length is n , then EL: C" -+ I",where C n is the set of all strings of length n over C. With this background, we shall now describe typical examples of classical systems. 1.3.1 Simple Substitution System

The simple substitution system is an example of a stream encryption. This system is perhaps the most ancient of all cryptosystems known to mankind. In it each letter of the plain text is transformed into another; this is also known as monoalphabetic substitution cipher. As an example, consider two substitutions SIand S2 in Table IV. In SIeach letter is replaced by the third letter in the natural sequence of the alphabet, but S2 is an arbitrary substitution. Notice that the key is a permutation of the alphabet. Thus once the permutation is fixed, both EL and Dk are fixed uniquely: Plain text, COMPUTERS; Ciphered text FRPSXWHUV; (using SIL

Plain text, COMPUTERS Ciphered text FTSIYWGJU (using S2),

Deciphering is again a table-searching procedure and we shall leave it to the reader to develop the deciphering table corresponding to the substitutions S1and S 2 . Often S , is called Caesar cipher in honor of the fact that Caesar used this kind of cipher in his communications. There are a total of 26! (which is at least 4 X permutations, which is also the number of keys. It might appear at first sight that with this size key space, it should be hard to cryptanalyze by exhaustive search, but the unfortunate part of this system is that all the statistical properties of the plain text language

56

S. LAKSHMIVARAHAN

TABLEIV EXAMPLES OF MONOALPHABETIC SUBSTITUTION Original alphabet

Substitution SI

D E F G H I J

A

B

C D E F G H I J K L M N ~

K L

M

Substitution

SZ C

X F N G H A M

In E

N 0 P

Q

Q

2

B S

I

Original alphabet

0 P

Q R S T

U V W

X Y Z

I“

Substitution

Substitution

SI

sz

R S T U V W

T J U W

X Y

Y K

2 16”

V P

A B C

D 0

I

L

R

~~~

I, blank spaces.

“permeate” through this transformation. By estimating the frequencies of the various characters in the ciphered text, one can invariably compromise this system. The amount of ciphered text needed on the average to uniquely solve for the key is called the unicity distance (Shannon, 1949). It can be shown that the unicity distance for this cipher is 28. In other words, just by using a ciphered text which is 28 letters long, substitution ciphers can be compromised by cipher text only attack. For a detailed treatment of various cryptanalytic methods, the reader is referred to Sinkov (1968) and Konheim (1981), and for a more recent and thorough treatment of the role of unicity distance in cryptanalysis, to Deavours (1977). 1.3.2 Polyalphabetic Substitution System

This stream system is best explained by converting letters into integers, say, according to Table 111. Let it4 = mlm2-.-mL and K = klk2 k,, be the + ki)(mod plain text and key. IfEk(M) = CIC2 CL, then Cjn+l= 27), where i = 1, 2, ... n and j = 0, I , 2, ..., such that jn + i IL. As an example, let M = COMPUTERS and K = HELP. Clearly, L = 9 and n = 4. Using Table 111, we obtain the following:


C

O

J

S

M

X

P

D

U

A

T

E

R

S

X

P

F

W

57

The number of keys is 27". Since the key is used repeatedly in cycles, the same letter of the plain text could get mapped into different letters in the ciphered text at a different time. For example, if M = ADVANCES and K = HELP, the first and the fourth A in M are transformed into H and P after encryption. In other words, in polyalphabetic substitution, the frequency characteristics of the ciphered text are, in general, very different from those of the plain text. Consequently, this system is much more resistent to statistical analysis. The cryptanalyst has to first estimate the length of the key and then solve for the actual key. If n = 1, this is a special case of simple substitution. If n 2 L and the key is chosen randomly (that is, if a key is drawn from the set of all possible keys under the assumption of uniform distribution), the resulting system is called the one-time pad system. For various methods to successfully cryptanalyze this system by cipher text only attack, when 1 In < L, refer to Sinkov (1968) and Konheim (1981).

1.3.3 Transposition System This is an example of a block-oriented encryption system. In this system each block of n characters are permuted o r transposed according to a fixed permutation. As an example, let

&=(

1 2 3 4 3 2 4 1

),

1 2 3 4

Dk=

(

4 2 1 3

)

Here Ek implies that the block length is 4, the first letter of the plain text block becomes the third letter of the ciphered text block, the second remains the second, the third becomes the fourth, and the fourth becomes the first. Thus

FIG.4. Basic idea of Lucifer: an example of a product cipher. The S and P boxes implement substitution and permutation, respectively.


59

There are n! keys or permutations. This transformation preserves the single-letter frequency characteristics of the plain text, but changes digram and trigram frequency characteristics. From this angle, this is again better than simple substitutions. We leave it to the reader to find the Dk deciphering transformation corresponding to Ek , which is a transposition. 1.3.4 Product Cipher System

This system is a combination of one or more of the basic systems described earlier. The well-known IBM Lucifer is a standard example of a product cipher system (Smith, 1971; Feistal, 1973). The basic idea of Lucifer is illustrated in Fig. 4. This is a block-oriented system consisting of layers of permutations and substitutions which are easily implemented in hardware (see Figure 5 ) . The length of the block, the input size of the substitution box, and the number of layers are the design parameters. It is

I

I

I

I

FIG.5 . Example of a substitution S box: BDC, binary-decimal converter; DBC, decimal-binary converter. It is a practical implementation of the following substitution table: Substitution

Input

Output

to

Of

S box

Output of BDC

Input of DBC

S box

OOO 001 010 01 I 100 101 I10 111

0

6 3 2

110 01 1 010 100

1

2 3

4

4 5 6

0

7

5

7 I

OOO 111 001 101

60

S. LAKSHMIVARAHAN

assumed, without loss of generality, that the plain text input to be enciphered is a binary sequence. Once the block length n and the input size m of the substitution are fixed, there are n! permutations and 2"! substitutions to choose from (see Fig. 5). Since 2m!grows much faster with m, from a practical standpoint of being able to mass-produce various substitutions in hardware, m generally is taken to be small, such as m = 4. The key for this product cipher is the specificationof the sequence of permutations and substitutions in various layers. These product systems (by proper choice of substitutions) correspond to nonlinear transformations and hence, in general, are much harder (compared to any single-component system) to crack. The idea of the product cipher is due to Shannon (1949).

Recently, in response to the growing interest in cryptography, the National Bureau of Standard (NBS) in collaboration with the National Security Agency (NSA) and IBM announced a block cipher called DES, the Data Encryption Standard (1977) for use by commercial and business organizations. This is a product block cipher of block length 64 bits with a key of length 56 bits, but expanded to 64 bits to provide error detection capabilities. The DES is a close cousin of Lucifer and has 16 layers of transformation. This standard is perhaps one of the most thoroughly analyzed systems of modem times (Diffie and Hellman, 1977; Morrits et al., 1977). While no one has yet compromised DES, there is a considerable debate over its security and the center of that controversy is the 56-bit key. The critics of DES claim that 56 bits is too small and provide arguments as to how a determined analyst with finite resources can compromise this system. As a means of strengthening DES, it is suggested that the key be increased to 128 bits and/or multiple encryptions be adopted. For a detailed account of-this controversy and other various aspects of DES, refer to Davis (1978), Diffie and Hellman (1977), Ehrsam et al. (1978), Konheim (1981), Merkle and Hellman (1981), Morris et al. (1977), Moms (1978), and Sugarman (1979). A discussion of the time-memory trade-off in cryptanalysis with special reference to DES is given in Hellman (1980). While multiple encryption in general need not increase security (Merkle and Hellman, 1981), recently Asmuth and Blakley (1981) have described a novel way to combine two different cryptosystems to produce a third one which is such that breaking the composite system is equivalent to breaking the two constituent systems. 1.4 Public Key Encryption System: Basic Concepts

The basic idea of the public key system was originally proposed by Diffie and Hellman in 1976a and it goes as follows. Given a set of n users


61

in a communication network, the set of all possible communicants grow as n2. The classical approach to ensure security of communication between all these communicants would necessitate the distribution of secret keys between all these communicants. Since the number of keys also grows as n 2 , for really large n this approach could be prohibitively expensive and time-consuming. The public key concept alleviates some, if not all, of the problems of the key distribution problem. The idea of this system is similar to the use of a telephone network using the telephone directory, except that each conversation or data exchange must be made secure. Thus each user in a public key system picks his own encryptiondecryption pair, publishes the encryption algorithm in a directory with his name and address, but keeps the decryption key secret. If a user B wants to send a message to user A, user B simply looks up the directory and encrypts the message with A’s encryption algorithm and sends it to A. Using his own secret decrypting key, A then deciphers the message. In other words, any user B without prior exchange of keys (with or without even knowing A) can send a message in secret to A. For this program to be feasible, it is necessary that each user must, with relative ease, be able to find the (E, D)pair. (Notice a slight change in our notation: the subscripts of E and D,which are keys, are suppressed for convenience.) The basic idea of the public key system requires that E be made public but that each user keep D to himself. This requirement, in turn, necessitates that it should be computationally infeasible (under the assumption of finite resources) to recover D from public knowledge of E. More formally, in a public key system, we have the following:

(Pl) E is a one-to-one transformation and if C = E ( M ) , then D ( C ) = M. (P2) The pair (E, D)can be easily found and both E(M) and D(C) can be efficiently computed. (P3) Knowing E, it should be computationally infeasible to discover

D. We urge the reader to compare conditions Pl-P3 with C1 and C2 of the classical systems discussed in Section 1.3. The ease of generating easily implementable pairs (E, D)but the infeasibility of recovering D from the knowledge of E lies at the heart of these systems. While it may seem impossible at first sight to meet these requirements, it should be quite gratifying to learn that within the short time after the announcement of these basic ideas, a number of public key systems were invented. Today there are examples of at least four different families of these systems. A detailed description of these examples is the principal theme of this ar-

62

S.LAKSHMIVARAHAN

ticle. In Section 1.5 we analyze the preceding requirements with a view to gain further insight into the design of these systems. 1.5 Mathematical Formalism for Public Key Cryptosystem

We begin by introducing some definitions. Let f :x + y be a function with f as its inverse: f is called a one-way function if f ( x ) is easily computed but f - ' ( x ) is hard or infeasible to compute. To get a feel for this definition, consider the following examples. Multiplication and division are inverses of each other, but multiplication is comparitively easier than division. (This is perhaps the reason why children learn to multiply much earlier than divide.) Given a polynomial f ( x ) = a2x2 + alx + a o , it is far easier to computef(x) for any given x than it is to find x for any given (real) y such that f (x) = y . It is this lack of symmetry between the computations off ( x ) andf-'(x) that the definition of a one-way function tries to capture. The key words in this definition are easily computed and hard or infeasible to compute. In an attempt to formalize these intuitive concepts of easy and hard, we again turn to computational complexity theory (a young and active branch of theoretical computer science) (Aho et al., 1974; Garey and Johnson, 1979). According to this theory, a function is said to be easy to compute if there exists an algorithm (or a program) which computes it in a number of steps bounded by a polynomial in the size of the input. For example, two square matrices each of order n can be multiplied using n3 multiplications and n2(n - 1) additions. A set of n distinct integers can be ranked in ascending order in no less than log2n! = n log2 n pairwise comparisons. On the other hand, a function is said to be hard to compute if there is no known algorithm which can compute it in a number of steps bounded by a polynomial in the size of the input. Let X = {XI , x2 , ..., x , } be a set of n Boolean variables and let P be a proposition in the conjunctive normal form over these variables; that is,

P

=

CI A C2 A

A C,

where Cj = (Uil v Ui2v

v uikj)

Us E { x i , iili = 1, to, n}, for s = il i k . , kj > 2 for e a c h j = I , to, m , J and A and v refer to the well-known logical AND and OR operators. To test whether P is satsifiable, that is, whether there exists a truth assignment over X such that P is true, we need to examine, in the worst case, all the 2" possible truth assignments over X.No algorithm with a polynomial bound on the number of steps which can test for the satisfiability of P is known to date. Accordingly, this is an example of a hard problem. With this definition of easy and hard, now it is simple to see that if we require E to be a one-way function, it should be easy to employ E but hard to

63


discover D from E, thereby meeting condition P3. But the catch is if D is hard to find, it should be hard for everybody, including the user who wants such an E. In other words, with E being a one-way function, how can we satisfy condition P2? To this end, the following definition is introduced. A one-way function f is called a trapdoor one-way function if (1) f ( x ) is easily computed for all x and (2) there exists certain hidden or trapdoor information using whichf-I(x) is easy, otherwisef is hard to compute. Thus requiring E to be a trapdoor one-way function and keeping that trapdoor information secret will meet the stringent requirements of conditions P2 and P3; in other words, trapdoor one-way functions provide a formal framework for the design of public key cryptosystems. At this point a novice might rightly wonder as to why one should settle for this definition, which identifies easiness with a polynomial bound when, after all, 2" < nlOOeven for n = 29. The class of all problems which have a polynomial bound on the number of steps needed for solution is collectively known as the P (for polynomial) class. The importance of this class stems from a variety of angles. First of all, complexity theory primarily emphasizes asymptotic complexity, that is, for n -+ m. Thus 2" < nlOOcan be true only for finitely many values of n and there are infinitely many values of n for which nlOOwill be smaller than 2". In other words, asymptotically polynomial algorithms require less computation compared to nonpolynomial (that is, exponential) algorithms. Furthermore, the polynomial class of problems has a very nice invariance property with respect to the choice of basic models of computation-the Turing machine model, the random access machine (RAM) model, etc. That is, if a problem can be solved in polynomial time using one model, it can be shown that the same holds true for other models (Ah0 et al., 1974). Also, almost all the polynomial algorithms that are known to date are of the type n 2 , n 3 , n log n , etc. Thus, on a machine that takes 1 X lop6sec for each basic operation, a problem of size n = 71 13 can be solved in 1 hr using an algorithm of complexity n3. On the same machine for a problem requiring 2" computations, in I hr, n is only 38. Supposing that the time for a basic operation reduces to I x lop9sec due to technological advances, then the size of the problem that can be solved by an n 3 algorithm increases 10fold, but for the 2" algorithm the size increases from n = 38 to n = 48. In other words, the class of problems which are not known to have polynomial algorithms are inherently difficult to solve. This latter class, among other things, contains large problems of great practical significance, such as the traveling salesman's problem and the satisfiability problem. In contemporary complexity theory, the class of problems that are not known to possess polynomial algorithms has been isolated and is called the N P class. This class contains problems which are solvable in polynomial time, but under unlimited parallelsim. The N P stands for nondeter-

-'

64

S. LAKSHMIVARAHAN

ministic polynomial, where the word nondeterministic is essentially used to imply the need for unlimited parallelism if the computations have to be finished in polynomial time. The satisfiabilityproblem is perhaps the most notorious problem in this class. If we have the ability to generate all the 2" truth assignments in parallel, satisfiability can be tested in polynomial time, since given a truth assignment, satisfiability can be tested in a number of steps proportional to mn, where m is the number of classes and n is the number of variables. But recall that unlimited parallel computation is not a reality. Also, a little reflection reveals that to test for satisfiability we need to exhaust all the 2" possible truth assignments only in the worst case. Herein lies the basic characteristic of an NP problem, namely, that any suspected solution to this problem can be checked in polynomial time, but the search for the right solution must continue until it is found. Also, all the known methods for converting an NP problem to one that is suitable for serial computation invariably lead to a problem of exponential complexity. Before describing these systems, we present the basic mathematical tools for the analysis of these systems. 2.

Mathematical Preliminaries

Ever since the announcement of the basic concepts of public key systems and the use of trapdoor one-way functions to realize them by Diffie and Hellman in 1976b, a number of examples of this system came into existence. Today there are at least four different classes of public key systems that are widely known. Three of these examples rely very heavily on basic results from number theory, especially modular arithmetic, and one example is based on a special class of linear error-correcting codes known as Goppa codes (McEliece, 1978). While there are a number of excellent textbooks on number theory (Vinogradov, 1961; Uspensky and Heaslet, 1939; LeVeque, 1961) and coding theory (McEliece, 1977; Peterson, 1961; Berlekamp, 1968), with a view to make this article self-contained we present in this section most of the basic results that are necessary for a thorough understanding of the mechanics of the existing examples of public key systems. The reader who is mainly interested in the examples may, without loss of continuity, go directly to Section 3. In this section all the lowercase letters a through z, with or without subscripts, refer to integers. 2.1

Divisibility

Given m , n, and n # 0, if there exists k such that m = nk, then we say m is divisible by n and is written as nlm. Notice that if n(m,then -nlm. If n


65

does not divide m , we denote this by nlm. If nlml and nlm2, then n is known as the common divisor of ml and m2. Then nl(mIx + m2y) for all x and y . If n(m and m(k,then nlk. Also, if n # 0, then nlO and Ilk for every k . If {dl , d2, ..., dk} is the set of all common divisors of m and n , then d = maxi{ldiI} is defined as the greatest common divisor (GCD) of m and n and is denoted by (m,n), where 1x1 refers to the absolute value of x. Clearly, (m,n) > 0 and is unique, for if d and d’ are two GCDs for m and n , then dld’, since d’ is the GCD, and likewise d’ld, from which it follows that d = d‘. We readily see that (m, km) = Iml, ( m , 0) = lml, ( m , I ) = 1 , and ( m , n ) = (Iml, In/). Any integer m > 1 is called a prime if 1 and m are its only divisors; otherwise m is called a composite. Clearly 2 is the only even prime number. If (m,n) = 1, then m and n are said to be relatively prime to each other; 1 is neither a prime nor a composite. Property 2.7.7. Every integer rn is uniquely represented in terms of a positive integer n such that m = nq + r , where 0 5 r < n. In other words, n q is the largest multipleiof n that is less than or equal to m . If m = nql + rl is any other representation, by subtracting we see that 0 = n(q - 41) + ( r - rl); that is, n ) ( J r- rll).But Ir - rll < n ; hence r = rl and q = q 1. Here q is called the quotient and r is called the residue or remainder. If m = _+42and n = 4; then -42 = -11 X 4 + 2, but 42 = 4 X 10 + 2. For other equivalent representations allowing negative remainders, see Uspensky and Heaslet (1939). Property 2.7.2. If m = nq + r with n > 0, then any common divisor of m and n is also a common divisor of n and r. In particular, ( m , n) = ( n , r). If m = n, then the previous conclusion is trivial and so let m # n . If alm and aln, then there exists k l f kZ such that m = kla, n = kza, and ( k l &)a = r. Thus r is a multiple of a. If n = rib and r = r26, then m = ( r l q+ r2)b. Thus all the divisors of m and n also divide n and r. Hence ( m , n) =

(n,4 . Algorithm 2.7.7. Given two positive intergers m , n with n < m , ( m , n ) can be easily computed according to the following scheme: m = nql + r l , n = r1q2 + r2, rl = r2q3 + r3, rk-3 = rk-2qk-1 + rk-l rk-2 = rk-lqk + rkq rk-1 = rkqk+l 9

O r2 > -.’> rk-1 > rk > rk+l = 0, the algorithm needs at most n steps where in each step one division is performed, that is, k + 1 5 n.

However, by exploiting the properties of the remainders a better upper bound on k + 1 can be derived. Consider rs = rs+1qs+2+ rs+2,where 0 < rst2 < r,+I < r , . While rs+l < r , , it could well be that r,+l = rJ2, r,+] > rs/ 2 , or rs+l < rJ2. If rs+l > rJ2, then (r.J2)(2 - qs+2)> rs+2.But q s + 2 2 I and rs+2 > 0 imply that rs+2 < rJ2. In the other two cases, clearly r,+2 < r s / 2 . Thus each nonzero remainder rs+2is less than rJ2. Property 2.1.3 summarizes this analysis. Property 2.7.3. The number of divisions in the Euclid’s algorithm is never greater than 2 log2 n. The GCD algorithm has been analyzed for well over a century, the earliest attempts being due to Lam6 (1775-1870) and Kronecker (18231881). For a very comprehensive and thorough treatment of various GCD algorithms, see Knuth (1966), where, in addition to the upper bound, the average number of divisions required is also discussed. Property 2.7.4. If (m, n) = k , then there exist integers x and y such that mx + ny = k. This can be easily seen by writing rl = m - nql , r2 = mq2 + n(1 + qlq2),and so on. Algorithm 2.1.2, in addition to finding ( m , n), will also find x and y such that mx + ny = (m, n). AIgorithm 2.7.2. Let (a1 , a 2 , u3), (bl ,b2, b3), and (cI, c2, c3)be three sets of vectors. Step 1 . ( a ~az, , a3) + (1, 0, m) and Step 2 . If b3 = 0, stop. Step 3. Set q = [a3/b3] and (CI

9

c2

7

c3)

+

@I,

b2, b3) +-(0,1 ,

4.

a2 a,) - (bl , b2, b3)q a2 a31 +- (bl 62 63) (bl bZ1 b3) + (c11 c2, c3) (01 1

3

(01 9

9

9

9

7

and go to Step 2. I 1x1 (Ixl) denotes the largest (smallest) integer less than (greater than) x; [XI denotes the integer part of x.


67

Algorithm 2.1.2 is known as the extended Euclid's algorithm and when the algorithm stops, al = x, u2 = y, and a3 = (m, n ) . Let m = 42 and n = 4: Q

at

10

I 0

2

I

a3

b,

0 4 2 I 4 -10 2

0

1

1

-10 21

a2

-2

b2

b3 4 2 0

Clearly, x = 1 , y = -10, and (m, n ) = 2. Again, the number of operations (divisions, etc.) required by Algorithm 2.1.2 is proportional to log2 n. Given m and n , the least common multiple of m and n is denoted by LCM(m, n) and LCM(m, n ) = Irnn)/(m, n ) . Thus, LCM(m, n ) L 0, and both n and m divide LCM(m, n ) . Also, if aln and bJn,then LCM(a, b)(n. Finally, GCD(ml,m2 m,) = GCD[ml , GCD(m2, mj)] and LCM(m1, m2, m3) . GCD(m1, m2, m3) = Irnlrntm3l. Extension to more than three numbers is obvious. Property 2.7.5. Every integer n > 1 is uniquely expressible as a product of the powers of prime numbers; that is, n = pylp~2pp-..pfk, where 2 5 P I < p2 < p3 < -.-< Pk and a;L 0 are all integers. This factorizability can be easily proved by induction and the uniqueness follows from the fact that if (n,y) = 1 and xlyz, then xIz (LeVeque, 1961). Given an arbitrary integer n , this property does not provide an algorithm for finding its factors. We shall discuss a typical algorithm for this problem later in this section. We hasten to add that there is no known efficient algorithm for prime factorization; in fact, one of the examples of the public key system due to Rivest et ul. (1978) very crucially uses the fact that given the prime factors, the number can be easily found by multiplying them, but given a number, it is very difficult to prime factorize. In other words, multiplication and prime factorization constitute a one-way function. Various methods for prime factorization and primality testing are discussed in Section 2.5. Property 2.7.6. Given a positive integer n, the total number of numbers less then n and relatively prime n is denoted by 4 ( n ) and is called the Euler function. Clearly2 442) = I , 4(3) = 2, 445) = 4, but 4(6) = 2. 4 ( n ) for n > 2 is always even, for if a < n and (a, n ) = 1, then so is ( n - a, n ) = 1 . +(p) = p - 1 if p is prime. The following properties of the 4 function are easily verified (LeVeque, 1961 ; Vinogradov, 1961):

(1) If n = mlm2 and (ml , m2) = 1, then 4 ( n ) = 4(m1)4(m2). (2) If n = pa, where p is prime and a 2 1 , then +(p) = pa - pa-'.

* 4(1) is defined as I .

68

S. LAKSHMIVARAHAN

(3) If n = p l ' p f 2 ... pzk, where pi are prime and ai2 1, then +(n) n(l - p i l ) ( l - p ~ ' ... ) (1 - p i ' ) . (4) +(m) 5 m - 1 and +(m) = m - 1 if and only if m is a prime.

=

From these properties, it is clear that +(n) can be easily calculated for any arbitrary n if its prime factorization is known. In other words, computing +(n) for large n is at least as hard as prime factorization. Also, given n, there is no known formula for solving +(x) = n. It should be interesting to note that the public key cryptosystem due to Rivest ef a!. (1978) exploits the properties of the r$ function in the design of a one-way function. 2.2 Congruences

Let m > 0. Given any integer a , from Property 2.1.1 we know a = mq + r , where r , known as the residue, can only take on values 0, 1,2, ..., m 1. If a and b are two integers having the same residue when divided by m , then we say a is congruent to b and it is written as a = b(mod m) and m is called the modulus; a f b(mod m ) denotes that a is not congruent to b(rnod m). It is easily seen from this definition that a = a(mod m) for all a (reflexivity), a = b(mod m) implies b = a(mod m) for all a and b (symmetry), and a = b(mod m) and b = c(mod m ) implies a = c(mod m) for all a, b, and c (transitivity). Thus congruent under modulo m defines an equiualence relation. Accordingly, two integers are equivalent if they have the same residue under modulo m. We shall denote the set of all integers equivalent to a(mod m) as [a] = {xlx = a f km, k = 0, 1,2, ...}, a = 0,1,2, ..., m - 1. The following properties are easily verified. Property 2.2.7. Let a = b(mod m) and c = d(mod m). Then we have the following: a 2 c = b 2 d(mod

m).

ac = bd(mod m). an = bn(mod m) for any integer n 2 0. ka = kb(mod m ) for any integer k 2 0. ka = kb(mod mk) for any integer k 2 0. If s is a common divisor of a, b, and m, then al = bl(mod m1), where a = als, b = bls, and m = mls. I f a = b(mod mi)for i = 1,2, ..., k, thena = b(mod m), where m = LCM(mt , m2, ..., mk)If a = b(mod m) and m = mlm2 mk, then a = b(mod mi) for each i = I , 2, 3, ..., k. If a b(mod m) and (r, m ) = 1, then a/r = (b/r)(mod m); that is, division in modulo arithmetic is restricted to those divisors which are relatively prime to the modulus.


69

The set CRS = (0, I , 2, ..., m - l}, of all distinct residues under mod m is called the complete residue system (CRS). The set RRS = {xlx E CRS and (x, m) = 1) of all residues which are relatively prime to m is called the reduced residue system. Clearly, c#J(m) = [RRSl < (CRSI = m for all m 2 2. Thus if m = 6, RRS = { I , 5). If ( a , m ) = I and x runs through the elements in CRS(RRS), then for any integer b, (ax + h)(mod m) also runs through CRS(RRS), perhaps in a different order. In other words, if we define A = (zlz = (ax + b)(mod m ) , ( a , m ) = 1 and x E CRS), then A = CRS, and similarly for RRS. Property 2.2.2. If ( a , p) = 1, where p is a prime, then ap-l = l(mod p ) . Thi is known as Fermat's theorem. Since ( a , p) = I , y = &mod p ) E RRS i f x E RRS. Therefore, flL-l'yi = n;=-l'axi(mod p ) , where xi and yi are nonzero elements of RRS. Thus ap-l = l(mod p ) , since fl!=-~'y; = flL=-l'x,'xi and cancellation is allowed, as these products are relatively prime to p. An extension of Fermat's theorem to nonprime modulus is known as Euler's theorem, which is also proved along the same lines as Fermat's theorem, and it is given in Property 2.2.3. Property 2.2.3. If (a, m) = 1 and m > 1, then a""') = l(mod m ) (Euler' s theorem). The converse of Fermat's theorem, namely, if ( a , m ) = 1 and urn-' = l(mod m) implies m is a prime, is not true, as the following counterexample shows: 4' = 4(mod 15) if i is odd and 4' = l(mod 15) if i is even. Thus 414 = l(mod 15) and (4,15) = 1, yet I5 is not a prime. Likewise, 390 = l(mod 91) and (3,91) = 1 and 91 = 7 x 13. Since c#J(m)5 m - 1, with equality holding if and only if m is a prime, the true converse of Fermat's theorem, stated below, also suggests an algorithm for primality testing. Property 2.2.4. Given m > I, if there is an x such that xm-' = l(mod m) but d r n - ' ) ' P f I(mod m), then m is a prime, wherep is a prime divisor of m - 1. Obviously, we need to test only for those x E RRS corresponding to the given m, but for each such x we must test through all the prime divisors of m - I . Thus even though the algorithm is straightforward, it is often very time-consuming. An efficient algorithm for primality testing is needed for the implementation of the public key system due to Rivest et al. (1978) referred to earlier. We shall discuss other algorithms for primality testing later in this section. Property 2.2.5. Solution to linear congruence, that is, finding y such that ay = I(mod m ) , is of extreme importance in the design of public key systems. Solving for y is essentially dividing the congruence by a , and in view of Property 2.2.1, we cannot divide by a unless ( a , m ) = 1, a condition which we shall assume to be true, and under this condition, the solution is unique. From Euler's theorem, a'#'("')= aa&"')-'= I(mod m), we obtain a closed form for solution, y = However, this is often

70

S.LAKSHMIVARAHAN

not very useful, since given any m ,computation of $(m)is not easy. For a more practical method, observe that the preceding congruence is equivalent to the equation mx - ay = 1 for some integer x. Given a and m [where ( a , m) = 1 and a < m],the integers x and y satisfying this equation can be readily obtained by using (extended Euclid's) Algorithm 2.1.2 in a number of steps proportional to log2 m. As an example, if 2y = l(mod 21), then $(m)= 12, 212 = 195 x 21 + 1 = l(mod 21), and y = 2'' = 97 x 21 + 11 = ll(mod 21). We urge the reader to check this answer using Algorithm 2.1.2. The problem of finding y when ay = b(mod m),b # 1 and (a, m) = 1 is a bit more involved. This is equivalent to mx - ay = b. A necessary and sufficient condition for the solution to exist is klb, where k = (a, m).In this case, there are k different solutions. T o find all these solutions, let a = a'k, b = b'k, and m = m'k, where ( a ' , m')= 1. Solve m'x' - a'y' = 1 as before and let xt, and yt, be the unique solution. Then xo = b'xO,yo = b'yo is the solution to m'x - a'y = b', and yo + (i - 1)m' for i = 1, 2, ..., k constitutes the k distinct solutions to mx - ay = b or ay = b(mod m).In other words, (extended Euclid's) Algorithm 2.1.2 provides a very practical way for solving the linear congruence. A closed-form expression for the solution of ay = b(mod m) can also be obtained by involving Euler's ' . an example, if 4y = 2(mod 42), theorem and it is given by y = b ~ " ~ ) -As then y = 11 and y = 32 are the two solutions. Property 2.2.6. If y = bi(mod mi), i = 1, 2, ..., k, and ( m i ,mj)= 1 for all i # j , then how do we solve for y? It can be shown that the solution to these simultaneous congruences exists and is unique up to modulo M = mlm2 ... mk (LeVeque, 1961). To find the solution define M , = nj+,mj, where s = 1, 2, ..., k. Let M,;' be such that M,M;' = l(mod m,).Since (m,,M,)= 1, M,;' exists and is unique up to modulo m,. Let yo = ECik,~MiM;'bi.Since m, divides all Mj ,j # s, from the definition of M;' it follows that ydmod m,)= b,T; that is, y = yo is the solution to y = bi(mod m,)for all i = 1, 2, ..., k. Now, the original system of k congruence is equivalent to y = yo(mod mi)and from Property 2.2.1 these latter congruences are equivalent to y = yo(mod M). In other words, y = yo(mod mi), i = 1, ...,k, and y = yo(mod M) have the same solution. As each bi runs through its respective complete residue system, yo runs through its complete residue system with respect to M. This result is known as the Chinese remainder theorem. As an example, if y bl(mod 5) and y = b2(mod 7), then M = 35, MI = 7, M2 = 5 , M;' = 3, MT1 = 3, and yo = 21bl + 15b2. If bl = 3 and b2 = 4, then yo = 123(mod 35) = 18. Algorithm 2.2.7. 'Euler's and Fermat's theorems involve raising an integer to an integer power. Furthermore, the test for primality of a , given

71


a number m (refer to Property 2.2.4), also involves raising an integer to an integer power. This raises a fundamental question as to what is a good way to compute y"? This algorithm, known as a binary algorithm for computing yfl, is quite simple and very easily implemented: Step 1 . Step 2. Step 3 . Step 4 . Step 5 . Step 6.

Set a = n , b = 1, and c = y . Set d = 1 if a is even and d = 0 if a is odd. Set a If d = 1, go to Step 6. Set b = b x c . If a is zero, stop and b is the answer. Set c = c X c and go to Step 2.

=

1~121.

If B(n) is the number of 1's in the binary expansion of n , then clearly B(n) 5 [log2nl. The total number of multiplications needed to compute y" by this method is [log2 n ] + B(n) and hence is proportional to log2 n . Adaptation of this method for computing y"(mod m ) is obvious. For a very detailed and thorough discussion of various methods for computing y R , see Knuth (1966). 2.3 Primitive Roots and Discrete Logarithms Property 2.3.7. Given a and m, in order for there to exist an integer h such that ah = l(mod m ) , it is necessary and sufficient that (a, m ) = 1 , for if ( a , m) = I , then h = +(m) by Euler's theorem. Now let (a, m) = d > 1 and ah = I(mod m ) for some h 2 1. Then d J a ,dlah,and dlm. From ah = 1 + mk for some integer k, it follows that dl 1, a contradiction. Hence d = 1 . If h is the smallestpositiue integer such that ah = l(mod m ) , then a is said to belong to the exponent h to mod m or h is called the order of a(mod m ) and is written as ord", Property 2.3.2. If h = ord", then a, a * , a 3 , ..., ahare all distinct mod m , for if I 5 s < r 5 h and a" = ar(mod m ) imply mIas(ar-s- 1). But ( a , rn) = 1 and hence ( m , a") = 1. From this it follows that rnl(a'-" - l ) , which when rewritten becomes ar-s = l(mod m ) and 0 < r - s < h. This contradicts the definition of h. Hence as and a' are distinct for all 1 5 s < r 5 h. Property 2.3.3. If ak = l(mod m ) , then ordklk; consequently, ordkl&m). Thus for any s, as = a'(mod m ) , where s = k - o r d i + r for some k 2 1 . In other words, the exponent or index of the powers of a under mod m are to be computed under mod ord; . Property 2.3.4. If g is an integer such that ( g , m ) = 1 and h = +(m) is the smallest integer for which g h = l(mod m ) , the g is called the primitive root of m. In other words, g is a primitive root of m if g belongs to the

72

S. LAKSHMIVARAHAN

TABLEV

EXAMPLES OF

2 3 4 5 6

I 2 2 4 2

I 2 3 2, 3

1 1 I 2

5

1

PROPERTY

2.3.4

7 8 9

6 4 6 4

10

3, 5 None 2, 5 3, 7

2 2 2 2

exponent +(rn). From Property 2.3.2, the g' terms are all distinct mod rn for i = 1, 2, ..., +(rn), which in turn implies that { g , g2, g3, ..., g$("')} constitutes the reduced residue system (RRS) for modulus rn. Furthermore, g" = gr(mod rn) implies and implied by s = k+(rn) + r for some integer k. Table V illustrates a number of examples. Clearly, when rn = 6, the only value of a for which (a, 6) = 1 and a > 1 is a = 5; 446) = 2 and 5' = 5(mod 6) and 52 = 25 E l(mod 6). Thus 5 is a primitive root for 6. However, if rn = 8, +(m)= 4 and the reduced residue system for mod 8 is (1, 3, 5, 7). But 32 = 9 = l(mod 8), 52 = 25 = l(mod 8), and 72 = 49 = 1 mod 8. Thus ordi = ord; = ordg = 2 < 4 = +(rn), from which it follows that 8 does not have any primitive root. Also, the primitive root of any modulus rn 2 3, if it exists, is greater than 1. All these examples bring to focus the following questions: Which of the positive integers have primitive roots and how many of these are there? Property 2.3.5. If ord; = h, then ord: = h/d, where d = (h, n). From the definition of h and 1 = ah(modrn) -= audd(mod rn) = an)h/d(modrn), we obtain h'l(h/d),where h' = ordg by definition. From 1 = (aa)h'(modrn) = anh'(modrn), it follows that hlnh'. Thus (h/d)l(n/d)h'.But (hd, nld) = 1; hence (h/d)lh'. Combining these, we see that h = h'd. Stated in other words, ord; = ord: = h if (h, n ) = 1. From Property 2.3.3, we need to consider only those values of n < h. For a given h, there are only +(h) distinct values n can take such that (n, h) = 1 and n < h. Furthermore, from Property 2.3.2, it follows that a" for all these values of n are distinct mod m. Thus if a belongs to exponent h to modulus rn, 4(h) distinct elements also belong to the same exponent h to modulo rn. Specializing the earlier agrument for primitive roots we can now answer the question on the number of primitive roots if they exist. Thus if g is a primitive root of mod m , then ordg = 4(m) for those (r, +(m))= 1 and r
1. If a = u1u2u3 .--a,, then J ( a , m) = J ( a l , m)J(a2, m) J(a,, m). If a = cb2, then J(a, m) = J(c, m). If n and m are odd and relatively prime integers, then J ( n , m) = ( - 1 ) b - 1V21lh- IY2lJ(m, n).

As an example, let m = 25. Then J(3,25) = L2(3,5 ) = (- 1)* = 1 by (1). But it can be checked by direct computation that 3 is not a quadratic residue of mod 25. But J(16,25) = J2(4,25) = L4(4, 5 ) = + 1, and it can be readily checked that 16 is a residue with 4 and 21 as its square roots. In other words, (1) needs to be applied with caution. Thus it is clearly necessary that L(a, p i ) = +1 for a to be a quadratic residue of m = pplp;* ..p:k. On the other hand, J(a, m) = - 1 implies that a is not a quadratic residue of m. Now, we state the general theorem for quadratic residues. Property 2.4.5. The necessary and sufficient conditions for a general congruence x2 = a(mod m), rn = 2a~p;2p;3 p z $ l l , 2 < p2 < p3 < < Pk+l, ai 2 1, i = 2, ..., k + 1, and GCD(a, m) = 1, to be solvable are the following: (1) a = l(mod 4) if a I= 2 or m = 4(mod 8) and a = l(mod 8) if a,2 3 or m = O(mod 8). ( 2 ) L(a, p i ) = 1 for i = 2 , 3 , ..., k + 1.

When these conditions hold, there are exactly 2k+6solutions, where k is the number of odd primve divisors and 0 8=[1 2

if a t = 0, 1 or m = l(mod2), n = 2(mod4) if a l = 2 or m = 4 ( m o d 8 ) if a l L 3 or m = O(mod8)

The proof of this theorem is rather involved and we refer the reader to LeVeque (1961) and Vinogradov (1961). If J(a, m) is defined as ? 1, depending on whether a is a quadratic residue or nonresidue of m, then one cannot have the quadratic reciprocity law for J(a, m). Since this latter law is very useful, J ( . , .) is defined using these properties.


79

This theorem is only concerned with the conditions for the existence and the number of solutions. Finding the actual solution is again nontrivial; it involves actually solving for x2 = a(mod 2"1) and x2 = a(mod pp'). Consider first x 2 = a(mod 2"1). If a I = 1 and a is odd, then x = l(mod 2) is the only solution. If a I = 2 and a = I(mod 4), then x = ? I(mod 4) are the only two solutions. If a , 2 3 and a = l(mod 8) and if x = b is a solution to the preceding congruence, then so are x = -b, x = b + 2"l-*, and x = -b 2"1r1(mod 2"1). For a method to find b, see Uspensky and Heaslet (1939). It can be shown (LeVeque, 1961) that for odd prime pi, a is a quadratic residue ofpp'if and only if it is a quadratic residue of pi.However, except for some special cases, there are no general methods for solving for x2 = a(mod p ) for large prime p . Thus if p = 4n + 3, by Euler's criterion (Property 2.4.2), a2"+I= l(mod p ) . Hence x2 = a2n+2= a(mod p ) , from which x = ?a"+l(mod p ) follows, and similarly for the case p = 8n + 5. Once the solutions to x 2 = a(mod 2"1) and x2 = a(mod p?) are known, they can be combined using the Chinese remainder theorem (Property 2.2.6) to obtain those of x2 = a(mod m).

+

2.5 Prime Factorization and Primality Testing

We have already noted that every positive integer is uniquely expressible as a product of powers of primes. However, the problem of actually finding the prime factors, given a number n, is computationally very hard in the sense it is excessively time-consuming. In this section, we shall describe some algorithms for both prime factorization and primality testing. Algorithm 2.5.7. Given an integer n > 1, we can continue to divide n by successive primes from the list 2 , 3 , 5 , 7 , 1 1 , 13, ..., up until finding the smallest p such that 0 = n(mod p). Now p is a factor and continue the process with n replaced by nlp. If 0 n(mod p ) and n 5 p 2 , at any stage, then the process terminates and n is a prime. Clearly, this method tries out all primes less than or equal to fi.If n I2h - 1 , then the maximum number of divisions needed by the algorithm is of the order of 2h'2,which is exponential, where b is the number of bits in the representation of n. Furthermore, this method also requires a table of primes. One way to circumvent this is to first factor all the powers of 2 from n, that is, express n = 2knl for some k 2 0. Then continue dividing n l by odd integers starting with 3 instead of odd primes. In this way, we need not store the table of primes. The literature on prime factorization is rather extensive and dates back well over three centuries to the days of Fermat and Legendre. Recently,

+

80

S. LAKSHMIVARAHAN

Pollard (1974) announced a factoring algorithm requiring a number of operations proportional to 2b’4.Rivest et al. (1978) quoted in their paper a factorization algorithm due to R. Schroeppel requiring only 2SQRT(b10gb), where SQRT(x) is the square root of x. While this algorithm also needs exponential time, it is much faster than all the algorithms known to date. For a comprehensive coverage of the factorization algorithms, refer to Knuth (1966) and a survey article by Guy (1975). Indeed, every algorithm for prime factorization can be used for primality testing. However, there are a number of special algorithms for the latter problem. We have already seen one such algorithm in connection with Euler’s and Fermat’s theorems in Property 2.2.4. We now describe a Monte Carlo method for testing primality (Solovay and Strassen, 1977). Algorithm 2.5.2. Step I . Given n, generate a random integer a with a uniform distribution on the set (1, 2, ..., n - 1). Step 2. Compute d , the greatest common divisor of a and n (using Euclid’s algorithm 2.1.1). If d > I , then n is a composite. Step 3. Compute = m(mod n ) (using Algorithm 2.2. I). Step 4. Compute J(a, n ) (using Property 2.4.4).’ Step 5. If J ( a , n) = m,n is a prime; otherwise n is composite.

If n is indeed a prime, then from Euler’s criterion (Property 2.4.2) and the definition of Jacobi’s symbol, it follows that this algorithm leads to the correct decision. If n is composite, then the probability of m = J ( a , n ) , that is, of erroneous decision, is at most 0.5. Thus if we repeat the preceding procedure independently k times and if in all these k trials the algorithm decides that n is a prime, then n is a prime with a probability of at least 1 - 2-k. Thus by proper choice of k , for large n one can make this probability as close to unity as desired. Furthermore, it can be shown that the number of operations in each of the steps of this algorithm is proportional to log2 n and hence the overall time is proportional to k log2 n . 2.6 Finite Fields

+

A set of F endowed with two (binary) operations (addition) and x (multiplication) is said to be afield if the following is true. Let a , b, and c be arbitrary elements belonging to F: From Property 2.4.4, it can be seen that the computation of J(a, n) is very similar to that of the GCD. For further details see Williams (1980).


81

Closure. a + b and a x b belong to F . Commutativity. a + b = b + a and a x b = b x a . Associativity. ( a + b ) + c = a + ( b + c ) and ( a x b ) x c = a x (b x c). Zero and unit elements. There exist elements 0 (called zero) and I (called unit) both in F such that a + 0 = a , a x I = a, and a x 0 = 0. Inverse. There exists -a (called additive inverse) and b-' (called multiplicative inverse) such that a + ( - a ) = 0 and b x b - 1 = I , provided that b # 0. Distributivity. a X ( b + c) = a x b + a X c and (a b) x c = a x c+bxc.

+

Clearly real numbers, rational numbers, and complex number are all fields. If the set F satisfying these properties is infinite, then it is an infinite field; otherwise, it is called a finite field. If q is a prime, the complete residue system mod q constitutes a finite field where both addition and multiplication are mod q. The latter field is often denoted GF(q). Let F be a field. Consider apolynominalf(x) = an + alx + a2x2 + + a,x", where all the coefficients are elements of F. If a,, # 0, thenf(x) is a polynomial of degree n . If a,, = 1, it is known as a monic polynomial of degree n. Polynomials can be added and multiplied in the usual way. If f ( x ) and gfx) are two polynomials of degree n and m ,with n > m , thenf(x) can be expressed asf(x) = g(x)h(x) + r ( x ) , where h(x) is called the quotient polynomial and r ( x ) is the remainder polynomial whose degree is less then m. If r(x) = 0 (the zero polynomial), then g(x) is said to dividef(x) or g(x) is a factor off(x). A polynomial f ( x ) is said to irreducible if it is not divisible by any polynomial with coefficients in F and of degree less than n and greater than 0. Irreducible polynomials play a role which is very similar to that of a prime number. Let g(x) be an irreducible polynomial of degree m (>O) with coefficients in GF(9) for some prime number q. Then the residue classes mod g(x) constitute a field (Berlekamp, 1968). This field is called the Galoisfield, which is an extension of the groundfield GF(q). Since the general form of the residue mod g(x) is a polynomial of the type bo + b l x + bzx2 + .--+ bm-,xm-l with each of the coefficients taking values in GF(q), there are a total of q" distint residue classes. Often this extension field is denoted as GF(q"). As an example, if q = 2, m = 2, and g(x) = x2 + x + 1, then GF(22)consists of (0, 1 , x, x + I}. Finite fields, especially GF(q"), play a very fundamental role in the analysis and design of error-correcting codes (Berlekamp, 1968; Peterson, 1961). There are very efficient ways for test-

82

S. LAKSHMIVARAHAN

ing irreducibility (Berlekamp; 1968). Furthermore, it can be shown that over any finite field a randomly selected polynomial of degree m (for large m) is irreducible with probability very close to llm.

3.

Examples of Public Key Cryptosystems

In this section, we describe four different classes of public key cryptosystems that came into existence since the inception of the basic ideas in 1975-1976. It is assumed that there are L (22) users in a network environment. Our aim is to describe examples of public key encryption algorithms that can provide security of data in communication between different pairs of communicants. In each of the examples every user picks his own ( E , D ) pair, where E is designed as a (trapdoor) one-way function with the trapdoor information kept secret. 3.1 Algorithms Based on Exponentiation and Discrete Logarithms

Historically, such an algorithm is one of the first examples of a public key (distribution) system6 Diffie and Hellman announced this scheme in 1976 (1976b). In this scheme, it is assumed that the manager or the corn-' pany that operates the basic network picks a large prime number p. Let a be its primitive root. For reasons that will become apparent, the prime numberp is chosen to be orders of magnitude larger than L , the number of users. The numbers p and a are the system parameters and are known to all users. Each user i, i = 1,2, ..., L, independently generates an integer n; randomly under uniform distribution on the set {I, 2, 3, ..., p - 1) and computesf(n;) = m; = afl;(modp), i = 1 , 2, ..., L. Since a is a primitive root of p , an odd prime, clearly f is a one-to-one function from { 1, 2, ..., p - 1) onto itself. User i now keeps n; secret but publishes m iwith his name in a directory. If user j wants to send a message to i, he (userj9 obtains m;from the directory and computes the key kj; (for communication fromj to i) as follows:

k.. ,=

,,(mod p )

=

a"i"j(mod p )

Similarly, user i for his communications to user j computes his key kii as k.. = - m?'( ,I mod p ) = aW(mod p) rJ

The other example of a public key distribution system based on one-way functions i s due to Merkle (1978).


83

Since ninj = n j n i , it follows that k, = kj;. While kii = k j j , users i and j independently compute them. Furthermore, the computation of k , by exponentiation mod p takes at most 2b operations (see Section 2), where b is the number of bits in the binary representation of p. An intruder, on the other hand, wanting to compute kij may obtain mi and mj from the public directory and express

But from Section 2 , it follows that computing the discrete logarithm nj = log, mj or ni = log, m; for larege p, in general, needs b2b'2operations, which is clearly exponential. In other words, the legitimate users can compute the right key in polynomial time, but by proper choice of p it is computationally infeasible for the intruder to obtain k, from the knowledge of mi and mj.Notice that there is no explicit trapdoor involved in this scheme. Having made the recovery of the secret key difficult for the intruder, the question is, How does u s e r j use kji in his encryptioddecryption? In a network environment with more than one user, it is generally very difficult to let each user pick his own encryption-decryption algorithms. The system as a whole will support a class of algorithms where the users will be allowed to pick their own choice of private key. With this constraint in mind, assume that the network uses a classical algorithm. We could let the users i a n d j use k, directly as the key for this algorithm or use k , to encrypt the key to be used in the algorithm. In either case, great caution must be exercised by the manager in the choice of the classical cryptographic algorithm for use in the network, for while the key may be very secure, most of the simple classical cryptosystems, such as polyalphabetic substitution, are vulnerable to ciphered text only attack; that is, they can be compromised using a variety of side information. In view of this, the manager should select product ciphers such as Lucifer and DES. We shall illustrate an example of a network using a DES algorithm where k , is used only for the secret transfer of the keys to be used in the DES algorithm. User j for his communications with i chooses a KDES, a key for the DES algorithm. U s e r j then encrypts KDES using the kj; computed earlier and sends it to user i. Now user i, using his k , , recovers KDES. From then on users i a n d j communicate in secret using the DES algorithm with KDES as the key. In other words, Diffie and Hellman's scheme is essentially a public key distribution scheme rather than a public key encryption scheme. For other classes of key distribution schemes, see Ehrsam et al. (1978), Matyas and Meyer (1978), and Merkle (1978).

84

S. LAKSHMIVARAHAN

To get an idea of the size of the public directory, assume that there are lo00 users and p = 2s21- 1 . Then clearly each mjis 521 bits long and in total the directory needs 5.21 X lo5 bits of memory. For this value of p exponentiation mode p can be done in about 1042 operations, but to compute discrete logarithms, it needs well over lo7*operations. As n; and nj are chosen at random and a is a primitive root, k" ,in principle, can take on any value in the set { 1 , 2, ...,p - 1). Since a larger key implies greater security, one can also require k" to be larger than a prespecified lower bound. We have already seen in Section 2 that if p is a prime of the form 2" + 1 or if p - 1 has small prime factors, discrete logarithms can be computed in a time proportional to (log2 p)*. In other words, primes of these forms must be avoided. Pohlig and Hellman (1978) have shown that if p = 2p I + 1, where p1 is a prime, then the preceding key distribution algorithm is computationally secure. Berkovits et al. (1979) announced a very interesting implementation of DBie and Hellman's public key distribution scheme along with DES algorithm for secure data transmission in a network environment. Thus as long as the computation of discrete logarithms continues to be infeasible, this class of system would continue to be of great cryptographic significance. As an offshoot of their analysis of discrete logarithm problems, Pohlig and Hellman (1978) described an encryption scheme that falls short of being used as a public key system but which has great potential as a classical system. The manager of the system selects a prime p. The message M and its corresponding ciphered text C are such that 1 s M and C s p - 1 . User i picks an integer ei in the range 1 s ei Ip - 2 and relatively prime to p - 1 and computes di such that eidi = l[mod (p - l)]. The encryption algorithm Eei is such that C

=

Eq(M)

Me;(mod p)

and the decryption algorithm Ddi is given by M = Dd;(C) = Cd;(mod p) = Meidi(rnodp ) = M(mod p ) If the encryption key ei is public, since p is known, di, the multiplicative inverse of e i , can be computed using a GCD algorithm in a number of operations proportional to log2 p. In other words, this choice of encryption algorithm does not constitute a good one-way function. On the other hand, if user i can secretly send dito userj (perhaps by registered mail), then both i and j can use Eei and Ddi for their secret communication. For further discussion of this cryptosystem, refer to the article by Pohlig and Hellman (1978); for another interesting variation of the Pohlig and Hellman scheme refer to Simmons (1979). Using essentially the same idea


85

of exponentiation modulo an integer for both the encryption and decryption, but choosing the modulus to be a composite instead of a prime, Rivest et al. (1978) independently arrived at a very elegant public key cryptosystem. We now proceed to describe this system.

3.2 Algorithms Based on Exponentiation a n d Prime Factorization The first algorithm of this type, known as the RSA algorithm (Rivest et al., 1978), proceeds as follows. Each user i (i = 1, ..., L ) picks two very large (of the order of 10%)random prime numbers pi and 4,. User i then makes n i , the product of pi and qi, public but keeps p i and qi secret to himself. For the purposes of finding these large primes, user i may use, for example, Algorithm 2.5.2 for primality testing due to Solovay and Strassen. Since there are no efficient factorization algorithms (see Section 2.5), by so doing user i has effectively concealed the factors p i and qi from everyone else. Compute the Euler function: +(ni) =

+(pi)+(qi) =

II

- (pi + 4i)

+1

User i then picks an integer dirandomly such that 2 cr di 5 n - 1 and di is relatively prime to + ( n i ) ; that is, ( 4 ,+(ni)) = 1. Since +(x) for x > 2 is always even, such a di must necessarily be odd. Let ei be the multiplicative inverse of &mod #(ni)]; that is, eidi = l[mod #@I;)]. Clearly such an ei exists and is unique. For reasons that will become apparent, all these computations of +(ni), ei , and d; are all to be done in secret. User i now makes the encrypting key ( n i , ei>public with her address, but keeps the decrypting key ( n j , dj), along with the factors pi and qi and 4(ni), all secret. Now suppose that user i wants to send a message' M (where M < n i ) to user i. The encryption is done as follows: C = Ei(M) = M'i(mod n;). U s e r j sends C to user i, who then decrypts it as M = D i ( O = Cdi(mod n i ) . To prove the validity of this scheme assume first that M is not a multiple of pi and q i ; that is, (M,pi) = (M,4;)= (M, n i ) = 1. By Fermat's theorem (Property 2.2.2), MPi-' = l(mod p i ) and Mqi-I = l(mod qi). Since p i - 1 and qi- 1 divide +(ni), we obtain Ms$("i)= l(rnod p i ) and Mf@("i) = l(mod qi)for some integer s. Using Property 2.2.1, we can combine these congruences to obtain

' Long messages are segmented and each segment is converted into an integer using an invertible transformation such as the one in Table 111. For example, the message MESSAGE can be segmented as ME, SS, AG, EB, which, using Table 111, becomes four integers, 1204, 1818,0006, and 0426. The M under consideration is one such integer.

86

S. LAKSHMIVARAHAN Ms#W = l(mod n;)

DiE;(M)= Meid;(modn i )

= M*+S+(ni)(mod n;),

by choice of e; and di

= M(mod ni), by the preceding argument. Now, since 0 IM < n i , there are qi - 1 possible messages such that ( M , n;) = p i , and pi - 1 possible messages such that (M, n;) = 4;. Thus if M is a multiple of pi or qi , the system is immediately compromised. However, if pi and q; are of the order lom or above, the probability of this event happening is (pi + qi - 2)/piqi, which is of the order of or less. In other words, disregarding this highly improbable event of M being a mulconstitutes a valid encryption-decryption tiple of p ; or 4;,( E ; , 0;) pair. Since e;d; = d i e i , it should be interesting to note in passing that E;[D;(M)]= M, a fact which is very crucial in obtaining digital signatures, which are discussed in Section 4. Notice that both encryption and decryption can be done in a time proportional to log n; (see Section 2.2). To illustrate the preceding ideas, let us pick an example involving very small integers. Let pi = 5, q; = 1 1 , ni = 55, and +(ni) = 40. Choose ei = 27 and one can easily find d; to be 3. If M = 2, then Ei(M) = 227(mod55) = 18 and Dj(18) = 183(mod55) = 2. A direct way to cryptanalyze this scheme is to factor ni . But by proper choice of pi and q; this problem can be made very difficult, even for very sophisticated factoring algorithms. In their article Rivest er al. (1978) suggested that p and q differ in length by only few digits and that both p - 1 and q - 1 contain large prime factors. Thus cryptanalysis of this scheme is at mosr as difficult as factorization. The next best thing is to compute +(n;) or d ; . If $(nil is known, since ( p + 4;) = n; - +(ni) + 1 and (pi - 4;) = [(pi + qJ2 - 4ni]”*, we can easily compute pi and 4;. Similarly, if di is known, then e;d; = 1 + s+(n;), a multiple of $(n;). Using any multiple of +(n;),Miller (1975) developed a method to factorize n ; . Thus computing #(n;) or d; is at least as difficult as factoring ni , Simmons and Norris (1977) proposed a method of cryptanalyzing this system by successively encrypting the ciphered text C until C is obtained. That is, if we define C1 = C and compute Ck+]= CP(mod ni) for k 1 2 = C, then clearly C,= M(mod ni), since C, = Me:(mod n;) = until CS+, M(mod nil. Here s is called the iteration exponent of M . However, if p i and qi are “properly” chosen, it can be shown that (Rivest, 1978) the probability of finding such an iteration exponent can be made of the order of or less. For other possible methods of cryptanalyzing this scheme and counterarguments refer to Herlestam (1978), Rivest (1979), and Williams and Schmid (1979).


87

Soon after the publication of this scheme. Rabin (1979) and Williams (1980) independently announced a class of encryption-decryption algorithms of the same type using the fact that deciphering without proper keys is indeed equivalent to prime factorization. In the following we present the algorithm due to Williams (1980) without proof. (For added simplicity, we also drop the subscript i). Let p and q be two large primes where p _= 3(mod 8), q = 7(mod 8), and n = pq. Define A(n) = LCM(p - 1. 4 - 1) and select e to be relatively prime to A(n). Let s = $ { f [ ( p- I)(q - I)] + 1) and find d as a solution to ed = s[mod A(n)]. As above, (n,e ) is the encryption key and (n,6)is the decryption key and both the encryption and decryption are done in two stages each. Let At be the set of all positive integers M such that 2(2M + 1) < n if J ( 2 M + 1 , n ) = -1and4(2M+ I ) < n i f J ( 2 M + I , n ) = I,whereJ(a,b) is the Jacobi symbol (see Section 2.4). It is possible that J(2M + I , n ) = 0. But for proper choice of n, this event can be made highly improbable. If M E At, E(M) is defined as &[El ( M ) ] ,where 4(2M M I = EI(M) = 2(2M

[

+ +

1) 1)

if J(2M if J(2M

+ +

I, n) = 1 1, n ) = -1

C = ,!$(MI) = M:'(mod n) Likewise the decryption D(C) is defined as D z ( D I ( C )where ) CI = Dl(C) = Cd(mod n ) and

DdCI) =

i

HtCI - 1) #(n - Cl) - 1 3 - 1)

#(n

-

CI) - 11

when when when when

CI = O(mod 4) Cl = I(mod 4) CI = 2(mod 4) CI = 3(mod 4)

It can be shown that D z ( DI (&(El (M))))= M if M E A. For a proof of this fact and that of the equivalence of deciphering without proper keys to prime factorization we refer the reader to Williams (1980). An important implication of this equivalence is that so long as prime factorization continues t o be a computationally hard problem, we can safely use the preceding algorithm for secrecy in communication. In an interesting paper Blakley and Borosh (1979) showed that for every pair p and q of distinct primes, there are at least nine distinct integers x less than p q such that x c = X(mod pq), where c is an odd positive integer. In other words, in the Rivest-Shamir-Adelman scheme there are at least nine unconcealable messages; these include 0, I , and p q - 1. The rest of the six unconcealable message are dependent on p and q . In a series of three related articles published in Cryptologia, Blakley and Blakley (1978,

88

S. LAKSHMIVARAHAN

1979a,b) showed that if any one of these six unconcealable p - or qdependent messages could be identified, then the system can be easily compromised. Blakley and Borosh (1979) also gave an example of a system which do not conceal any message: p = 97, q = 109, n = 10,573, e = 865, and d = 9505, where x865= X(mod 10,573)for all x. For the samep, q, and n, ife = 169 and d = 3865, then it can be shown that there are only 325 unconcealable messages. These examples raise a fundamental question, namely, Under what conditions does a scheme of the same type as the Rivest et al. (1978) scheme ensure maximum possible concealment of messages? In the following we merely summarize without proof the major conclusions from Blakely and Borosh (1979). Let N = p1p2p3 .-.P k , where k 2 2 and the p i are all distinct primes ( k = 2 for Rivest, Shamir, and Adleman’s scheme). Choose e and d to be two integers where 2 Ie 5 n - 1 and 2 Id In - 1 such that e is odd, [e - I , A(n)] = 1, ed = I[mod A(n)],and A(n) = LCM(p1 - 1, p2 - 1, ..., Pk - 1). If such an e and d are used as the encrypting and decrypting exponent in a Rivest-Shamir-Adleman-type scheme, then the number of unconcealable messages is minimum and equal to 3k. We now conclude this section by computing the size of the public directory. I f p and q are of the order of lo”, then n is of the order of = 2600.Since ( n , e ) is the public key, each user needs about 1200 bits, and 10o0 users need 1.2 Mbits. From the angle of the size of the public directory, this scheme compares very favorably with the algorithms in Section 3.1 based on exponentiation and discrete logarithms. 3.3 Algorithms Based on the Knapsack Problem

Let us begin by describing a version of the basic 0-1 knapsack problem? Given a set {a, , a2 , ..., a,} of positive integers and a positive integer S, the problem is to find a binary m-vector M = ( M ,IM2 , ..., M,)such that S = z=laiMi, given that the solution exists. Often the vector a = (ai, a2, ..., a,) is called the knapsack vector. We can imagine S to be the size of a knapsack (in cubic feet) and ai(i = 1, ..., n) to be the size of the ith object (in cubic feet). The problem is to find the subset of n objects that will exactly fill the knapsack given that such a subset exists. Any suspected solution can be tested if it is indeed a solution in polynomial time (just n 1 additions), but finding the solution in the worst case may result in trying out all the 2” possible binary n-vectors. It is well known (Ah0 et al., 1974; Garey and Johnson, 1979) that the general knapsack problem is NP complete, in the sense that it is one of the notoriously difficult problems. Even the best known algorithms for solving such NP-complete problems need

* See Note Added in Proof,p. 107.


89

time of order P2and space of the order P4(Schroeppel and Shamir, 1980), where n is a measure of the “size” of the problem. However, as we have noted in Section 1, NP completeness theory deals only with the worst case difficulty. There are a number of special instances of the knapsack problem which are polynomially solvable. For example, if the knapsack vector satisfies the condition ai > &:laj for all i, then the solution m can be easily found: M,, = 1 if and only if S 2 a,, . M i = 1 if and only if s - Xy=i+I M ~ UL , ai for i = n - I , n - 2, ..., 2, 1. In particular, if ai = 2i-’,then M is clearly the binary expansion of S. Any knapsack vector a whose ith component is greater than the sum of all the previous components is called a superincreasing vector. For a discussion of various other versions of the knapsack problem, refer to Garey and Johnson (1979) and Karp (1972). The basic idea of the knapsack-based cryptosystems may be stated as follows. Each user picks and publishes their own knapsack vector with their name and address. It is assumed that the messages are n-bit binary strings. (Because of this, the problem under description is called 0-1 knapsack). If user j wants to send a message M = ( M I ,M 2 , ..., M , ) to user i, he (userj ) obtains a copy of ith knapsack vector, say, a, from the U , ~ Mand , , communicates S to i. published directory, computes S = Now if i had chosen a to be superincreasing, since a is public, by wire tapping S anyone can recover very easily the message M and hence there is no secrecy. On the other hand, if i had chosen a to be nonsuperincreasing (that is, components of a are random integers), then the recovery of the message M from S is a very difficult problem to everybody including i. The question is, How, then, should i pick his knapsack vector in such a way that it is easier for him to recover M from S but extremely difficult for all others? Merkle and Hellman (1978)provided a very elegant solution to this problem and it runs as follows. Let the users each pick a superincreasing knapsack vector a’ = ( a ; ,a ; , ..., a;) and choose an integer r 2 x,v=,aj and another positive integer w such that ( w , r) = 1. Then they computes w-’,where ww-I = l(mod r ) . Notice that w and w - ’ can be found in polynomial time using the methods of Section 2. The users then transform the superincreasing knapsack vector a’ to another knapsack vector a; = ( a l , a 2 , ..., a,,), where a; = waf(modr), i = 1, ...,n. Each of the users now publishes this transformed vector a in the public file with their name and address and keeps a’, r, w , and w - I all secret to themselves. One can look upon this transformation as a multiplicative pseudo-random generator. This effectively conceals the structure of a’ and the components of a look more like a bunch of random integers. The encryption is extremely simple and consists in findaiMi = S, where a is obtained from the directory and M is the ing the

c:=,

x;=,

90

S. LAKSHMIVARAHAN

message. The intended user, on receiving S, decrypts as follows: n

S' = w-IS(mod r)

=

w-I

2 aiMi(mod r ) i= I

=

n

n

i= 1

i= I

2(w-'ai)Mi(mod r) =

a/Mi.

In other words, computing S' and using a', the message M can be recovered uniquely in polynomial time. Since r, W, and a' are all secret, any wire tapper having a copy of only S and a is left with no choice but to exhaust all possible solutions, which is indeed very difficult, as seen earlier. Thus the role of the concept of a trapdoor one-way function in this scheme is abundantly clear. The following example, taken from Merkle and Hellman (1978), illustrates the preceding ideas. Let a' = (171, 196,457, 1191, 2410), r = 8443, w = 2550, and w - ' = 3950. Let M = (0, 1 , 0, 1 , 1). Then S = 15115. Clearly S' = Sw-'(mod r ) = 3797. Using S', a', and the algorithm given in the first paragraph of this section, one can easily check that M s= 1, M4 = 1 , M4 = 1 , MS = 0, M2 = 1 , and M I = 0. In order to render cryptanalysis by the brute-force (exhaustive) method computationally infeasible, Merkle and Hellman (1978) suggested that n be at least 100, r be chosen randomly under uniform distribution in the interval [220'+ 1, 2202- 11, while a/ be chosen randomly under uniform for i = 1, 2, ..., distribution in the interval [(2(i-')- 1)21w + 1 , 2(i-1)21w] 100; that is, a; is from [I, 2100],a; is from [2Iw + 1 , 2 x 2'O0], and so on. Then pick a w I randomly under definition in the interval [2, r - 21 and let d be the greatest common divider of w I and r; w is taken as wild. This choice ensures all the conditions stated earlier. Furthermore, it makes any attempt to recover w by exhaustive analysis infeasible. If r and a' are chosen according to this plan, all the integers in the system are represented as a 202-bit number. When M is such that all its components are unity, then S 5 100(2202- 1) < (2209- I). In other words, S is essentially a 209-bit number. Thus a 100-bit message is transformed into a 209-bit message for reasons of security. This type of data expansion is an inherent feature of the 0- 1 knapsack-based cryptosystems. Let us now compute the size of the public key per user. Since ai = waf(modr), each of the ai is a 202-bit number. The public knapsack vector a as a whole per user needs 202 x 100 = 2012 kbits, as opposed to 1.2 kbits for the RSA-type algorithm (see Section 3.2) or 512 bits for the Diffie-Hellman type of scheme (see Section 3.1). To reduce the size of the public file Merkle and Hellman (1978) suggested that a one-way hash total, that is, h[a], which is of the order of 100 bits, be kept in the public


91

file instead of the knapsack vector a itself. In this modified scheme, u s e r j who wants to send a message to i calls i, requesting him to send his a vector. U s e r j , on receiving it, computes h[a] and compares it with user i’s hash value published in the directory. If there is a match,jencrypts his message for transmission to i; otherwise j takes no action, since the person who claims to be i is indeed an impostor. Notice that for this modification to be practical, h [ . ]must be a true one-way hash function. Use of this kind of one-way functions has been analyzed extensively by various authors in the context of authentication in time-sharing systems (Evans er al., 1974; Ingemarsson and Wong, 1980a; Purdy, 1979; Wegman and Carter, 1979, 1981). Having understood the basic mechanics of the knapsack-based cryptosystems, we shall in the following paragraphs mention a number of further improvements and modifications of this basic idea. Merkle and Hellman (1978) suggested that the trapdoor transformation could be iterated more than once. For example, choose arrto be superincreasing, rl 2 ~ . F l a ~ , (w, ,r l ) = 1, w l w l l = I(mod r l ) , and a: = wlaf‘(modr l ) , where a’ = ( a ; , a ; , ..., a;). Now let r2 2 x : = l u : , (wt , r z ) = 1, W Z W T I = l(mod rt), and ai= w2al(mod r 2 ) ,where a = (a,, a z , ..., un).This a is made public. In fact, such a transformation could be used any finite number of times. While there is no conclusive proof, still there is a growing consensus that such iterated knapsack-based cryptosystems are “secure” (Shamir and Zippel, 1980). However, one offshoot of such an iterative transformation is that it contributes to further data expansion. In addition to the preceding transformation, one could scramble the order of the components of the vector and perhaps add random multiples of r to various components of a to further confuse the cryptanalyst. The second modification is due to Shamir and Zippel (1980). They provided an interesting cryptanalytic attack on the original Merkle and Hellman knapsack system (where the superincreasing sequence is transformed only once) if the modulus r is known. They showed that the attack fails if the original superincreasing knapsack vector is transformed more than once. They also suggested a scheme for the selection of the public knapsack vector a , as shown in Fig. 6. The blocks of zeros between the lower order and the diagnonal matrix is logz N bits. To further obscure this structure one could use a trapdoor one-way transformation of the kind used earlier. A third interesting variation of the knapsack-based system was discussed in Arazi (1980), wherein a message is enciphered by adding to its numerical value the sum of a subset of elements picked at random from the publicly known knapsack vector. This scheme has an advatage when the same message is to be transmitted at different times. In such a case,

92

S. LAKSHMIVARAHAN

..*OOR;' OlooOO ..* OOR? loo000 OOR;

a ; = R;oooo ... OOlOOo = R$OOOO - * a; = R@OOO

a:, = R:,I(y)(J ... 000000 -*

... .b

n

riog2nl

FIG.6. A choice for the superincreasing sequence: R,!and R:'are random binary strings of appropriate length where i = 1 , ..., n.

this modification enables multiple mappings to be realized by selecting random subsets of the elements of the knapsack to be added to the message. At this point the reader might wonder, Why use 0-1 knapsack? Also, what are the effects of allowing Mi to be an integer in a fixed range? In other words, what are the potentialities of integer knapsack problems for use in public key cryptosystems? In a series of two reports Shamir (1980, 1981) analyzed various aspects of the knapsack problems in general, including the integer knapsack systems. It is shown that small knapsacks are associated with totally unacceptable security risks and that large values of n (the size of the knapsack vector) are often necessary for this class of systems. An interesting consequence of this result is that any attempt to reduce the size of the public file in the original 0-1 knapsack problem by allowing integer values to be message componentsMi (i = 1, ..., N ) and reducing the size of the knapsack vector could lead to loss of security. That is, between a 0-1 knapsack with N = 100 and an integer knapsack where each M iis a 25-bit integer and n = 4, a conservative choice is undoubtedly the former (Merkle and Hellman, 1978; Shamir, 1980). Another interesting story on the knapsack problem's cryptographic significance is due to Ingemarsson (1980). A knapsack S = xy=laiMi is said to be partially solvable if there exists at least one index t such that at > &+,ai. Under this condition M I can be easily obtained, and by subtracting a,M, from S, we obtain a reduced knapsack problem. There is evidence to believe that one can obtain a partially solvable knapsack from a given knapsack by multiplying it modulo another integer (Herlestram, 1978; Ingemarsson, 1980). From a cryptographic security point of view, partial solvability is a definite weakness. Ingemarsson has analyzed the class of knapsack problems which are not partially solvable after multiplication by an integer modulo another integer.


93

3.4 Algorithms Based on Algebraic Coding Theory

The general problem of designing a decoding algorithm for linear errorcorrecting codes (Peterson, 1961) has been known to be a difficult problem. Recently, Berlekamp et al. (1978) proved that the problem is indeed NP complete (Garey and Johnson, 1979); that is, it belongs to an equivalence class of notorious problems such as the traveling salesman’s problem and the satisfiability problem. Based on this fact and the existence of a very efficient decoding algorithm (Patterson, 1975) for a rather special class of error-correcting codes known as Goppa codes (McEliece, 1978), McEliece announced an interesting public key cryptosystem. Given an irreducible polynomial of degree t over GF(29, (see Section 2.6), there exists an irreducible Goppa code of length r = 2s and dimension n 2 r - st which is capable of correcting any pattern of t of fewer errors (McEliece, 1977). Each user in the system selects a suitable value of n , t , and an irreducible polynomial of degree t. Based on this, each then produces a generator matrix G (a binary matrix of order n X r). Each user now randomly selects a nonsingular binary matrix S of order n x n (called the scrambling matrix) and a permutation matrix P of order r x r, computes G = SGP, and publishes in a directory the matrix G and the constant t with his or her name and address. The generator matrix G , the scrambling matrix S, and the permutation matrix P are all held secret and constitute the trapdoor information. Suppose u s e r j wants to send a message M, considered as an n-bit vector, to user i. He or she gets a copy of G and t corresponding to user i from the directory and encrypts M as C = E(M) = MG @ z , where z is a random binary vector of dimension r which contains 1’s in exactly t positions and @ is modulo 2 addition. Notice that the encrypted message C is a binary vector of dimension r and z is locally generated at the time of encryption. User i , on receiving C, decrypts as follows. First he computes C = CP-I. He can rewrite C = M G + Z,where M = MS and 2 = ZP-I. This implies that C is a code word corresponding to a message A?. User i now recovers M from C, using the well-known decoding algorithm due to Patterson (1975). It can be shown that this recovery requires a number of operations proportional to nt. The original message M is obtained as M = MS-1. Any attempt to recover G from G or directly decode C is infeasible if the parameters are carefully selected. McEliece has suggested a choice of n = 524, s = 10, r = 1024, and t = 50. An important characteristic of this algorithm is that it can be readily implemented in hardware and a very

94

S. LAKSHMIVARAHAN

high rate of data transfer ( lo6 bitslsec) can be achieved. Clearly the size of the directory per user is 0.5 Mbits. 4.

Applications

The primary motivation for the development of public key cryptosystems is to provide secrecy in communication. However, the concept of one-way functions, which constitute the basic building block for these systems, has found widespread applications in many other related areas. In this section we provide a survey of these applications. 4.1 Authentication

In the general context of communication, authentication is a process by which each party involved in the communication verifies the identity of the other party. We generally authenticate ourselves in our daily activities, while banking, shopping, and crossing international boarders, etc., by using driver’s licenses, credit cards, passports, etc. All these means of authentication are unique and private to the party being authenticated. In the context of computer communication there are basically two kinds of authentication problems: (1) message authentication and (2) user authentication. The latter may be subdivided into user-to-central computer authentication and user-to-user authentication. Message authentication is the verification of the integrity of the message, that it is not forged or modified by a third party. Definitive work in this direction is due to Feistal et al. (1975). In their work message authentication is obtained by introducing redundancy with multiple encryption and block chaining. A good encryption scheme lies at the heart of this proposal. Wegman and Carter (1981) have developed an interesting authentication scheme using hash functions. This is done by attaching what is known as an authentication tag. Let At be the space of all messages and Tbe the space of tags. For example, Tcould be the set of all binary strings of length up to 100 bits, whereas messages could be much longer strings. Let F be a set of hash functions such that iffis in F, thenf:At+ T. This set F is public information. Two users i a n d j agree secretly on a functionf E F for use in authentication. If M is the message i wants to send toj, she then computesf(M), the tag for m ,and sends the pair M , f ( M ) toj. Now since f is secret between them, j can easily verify f(M).An important result proved by Wegman and Carter (1981) is that any third party knowing M andf(M) can find another message M’ for which the probability of guessing the correct authentication tagf(M’) is at most p. It is shown that


95

by properly designing the set F, choosing the length of the tag, randomizing the choice off in F , the value of p can be made as small as desired. In other words, using a class of one-way hash functions Wegman and Carter (1981) have developed a provably secure authentication technique for sending messages over an insecure channel. For an interesting comparison of the use of redundancy in authentication and error-correctinddetecting codes see Simmons (1979). User-to-central computer authentication is often done by the use of a password scheme. The most vulnerable part of this scheme is the password table itself. Even if the password entries are encrypted using a secret key resident in the system, such a key is often accessible t o a privileged few, such as the manager or operator. For effective user authentication one can use one-way functions as follows. If w is a password, in the password table storef(w), wheref(.) is a carefully chosen one-way function. In this case computing w fromf(w) would be infeasible to everybody, including the manager and operator. Also, if the user should forget his w, he too cannot recover his own w fromf(w). The only alternative for him is to pick a new w' and storingf(w') in the table against his name. Purdy (1974) and Evans et al. (1974) have independently developed different one-way functions for password protection. In the following we describe the scheme due to Purdy (1974). Let 4 be a prime and p ( x ) = x" + aIxfl-'+ + I.- + a, be a polynomial with integer coefficients. If w (assumed to be an integer) is a password, then f(w) = p(w)(mod 4). In other words, f(w) is the remainder when the value of the polynomial at point w is divided by 4.For proper choice of n and 4,finding w fromf(w) using polynomial root-seeking procedures can be computationally difficult. Purdy (1974) suggested the following example: 4 = 224 - 59, n = 224 17, n l = 224+ 3, p(x) = xfl + alx" + a2x3+ u 3 x 2+ a4x + U S , where ul , ...,a5are all arbitrary random integers. For this choice finding w from f ( w ) needs n2(logq)2;= loi6 operations. If one operation is done in sec, this would require 400 years! But computingf(w) from w is comparatively very cheap. It should be interesting to note that the use of one-way functions for user-to-computer authentication predates the emergence of the public key cryptosystem. Evans et al. (1974) transformed w through a sequence of password-dependent iterative (nonlinear) transformations, which, in concept, is very similar to a product cipher system such as Lucifer or DES. For the establishment of a user-to-user authenticated interactive communication link in a computer network, Needham and Schroeder (1978) developed an encryption-based authentication scheme. They developed an interesting key distinction protocol under both classical and public key encryption schemes. The key that is being distributed is used for authenti-

+

96

S. LAKSHMIVARAHAN

cation through encryption. A noteworthy conclusion of their study is that the key distribution protocol for the public key system is no simpler than its classical counterpart. In addition to authentication, they also developed protocols for network mail delivery and signed communication. For various other approaches to user-to-user authentication, key distribution, network mail, etc., refer to Branstad (1978), Denning and Sacco (1981), Matyas and Meyer (1978), Merkle (I978), Ehrsam et al. (1978), Kline and Popek (1979), Popek and Kline (1979), and Sendrow (1978). Ingemarsson and Wong (1980a,b) presented an application of one-way functions in providing user authentication for shared data and in on-board satellite communication systems. 4.2 Digital Signatures Authentication simply helps to verify the identity of the parties involved in a communication. In many applications such as point-of-sale transactions, the sale of stocks and bonds, and interbank money transfers, in addition to authentication, we need an electronic means of being able to sign a message so that the parties involved in a transaction can protect themselves in case of disputes such as alleged forgery or repudiation by the sender. Digital signatures have been developed based on both classical and public key encryption schemes. In this section we describe a method of obtaining digital signatures using the public key approach and it is due to Diffie and Hellman (1976b, 1979). Let (E, D ) be a pair of encryption and the corresponding decryption algorithms in a public key cryptosystem satisfying conditions Pl-P3 of Section 1.4. Recall that E is public but D is private. Condition P1 requires E to be a one-to-one function. This in turn implies that there is a D such that D[E(M)] = M, where M is in the domain of E. In addition to conditions Pl-P3: (P4) If E is also onto, that is E(D(M))= M is also true, then E is one to one and onto, that is,'E is a permutation. Using conditions Pl-P4, Diffie and Hellman (1976a) made the following proposal for incorporating security and signature. Let (Ei , Di)and (Ej ,Dj) be the encryption-decryption pairs for i andj, respectively, where E's are public and D's are private. If M is a message that i wants sent toj, he first signs M to obtain the signature S = Di(M)by decrypting M with his secret Di . He then attaches his name and address to S to attain S ' . Now he encrypts S' using userj's encryption algorithm and obtains C = 4 ( S ' ) and communicates C t o j . The latter, on receiving C , first decrypts C with his own secret decryption algorithm Dj to


97

recover S‘ = Dj(C).From the name and address filed, he identifies that i has sent him this message and obtains Ei from the public file and recovers M = Ei(S). The pair (S’, C) indeed constitutes a signature. Notice that onlyj can recover S from C , since Dj is known only to him. Furthermore,j can be sure that it definitely came from i, since Di is known only to i. Thus so long as Di continues to be known only to user i, j can prove, say, in a court of law, that no one other than i is the originator of the message M. We wish to emphasize the necessity of Di being known only to i in this argument, for if Di is either accidentally or otherwise disclosed, then all the messages signed using Di become invalid and i will tell the court that he is being set up by someone who now knows D i . Some of the other interesting features of this proposal include the fact that the signature is message dependent and if the same M is to be sent more than once, one can attach a serial number or time stamp. In this way one can prevent the replay of a previous conversation by an intruder. To obtain signature and secrecy we decrypt and then encrypt. If only signature is needed without secrecy, user i finds S = Dj(M)and obtains S’ by attaching his name and address to S and sends S’ in clear t o j . Notice that in this case anyone, includingj, knows that i is the originator; but so long as Di is secret to i , no one could have generated S’. Thus the role of condition (P4) in the proper signing of a message is clear. The immediate question is, Which of the examples in Section 3 satsify all conditions (Pl)-(P4)? Only the RSA algorithm satisfies all these conditions. Recall that the encryption exponents e and d in the RSA algorithm are such that ed = de = l[mod + ( n ) ] . Hence one can decrypt or encrypt with suitable algorithms, depending on whether one needs secrecy alone, signature alone, or signature and secrecy. The knapsack algorithm is anything but onto and hence it cannot be used as such for signing a message. For a very thorough discussion of the suitability of knapsackbased algorithms for signature and security see Shamir (1980). Similarly, the encryption scheme due to McEliece (1978) is not onto, since the decoding algorithm for this coding scheme will not produce an output unless the input to the decoder is a vector which differs from a code word in at most t bits. Thus only a very small fraction of the 2‘ binary vectors have the preceding property and are not suitable for signature. We conclude this section on signature by describing a scheme due to Rabin (1979), wherein forging a signature is as hard as factorization. Let M’ be a binary message of, say, a bits. To sign this message generate a random binary sequence R of, say, b bits for some integer b. Concatenate M’ and R to obtain a new binary sequence M of length a + b bits. Compute h(M) = p, an integer, where h(.) is a suitably chosen hash or compression function. Find a = p(mod n), where n = p q the product of two primes. As

9a

S.LAKSHMIVARAHAN

in the RSA algorithm, p and q are secret but n is public. Check if a! is a quadratic residue of n. This can be done by raising a to the power p - 1)/2 and (q - 1)/2 (see Section 2.4). If a! is a quadratic residue of n, then (R, a) constitutes the signature; if not, generate another R and repeat the process. Under mild conditions on h(.),it can be shown that the probability of selecting an R such that a! is a quadratic residue is about 0.25. Rabin (1979) has shown that forging this scheme is equivalent to factorization. For other interesting schemes refer to Rabin (1978) and Lieberherr (198 1). 4.3 Read-only Secure Communications

This application is due to Simmons (1979). Suppose two countries U and R are parties to an agreement on banning underground nuclear tests. As a part of the treaty each one wants to install in the land of the other seismic devices for monitoring such activities. The basic problem is one of ensuring the authenticity of the measurements that are communicated from these devices. It is conceivable that a country being monitored may try to cheat the other by unauthorized injection-delection. In fact, when the tests are really being conducted one can delete the actual signals and inject more innocent looking ones. One solution is to encrypt the measurements using a good encryption algorithm (say, for example, DES with 128 keys). Since decryption without the knowledge of the keys is difficult, injection and delection cannot be done. While this would ensure the authenticity of the measurements, the country being monitored, since it does not know the contents of the encrypted message, might accuse the other that it is also trying to use this monitoring station for other kinds of activities such as spying. One way to take care of this objection is to surrender the encryption key to the country being monitored at frequent intervals, say, once in a month, so that the latter can verify the messages that were sent in the previous month. But this involves the continuous generation and distribution of keys, which is often very expensive; also, the country being monitored may not wish to wait for a month to know what went on earlier. Use of public key system would answer most of these problems in a rather natural way. Let each country generate the pair (E, D)of encryption-decryption algorithms satisfying conditions PI -P3. Implement E, the encryption algorithm, in the measuring systems so that the output of the instruments are encrypted by E. Keep the details of E secret but give a copy of D to the country being monitored. With D on hand, each country can on-line verify the contents of transmission, yet cannot compute E from D.This ensures that injection and delection can be detected with certainty and all constraints are satisfied.

99


4.4 Conference Key Distribution Systems

The public key distribution system (see Section 3. I) as it exists is used for communication between two individuals. An extension of this concept is called the conference key distribution system, which admits a group of more than two users to share the same encryption and decryption key and subsequently use it for secret communication among members of the group. This extension, due to lngermarsson et al. (1980), is potentially applicable to secret computer conferencing. The basic idea of this scheme goes as follows. Choose a large prime p. Let there be L users in the conference numbered from I to L. The ith user generates a random integer Ri under a uniform distribution over integers in the interval [ I , p - I] and keeps Ri secret. To illustrate the mechanism by which a common key is derived, let I = { I , 2, ..., L} and J C I. Define S(")(J)as the sum of all possible products of x integers from the set J with distinct subscripts. For example, if J = {I, 2}, then S(2)(J)= R ,R 2 . If J = { 1, 2, 3}, then S(2)(.l)= R I R z + RzR3 + R , R 3 ,and so on. A conference key of order n is K(")= &")(I) (mod P), where n 5 ( I ) = L and g is the primitive root of p (see Section 2.3). When L = 2 = n, we obtain the Diffie-Hellman scheme of Section 3.1. Since each R; is secret to user i , computation of K(") is nontrivial and involves many stages of communication and computation. An illustration of the main idea for n = 2 and L = 3 is given in Table VIII. It is asumed that the stations are connected in a loop where I sends the result of its computation to 2, 2 to 3, and 3 to 1. For an analysis of the security of this scheme refer to Ingemarsson et al. (1980). There it is shown that the Lth-order key is most secure and involves the least amount of computation. Once the common key is arrived at, the secret conferencing can continue by encrypting all the messages with the common key. TABLEVIII Stations (all computations are mod P ) Time sequence Generate Compute and send

I

R,

2

3

R2

aR3

aR2

R3

100

S. LAKSHMIVARAHAN

Another very interesting application of the public key system is due to Chaum (1981). He has developed a method for an electronic mail system where one correspondent can remain anonymous to a second, yet the second person can respond to the first through what is known as an untraceable return address. Application of this idea includes the concept of computer election, where any interested party can verify that the votes have been properly counted, yet the identity of the voters can be kept secret. 4.5 Data-Base Security

Data-base security has been addressed at two different levels. First there is the access control or operating system security level, where the access is restricted to only legitimate users. Clearly, legitimacy determination involves authentication. For a detailed treatment of the access control techniques see Hsiao et al. (1979), especially Chapter 8 and the references therein. The second level involves using cryptographic transformations to store data in encrypted form. In this case even if access is gained illegally, immediate exposure is prevented. For greater security, often these two levels of security need to be combined. In this section we shall review some of the special encryption algorithms developed in the context of data base security. The relevance of the classical encryption algorithms for data base security has been analyzed by Gudes et al. (1976) in the context of multilevel models for a data base. They developed a scheme where cryptographic transformations can be handled either by the system or by the user. At the IBM Thomas J. Watson Center, for the purposes of protecting the secrecy of their own sensitive data a package called Information Protection System (IPS) was developed. This package is a collection of DES-based application programs and was developed for exclusive use by IBM. For a very readable account of IPS and other commercially available cryptographic products, refer to Konheim et al. (1980) and the references therein. Any discussion of data base security would be incomplete without citing to the work of Davida e? al. (1981). These authors have developed a new class of encryption algorithms called the subkey system. The system is record oriented and differs from the conventional block ciphers in a very basic way, in the sense that while the whole record is enciphered as a block, the decryption of separate fields of the record is possible without having to decrypt other fields. One can classify this system as an asymmetric cryptosystem in the sense of Simmons (1979), where the encryption and the decryption keys are different but related. Let a record contain n fields and let 5 be the content of the ith field.


101

Without loss of generality, assume that f, is an integer. To encrypt a record, pick n distinct large primes p I ,p 2 , ...,p , and let p = p I p 2 p 3--.p, . All the computations for encryption are done under mod p. The encryption key ei for the ith field is obtained as follows: Dkfine zi by (p/pi)zi= l(mod p i ) by Fermat’s theorem zi = (p/pi)Pi-2(modp i ) . The encryption key ei = (p/pi)zi.To encrypt a record, first generate n random integers h , , h 2 , ..., h, . Let h;llJ be the concatenation of the two integers hi andf, , where hillJ < p i . The encrypted record c is defined as N

c=2 (hillJ)ei (mod P I i= I that is, C is the sum of the products of hillJ and ei(mod p ) . By the Chinese remainder theorem (Section 2.2) we obtain [since ei = l(mod pi)]C = (hil(J;)(modp i ) . In other words, to decrypt the ith field, find the remainder when C is divided by p i . By discarding the random part of the remainder, we obtain J . For a detailed discussion of the security, advantages, and disadvantages refer to Davida et al. (1981). 5.

Conclusion

The basic premise in cryptography is security. From this point of view cryptosystems have so far been characterized in two different ways: (1) as unconditionally secure systems and (2) as computationally secure systems. The only known system that belongs to the former category is the now well-known onetime pad system. The computational security of public key systems is centered around the fact that they are derived from problems (belonging to the NP class) which have resisted efficient (polynomial) solutions for decades. The only public key system for which decryption without proper keys has been shown to be equivalent to the well-known hard problem (factorization) is due to Rabin (1979) and Williams (1980). This proof is a constructive proof, and as every constructive proof it is susceptible to chosen plain text attack. Furthermore, there is no known public key system to date that is unbreakable against an attack with unlimited computing resources, for all these systems are derived from the NP class, which only needs nondeterministic polynomial time. Recently Brassard (1979) has proved that the proof of the cryptanalysis of any system based on one-way functions is indeed NP complete would imply that NP = Co - NP. But this latter equality is either false or very difficult to establish. Also, an example of an easy to break N P complete cipher was given in Lampel (1979). All these results in no way undermine the importance of the theory of public key systems, but very conclusively

102

S. LAKSHMIVARAHAN

point to the fact that a new way of quantifying cryptanalytic complexity needs to be developed. For some of the recent attempts, see Shamir (1980), Brassard (1981), and Bennett and Gill (1979). The unprecedented growth in cryptographic research by nonfederal agencies, universities, and private organizations as witnessed by the post1975 literature has evoked concern by the National Security Agency (NSA), the sole agency for making and breaking cryptographic codes in the United States. It is believed that if the design details of the highly secure cryptosystems are published, other countries may adopt them, thereby making the NSA’s job more difficult, if not impossible. Furthermore, publication may reveal better ways to break codes used by the United States. Accordingly, the NSA is advocating some kind of a restraint regarding the open dissemination of results of great cryptographic significance. A number of professional societies, including the American Association for the Advancement of Science, the Institute of Electrical and Electronic Engineers, the Associations of Computing Machinery, as well as the National Science Foundation, have appointed special committees to evaluate critically both the short- and long-term impacts of any such restraint. For a discussion of these questions the reader Davida (1981), Deavours (1981), Denning (1981a,b), Kahn (1981), Kolata (1978), Report of the Public Cryptography Study Group (1981), Shapley and Kolata (1977), Shapley (1977, 1978), Sugarman (1978), the U.S. Senate Select Committee on Intelligence: Summary of Report (19781, Walsh (1981), and Weingarten (1981). ACKNOWLEDGMENTS

Our thanks are due to Dr. Seun K. Kahng, the Director of the School of Electrical Engineering and Computer Science at the University of Oklahoma, for his continued support and encouragement. The second half of this article was completed when the author was visiting the Institute of Applied Mathematics, University of Bonn, West Germany, during May-June 1982, under the auspices of Sonderforschungsbereiches 72. We wish to record our gratitude to Dr. U. Herkenrath and Dr. D. Kalin for the invitation and hospitality. We are indebted to Dr. Suresh Kothari for reading the entire manuscript and suggesting various improvements. Finally, we wish to thank Chris Barnett and Linda Tahsequah, who typed various portions of the manuscript in record time.

REFERENCES Adleman, L.M., and Rivest, R.L. (1978). The use of public key cryptography in communication system design. IEEE Commun. SOC.Mug. November, pp. 20-23. Aho, A.V., Hopcroft, J.E., and Ullman, J.D. (1974). “The Design and Analysis of Computer Algorithms.” Addison-Wesley, Reading, Massachusetts.


103

Arazi, B. (1980). A trap door multiple mapping. IEEE Trans. Inf IT-26, 100-102. Ash, R.B. (1965). “Information Theory.” Wiley (Interscience), New York. Asmuth, C.A., and Blakley, G.R.(1981). An efficient algorithm for constructing a cryptosystem which is harder to break than two other cryptosystems. Cornput. Math. Appl. 7, 447-450. Bennett, C.H., and Gill, J. (1979). Relative to a random oracle A, PA # NPA # Co - NPA with probability one. IBM Res. Rep. RC 7925. Berkovits, S . , Kowalchuk, J., and Schanning, B. (1979). Implementing public key scheme. IEEE Commun. SOC.Mag. May, pp. 2-3. Berlekamp, E.R. (1968). “Algebraic Coding Theory.” McGraw-Hill, New York. Berlekamp, E.R., McEliece, R.M., and Van Tilborg, H.C.A. (1978). On the inherent intractability of certain coding problems. IEEE Trans. Inf. Theory IT-24, 384-386. Blakley, B., and Blakley, G.R. (1978). Security of public key cryptosystems against random attack. Part I. Cryptologia 2, 305-321. Blakley, B., and Blakley, G.R. (1979a). Security of public key cryptosystems against random attack. Part 11. Cryptologia 3, 29-49. Blakley, B., and Blakley, G.R. (1979b). Security of public key cryptosystems against random attack. Part 111. Cryptologia 3, 105-118. Blakley, B., and Borosh, I. (1979). RSA public key cryptosystem d o not always conceal messages. Compur. Math. Appl. 5 , 169-178. Booth, K.S. (1981). Authentication of signatures using public key encryption. Commun. ACM 24, 772-774. Branstad, C.K. (1978). Security of computer communication. IEEE Commun. SOC. Mag. November, pp. 33-40. Brassard, G . (1979). A note on the complexity of cryptography. IEEE Trans. lnf. Theory IT25, 232-233. Brassard, G . (1981). A time-luck trade off in relativized cryptography. J . Compur. I f . Sci. 22, 280-3 11. Campbell, C.M. (1978). Design and specification of cryptographic capabilities. IEEE Commun. SOC.Mag. November, pp. 15-19. Chaum, D.L. (1981). Untracable electronic mail return address and digital pseudonyms. Commun. ACM 2 4 , 8 4 4 8 . Data Encryption Standard (1977). “Federal Information Processing Publication,” N.B.S. Publ. 46. Natl. Bur. Stand., Washington, D.C. Davida, (3.1. (1981). The case against restraints on non-governmental research cryptography. Commun. ACM 24, 445-450. Davida, G.I., Wells, D.L., and Kam, J.B. (1981). A data base encryption system with subkeys. ACM Trans. Database Syst. 6, 312-328. Davis, R.M. (1978). The data encryption standard in perspective. IEEE Commun. SOC. Mag. November, pp. 5-9. Deavours, C.A. (1977). Unicity points in cryptanalysis. Cryptologia 1, 46-68. Deavours, C.A. (1981). The black chamber: A column shunting off the spigot in 1981. Cryprologia 5 , 43-45. Denning, D.E., and Sacco, G.M. (1981). Time stamps in key distribution protocols. Commun. ACM 24, 533-536. Denning, P.J. (1981a). Government classification of private ideas. President’s Letter. Commun. ACM 24, 103-105. Denning, P.J. (1981b). Report of the public cryptography study group. Commun. ACM 24, 434.

104

S. LAKSHMIVARAHAN

Dime, W., and Hellman, M.E. (1976a). Multiuser cryptographic techniques. AFIP Proc. NCC pp. 109-112. Dime, W., and Hellman, M.E. (1976b). New directions in cryptography. IEEE Trans. lnf. Theory lT-22,644-654. m e , W., and Hellman, M.E. (1977). Exhaustive cryptanalysis of NBS encryption standard. Computer pp. 74-78. D B e , W., and Hellman, M.E. (1979). Privacy and authentication: An introduction to cryptography. Proc. IEEE 67, 397-427. Ehrsam, W.F., Matyas, S.M., Meyer, C.H., and Tuchman, W.L. (1978). A cryptographic key management scheme for implementing the data encryption standard. IBM Syst. J . 17, 106-125. Erdos, P. (1950). On almost primes. Am. Math. Mon. pp. 404-407. Evans, A., Jr., Kantrowitz, W., and Weiss, E. (1974). A user authentication scheme not requiring secrecy in the computer. Commun. ACM 17, 437-442. Feistal, H. (1973). Cryptography and computer privacy. Sci. Am. 228, 15-23. Feistal, H., Notz, W.A., and Smith, J.L. (1975). Some cryptographic techniques for machine to machine data communications. Proc. IEEE 63, 1545-1554. Gardner, M. (1977). A new kink of cipher that would take millions of years to break. Sci. Am. 237, 120-124. Carey, M.R., and Johnson, D.S. (1979). “Computers and Intractability-A Guide to the Theory of NP-Completeness.” Freeman, San Francisco, California. Gersho, A. (1978). Communications privacy. Editorial. IEEE Commun. SOC.Mug. November, pp. 2-4. Gudes, E., Koch, H.S., and Stahl, F.A. (1976). The application of cryptography for data base security. AFIPS Conf. Proc. 45,97-107. Guy, R.C. (1975). How to factor a number. Proc. Manitoba Con$ Nuner. Math., Sth, 1975 pp. 49-89. Hellman, M.E. (1977). An extension of the Shannon theory approach to cryptanalysis. IEEE Trans. Inf. Theory lT-23,289-294. Hellman, M.E. (1978). An overview of public key cryptography. IEEE Commun. SOC.Mag. November, pp. 24-32. Hellman, M.E. (1979). The mathematics of public-key cryptography. Sci. Am. 241,146-157. Hellman, M.E. (1980). A cryptanalytic time-memory trade off. IEEE Trans. lnf. Theory IT%, 401-406.

Herlestam, T. (1978). Critical remarks on some public key cryptosystems. BIT pp. 4934%. Hill, S. (1978). Gentle diversions. IEEE Commun. SOC.Mag. November, pp. 56-58. Hoffman, L.J. (1977). “Modem Methods for Computer Security and Privacy.” PrenticeHall, Englewood Cliffs, New Jersey. Hsiao, D.K., Kerr, D.S., and Madnik, S.E. (1979). “Computer Security.” Academic Press, New York. Hyviirinen, L.P. (1968). “Information Theory for Systems Engineers.” Springer-Verlag. New York. Ingemarsson, I. (1980). Knapsacks which are not partly solvable after multiplication modulo Q. IBM Res. Rep. RC 8515. Ingemarsson, I., and Wong, C.K. (198Oa). A user authentication for shared data based on trapdoor one-way function. IBM Res. Rep. RC 8291. Ingemarsson, I., and Wong, C.K. (1980b). Encryption and authentication in on-board processing satellite communication systems. IBM Res. Rep. RC 82!l2.


105

Ingemarsson, I., Tang, D.T., and Wong, C.K. (1980). A conference key distribution systems. IBM Res. Rep. RC 8236. Kahn, D. (1966). Modern cryptology, Sci. Am. 219, 38-46. Kahn, D. (1%7). “The Code Breakers, The Story of Secret Writing.” Macmillian, New York. Kahn, D. (1980). Cryptology goes public. IEEE Commun.Soc. Mag. pp. 19-28. Kahn, D. (1981). The public’s secret. Crypfologia 5 , 20-26. Karp, R.M. (1972). Reducibility among combinatorial problems. In “Complexity of Computer Computations” (R.E. Miller and J.W. Thatcher, eds.), pp. 85-104. Plenum, New York. Katzen, H., Jr. (1977). “The Standard Data Encryption Algorithm.’’ Petrocelli Books, Inc. Kline, C.S., and Popek, G.J. (1979). Public key vs. conventional key encryption. AFIP Proc. NCC pp. 831-837. Knuth, D.E. (1966). “The Art of Computer Programming,” Vol. 11. Addison-Wesley, Reading, Massachusetts. Knuth, D.E. (1973). “The Art of Computer Programming,” Vol. Ill. Addison-Wesley, Reading, Massachusetts. Kolata. G.B. (1977a). Computer encryption and the national security agency connection. Science 197,438-440. Kolata, G.B. (3977b). Cryptology: On the brink of revolution? Science 197, 747-748. Kolata, G.B. (1978). Cryptology: A secret meeting at IDA? Science u)o, 184. Konheim, A.G. (1980). Digital signature and authentication. IBM Res. Rep. RC 8074. Konheim, A.G. (1981). “Cryptography: A Primer.” Wiley, New York. Konheim, A.G., Mack, M.H., McNeill, R.K., and Tuckerman, B. (1980). The IPS cryptographic programs. IBM Sysf. J. 19, 253-283. Kraemer, K.L., and Colton, K.W. (1979). Policy, values and EFT research: Anatomy of a research agenda. Commun. ACM 22, 660-671. Lampel, A. (1979). Cryptology in transition. Cornput. Sum. 11, 285-303. Lehmer, D.H. (1966). Computer technology applied to the theory of numbers. In “Studies in Number Theory” (W.J. LeVeque, ed.), Vol. 6, MAA Stud. Math. Lennon. R.E. (1978). Cryptography architecture for information security. IBM Syst. J. 17, 138-150. LeVeque, W.J. (l%l). “Elementary Theory of Numbers.” Addison-Wesley, Reading, Massachusetts. Lieberherr, K. (1981). Uniform complexity and digital signatures. Theor. Compur. Sci. 16, 99-1 10. McEliece, R.J. (1977). “The Theory of Information and Coding,” Vol. 3. Addison-Wesley, Reading, Massachusetts. McEliece, R.J. (1978). “A Public Key Cryptosystem Based on Algebraic Coding Theory,” DSN Rep., Jet Propulsion Lab., California Institute of Technology, Pasadena. Martin, J. (1973). “Security, Accuracy, and Privacy in Computing Systems.” Prentice-Hall, Englewood Cliffs, New Jersey. Martin, J. (1978). “The Wired Society.” Prentice-Hall, Englewood Cliffs, New Jersey. Matyas, S.M., and Meyer, C.H. (1978). Generation, distribution and installation of cryptographic keys. IBM Syst. J. 17, 126-137. Merkle, R.C. (1978). Secure communication over insecure channels. Commun. ACM 21, 294-299. Merkle, R.C., and Hellman, M.E. (1978). Hiding information and signatures in trap door 525-530. knapsacks. IEEE Trans. In$ Theory IT-24,

106

S. LAKSHMIVARAHAN

Merkle, R.C., and Hellman, M.E. (1981). On the security of multiple encryption. Commun. ACM 24,465-467. Miller, G.I. (1975). Riemann’s hypothesis and test for primality. Proc. ACM Symp. Theory Comput. 7 , 234-239. Moms, R. (1978). The data encryption standard-retrospective and prospects. IEEE Commun. SOC.Mag. November, pp. 11-14. Moms, R., Sloane, J.J.A., and Wyner, A.D. (1977). Assessment of the National Bureau of Standards proposed federal data encryption standard. Cryptologia 1, 181-191. Needham, R.M., and Schroeder, M.D. (1978). Using encryption for authentication in large network of computers. Commun. ACM 21, 993-998. Patterson, N. (1975). The algebraic decoding of Goppa codes. IEEE Trans. Inf. Theory IT21, 203-207. Peterson, W.W. (1961). “Error Correcting Codes.” Wiley, New York. Pohlig, S.C., and Hellman, M.E. (1978). An improved algorithm for computing logarithms over GF (p) and its cryptographic significance. IEEE Trans. I f . Theory IT-24,106110. Pollard, J.M. (1974). Theorems on factorization and primality testing. Proc. Cambridge Philos. SOC. 76, 521-528. Popek, G.J., and Kline, C.S. (1979). Encryption and secure computer networks. Cornput. Sum. 11, 331-356. Purdy, G.B. (1974). A high security log-in procedure. Commun. ACM 17, 442-445. Rabin, M.O. (1978). Dignitalized signatures. I n “Foundations of Secure Computation’’ (R.A. DeMillo, D.P.Dobkin, A.K. Jones, and R.J. Lipton, eds.), pp. 155-168. Academic Press, New York. Rabin, M.O. (1979). “Digitalized Signatures and Public Functions as Intractable as Factorization,’’ LCS/TR-ZlZ. Massachusetts Institute of Technology, Cambridge. Report of the Public Cryptography Study Group (1981). Commun.ACM 24, pp. 435-445. Rivest. R.L. (1978). Remarks on a proposed cryptanalytic attach on M.I.T. public key cryptosystem. Cryptologia 2, 62-65. Rivest, R.L. (1979). Critical remarks on “Some critical remarks on public key cryptosysterns,” by Herlestam. BIT 19, 274-275. Rivest, R.L., Shamir, A., and Adlman, L. (1978). On digital signatures and public key cryptosystems. Commun.ACM 21, 120-126. Schroeppel, R., and Shamir, A. (1980). “AT = 0(2“*) and S = O ( P 4 ) Algorithm for Certain NP-complete Problems,” LCS/TM-147. Massachusetts Institute of Technology, Cambridge. Sendrow, M. (1978). Key management in EFT networks. Proc. Compcon pp. 351-354. Shamir, A. (1979a). How to share a secret. Commun.ACM 22, 612-613. Shamir, A. (1979b). “On the Cryptocomplexity of Knapsack Systems,” LCS/TM-129. Massachusetts Institute of Technology, Cambridge. Shamir, A. (1980). “The Cryptanalytic Security of Compact Knapsacks,” Preliminary Report, LCS/TM-164. Massachusetts lnstitute of Technology, Cambridge. Shamir, A., and Zippel, R.E. (1980). On the security of Merkle-Hellman cryptographic scheme. IEEE Trans. I f . Theory IT-26,100-102. Shannon, C.E. (1949). Communications theory of secrecy systems. Bell Sysr. Tech. J . 28, 656-715. Shapley, D. (1977). Telecommunications eavesdropping by NSA on private messages alleged. Science 197, 1061-1064. Shapley, D. (1978). Intelligence agency chief seeks dialogue with academics. Science 202, 407-410.


107

Shapley, D., and Kolata, G.B. (1977). Cryptology: Scientists puzzle over threat to open research, publication. Science 197, 1345-1349. Simmons, G.J. (1979). Symmetric and asymmetric encryption. Comput. SUN. 11, 305-330. Simmons, G.J., and N o m s , M.J. (1977). Preliminary comments on the M.I.T. public key cryptosystems. Cryptologia 1, 406-414. Sinkov, G.J. (1968). “Elementary Cryptanalysis-A Mathematical Approach.” Random House, New York. Smith, J.L. (1971). The design of Lucifer, a cryptographic device for data communication. IBM Res. Rep. RC 3326. Solovay, R., and Strssen, V. (1977). A fast Monte-Carlo test for primality. SIAM J. Comput. 6, 84-85. Sugarman, R. (1978). Freedom to research and publish on cryptography remains unresolved. IEEE Spectrum News Suppl. Sugarman, R. (1979). On foiling computer crime. IEEE Spectrum 16, 31-41. U.S. Senate Select Committee on Intelligence: Summary of Report (1978). Unclassified summary: Involvement of NSA in the development of the data encryption standard. IEEE Commun. Soc. Mag. pp. 53-55. Uspensky, J.V., and Heaslet, M.A. (1939). “Elementary Number Theory.” McGraw-Hill, New York. Vinogradov, I.M. (1961). “An Introduction to the Theory of Numbers.” Dover, New York. Walsh, J. (1981). Shunning crypto-censorship. Science 212, 1250. Wegman, M.N., and Carter, J.L. (1979). New classes and applications of hash functions. IBM Res. Rep. RC n26. Wegman, M.N., and Carter, J.L. (1981). New hash functions and their use in authentication and set equality. J. Comput. Sysr. Sci. 22, 265-279. Weingarten, F.W. (1981). Cryptographic research and the computer science community. Commun. ACM 24, 851-853. Willet, M. (1980). Deliverate noise in a modem cryptographic systems. IEEE Trans. lnf. Theory IT-26, 102-105. Williams, H.C. (1980). A modification of the RSA public key encryption procedure. IEEE Trans. lnf. Theory IT-26,726-729. Williams, H.C., and Schmid, B. (1979). Some remarks concerning the M.I.T. public-key cryptosystem. BIT pp. 525-538.

NOTE ADDEDI N PROOF Since the submission of the final manuscript for this article, a number of papers and books have come into existence (see Additional References, below). Perhaps the most notable of these are the papers due to Shamir (1982) and Adleman (1982). Using some of the very recent results from the theory of integer programming, Shamir (1982) has developed a polynomial time algorithm for breaking the knapsack-based public key cryptosystem (see Section 3.3) in which the public key knapsack vector is obtained from a superincreasing sequence after a single application of modular multiplicative transformation. In particular, the method finds a modulus (rf-multiplier (w)pair, using which the original superincreasing sequence and hence the plaintext message can be easily obtained. Shamir has shown that even if the number of knapsack elements as well as their size grow without bound, the fraction of the keys for which this algorithm does not work is very small. Although it is interesting to note that the above algorithm due to Shamir does not compromise the system if the public key knapsack vector is obtained after multiple (instead of one) applications of

108

S. LAKSHMIVARAHAN

modular multiplicative transformations; this hope, however, was short lived. Adleman (1982) has developed another interesting scheme by which one can in fact break even the iterated knapsack system. The saying that history repeats itself is very true in cryptography even today.

Additional References Adleman, L.M. (1982). “On Breaking the Iterated Merkle-Hellman Public Key Cryptosystern.” Tech. Rep., Computer Science, Univ. of Southern California, Los Angeles, California. Denning, D.E.R. (1982). “Cryptography and Data Security.” Addison-Wesley, Reading, Massachusetts. Kak,S., ed. (1983). Special Issue on Data Security in Computer Networks. Computer, pp. 8-62,

Meyer, C.H., and Matyas, S.M. (1982). “Cryptography.” Wiley, New York. Shamir, A. (1982). “A Polynomial Time Algorithm for Breaking the Basic Merkle-Hellman Cryptosystem.” Tech. Rep., Applied Mathematics. The Weizmann Institute, Israel.

Software Engineering Environments ANTHONY I. WASSERMAN Medical Information Science University of California. San Francisco San Francisco. California

1 . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . The Software Life Cycle . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Functional Specification . . . . . . . . . . . . . . . . . . . . . 2.3 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Coding and Implementation . . . . . . . . . . . . . . . . . . . . 2.5 Quality Assurance: Testing and Verification . . . . . . . . . . . . . 3. Management Procedures . . . . . . . . . . . . . . . . . . . . . . . . 3. I Personnel Deployment . . . . . . . . . . . . . . . . . . . . . . 3.2 Cost and Schedule Estimation . . . . . . . . . . . . . . . . . . . 3.3 Evaluation of Project Progress . . . . . . . . . . . . . . . . . . . 3.4 Release Control . . . . . . . . . . . . . . . . . . . . . . . . . 4 . Software Development Methodology . . . . . . . . . . . . . . . . . . 5 . Automation in the Development Environment . . . . . . . . . . . . . . 5.1 Software Tools . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Programming Systems. . . . . . . . . . . . . . . . . . . . . . . 5.3 The Tool Kit Approach: Unix . . . . . . . . . . . . . . . . . . . 5.4 Tools for Supporting Ada . . . . . . . . . . . . . . . . . . . . . 6 . An Example: The User Software Engineering Methodology . . . . . . . . 6.1 USE Methodology Overview . . . . . . . . . . . . . . . . . . . 6.2 The USE Specification . . . . . . . . . . . . . . . . . . . . . . 6.3 The Role of Prototypes in USE . . . . . . . . . . . . . . . . . . 6.4 Design and Implementation . . . . . . . . . . . . . . . . . . . . 7 . The Software Development Environment . . . . . . . . . . . . . . . . 8. The Physical Environment . . . . . . . . . . . . . . . . . . . . . . . 8.1 Support Services . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 The Developer's Office . . . . . . . . . . . . . . . . . . . . . . 8.3 Computer Terminals . . . . . . . . . . . . . . . . . . . . . . . 8.4 Summary: The Physical Environment . . . . . . . . . . . . . . . . 9 . Toward Improved Software Engineering En vironments . . . . . . . . . . 9.1 Toward Improved Methodologies. . . . . . . . . . . . . . . . . . 9.2 Toward Improved Software Tools . . . . . . . . . . . . . . . . . 9.3 Toward Improved Computing Support . . . . . . . . . . . . . . . 10. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . ........

110 111

112 112 113 114 115 115 116 117 117 118

119 123 123 125 129 131 133 134 135 136 137 138 141 142 143 147 149 149 149 152 155 158 159

109 ADVANCES IN COMPUTERS. VOL . 22

Copyright 0 1983 by Academic Press. Inc. All rights of reproduction in any form reserved. ISBN @12-012122-0

110

ANTHONY I. WASSERMAN

1.

Introduction

Every organization has a software development methodology and a software development environment. What this statement means is simply that there is a vast body of software developed in an almost unbelievable variety of ways. Indeed, vast segments of our entire postindustrial civilization are highly dependent upon the proper functioning of this software and the machines on which it executes. While some of the software was developed according to modem practices for software development, most of it was built in an ad hoc way and is kept working through an unstructured process of “iterative enhancement.” There is ever-increasing emphasis on software. Many organizations that used to “hard-wire” their systems have replaced part of the circuitry with software, often in the form of embedded microprocessor-based systems, where a program is stored in read-only memory. With this tremendous growth in the volume and criticality of software has come a growing need to improve the quality of software and the process by which it is produced. The problems of developing new software systems are accompanied by the problems of maintaining and enhancing existing software systems. Many software systems go through many versions during their lifetimes, as they are adapted to provide new functions and to work in different machines or with different operating systems. It is widely acknowledged that the costs of maintenance far exceed the costs of development in terms of costs allocated to each. Accordingly, it is necessary to create and to use techniques that improve the productivity of both individual software developers and the organizations to which they belong. Furthermore, it is necessary to assure the quality of such systems, both in terms of their conformance to user requirements and their reliable operation. These problems were first recognized in the mid- 1960s, when developers failed in several significant attempts to deliver complex software systems. These problems, and the subsequent search for solutions, led to the creation of the discipline of “software engineering” in the late 1960s. The term software engineering was chosen as a provocative term to indicate the need to bring an “engineering-type” approach to the entire software development process. Since then, people have sought to identify and use tools and techniques that would yield a demonstrable, i.e., measurable, improvement in the software production process and/or the software products themselves. Many ideas have been proposed, and some important concepts and approaches have emerged, but software engineering still lacks the history

SOFTWARE ENGINEERING ENVIRONMENTS

111

and experience that one finds in traditional engineering fields. As a result, software engineering concepts are just beginning to come into widespread use. An important consideration in the development of software systems is the entire development environment. In its most general sense, the development environment includes the technical methods, the management procedures, the computing equipment, the mode of computer use (batch or interactive, centralized or distributed), the automated tools to support development, the software development staff, and the physical work space. An ideal development environment should enhance the productivity of the information system developers and provide a set of tools (both manual and automated) that simplifies the process of software production. The environment should contain facilities both for the individual member of a development group and for the overall management of the project. The purpose of this article is to explore some of the major issues associated with a development environment. Section 2 describes the major phases of the software development life cycle. Section 3 presents an overview of some of the management procedures that must be followed throughout these phases, stressing the necessary interrelationship between technical methods and management procedures, which is at the heart of the discussion of software development methodology in Section 4. Section 5 treats the automated support that can be provided for a software development methodology. Section 6 describes the integration of a methodology with automated support for the User Software Engineering methodology. The discussion then turns to the integration of a methodology and tools into a software development environment. Section 7 provides a brief overview of the components of an environment. Section 8 discusses aspects of the physical environment, making suggestions for a hospitable software development environment. Finally, Section 9 suggests some necessary future work that can improve methodologies, tools, and computing support, thereby leading to improved software engineering environments.

2.

The Software Life Cycle

Software development typically occurs as a sequence of steps comprising a “life cycle” starting with the original system concept and proceeding through analysis, specification, design, implementation, testing, and operation. During system operation, the system is typically modified to fix problems, enhance capabilities, and/or adapt to new execution environ-

112


ments. This activity, termed evolution, often takes two to four times the original development effort.

2.1 Analysis

Analysis of the problem at hand is the essential first step of any software development activity. Without such analysis, it is impossible to proceed; furthermore, an inadequate job of analysis is virtually certain to lead to project failure, since poor understanding of the problem makes it impossible to produce a good specification. Successful analysis involves communication with users and customers for the system, who can describe their needs. Analysis also involves communication with the eventual developers of the system, who must be able to evaluate implementation feasibility and describe any design or implementation constraints. Because of the complexity of systems, key tools for analysis support problem decomposition, through any of a variety of schemes, including procedural decomposition, data abstraction, data flow, processing sequtnce, or transactions. Graphical notations help to show the interrelationships of system components to one another and facilitate the communication process. User involvement is particularly important in this phase, since users possess substantial expertise about the problem domain. There are a variety of approaches that can be used effectively to improve problem understanding and communication between the analyst(s) and user(s). These approaches include interviewing (in the journalistic sense), doing the user’s job (where feasible and permissible), building mock-ups or prototypes of the system, and/or writing user level documentation for the proposed system.

2.2 Functional Specification A functional specification is a description of “what” the system will do. Whereas analysis serves to describe the problem and to identify requirements, the functional specification is the first statement of the system’s intended behavior. Thus, it contains a statement of the system functions, the external interfaces, the required data items, and requirements upon performance, design, and implementation. Functional specifications have many different roles within the software life cycle, including the following:


113

1. The functional specification is a means for precisely stating the software system requirements. At this stage, the technical realization of the system model is documented in as much detail as possible. The functional specification can be compared against the requirements definition to ascertain the correspondence between the specification and the needs. 2. The functional specification provides insight into the problem structure and is used during the design phase as a checkpoint against which to validate the design. Typically, there will be an iteration between specification and design, as insight into some of the system construction problems help to clarify the functional specification. 3. The functional specification is the basis against which testing and verification are performed. Clearly, one cannot prove that a program is correct in the absence of a clear understanding of the program’s behavioral characteristics. Similarly, although certain kinds of testing can locate clerical errors and other low-level problems, system testing and acceptance testing require comparison of the system against an objective specification. 4. Modifications and enhancements to a system throughout its operational lifetime require an understanding of the system functions, as documented in the functional specification. During evolution, the functional specification can help to locate those system functions that must be changed, and can then be revised accordingly.

The implication of this multifaceted role is that a functional specification must be able to serve (to some degree) each of these four different functions. In practice, this means that functional specifications must have both a formal and an informal component.

2.3 Design The process of software design permits the developer to determine the feasibility of implementing the specification,to analyze alternative system structures, to develop algorithms for the solution of the problem, and to define any constraints upon the implementation. In summary, it is a stage at which the primary concern is with the means by which the system will be built. The design activity was originally largely implicit in the system development process, and any design was done either in conjunction with the specification process or with code production. In the former case, design issues often became entangled with more basic issues concerning the functions to be performed by the system; in the latter case, it was often impossible to make global design decisions.

114


More recently, though, the design process has become more explicit. Indeed, the design activity can be separated into two phases: architectural design and detailed design. Architectural design is concerned with recognition of the overall program structure and the interconnection of program pieces, whereas detailed design is more concerned with the selection of algorithms and data structures that are appropriate to the fulfillment of specific system functions. One of the key goals of the design process is to simplify the subsequent stages of coding and testing. At the end of the design phase, virtually all of the key decisions concerning program organization, logical data structures, and processing algorithms will have been made, with the intent of making code production into a straightforward activity. The primary concerns of coding then focus on proper programming style and the mapping of logical data structures to physical data structures. In summary, though, the importance of the design process has become evident, since the output of the design activity is a software blueprint that can be used by the programmer(s) to implement the system without having to refer back to the specification and without having to make unwarranted assumptions about the requirements.

2.4 Coding and Implementation

The relationship between programming languages and programming style became apparent during the 1970s and was reflected in advances in both areas. The most visible aspect of software engineering was termed “structured programming.” Although this term originally referred to the entire process of program development, its meaning was corrupted to refer almost exclusively to the implementation phase. “Structured coding,” then, was marked by a set of general and language-specific guidelines for programming, based on the recognition that programs must be read by humans as well as by computers and that programming style strongly influences the ease with which programs can be read, tested, and/ or modified. This observation was reflected in the area of programming language design. Beginning with Pascal, a large number of programming languages and dialects were designed and implemented, with many different application areas in mind (Wirth, 1971; Liskov et al., 1981; Wasserman et af., 1981; Brinch Hansen, 1975; Lampson et af., 1977). The design of Ada drew upon a number of these other language designs, incorporating many of the key advances in programming language design, as well as making several innovations of its own (Ichbiah, 1981).


115

2.5 Quality Assurance: Testing and Verification

Attempts to assess the correctness of programs may be divided into two separate categories: testing and verification. Verification is a formal mathematical proof that the program is in conformity with its specification; testing, by contrast, is a series of controlled experiments that seek to provide empirical evidence that a program behaves properly (and provides the desired results for broad classes of anticipated inputs). Verification, for the most part, is limited to small, mathematically oriented programs in which it is possible to provide a precise specification; these constraints presently rule out verification for artificial intelligence programs and information systems, which typically involve exceptional conditions (such as missing values), data bases, user interaction, and other such complications. Progress in verification has been hampered by the shortage of people who are able to construct program proofs, by the small number of automated verification aids, and by the general lack of acceptance of program proofs in the computer science community (DeMillo et al., 1979). While greater progress has been made in the area of testing, it, too, is far from being a well-understood activity. Testing is normally done in three stages: module testing, integration testing, and acceptance testing. In module testing, individual program units are tested for correctness. In integration testing, two or more modules are joined and tested together, to see if they work properly together and to make certain that the interfaces mesh. Finally, acceptance testing determines whether the system conforms to its specification (Myers, 1979). In general, errors found in module and integration testing reflect errors made during design or implementation, whereas errors found during acceptance testing reflect specification errors-incomplete, inconsistent, incorrect, or ambiguous statements of what the system was to do. The most serious aspect of this situation is that the errors that were made first are detected last! An error in the requirements definition may not be caught until the entire system has been constructed and tested; such an error may require massive changes in the system design and implementation. It is for this reason that analysis and design errors are the most expensive kind of errors, and that efforts such as formal reviews of design and code have a significant payoff in terms of development costs. 3. Management Procedures A set of management techniques is necessary to carry out the various phases of the life cycle and to assure that the desired system is produced.

116

ANTHONY 1. WASSERMAN

Proper management can result in effective deployment of project personnel, predictability of project schedule, budget, and outcome, accurate estimation of software properties, and a product that meets the needs of its users throughout the lifetime of the system. Management of software development involves both management of people and management of the software product. The former involves issues of supervision of individual and project progress and selection of appropriate team organizations and individual assignments. The latter focuses on reviewing a series of work products, deciding when a system is ready to be released, and controlling the means by which it (and its subsequent versions) are modified and released. Finally, the management activity includes selection and revision of the technical procedures that are to be used for system development. 3.1

Personnel Deployment

Within any given development organization, the management must determine how to structure the organization and what assignments to give to individuals within that organization. Historically, most organizations have had similar structures, with project teams consisting of 5 to 10 persons. The patterns of promotion have been from programmer to analyst to successive levels of management. Such promotion patterns often fail because the skills required for the different tasks vary greatly. A person who is a good programmer is not necessarily a good analyst or a good manager. A programmer who is knowledgeable about all of the low-level details of a specific machine and the characteristics of a specific programming language processor is often happiest when permitted to continue with nuts-and-bolts tasks. When that person is promoted to a job that involves interpersonal skills or dealing with nontechnical issues, the programmer does poorly. Similarly, persons with good managerial skills or problem-solving abilities are frequently unable to follow through with the task of constructing a correctly functioning program. Finally, there is wide variation in the individual skills of programmers-often an order of magnitude or more-in the performance of specific tasks. Thus management must find appropriate ways to use people on different projects and must accommodate the variation in staffing levels that occurs on most projects. In addition, organizations must create career paths for persons who do not wish to switch between the technical and managerial side of projects. A person with a very specific set of skills can be valuable to a large number of projects over time and can be most valuable to an organization by using those skills on successively more


117

challenging projects. (Many of these topics are treated in more detail in the collection compiled by Curtis, 1981.) 3.2 Cost and Schedule Estimation

In the eyes of customers and top management, accurate estimation of delivery dates and costs for a software development project are the most critical tasks of software project management. Product marketing, integration of software into an embedded system, and other activities are heavily dependent upon the accuracy of these estimates. Historically, though, these estimates have been extremely poor, largely because there was little prior experience from which to draw on a specific system and because there were very few metrics. Delivery dates for completed software were often set arbitrarily, with little awareness of the time that it would take to design, build, and test a complex software product. Such unrealistic deadlines could not be met, and efforts to do so led to poor-quality software. In attempting to minimize development time, many important development steps had been performed hurriedly, if at all. Systematic testing, in particular, was often given short shrift, and many errors were discovered by system users after product release, often resulting in extensive maintenance expenses for the developers. Estimates of effort were based either on “required” delivery dates or on a measure such as lines of code. The “lines of code” measure was derived from the fact that programmers could be expected to deliver a fixed number of lines of code per day, depending upon the nature of the application. Thus an estimate of the total number of lines of code for the system could be used to estimate the total number of persorbdays for the project. This scheme was most undependable for innovative applications, since it was difficult to estimate the eventual size of the system. Furthermore, estimates of size tended to be low, as the estimates failed to account for error handling, user interfaces, and other necessary code that was not a central part of the system. Recent work in this area has led to economic models of software development (Boehm, 1981) and to numerous metrics intended to help assess a project size and complexity (Basili, 1980). 3.3 Evaluation of Project Progress

A key advance of the phased approach to software development is that there may be milestones (work products) associated with each project phase. Without such products, it is difficult for a manager to see how well either individuals or the entire project team is progressing. In the past,

118


programmers worked independently and only gave subjective estimates of their progress; their code and documentation was “private property,” rather than shared among the members of the group. The intended effect of a life cycle approach is to make the intermediate products of the development process more visible, permitting teamwork within a group, including sharing of designs and code and allowing formalized review procedures. Review and evaluation of project progress can be conducted either formally or informally and may take place at one or more points during the system development activity. In any event, the objective of the review is to identify errors, locate potential problem areas, and measure progress against objectives. Different techniques are appropriate at different stages. During the analysis and specification stage, evaluation can be performed by the customers against the developer’s functional specification or a prototype version of the system. An author-reader cycle (Ross and Schoman, 1977) provides another approach to evaluating work and recommending changes. During the design stage, it is common to conduct design reviews and structured walk-throughs (Yourdon, 1979; Myers, 1978) to locate errors. During implementation, code inspections (Fagan, 1976) are useful to locate instances of poor programming style or incorrect logic. These manual, team-oriented techniques can serve as a valuable check against both major and minor errors in system specification, design, and implementation. 3.4 Release Control

The development of any medium to large software system involves the coordination of a large number of items, including user requirements, life cycle requirements, budgets, schedules, formal specifications, contracts, design documents, program code, test data and test results, user documentation, change requests, and so on. Furthermore, the software product release may include hundreds of separate files. Even if a single individual is responsible for all of these items and even if all of these documents are stored on-line in compatible format(s), it is very easy to become overwhelmed by the volume of the documentation and the complexity of the system. The problem is complicated by such things as the discovery and repair of errors in the middle of the distribution process. The problem of control becomes even more complex when a given system is intended to run in more than one execution environment and when the system is being modified during its lifetime. In such a situation, the assemblage of documentation and code representing a specific release


119

of the system is a significant task, as is the decision concerning when to release new program modules or a new system to replace existing versions. Effective management of this problem requires some form of configuration management (Bersoff et a f . , 1979), a formalized procedure that identifies every item associated with the project and keeps tracks of it. The configuration management process makes it practical to keep track of multiple versions of a system and to maintain project control that gives authorization for each modification to the “current” system.

4.

Software Development Methodology

The technical methods and management procedures are at the heart of a software development methodology, which, in turn, is at the heart of the software development environment. Many organizations are presently working to create their own methodologies that they can follow from the original concept of a system through its specification, design, development, operation, and evolution (Lundeberg et al., 1981; O’Neill, 1980; Wasserman, 1980a,b; Yourdon, 1982). A methodology typically consists of a sequence of steps combining management procedures, technical methods, and automated support to produce information systems, as shown in Fig. 1 . For some organizations, the methodology is well defined and well structured, with review procedures and intermediate products prior to completion of the system: in other organizations, the methodology is ill defined and poorly structured. In any event, though, it can be seen that technical methods, management procedures, and automated support form the cornerstones of the information system development methodology. Such a methodology includes technical methods to assist in the critical tasks of problem solving, documentation, hierarchical decomposition, design representation, coding, systematic testing, and software configuration management. Such a methodology also includes management procedures to control the process of development and the deployment of these technical methods. The management and technical aspects of the methodology have a synergistic relationship in that the technical methods provide the intermediate results that are needed for effective managerial control, while the management procedures serve to allocate technical resources and support the development organization. Finally, automated support exists for the purpose of enhancing the effectiveness of the developer, with technical needs serving to drive the

120


MANAGEMENT PROCEDURES

1

I Provide Visible Structure

Selected Automatec Tools Provide Management Reports

Coordinatc and Guide

DEVELOPMENT METHODOLOGY

1 AUTOMATED TOOLS

TECHNICAL METHODS

I

Determine Needed Tools

FIG.1. Components of a software development methodology.

development of new automated tools and the acquisition of new computer systems. Management procedures determine the nature of the automated support that is provided, both in terms of the computer system to be used and the languages and tools that comprise the software to be used in support of the system development effort. Regardless of the item being produced or the details of the methodology, it is nevertheless possible to identify a number of desirable characteristics for an information system development methodology. These desirable characteristics include the following. (1) Support for Problem Solving. The methodology should support effective problem-solving techniques. It should encompass intellectual processes such as abstraction, partitioning (modularization), classification, and hierarchical decomposition. The methodology may be oriented to a specific universe of discourse, e.g., message-based systems. ( 2 ) Life-Cycle Coverage. The methodology should cover the entire software development cycle. It does relatively little good to have a meth-


121

odology for software design if there is no systematic procedure to produce the specification used for the design and/or the executable program that must be created from the design. Thus a methodology must assist the developer at each of the stages of the development cycle. (3) Support f o r Phase Transitions. The methodology should facilitate transitions between phases of the development cycle. When a developer is working on a particular phase of a project (other than requirements analysis), it is important to be able to refer to the previous phase and to trace one’s work. At the design stage, for example, one must make certain that the architecture of the software system provides for all of the specified functions; one should be able to identify the software module(s) that fulfills each system requirement. During implementation, it should be easy to establish a correspondence between modules in the system design and program units, and between the logical data objects from the design stage and the physical data objects in the program. It is important to note that one must be able to proceed not only forward to the next phase of the Life cycle, but also backward to a previous phase so that work can be checked and any necessary corrections made. This phased approach to software development makes it clear that information lost at a particular phase is generally lost forever, with an impact on the resulting system. For example, if an analyst fails to document a requirement, it will not appear in the specification. Eventually, during acceptance testing (or perhaps during system operation) that failure will be recognized and it will be necessary to make modifications to the system. (4) Support f o r Validation. The methodology must support determination of system correctness throughout the development cycle. System correctness encompasses many issues, including not only the correspondence between the system and its specifications, but also the extent to which the system meets user needs. Accordingly, the methodology must not only be concerned with techniques for validation of the complete system, but also must give attention to obtaining the most complete and consistent description of user needs during the early stages of the project. For example, the methods used for analysis and specification of the system should aid problem understanding by the developers, the users, and other concerned parties, and make it possible to trace later system development back to the requirements and specification. ( 5 ) Support f o r the Software Development Organization. The rnethodology must support the software development organization. It must be possible to manage the developers and the developers must be able to work together. This requirement implies the need for effective communication among analysts, developers, and managers, with well-defined steps

122


for making progress visible throughout the development activity. The intermediate products generated by the methods and tools, such as a detailed design or an acceptance test plan, can be reviewed by the organization so that progress can be effectively measured and so that quality can be assured. (6) General Applicability. The methodology must be repeatable for a large class of software projects. While it is clear that different methodologies will be needed for different classes of systems and for different organizational structures, an organization should be able to adopt a methodology that will be useful for a sizable number of programs that they will build. Certainly, it makes little sense to develop a methodology for each new system to be built. (7) Teachabilify. The methodology must be teachable. Even within a single organization, there will be a sizable number of people who must use the methodology. These people include not only those who are there when the methodology is first adopted, but also those who join the organization at a later time. Each of these people must understand specific techniques that comprise the technical aspects of the methodology, the organizational and managerial procedures that make it effective, automated tools that support the methodology, and the underlying motivations for the methodology. (8) Automated Support. The methodology must be supported by automated tools that improve the productivity of both the individual developer and the development team. This collection of tools, and the way in which they are used, constitute a “programming support environment.” (9) Support for System Evolution. The methodology should support the eventual evolution of the system. Systems typically go through many versions during their lifetimes, which may last 8 to 10 years or more. New requirements arise from changes in technology, usage patterns, or user needs, and these changed or additional requirements must be reflected in a modified system. The development methodology can assist this evolutionary activity by providing accurate external and internal system documentation, and a well-structured software system that is easily comprehended and modified by those making the system changes. The software development environment, which provides a framework for describing the way that software developers make use of a software development methodology, can be seen to be different for every organization, as it is dependent upon the individuals who comprise that organization. More importantly, it can be seen that the environment is affected by every change, no matter how minor, ranging from the use of new computer equipment to new hirings.


123

5. Automation in the Development Environment

The notion of an automated development environment, an integrated collection of computer-based tools that support software development, is subsumed within the notions of a software development methdology and a software development environment. This situation is as it should be, since automated tools do not normally exist as an end in themselves but, rather, as a means to an end. In its most primitive form, automated support for software development is simply a processor that executes code written in some machineprocessable notation (perhaps assembly code). A batch programming environment provides little more: some operating system support, some device handling, and language processing. At the other end of the spectrum, an automated development environment provides the developer with a variety of software tools, communications facilities, and functions of the “electronic office” that go well beyond the basic task of program construction.

5.1 Software Tools

Modem development environments are typically implemented on a time-sharing system and provide the developer with a collection of tools. This tool collection usually takes one of two forms; a tool system or a tool kit. In a tool system, all of the tools are organized in support of a development task, typically in support of a single programming language. Each tool has a single purpose and is expected to be used in a certain way. In a tool kit, the environment contains a large number of tools, possibly in support of a variety of programming languages, but the developer has considerable freedom in deciding when and how to use the various tools. In either case, though, the key objective is to provide programmers with tools and techniques that can both improve the process of software production and the quality of the resulting software. Among the most common examples of such tools are text editors and compilers; file copying and printing routines are among the most common utilities. Examination of a number of tools, development environments, and proposed environments is useful in enumerating some specific tools that should be part of an information system development environment. These tools include the following. Operating system level support. An operating system that provides access to shared resources, standard interfaces to input-output devices,

124


and program linkage and loading facilities is basic to any system development environment. Data base management system. A data base management system can be used both by the tools in the development environment and by the systems developed in the environment. Language processors. Compilers andor interpreters, along with runtime support are needed for all of the programming languages that are to be used in the information system development environment. Text editor. A text editor is essential for all of the documentation associated with the system development, as well as for the program code. Some programming environments may provide syntax-directed editors that are able to check program input for conformity to the syntax rules of one or more languages which are used during the development process. Formatters. Formatting programs are useful for both documentation and programs. Formatting programs support processing of tables, equations, and text, providing a means to produce documents for reproduction andor phototypesetting. A “pretty printer” accepts program text as input and produces a reformatted program text as output, performing any needed indenting or line splitting to enhance the readability of the program text. Static analyzer. A static analyzer is able to examine program text to determine whether there are any program variables that are used before they are assigned a value, any program variables that are assigned a value and never used, or any segments of “dead code” that cannot be reached through the program’s control flow. Such a tool is valuable in uncovering errors and improving program structures. Dynamic anatyzer. A dynamic analyzer is able to provide information about a program during its execution. One can obtain a snapshot giving the values of program variables, a trace that shows the program units that have been invoked, an analysis of the execution time spent in each program unit, or a count of the number of times that specific lines in the program have been executed. With a suitable graphics-based tool, one could observe the dynamic execution of a program seeing accesses to program units or data objects. All of this information is useful for uncovering errors and tuning the system to enhance performance. Similar dynamic analysis tools exist for data base management systems, making it possible to count the number of input-output operations, the number of pages fetched, or the elapsed time for various data base operations. Configuration management. A configuration management tool may be used to keep track of the myriad documentation associated with any system development project, including one or more versions of the source


125

and object code. Such configuration management, as noted above, is an important aspect in controlling changes to the emerging system. Logging tools. Logging aids may be used for system auditing and as a way of determining the use of various programs in the development environment. Logs may keep track of user access to the system and can assist in uncovering any security violations of the system. Development dictionary. A development dictionary may be used to store information on data items, data structures, functions, and other information gathered through the development process. Information can be entered into a dictionary beginning at the analysis phase, as one identifies data elements and data flows. System functions specified early in the development process can be linked to modules in the design phase and procedures or functions in the implementation phase through the development dictionary. This list is far from complete, since one could provide useful tools for specification, architectural design, detailed design, data base design, program verification, project management, and library maintenance, but it is intended to be suggestive of the types of tools that can aid the developer in the effective production of high-quality software systems. If such a collection of tools can be designed in a harmonious way, then the automated development support system should have a significant favorable impact on the development process. We shall next look at some tool systems, beginning with an overview of programming systems and continuing with an overview of INTERLISP and Smalltalk, two particularly powerful programming systems. We will then look at the tool kit approach, concentrating on Jhe Unix‘ environment and its tools. Finally, we describe the requirements for supporting the construction of Ada2 programs, which may be achieved with either the tool system or the tool kit approach. 5.2 Programming Systems

Among the oldest concepts in programming environments is that of a “programming system.” A programming system is an integrated collection of facilities in which all of the facilities available to the programmer are those in support of the development and execution of program written in a single programming language. Early systems such as JOSS (Shaw, 1964) and QUIKTRAN provided the user of a time-sharing system with a single language processor. MUMPS (Greenes et al., 1%9) was a success1

Unix is a trademark of Bell Laboratories.

* Ada is a trademark of the U.S. Department of Defense.

126


ful effort to provide time sharing in a high-level language on a minicomputer. More recently, high-level language support on microprocessor-based systems, including BASIC and Pascal, has been organized into a programming system. In addition, some of the most sophisticated programming environments, including INTERLISP and Smalltalk (Kay, 1982; Tesler, 1981), have taken a similar approach. The earliest models of programming systems offered two different modes of operation: “direct mode,” a desk-calculator-like approach in which statements would be directly interpreted and executed, and “program mode,” in which statements were stored for later execution. If the statement were just typed, it would be executed at once; if it were preceded by a special character or a numeric label, it would be stored. The options available to the developer were severely restricted, sometimes limited to the following minimal set:

(I) Save a program by name. (2) Retrieve a program by name into a work space. (3) Execute the program in the work space. (4) Display the program (or some portion of it). ( 5 ) Quit. The text editor was very rudimentary and operated in a line-oriented manner. Every line to be saved was numbered, and lines would be stored in numerical order. If a line was to be corrected, one simply typed the line number and the corrected code. If insertions were needed, then an appropriate line number was chosen. If deletions were to be made, then the line number alone was typed. Despite these limited capabilities, such an approach has some very definite strengths. Two of these are worth noting. First, the programming system approach makes it possible to direct all of the resources of the computer system toward the design and execution of programs written in a single language. Second, the programming system itself can be simple and small, making it easy to learn, to use, and to run on a machine having limited resources. As a result, there have been numerous efforts to create more powerful programming systems. Most of these efforts may be characterized as the assemblage of a powerful, unified collection of tools in support of the programming task. These tools include editors that “know” the syntax of the programming language, debugging tools that can save and restore information about a program at arbitrary points during its execution, and specialized processors to support the efficient execution of suqh programs.


127

5.2.1 INTERLISP

INTERLISP and Smalltalk provide particularly sophisticated facilities among the multitude of programming systems. The INTERLISP system builds upon an interpreter for the LISP programming language, which has been widely used for more than a decade for the development of expert systems. The system is used in dozens of sites, almost exclusively within the artificial intelligence community. Warren Teitelman, the principal architect of INTERLISP, based the system on several key perceptions (Teitelman and Masinter, 1981): (1) Typical LISP users were engaged in experimental, not production, programming. (2) Users were willing to expend computer resources to improve human productivity. (3) The application development paradigm did not follow the life cycle model, since the desired behavior of these expert systems could not be specified in advance.

Instead, programs followed what Sandewall (1978) has termed “structured growth” and programmers primarily needed tools that supported these coding and evolution activities. Thus the programming system approach was ideal, and an integrated set of tools was developed. The unique properties of INTERLISP hinge on the extent to which the tools are integrated and the ease with which individual programmers can make personalized extensions to the tools. In addition, the LISP language facilitates such tool building. Since LISP makes no distinction between programs and data, it is then very easy to treat programs as data. One can write a LISP editor in LISP and embed within the editor the fact that the text being edited should be correct and readable LISP. Such an editor can even free the LISP programmer from typing all of the parentheses required in pure LISP, permitting the programmer to type a statement or function in one format and editing it into proper LISP, arranging it into a “pretty-printing” indentation scheme at the same time. Among the more powerful facilities in INTERLISP (Teitelman, 1978) (all of which would be useful in other programming systems) are the following: (1) The Programmer’s Assistant. A tool that “listens” to the user input, keeping a record of user inputs and effects of operations, including side effects. The programmer can redo a sequence of steps, with or without changes, using this history facility.

128


(2) DWIM (Do What I Mean). An error-correcting “package” that attempts to make “reasonable interpretations” of user input, ranging from spelling corrections to provision of default parameter values. DWIM may be seen as an extension of the programmer’s assistant concept, in that both embody the idea that the user is interacting with an active intermediary (agent). ( 3 ) Masterscope. An interactive program for analyzing and cross-referencing programs using a data base of results from its analyses. It permits programmer queries and can assist in low-level modifications to LISP programs. 5.2.2 Smalltalk

Whereas INTERLISP is aimed at the expert programmer, Smalltalk is aimed at the novice and the casual user. Smalltalk was developed as a “language” for the Dynabook, a proposed portable personal computer with substantial processing and storage capability. Smalltalk (apd the Dynabook) were the ideas of Alan Kay, who developed them throughout the 1970s while he was at Xerox Palo Alto Research Center. Although Smalltalk has not been widely used until now, its design has influenced many other systems, and several organizations have now been licensed to manufacture and distribute Smalltalk systems. For example, the new Apple Lisa computer system draws heavily on the Smalltalk concepts. The organization of Smalltalk is object oriented, based on the concept of objects that communicate with one another through messages. Memory in Smalltalk is also in terms of objects. Each Smalltalk object belongs to a class hierarchy. Everything in the Smalltalk universe is a legal object, so that they may all send and receive messages. The result is a highly uniform system. Th’e power of Smalltalk lies not so much in the language, but in the higher-level collection of facilities, termed “kits.” A kit may be seen as a collection of objects and messages that make it possible for the user to perform useful tasks. Smalltalk kits exist for such diverse tasks as algebra, document editing, animation, and programming. Kits may be used and/or modified to create applications. Thus Smalltalk systems are structured into several layers. Applications can be modified by fixing the kit without descending to the lower levels. In that sense, Smalltalk may be seen as a kit-writing language and thus carries the idea of a programming system to a logical conclusion. As with INTERLISP, Smalltalk deviates from the life cycle model, assuming that the userfprogrammer is simply seeking a problem-solving medium. In a11 cases, though, the desired result is a properly functioning program.


129

5.2.3 Summary: Programming Systems

Thus the programming system is an extremely powerful means of building systems, particularly when the systems are extremely well specified or, for some reason, impossible to specify. If a specification must be developed, though, the user of a programming system must rely upon other methods, using the programming system just for implementation. Especially for systems to be developed by a single individual, a programming system is very useful, permitting rapid implementation of systems with relatively little overhead. Larger systems, though, tend to involve more extensive teamwork and documentation, and the value of the programming system approach seems to diminish with increasing system size. (This is not to say that one cannot build large systems in this way, but only that the tools provided by the programming system alone may be insufficient .) A second potential disadvantage to programming systems is posed by the restrictions of a single high-level language. If the particular language provided by the programming system is unsuitable for a given application, that is just unfortunate. If one needs access to lower-level machine resources, that is often impossible. Attempts to develop sophisticated applications in unsophisticated programming systems have frequently led to failures. 5.3 The Tool-Kit Approach: Unix

Rather than providing the developer with a collection of tools aimed at use of a single language, one can envision a more general time-sharing setting in which developers may use various languages along with generalpurpose tools. Rather than having a language-knowledgeable editor, for example, a general-purpose text editor is present. Such an editor may be used for cpmposing programs, writing documentation, or corresponding with friends. Most widely available time-sharing systems are based on this approach: the programmer has great latitude for many choices, since the system is designed to satisfy a broad range of needs, rather than the more specialized needs of a programming system. The Unix operating system (Ritchie and Thompson, 1974; Kernighan and Mashey, 1981) and its associated tools are an excellent example of such a toof kit. All of the tools and the underlying operating system were built with a highly consistent design philosophy that uses a single, uniform file format almost exclusively. This design facilitates the use of various tools in conjunction with one another, often in unpredictable ways.

130


In considering the structure and features of future software engineering environments, it is important to observe the reasons for the widespread acceptance and use of Unix and its tools. These reasons include the following: (1) Unix is quite small. It was developed to run on small systems and to use a small amount of memory (under 50 kbytes). Accordingly, it is a relatively easy task to read, understand, and modify the operating system itself, which is not the case with most other operating systems. (2) Unix is written almost entirely in a high-level language, contributing to system comprehensibility and modifiability. Source code is routinely distributed with the Unix license. Portability is also greatly enhanced. (3) Unix comes with a very large set of tools, with tools for word processing (including phototypesetting and mathematical formulas), language development, and interprocess communication. The basic Unix system includes processors for C , Fortran 77, Snobol4, and BASIC. In addition, many other useful tools, including screen-oriented editors, data base management systems, programming language processors, and graphics support, as well as elaborate games, are available at little or no cost through the Unix community. Many Unix sites, particularly in research environments, are linked by uucp (Unix-to-Unix copy) programs that support network electronic mail and news. (4) The basic user interface is through a program called the “shell.” Different shells can be provided to different users and users can customize their own shells. It is very easy to use a shell as a programming language and to create scripts representing a sequence of Unix commands. (5) The environment supports the combination of small, reliable software components into larger pieces, with reduced need for the construction of large software systems from scratch.

These features and others have made Unix a popular base for additional tools and for the integration of these tools into a tool system. One major extension to Unix is the Programmer’s Workbench (PWB) (Ivie, 1977), which was intended to make Unix more hospitable to the tasks of program development and maintenance, even if the program being developed were to be run on a machine other than the development system (hostltarget machine). The PWB/Unix environment then becomes a machine for software development. The Programmer’s Workbench added several features to the Unix environment to support this role:


131

(1) Job submission. Linkage with a remote computer system, permitting transmission of data to and from the development system. (2) Source code control. The ability to keep track of versions and releases of source code routines (Rochkind, 1975). ( 3 ) Program construction. The ability to declare dependencies among program units so that necessary recompilation and linking will be performed automatically in response to the “make” command when changes have been made to some part of a system (Feldman, 1979). Unix has been used as the underlying support for tool system efforts as well. As one example, INTERLISP is now available in the Unix environment. Similarly, projects such as Gandalf (Habermann et al., 1979) and User Software Engineering (Wasserman, 1982) have been built upon Unix by adding a “controlling” tool that supports a specific methodology for software development. As described in greater detail below, Unix has also been used as the operating system and tool framework for several “personal development systems.”

5.4 Tools for Supporting Ada

Much recent effort has been placed into specifying the characteristics of a programming support environment, i.e., tools, for Ada. Ada is a highlevel programming language designed to meet the U.S. Department of Defense’s requirements for a standard language for the construction of “embedded” computer systems, such as those on shipboard, in aircraft, and in electronic toys. Ada is intended to represent the “state of the art” in programming language design and incorporates facilities for modularity (package), exception handling, tasking, and interrupt handling. Code production, of course, is a relatively small factor in the overall process of software development and maintenance. Accordingly, the U.S. Department of Defense sponsored the development of a set of requirements for Ada Programming Support Environments (“Stoneman”) (Advanced Research Projects Agency, 1980). The Ada Programming Support Environment (APSE) is intended to be a machine-independent tool set that supports the construction of Ada programs and, optionally, a methodology for system development. The underlying philosophy includes a hostkarget machine approach to software development, in which programming support for development may be provided on a machine (host) other than that where the Ada program will be executed (target). A key integrating notion is that of a data base, which stores all relevant

132


information concerning a project throughout its life cycle.3 The APSE data base may hold a number of different objects, such as specifications, source code, object code, and user documentation, which may be grouped to form “version groups.” As shown in Fig. 2, the APSE may be viewed as having three levels. The first level, termed the Kernel (KAPSE), provides basic operating system and data base functions, the framework in which the tools are used. The KAPSE provides the data base, communication, and run-time support functions to enable the execution of an Ada program. The KAPSE yields a view of a virtual machine, providing the tool developer with a set of primitive functions that programs (tools) can invoke and the data base operations available to these programs. In other words, the KAPSE provides a level of system and tool portability, since tools do not have access to low-level machine-dependent functions. The second level, termed the Minimal Tool Set (MAPSE), gives the developer the basic set of tools needed for the development and execution of Ada programs. Tools provided in the MAPSE include a text editor, an Ada compiler, a linker, a loader, a source language debugger, static and dynamic analysis tools, a configuration management tool, and a command interpreter (similar to the Unix shell). The MAPSE should provide project portability, in the sense that the entire development project could be moved to a new machine; this becomes possible since the tools provide a consistent user interface, regardless of their host machine and implementation details. The feasibility of this approach has already been demonstrated by the Software Tools Project (Hall et al., 1980). The APSE is then a full environment based upon a particular MAPSE. An APSE should provide tools above and beyond those of the MAPSE and may support a particular methodology. Thus the APSE may provide graphical tools to support the data flow diagrams and structure charts of structured design (Yourdon and Constantine, 1979) or to check the consistency and interfaces of a program design language (Caine and Gordon, 1975). Since the MAPSE and KAPSE focus almost entirely upon the coding process, any tool support for performance measurement, project control, or specification aids would be part of the APSE. Taken together, the three levels of the APSE should improve the productivity of the developer by providing effective tools, and should en-

’

The Stoneman usage of the term data base should not be used in the same sense as it is used in the data base community. The notions of data modeling and data independence have not been advanced in discussions of an APSE. Furthermore, preliminary investigations indicate that existing data base management systems would not be especially useful for storing all of the information gathered during a programming project.


133

FIG.2. Multilevel structure of the Ada Program Support Environment (reprinted from Advanced Research Projects Agency, 1980).

hance project and programmer portability by providing compatible tools and user interfaces. Work is proceeding (Ada Joint Program Office, 1982) to identify the characteristics of methodologies that should be used in conjunction with these support environments to yield the best possible systems in Ada.

6. An Example: The User Software Engineering Methodology

Many of the ideas described so far can be illustrated by a brief overview of the User Software Engineering (USE) Project, developed by the author at the University of California, San Francisco. User Software Engineering is a methodology, supported by automated tools, a Unified Support Environment, for the specification, design, and development of interactive information systems. An interactive information system (11s) may be characterized as a program that provides its users with conversational access to data. In addition to the general goals of software development methodologies

134


described above, the goals of the USE methodology include support for effective user involvement in the development process. User Software Engineering is a synthesis of ideas, combining some ideas used successfully in other methodologies with some original aspects. User Software Engineering attempts to combine the systematic approach to software inherent in the life cycle approach with the rapid construction approach followed in the INTERLISP and Smalltalk environments. The problem with the life cycle approach is that there is typically a long delay between the specification of a system and its completion, leaving the user without a useful tool. When the system is built, it is often quite different from what the user expected or wanted, making it necessary to redo a large portion of the system. On the other hand, the rapid construction approach often fails to generate the documentation and intermediate work products that are needed for effective management of the software development process. User Software Engineering tries to strike a balance by using a prototype system as an aid to analysis and specification and making that system useful to users throughout subsequent stages of a life cycle approach until a production version of the system is completed. (Note that the prototype system might be satisfactory for the users, and that it might not be necessary to proceed to a final system, at least in some cases.) 6.1 USE Methodology Overview

The steps of the USE methodology can thus be summarized as follows: Preliminary analysis-activity modeling and data modeling, identification of user characteristics Initial data base and dialogue design Creation of a “facade,” a mock-up of the user-program dialogue with revisions as needed Informal specification of the system operations using narrative text Creation of a “prototype” system, providing at least some, and possibly all, of the system’s functions Formal specification of the system operations Architectural design Detailed design Implementation in PLAIN Testing and verification Some of these stages are handled by existing techniques, while others are specific to the USE methodology. For example, there are many effec?


135

tive techniques for analysis; we have successfully used Structured Systems Analysis (Gane and Sarson, 1979; DeMarco, 1979)for activity modeling and the Entity Relationship (ER) model (Chen 1976) or the Semantic Hierarchy model (Smith, 1980) for data modeling. In short, USE is not prescriptive about the analysis method(s) used-analysis is simply a means to obtain enough information to create a facade of the IIS, to begin creation of the USE development dictionary, and to make a preliminary specification.

6.2 The USE Specification

The format of the specification is fixed, however, A specification of an IIS is seen to consist of three parts: the user-program dialogue, the data base design, and the operations (transactions) associated with various user inputs. The interaction is described in a set of augmented state transition diagrams, each of which is termed a conversation. Various user inputs may cause state transitions, including the invocation of a “subconversation” (another diagram). Actions may be associated with a transition, so that all of the operations may be attached to transitions. We therefore took advantage of the easy encoding of transition diagrams to build an automated tool, the Transition Diagram Interpreter (TDI), that could “execute” the diagrams, thereby providing the user with the user interface to the program at a terminal. The Transition Diagram Interpreter accepts the encoding of one or more diagrams and produces an executable program simulating the specified interface. This “mock-up” of the user-program dialogue can be easily modified to meet user needs. The data base is described as a set of normalized relations (Codd, 1970), mapped from either the ER or the Semantic Hierarchy model. Both the transition diagrams and the data base design are comprehensible by users with some assistance from the analysts who create them. For operations, though, the comprehensibilityproblem was much more difficult. An informal means of specification, such as narrative text, would be comprehensible by end users and by other computer-naive persons needed to understand the specification; however, the informality was not good for software development, where the precision of a more formal notation is useful. On the other hand, such a formal notation would be rejected by the users, who would not be able to understand a mathematically based notation. The solution taken is to provide both an informal and a formal specification for each operation. The informal approach is simply a short para-

136


graph (two or three sentences maximum) of narrative text, while the formal approach follows the Basis approach (Leveson, 1980; Leveson et al., 1983). Basis uses a formal notation employing preconditions and postconditions in conjunction with a description of the behavior of operations similar to that developed for Alphard (Wulf et al., 1976). Each operation is placed on a single page, showing first the informal description and then the formal description. For those operations involving data base access andor modification, the data base operations may be shown in a data manipulation language, either relational calculus or relational algebra.

6.3 The Role of Prototypes in USE

Experience with this specification approach showed that users were indeed able to review the specification, and that it was sufficiently precise to guide the subsequent development. However, it seemed that the users did not really get a true sense of how the system would work from the written specification alone. Furthermore, attempts to show error handling and on-line assistance in the diagrams greatly increased diagram complexity and reduced their understandability. These observations led to the creation of TDI and the inclusion of the facade as a methodology step. Although the facade is extremely useful and gives the users a much improved sense of what the system will be like, it had several shortcomings. Among these shortcomings were the inability to provide realistic output messages with TDI and the inability to program the system functions easily. Although all of the system functions could be programmed directly through the actions associated with transitions, that approach yielded few benefits over direct programming of the system by traditional means. The goal was to quickly and reliably perform the major system functions. A key observation was that many of the operations involve data base access andlor modification, so that the desired functionality can be provided by combining TDI with a data base management system. One of the USE tools, used for several purposes in the methodology, is the Trow USE relational data base management system (Kersten and Wasserman, 1981). By linking TDI with Troll, it becomes possible to store actual data in the data base, so that user input can cause actual operations to be performed. In practice, it is necessary to provide some additional operations beyond those of the data base management system, so the linkage mechanism permits routines written in the Troll data manipulation language or in a variety of programming languages.


137

This tool, called RAPID (Rapid Prototypes of Interactive Dialogues), permits a rapid implementation of a significant percentage of the IIS specification with a notation that provides a close match to the specification method itself (Wasserman and Shewmake, 1982). While the system created from RAPID may suffice in some cases, it is frequently necessary to proceed with a production version of the implementation providing a complete set of functions. The goal is to make the interface of the production version identical to that of the precursor version so that the user will not have to learn a different system. While this implementation proceeds, however, the prototype system can be put to good use, for both productive work and user training.

6.4 Design and implementation

The methodology proceeds with architectural design, mapping the highest level transition diagram (main conversation) into the transaction model of structured design. Detailed design then provides a program design language for each module, associating an operation (action) in the transition diagram with a module in the detailed design. The preconditions and postconditions derived during the specification phase are similarly carried over into the modules. As with the specification, the design is reviewed using a structured walk-through before proceeding with implementation. Implementation is then straightforward in PLAIN, since the primitives of the specification method, including strings, pattern matching, relational data bases, transaction, preconditions, and postconditions, have a corresponding primitive in PLAIN. Not only is the implementation straightforward, but the encoding of preconditions and postconditions as assertions makes it easier to verify the correctness of the implemented system. Thus the USE methodology provides a series of steps to support the process of creating an IIS, from its original conception through implementation and maintenance. The methodology is supported by a Unified Support Environment, including TDI, RAPID, Troll, and PLAIN. In addition, other tools exist to assist with project management, including the USE development dictionary, containing information on all data items and functions, and the USE Control System, a version control and configuration management tool that guides the developer in the use of the other tools. All of these tools have been designed and developed to be used in the Unix environment, taking advantage of many of the underlying Unix tools. Future work will make these tools available on personal develop-

138


ment systems (see Section 9), leading to the concept of a User Software Engineering machine, USE2. We therefore see USE as an instance of a methodology comprising technical methods (the methodology steps) and management procedures (e.g., walk-throughs) supported by automated tools. As with the Ada Program Support Environment, the USE environment can be seen to have several levels: Unix and Troll are at the Kernel level, most of the standard Unix tools are at the Minimal Tool Set level, with the USE development dictionary, the USE Control System, and RAPID among the tools at the methodological support level. In this regard, USE is typical of methodologies, addressing some aspects of the development process while ignoring others. Little is said about analysis or testing, and procedures for project management are only lightly sketched. At the same time, the specification method and use of PLAIN are carefully prescribed.

7. The Software Development Environment

It is important to note, though, that success in building systems is dependent as much on the application of the methodology as on the methodology itself. Improper use of the methodology or its tools, unskilled developers, or a poor working environment can all hamper the system development process. It is therefore necessary to give attention to some of these other environmental factors as well, particularly those related to the computing environment and the physical work space. We may think of a software development environment as consisting of a computing environment and a work environment. The computing environment consists of the computer system(s) on which the work is done, the systems programs, the available programming languages and their processors, and any other software tools. The software development methodology is applied within this environment, as shown in Fig. 3. The work environment has an organizational aspect and a physical aspect. The organizational aspect involves project management practices, support personnel such as clerical staff, organizational structure, and technical staff. The physical aspect involves facilities, e.g., libraries, office layout, and ergonomic considerations in the workplace. Thediscipline of ergonomics refers to the mutual adjustment of people and machines; in the subsequent discussion, we shall be primarily concerned with the extent to which the working environment can maximize the comfort of the individual developer. All of these components of the environment may have an effect upon


139

SOFTWARE DEVELOPMENT ENVIRONMENT

YIELDS SOFTWARE PRODUCTS

FIG.3. The software development environment.

productivity of the individual, productivity of the organization, and the quality of the systems produced by that organization. Both individual and organizational developer productivity may be affected by a diversity of factors, including the following:

( I ) The availability of computing resources, including the turnaround time in a batch processing setting or response time in an interactive development setting (2) The software development methodology being employed (3) The administrative overhead of the organization, involving time spent on reporting, planning, and attending meetings

140

(4) (5)

(6) (7) (8) (9)


The availability of in-house meal facilities, which may reduce the time spent at lunch The developer’s typing speed and accuracy in an interactive development setting The complexity of the system being developed The phosphor used in the developer’s video display terminal The similarity of the system being developed to others previously developed The noise level in the working areas

This list makes it clear that a vast number of factors have an effect on the individual developer within a software development environment. Changes in the available operating system, the programming language, the project organization, the format of specifications, or project-reporting standards can all affect how the developer works and the amount of work that can be accomplished. The relationship between the methodology, the organization, and the physical environment is extremely complex. The software development organization must select upon an almost limitless number of possibilities and combinations of management practices, development techniques, and automated support to create and evolve the software development methodology. This methodology is also influenced by a wide variety of other considerations, including the size and skills of the software development organization and the intended applications of the programs being developed, the number of places in which they will be used, the criticality of the applications, and the projected needs for maintenance and modification. Because of the large number of possibilities and the great variation in uses and applications of software, it is not reasonable to expect different organizations to use exactly the same software development methodology. Even groups that follow a specific methodology will modify it to adapt to the particular characteristics of their organizations and projects. Furthermore, methodologies will be changing constantly to reflect new developments in hardware and software, as well as changes in staffing and management structures. The problem, then, for a given organization is how to optimize the effectiveness of the organization for the production of their software. This involves, among other things, selecting the appropriate software development techniques, using developers and support staff effectively designing the physical work space, and choosing suitable computing equipment. This optimization process frequently involves trade-offs (often economic), and changes to one aspect of an environment may have not only


141

first-order but also higher-order effects, since the environment is affected by every change, no matter how small. In short, there is an ecology of software development environments. Changes in the environment have measurable direct and indirect effects. The challenge is to understand those effects and to create environments that promote cooperative and creative effort. An interesting and emerging area of interest in software development environments is the physical work space in which development occurs. In Section 8 we outline some of these considerations, giving particular attention to support services, the developer’s office, and the characteristics of computer terminals.

8. The Physical Environment

Physical work space considerations are important to the creation of a good programming environment; however, there is still much disagreement as to what constitutes a good physical workplace, and there is virtually no solid data to utilize in designing workplaces. Almost all of the opinions offered are highly subjective. Furthermore, there is often significant disagreement on what appear to be simple issues. As an example of this dilemma, consider the problem of an organization providing in-house meal facilities. There are several reasons why an organization would want to arrange for food service, and even subsidize the cost, including the following:

(I) Convenience for the employees (2) Overcoming the lack of suitable nearby alternative facilities (3) Reducing the amount of time that employees spend at lunch (4) Reducing the likelihood of disclosing proprietary information to employees of other organizations (5) Reducing the likelihood that employees will drink alcoholic beverages during lunch

On the other hand, there is a sizable percentage of employees who have neutral or even negative feelings toward on-site food service. Such employees object to the quality of the meals, finding them institutional and bland, and prefer the notion of clearing their heads by getting away from the workplace for an hour, with no thought or discussion of work. Yet even some of these employees will occasionally opt for the food service if it is provided, particularly in inclement weather or when project deadlines loom.

142


In summary, there are likely to be two sides to virtually every issue concerning the physical work space, with the possible exceptions of employer provision of free and convenient parking and a smoke-free environment for nonsmokers. Accordingly, in the subsequent discussion of work space factors, it should be noted that the work space qualities that seem desirable to the author may seem less desirable to others, and that some of the work space factors described as important are not significant in every setting. The underlying assumption is that professional software developers are extremely valuable resources and that organizations should be prepared to invest substantially in their support and professional development. Organizations that skimp on support services, computing equipment, or physical workspace may find that much of a developer's time is spent on unproductive, low-level work, such as making photocopies, or that the developer is frequently interrupted from productive tasks by administrative chores or noise.

8.1 Support Services

The support services of a software development organization are those services that aid the organization's professional staff members in performing their work. These services include computing equipment and software support, as well as clerical help, telephones, reproduction and printing, mail, reference materials, and various other organizational services, including payroll, personnel, and educational services. As one example, the conversion from a batch-oriented development environment to an interactive development environment entails many changes. Typically, computing cost per user and productivity both increase as a result. In the changeover process, though, numerous changes occur, including the following: (1)

(2)

(3) (4) (5)

Reconfiguration of the computer system to accommodate terminals (or replacement of an old system) Changing the operating system or the user interface to the operating system Teaching developers to use terminals (which may include instruction in touch typing) Teaching developers to use new tools in the interactive environment Changing communication patterns through the availability of electronic mail


143

In the interactive environment, it is apparent that system load is a significant determinant of developer productivity, since heavily loaded systems increase response time and reduce machine resources available to the individual developer. Similarly, low-speed communications lines may impair developer productivity, particularly when using highly interactive tools. An investment in additional computing and communications facilities may create a more productive development environment. Certainly, it is difficult to quantify the effect of these support services on developer productivity; yet some of them can have a clear effect. Consider the availability of a telephone system providing automatic call forwarding combined with a message service so that all telephone calls during normal working hours are answered. On a project involving several individuals at several sites, this level of communication is essential over and above any electronic mail service that may be provided by a computer network. Understandably, then, poor support for typing, mail delivery, photocopying, and other similar services can be a source of frustration to a developer, especially when it is necessary to take time from a project to deal with the resulting problems. Such problems are compounded by the psychological makeup of many programmers, whose training and orientation are strongly colored by the need for precision and correctness in computer programs and who often expect similar standards in other areas. 8.2 The Developer’s Office

Since most developers spend a large share of each working day in an office, it is important to give particular consideration to the desirable characteristics of the office. Among the important factors to be considered are the following: Privacy Noise level Size and physical comfort Furniture Lighting Storage facilities Computer connections Communication Terminals Each of these factors is worth some discussion, and the characteristics of terminals deserve a separate section.

144


8.2.1 Privacy

Each developer needs a “home” in the work setting. The home must provide privacy in the sense that the developer can work alone and store personal items securely. The developer’s office is intended to serve this basic human need, providing a shelter as well as a place where that person may be found. There are strong arguments that can be made in favor of giving each developer a private office, i.e., with no one sharing the office and with the ability to close a door against persons seeking entry. First, there are occasions when the developer needs to concentrate on a project for an extended period of time and to be as free from interruptions as possible. Second, as a home away from home, developers often like the ability to personalize their office with photographs, posters, or other possessions; this notion is somewhat akin to the anthropological concept of “territorial imperative.” Third, there are many occasions-whenthe developer simply wants some privacy, perhaps simply to take a break or to conduct some personal business. Both the IBM Santa Teresa architectural design (McCue, 1978) and the TRW Software Productivity Project (Elwell, 1982) concluded that private offices were an aid to productivity. These conclusions run counter to modem trends in office design, which divide large open areas by movable partitions; it increasingly appears that the savings in construction costs are lost in reduced developer productivity. Similarly, shared offices tend to detract from productivity. A corollary of these conclusions is the need for stability. In addition to the time taken for packing and unpacking in office moves, there is a period of readjustment in the new office setting. While some groups joke about their nomadic existences, the time taken in the move frequently amounts to thousands of dollars per person. When a major computer company opened a new building 35 miles from their old site and transferred a software development group there, all of their project delivery dates were moved back, some by as much as three months, while employees moved offices, bought and sold homes, sought out schools for their children, and established new living patterns. There were certainly many additional effects as well, involving friendships, spouse employment, etc. 8.2.2 Noise Level

Different individuals are sensitive to different sounds, frequencies, and decibel levels. Certainly the response of an urban dweller is different from


145

that of someone living in the countryside. Thus, while some programmers can work while listening to loud music, others require absolute silence. In any event, changes in the noise level or continuing high decibel levels (above 80 dB) can be extremely disturbing to an individual’s work. Again, modern office design is not conducive to the occasional need for a quiet working environment. Persons in partitioned office environments are frequently disrupted by conversations in adjacent cubicles, and it is not relevant whether the conversation is concerned with software design, money market accounts, technical documentation, or dinner reservations. Disturbances also come from passing traffic, whether foot traffic outside the office space or vehicular traffic on the street or in the sky. Certain brands of video display terminals emit annoying high-cycle hums as well, disturbing individuals who can hear those frequencies.

8.2.3 Office Size and Physical Comfort

Most studies of space requirements in North America recommend 100 ft2as the minimal size for the individual office, with a minimal ceiling of 7 ft. This amount of space provides enough room for furniture, storage, and a working area. Depending upon the layout, it is often possible to accommodate a visitor as well. Office proximity to vents, fans, and windows often leads to significant variations in office temperatures in a building. Furthermore, there is a wide range of individual preferences for temperature levels in an office. Ideally, each office should contain thermostat adjustments that allow the occupant to adjust the temperature level as much as possible within approved limits. Many organizations have provided as little as 50 ft2 to developers. Limited space, combined with partitioned work areas and a high noise level, can seriously detract from the ability of the developer to accomplish productive work. Furthermore, such inhospitable work facilities make it extremely difficult for such organizations to attract the best people to work for them. The net result is that the skill level and productivity rates are so low that the organizations are at a competitive disadvantage and the quality of their software products suffers by comparison.

8.2.4 Furniture

The basic furniture for an individual’s office includes a large working surface, e.g., a desk or table, a chair, bulletin board and/or blackboard,

146


bookcases or storage shelves, and a surface to accommodate a computer terminal and/or a typewriter. The height of the latter surface must be lower than that of the desk to permit comfortable typing. Of these items, the chair is the most important, since software development is a largely sedentary job. Developers typically spend five to six hours a day seated at their desk or terminals and should therefore have especially comfortable chairs. Very few manufacturers provide ergonomically designed chairs that provide the individual with a wide range of adjustments, and few organizations have purchased such chairs for their employees. (A notable exception is the FAA, which provides each air traffic controller with his own chair.) Strangely enough, such chairs are widely available as an option for many automobiles, but have seen little use in the office. Another consideration is the size of the work surface. Unless it is deep enough and wide enough to permit the developer to open up and work with a computer listing, it will not be useful. In general, the minimal surface for the listing alone is 2 ft2. Within the past year, several manufacturers of office furniture have begun to incorporate these considerations into their product designs and to make it possible for development organizations to provide furnishings with greater comfort. 8.2.5

Lighting

Two kinds of lighting are needed in the office: overall lighting and task lighting. This is generally accomplished with overhead lighting and a separate desk light that can be aimed to provide additional lighting in a specific area. Natural lighting is highly desirable for all work areas, and the IBM Santa Teresa statement of requirements noted that “outside awareness,” i.e., a window, should be provided for as many offices as possible. Indeed, building requirements in some countries state that all offices shall have windows. Once again, most modern partitioned office designs fail to achieve this goal. 8.2.6 Storage Facilities

Developers must store a variety of items in their offices, including computer system manuals, software development guidelines, computer printouts, magnetic tapes, miscellaneous notes, books, and office supplies. The design of the Santa Teresa Laboratory estimated that program-


147

mers, managers, and clerical help each needed about 100 ft3 of storage space, although their specific requirements would vary considerably. 8.2.7

Computer Connections

In most software organizations, software development is done interactively, with each developer requiring access to a terminal. If the terminal is not located in the developer’s office, then the developer must go to another site in the building and contend with others for use of the terminals. In addition, necessary documentation, such as listings and manuals, must either be left behind or carried tothe terminal area. Unless there is good telephone support, the developer may then miss important telephone calls and/or visistors while working at the terminal. The terminal room also tends to be noisier than the individual office. Accordingly, it is desirable to provide each developer with a terminal in the office, or, at a minimum, to provide the connections so that the developer can place a terminal in the office as needed. (IBM’s Santa Teresa Laboratory provided both terminal connections and terminal rooms.) 8.2.8 Communications

In addition to the telephone and computer communications already discussed, human communications are essential in a software development organization. Thus the physical work space should contain meeting rooms of various sizes, ranging from small conference rooms (up to 10 people) to a seminar room (30-40 people) to a larger lecture room (100200 people). There should be minimal bureaucratic overhead associated with obtaining access to these rooms-at most a reservation system with no more than a couple of priority levels. 8.3 Computer Terminals

A large share of the developer’s time is spent at a computer terminal. Thus it is important to design terminals to minimize the occurrence of errors, reduce visual fatigue (a cause of eye strain and headaches), and increase the effective user input and system output speeds. There are a number of ergonomic considerations in the design of the visual display and the keyboard that can affect the developer. The first set of criteria applies to the visual display. Most alphanumeric terminals use a dot-matrix approach to display, ranging from 5 by 7 minimum upward, with 7 by 9 being the most common format for charac-

148


ter generation. The higher the dot density, the more readable the character will appear. The use of ascenders for tall letters and descenders for letters below the base line, such as g and p, is particularly helpful. Character size and spacing are equally important. Character width should be 70-80% of character height, and line thickness should be 1012% of character height. Assuming that one views a terminal from an average distance of 2 ft, the minimum acceptable character height is about Q in. although 6 in. provides substantially better reliability. Thus terminals that display 20 lines of text on a screen 2 inches in height are sufficiently uncomfortable that one cannot use them for very long. There are also important minimum levels for horizontal and vertical character separation. Another characteristic of visual display units is the refresh rate. It is important to maintain a refresh rate that eliminates flicker. While the minimal refresh rate is dependent upon the type of phosphor and the number of lines on the screen, the normal North American refresh rate of 60 Hz is quite good, and the European 50 Hz rate is only minimally acceptable. Finally, chromaticity is important, with the ideal screen color falling into the range where the eye is most sensitive. Studies have indicated that greens are the best colors, but that color may be less important than the level of contrast, making dark colors such as red and blue less suitable for screens. The next set of criteria applies to the keyboard. In addition to the positioning of various keys, a number of physical parameters influence user performance, including the following: The shape of the keys (2) The glare or reflection from the keys (3) The force required to displace the key (bounce) (4) The key displacement (travel)

(1)

As might be expected, there are significant individual differences in keyboard preferences. The last set of criteria relates to the placement of the terminal in the working environment. It is important to strike a balance between the proper amount of lighting for work and the reduction of bothersome glare on the terminal. Glare can often be reduced by adjusting the task lighting or coating the screen. The angles of the screen and keyboard are important to support readability and prevent poor posture leading to backache and headache.


149

8.4 Summary: The Physical Environment

This discussion only scratches the surface of the ergonomic and human factors aspects of the physical work space, but it serves to highlight many of the important considerations and to point toward needed future improvements, some of which are discussed below. This entire area is in need of a great deal of additional study and experimentation so that suitable work spaces may be devised and used in conjunction with modern software development methodologies.

9. Toward Improved Software Engineering Environments

The entire area of software engineering environments is still in its infancy and is characterized by requirements documents, such as the Stoneman, and speculation (Branstad and Adrion, 1981; Wasserman and Gutz, 1982). Nonetheless, present experience has identified many areas for change and improvement, so that it is possible to point out some of the shortcomings of present environments and some of the needed improvements for the future. The purpose of this section is to address these needs in the areas of methodology, tools, and computing support. 9.1 Toward Improved Methodologies

The previous decade laid the groundwork for software engineering and has shown the importance of software development methodologies. Recent work has focused on pulling together techniques used for different phases of the life cycle and has identified goals for methodologies, as described in Section 4. As yet, though, very few organizations have put methodologies into place and fewer have integrated the methodology with organizational structure. While these advances provide a useful framework for software development and a basis for methodological improvements, the present state of affairs leaves much to be desired. Among the questions that must be answered and reflected in new methodologies are the following: (1) Are there suitable alternatives to the traditional life cycle model for software development? In application areas such as artificial intelligence/expert systems, it is often impossible to provide the same kind of specification that one might provide in a hospital admission-dischargetransfer system or in a payroll package. In such systems, the life cycle

150


notions of specification and design do not apply very well. Consider the problem of trying to create a chess program to play at the grand master level or of inventing a new electronic game. In the chess example, there is no practical way to specify the behavior of the program in all possible cases. In the second example, the requirements are that the game be fun to play, challenging, and implementable within the severe time and space constraints of the embedded system. In designing a game and in building other innovative systems, the only effective way to proceed at present is to develop an initial version of the system, gaining some experience with its use, and to then modify it until it is satisfactory for all concerned. If the system is highly interactive, perhaps involving graphics, voice, and/or color, much of the effort will go into creating the human-computer interface, which evolves on the basis of general guidelines for design followed by experimentation. Once again, it may be impossible to write a traditional specification. In short, neither the life cycle notion nor most of the software engineering management concepts apply to such systems, and there is a need to develop effective methodologies for such increasingly common systems. (2) How can design, performance, and implementation constraints be accurately treated during the design and implementation process? At present, it is possible to specify restrictions on the design and implementation of systems, but difficult to ensure that they are handled during the later phases of development. While mathematical analysis can be used on algorithms, and access times calculated for file and data base applications, this form of performance analysis covers a relatively limited set of applications. In graphics applications, many user requests must receive virtually instant response. In embedded systems, the completed system must fit within a specified amount of memory. In real-time applications, signal processing must occur within a specified time limit. Present models for software design and development typically fail to address these constraints, so that failure to satisfy the constraints is not discovered until too late. (3) How can methodologies be improved to handle parallelism and concurrency more effectively? It is well understood that problems involving concurrent control are significantly more complex than are problems involving sequential control (Brooks, 1975); however, despite the growing number of systems programs and applications involving concurrency, there remain relatively few methods for specifying and representing such systems. Existing software design methods can be adapted for such applications, but often fail to show both shared access to data and concurrent execution of functions.


151

In addition, it is difficult to include timing and synchronization constraints for such systems, except those that can be represented by semaphores or their theoretical equivalents. With the trend toward distributed systems and data bases, it is important to address this problem and devise better methodologies. (4) What should be the relationship between the complexity of the methodology and that of the system being built? If one is building a small system, very little is needed in the way of methodology, as long as the program is properly specified and the programmer follows a reasonable set of guidelines for documentation, coding, and testing. On the other hand, if one is building an extremely large and complex system, perhaps involving different computer systems and different development sites over a period of years, a handful of simple methods is inadequate. In short, what is required is not a single methodology, but afamily of methodologies, in which use of the features of the methodology can be used or not, depending upon the scale of the system and its complexity. Thus the methodology can be scaled up and down in accordance with a particular use. (5) H o w can one quantiJL the productivity or system quality benejits that accrue from using a methodology, a specijiic technical procedure, a management technique, or a tool? Organizations have adopted new software development practices and tools primarily on faith rather than on any metrics. There is a need for measures in virtually every aspect of a methodology to replace the present subjective approach. Without such measures, it is difficult to determine which methods help or hinder the process of software development. It is also difficult to evaluate the trade-offs associated with specific techniques, such as the cost of adopting a method and training developers in their use versus the cost savings and/or quality improvement that may accrue after a particular method is in use. (6) H o w can one more accurately estimate the cost and schedule for a project at its outset? At present, many of the measures of development are in terms of lines of code. However, there is no code at the beginning of a project, and therefore cost projections must be made on the basis of the estimated number of lines of code. Even so, such numbers are notoriously inaccurate and fail to take into account such factors as the complexity of the system, individual differences among developers, and similarities to previous projects in the organization. Organizations are gradually accumulating hard data from their own development projects, and hence improving their ability to estimate costs and schedules. In addition, estimates of modules or transaction types can be made during analysis or specification and extrapolated to lines of code

152


measures. Most significantly, a discipline of software engineering economics (Boehm, 1981) is emerging, tying together some of the needs of project management with the information provided by the technical characteristics of a project. These questions are indicative of the work that must be accomplished to increase the effectiveness and scope of software development methodologies. Until some of the quantitative measures can be provided, and until suitable methodologies can be developed for a broader range of applications, the acceptance of these ideas will remain limited. 9.2 Toward Improved Software Tools

Even though much effort has been put into the design, development, enhancement, and evolution of tools, it should be remembered that the purpose of a tool is to support or facilitate some other activity. The inventor and developer of new and/or improved tools must have a solid understanding of the activities carried out by developers and be able to produce tools that fit into these activities. At present, the state of the art of software tools leaves much to be desired. This is not to say that there are no good tools. Indeed, there are a number of sophisticated and effective tools (Miller, 1979; Riddle and Fairley, 1980; Kernighan and Plauger, 1981) and there are several programming environments, as discussed above, that include many such tools. The point, though, is that there are few settings in which the tools actually work effectively in harmony with one another and in support of a software development methodology. There are two extremely serious problems with most of these tools: First, they fail to support a software development methodology or capture any data that assists in control of the software development process. There is a need for a qualitative improvement in tools to provide this critical missing dimension and augment the power of the tools. Second, present tools fail to support the software development life cycle in its entirety, primarily supporting coding activities. There is a need for more tools to assist with software specification, design, and testing, as well as with management of software projects. For example, there are tools that assist with problem specification, such as PSLRSA (Teichroew and Hershey, 1977), or with detailed design, such as a program design language, but they address only a portion of the development process and must be incorporated into methodologies for software development.


153

In addition to these primary problems, there are a number of secondary problems with many present tools, including the following: (1) Lack of computability. Tools are difficult to combine with one another, both at the direct interface level and at the user level; some tools cannot even use the same data or file formats. ( 2 ) Lack ofuniformity. The available set of tools differs among different machines, operating systems, and languages, making it difficult for a programmer to move easily from one development environment to another. ( 3 ) Lack of tailorability. Most tools are designed to be used in predetermined ways and cannot easily be customized to support different patterns of use by different developers. Preliminary efforts are underway to develop tools that overcome some of these deficiencies, and to gather some evaluative data on the use of tools.

In considering automated support for system development methodologies, one must first consider the framework in which tools will operate, specifying their general properties, and then proceed to identify tools that can be of assistance in the development process. A future automated support framework may be based on the introduction and unification of three new concepts within the development environment: (1) The use of data bases in conjunction with tools to provide a “knowledge base” about programmers and their use of specific tools (2) The ability to capture information about program structures, design decisions, and the software process itself in real time ( 3 ) The utilization of sophisticated human interfaces for software development, based on “personal development systems” (Gutz et al., 1981) As methodologies for software development evolve, new tools will be introduced to support those methodologies so that use of the associated tools will facilitate adoption of improved methods. The outcome of this process will be a whole spectrum of new software tools, based on the three concepts outlined. While these tools will incorporate the essential properties of current tools, their character will change so that they will be compatible with each other in an integrated automated development support system (ADSS). In particular, it will be necessary to make certain that new tools and the encompassing ADSS possess the following characteristics:

154


Singularity ofpurpose. Each tool should carry out one and only one well-defined function, or a small number of closely related functions. Ease ofuse. The user must not need elaborate knowledge in order to be able to use a tool. The features of a tool should be independent of one another, so that a user need not be aware of other features in order to use a specific feature. Self-documenting. The user must not need to have voluminous hardcopy documentation within arm’s reach in order to be able to use a tool. Although some compact hard-copy documentation should be available, most assistance would be provided to the user through structured HELP operations available interactively or through facilities that permit the user to make queries of an on-line reference document. Ideally, though, the tool should be human engineered so that its user interface is intuitive and it operates in such ii way as to provide intelligent guidance through the use of the tool. Consistency with other tools. Tools should interact with one another in a consistent way, typically by maintaining standard interfaces and communication protocols. Furthermore, tools within an environment should conform to a set of well-understood conventions so that a user familiar with tool x in that environment can easily become accustomed to another tool y. One of the maxims in the creation of the Unix environment is to “[expect] the output of every program to become the input to another, as yet unknown, program” (McIlroy et al., 1978). Adaptability. Tools should be adaptable to specific user requirements. For example, they should operate in a variety of modes to support user classes such as “novice,” “regular,” and “expert.” Different command formats, capabilities, and system messages may be associated with each user class. There should be an intuitive, smooth transition between these classes. Similarly, tools should provide for a meaningful set of defaults that can be altered to satisfy the needs of individual users. In this way, each user can start with a generic tool and then adjust parameters and options to tailor it to his or her own mode of use. Local Intelligence. Intelligence can be provided in the tool by designing it to collect useful information to be stored in a private data base, perhaps using a general-purpose data base management system as a tool in that process. Such data may be captured automatically from the user and can be gained from a community of tool users to provide profiles of usage that could assist in the evolution of that tool and the development of new ones. The data base for an individual tool is private to that tool and under both the user’s and the tool’s control. In this way, it will be possible to


155

provide powerful, useful, and personalized services and permit query requests for information stored in the data base. Support for the software life cycle. Tools must be developed to provide support to cover the entire life cycle, from requirements analysis through testing and evolution over time. Tools must therefore provide not only technical support for a specific phase of the life cycle, but should also provide management assistance and facilitate the transition from one phase of the life cycle to another (both forward and backward!). Support f o r management of software development. Two kinds of management are essential: software configuration management and management of software development personnel. Software configuration management involves keeping track of the emerging product, possibly in multiple versions. An effective configuration management facility can be distributed across a set of tools to capture program structure, interface requirements, problem reports, and updates to source and object versions of programs. Similarly, information about individual productivity and the amount of effort associated with various parts of a software project, organized by either project phase or module, can be easily collected, usually in a manner that is largely invisible to the developer. Such information is particularly valuable in gaining a better understanding of the software development process itself, which can lead both to better techniques for estimating the effort required for new projects and to identification and development of improved software tools. 9.3 Toward Improved Computing Support

A common means of software development is through computer terminals connected to a time-sharing system. For some time now, time sharing has served as a reasonable way to share the resources of a computer system. High-speed communication lines and improved display terminals have led to a significant improvement in the productivity of the individual developer. However, time sharing suffers from significant economic, operational, and technical weaknesses, and it is now an idea whose time has past. Future development systems will be based on a network of “personal development systems” (PDS), which overcome many of the disadvantages of time-sharing systems: (1) Time sharing is relatively expensive on a developer-year basis, typically costing about $10,000 per developer-year. Powerful personal development systems presently sell for $16,000 to $40,000. Even if the

156


cost of such systems were to remain constant over the next few years, the costs would be comparable. (2) Time sharing has some operational weaknesses, including the effect of system load on the individual developer, the catastrophic effects of a system failure, and the difficulty of individually customizing hardware or systems software. All of these problems are overcome by the PDS approach, where the load is determined by the individual developer, and not by a community of users. The only point at which system load is a factor is when the developer requires access to some shared resources. Most work, though, can be done “locally.” (3) Time sharing cannot easily support high-bandwidth input-output and extensive real-time processing. Accordingly, time-sharing systems do poorly at supporting video or nonkeyboard communications. If future development environments are to support such communications media as graphics, light pens, image processing, optical character recognition, movable devices, e.g., joysticks, and voice, then greater dedicated computing power will be needed to support them. In some respects, then, the alphanumeric terminal used by most developers is an artifact of timesharing systems and it can only be replaced with more sophisticated terminals when additional computing power is provided to the individual developer. Bitmap graphics terminals with multiple-window facilities are an especially attractive approach. The increased computing support comes from a PDS, which can provide improved reliability against system failure, greater control of the physical resources available to a developer, and better human interfaces for system development and use. The typical configurationfor a PDS will provide the programmer with at least the computing power available in a present-day, medium-scale timesharing system. The components of this environment include the following: (1) An intelligent terminal, with approximately I Mbyte of primary memory and the computing power and address space of today’s 32-bit minicomputers. (2) Local secondary storage, e.g., a Winchester disk, with upward of 40 Mbytes of storage capacity, as well as some removable media, such as a floppy disk or cassettes. (3) Graphics capability, including multicharacter fonts, reverse video, variable intensity, split screen, color, and the ability to display a complete 84-by-11-inch (or A4) page of text on the screen. (4) Networking capability, to connect with other PDSs in a local net-


Other computer systems

Controller

Archive

157

Data base machine

work, perhaps an Ethernet-like (Metcalfe and Boggs, 1976) medium, and to geographically dispersed systems, as shown in Fig. 4. (5) Audio input-output, with at least a rudimentary input vocabulary (200-500 words) and speech synthesis; at least initially, continuous speech recognition is not expected. The overall intent is to bring advances in hardware to assist the software developer. The first generation of such systems is already commercially available. For example, several vendors are selling PDSs using powerful microprocessors, high-resolution bit-map terminals, Winchester disks, and standard network and device interfaces, supporting the Unix operating system and its tools. For the first time, it is possible to use inexpensive hardware for the express purpose of improving human-machine communications. Such a system can support a variety of work patterns, allowing the developer to interrupt activities, switch among tasks, and save the state of each of these activities, as is done in Smalltalk. There is a synergistic effect between these hardware advances and the features of software tools. The previous discussion of tools made little mention of graphics, even though designers and programmers frequently use pictures to communicate among themselves and many software development methods use graphic representations. The availability of this hardware will promote the development of the next generation of software tools, in which graphics and nonkeyboard interfaces can play a major role. Such hardware will therefore be central to the creation of the next generation of software engineering environments.

158


10. Conclusion

Throughout history, we have seen that adyances in tools have led to higher quality products, improved productivity of workers, and occasionally to social and cultural changes. We may expect the same thing to happen with tools for system development, as newer, more sophisticated tools are build and integrated into methodologies and environments for system design and development. The challenge of system development for the near future is to draw effectively upon the experience of the past and develop more tools that embody the characteristics described in this article. These tools must then be integrated with technical methods and management practices to create methodologies that can improve both the quality of software systems and the process of system development. These methodologies must then be embedded in environments that provide the greatest possible support for both the individual developer and the software development organization. Many of the organizational and physical work space considerations are easily as important as the tool and methodological considerations. In particular, much attention must be given to the individual work space, building upon the ergonomic issues such as terminal design, lighting, office size, and furniture. Many office areas should be redesigned to provide developers with privacy and comfort. The office should be equipped with a personalized development environment that combines the need for alphanumeric text with the need for graphics. The idealized hardware support for the individual developer may have two terminalsone a monochrome alphanumeric terminal and the other a color graphics terminal-and multiple input media, including both a keyboard and a nonkeyboard device, such as a “mouse.” In considering the salary level for professional software developers and the cost of replacing such persons, substantial investments in support staff, computer hardware, and office space, well above the current levels in most organizations, seem to be justified. The software engineering environment, then, pulls together the central notions of software development methodologies with organizational, managerial, and ergonomic issues. Because of the complex interrelationships among these environmental components, an organization must address all of these issues in designing and building effective software development environments. It is only in this way that it will become possible to achieve the goal of increased productivity with modem software engineering environments.


159

ACKNOWLEDGMENTS

The author gratefully acknowledges discussions with Peter Freeman of The University of California, lrvine and Steven Gutz of Digital Equipment Corporation that have helped to sharpen the ideas presented here. Many of the ideas have evolved from work presented in earlier papers (Wasserman, 1980a,b).

REFERENCES Ada Joint Program Office (1983). Ada methodologies-Concepts and requirements. Software Eng. Notes 8(1), 33-50. Advanced Research Projects Agency (1980). “Requirements for Ada Programming Support Environments-”STONEMAN.” U.S. Dept. of Defense, Washington, D.C. Basili, V., ed. 1980. “Tutorial: Models and Metrics for Software Development.” IEEE Computer Society, Los Alamitos, California. Bersoff, E.H., Henderson, V.D., and Siege], S.G. (1979). Software configuration management: A tutorial. Computer =(I), 6-14. Boehm, B.W. (1981). “Software Engineering Economics.” Prentice-Hall, Englewood Cliffs, New Jersey. Branstad, M., and Adrion, W.R. (1981). NBS workshop report on programming environments. Software Eng. Notes 6(4), 1-51. Brinch Hansen, P. (1975). The programming language Concurrent Pascal. IEEE Trans. Software Eng. SE-1(2), 199-207. Brooks, F.W., Jr. (1975). “The Mythical Man-Month.’’ Addison Wesley, Reading, Massachusetts. and Gordon, E.K. (1975). PDL-A tool for software design. Proc. AFIPS 1975 Caine, S.H., NCC 44,271-276. Chen, P.P.3. (1976). The entity-relationship model-Toward a unified view of data. Trans. Dafabase Syst. I( I ) , 9-36. Codd, E.F. (1970). A relational model of data for large shared data banks. Commun. ACM l3(6), 377-387. Curtis, B., ed. (1981). “Tutorial: Human Factors in Software Development.” IEEE Computer Society, Los Alamitos, California. DeMarco, T. (1979). “Structured Analysis and System Specification.” Prentice-Hall, Englewood Cliffs, New Jersey. DeMillo, R., Lipton, R.J., and Perlis, A. (1979). Social processes and proofs of theorems and programs. Commun. ACM 22(5), 271-280. Elwell, J.F. (1982). An approach to the definition and implementation of a software development environment. Proc. AFIPS 1982 NCC 51, 309-318. Fagan, M. (1976). Design and code inspections to reduce errors in program development. IBMSysr. J . 15(3), 182-211. Feldman, S.I. (1979). Make-A program for maintaining computer programs. SofiwarePract. Exp. 9(4), 255-265. Gane, C., and Sarson, T. (1979). “Stuctured Systems Analysis.” Prentice-Hall, Englewood Cliffs, New Jersey. Greenes, R.A., Pappalardo, A.N., Marble, C.W., and Barnett, G.O. (1969). A system for clinical data management. Proc. AFIPS I969 FJCC 35, 297-305. Gutz, S., Wasserman, A.I., and Spier, M.J. (1981). Personal development systems for the professional programmer. Computer 14(4), 45-53.

160


Habermann, A.N., Notkin, D.S., and Perry, D.E. (1979). “Report on the use of Ada for the development and implementation of part of Gandalf,” Tech. Rep. Department of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania. Hall, D.E., Scherrer, D.K., and Sventek, J.S. (1980). A virtual operating system. Commun. ACM 23(9), 495-502. Ichbiah, J.D., ed. (1981). “Reference Manual for the Ada Programming Language.” Springer-Verlag, Berlin and New York. h ie , E.L. (1977). The Programmers Workbench-A machine for software development. Commun. ACM 20(10), 746-753. Kay, A. (1982). New directions for novice programming in the 1980’s. I n “State of the Art Report: Programming Technology” (P.J.L. Wallis, ed.), pp. 209-247. Pergamon Infotech, Maidenhead, England. Kernighan, B.W., and Mashey, J.R. (1981). The Unix programming environment. Computer 14(4), 12-24. Kernighan, B.W., and Plauger, P.J. (1981). “Software Tools in Pascal.” Addison-Wesley, Reading, Massachusetts. Kersten. M.L., and Wasserman, A.I. (1981). The architecture of the PLAIN data base handler. Software-Pract. Exp. 11(2), 175-186. Lampson, B.W., Homing, J.J., London, R.L., Mitchell, J.G., and Popek, G.L. (1977). Report on the programming language Euclid. ACM SIGPLAN Nor. l2(2), 1-79. Leveson, N.G. (1980). Applying behavioral abstraction to information system design and integrity. Ph.D. Dissertation, University of California, Los Angeles, 1980. (Avaiable as Technical Report No. 47, Laboratory of Medical Information Science, University of California, San Francisco.) Leveson, N.G., Wasserman, A.I., and Berry, D.M. (1983). BASIS: A behavioral approach to the specification of information systems. Information Sysr. 8(l), in press. Liskov, B., Atkinson, R., Bloom, T., Moss, E., SchafTert, C., Schiefler, B., and Synder, A. (1981). “CLU Reference Manual.” Springer-Verlag, Berlin and New York. Lundeberg, M., Goldkuhl, G., and Nilsson, A. (1981). “Information Systems Development-A Systematic Approach.” Prentice-Hall. Englewood Cliffs, New Jersey. McCue, G.M. (1978). IBM’s Santa Teresa Laboratory-Architectural design for program development. IBM Syst. J. 17(1), 4-25. McIlroy, M.D., Pinson, E.N., and Tague, B.A. (1978). Unix time sharing system: Foreword. Bell Sysr. Tech. J . 57(6), 1899-1904. Metcalfe, R.M. and Boggs, D.R. (1976). Ethernet: Distributed packet switching for local computer networks. Commun. ACM 19(7), 395-404. Miller, E.F., Jr., ed. (1979). “Tutorial: Automated Tools for Software Development.” IEEE Computer Society, Long Beach, California. Myers, G.J. (1978). A controlled experiment in program testing and code walkthroughs/ inspections. Commun. ACM 21(9), 760-768. Myers, G.J. (1979). “The Art of Software Testing.” Wiley, New York. O’Neill, D. (1980). The management of software engineering. Part 11. Software engineering program. IBM Sysr. J. 19(4), 421-431. Riddle, W.E., and Fairley, R.E., eds. (1980). “Software Development Tools.” SpringerVerlag, Berlin and New York. Ritchie, D.M., and Thompson, K. (1974). The UNIX time-sharing system. Commun. ACM 17(7), 365-375. Rochkind, M.J. (1975). The source code control system. IEEE Trans. Software Eng. SEl(4). 364-369.


161

Ross, D.T., and Schoman, K.E., Jr. (1977). Structured analysis for requirements definition. IEEE Trans. Software Eng. SE-3(1), 6-15. Sandewall, E. (1978). Programming in the interactive environment: The LISP experience. Cornput. Suru. 10(1), 35-71. Shaw, J.C. (1964). JOSS: A Designer’s View of an Experimental Online Computer System. Proc. AFIPS 1964 FJCC 26,455-464. Smith, D.C.P. (1980). Conceptual database design. In “Tutorial: Software Design Techniques” (P. Freeman and A.I. Wasserman, eds.), pp. 333-356. IEEE Computer Society, Los Alamitos, California. Teichroew, D., and Hershey, E.A. (1977). PSL/PSA: A computer aided technique for structured documentation and analysis of information processing systems. IEEE Trans. Software Eng. SE-31). 41-48. Teitelman, W. (1978). “The INTERLISP Reference Manual.” Xerox Palo Alto Research Center, Palo Alto, California. Teitelman, W.. and Masinter, L. (1981). The INTERLISP programming environment. Computer 14(4), 25-33. Tesler, L. (1981). The Smalltalk programming environment. Byre 6(8), 90-147. Wasserman, A.J. (198Oa). Information system development methodology. J . A m . Soc. In$ Sci. 31(1), 5-24. Wasserman, A.I. (1980b). Toward integrated software development environments. Scientia 1l5, 663-684. Wasserman, A.I. (1982). The User Software Engineering Methodology: An Overview. In “Information System Design Methodologies-A Comparative Review” (T.W. Olle, H.G. Sol, and A.A. Venijn-Stuart, eds.), pp. 589-628. North-Holland Publ.. Arnsterdam. Wasserman, A.I., and Gutz, S. (1982). The future of programming. Comrnun. ACM 233). 196-206. Wasserman, A.I., and Shewmake, D.T. (1982). Automating the development and evolution of user-program dialogue in an interactive information system. In “Evolutionary Information Systems” (J. Hawgood, ed.), pp. 159-172. North-Holland, Amsterdam. Wasserman, A.J., Sherertz, D.D., Kersten, M.L., van de Riet, R.P., and Dippe, M.D. (1981). Revised report on the programming language PLAIN. ACM SIGPLAN Nor. 16(5), 59-80. Wirth, N. (1971). The programming language Pascal. A d a Inf. l ( 1 ) . 35-63. Wulf, W.A., London, R.L., and Shaw, M. (1976). An introduction to the construction and verification of Alphard programs. IEEE Trans. Sofrware Eng. SE-2(4), 253-265. Yourdon, E. (1979). “Structured Walkthroughs,” 2nd ed. Prentice-Hall, Englewood Cliffs, New Jersey. Yourdon, E. (1982). “Managing the System Life Cycle.” Yourdon Press, New York. Yourdon, E., and Constantine, L.L. (1979). “Structured Design.” Prentice-Hall, Englewood Cliffs, New Jersey.


Principles of Rule-Based Expert Systems BRUCE G . BUCHANAN Department of Computer Science Stanford University Stanford. California

RICHARD 0. DUDA Laboratory for Artificial Intelligence Research Fairchild Camera and Instrument Corporation Palo Alto. California

1. Introduction: What Is an Expert System? . . . . . . . . . . . . . . . . . 1.1 Example: The MYCIN Program . . . . . . . . . . . . . . . . . . . 1.2 Key Components . . . . . . . . . . . . . . . . . . . . . . . . . 2 . Representation of Knowledge . . . . . . . . . . . . . . . . . . . . . . 2.1 Rule-Based Representation Frameworks . . . . . . . . . . . . . . . 2.2 Alternatives to Rule-Based Representation of Knowledge . . . . . . . 2.3 Knowledge Representation Issues . . . . . . . . . . . . . . . . . . 3. Inference Methods in Expert Systems . . . . . . . . . . . . . . . . . . 3.1 Logical and Plausible Inference . . . . . . . . . . . . . . . . . . . 3.2 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Explicit Representation of Control Knowledge . . . . . . . . . . . . 4 . Reasoning with Uncertainty . . . . . . . . . . . . . . . . . . . . . . . 4.1 Plausible Inference . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Bayesian Probability Theory . . . . . . . . . . . . . . . . . . . . 4.3 Certainty Theory . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Possibility Theory . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 The Dempster-Shafer Theory of Evidence . . . . . . . . . . . . . . 5 . Key Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Nature of the Problem . . . . . . . . . . . . . . . . . . . . . . . 5.2 Representation . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Explanation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Knowledge Acquisition . . . . . . . . . . . . . . . . . . . . . . . 5.6 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 Classes of Problems for Expert Systems . . . . . . . . . . . . . . . 163 ADVANCES IN COMPUTERS. VOL. 22

164 166 173 173 176 180 182 184 184 185 189 190 190 190 194 196 197 198 199 199 200 200 201 201 201

.

Copynghl 8 1983 by Academic Press Inc . All rights of reproduction in any form reserved . ISBN 612-012122-0

164

BRUCE G. BUCHANAN AND RICHARD 0. DUDA

5.8 The Data. . . 5.9 The Expertise.

.......................... .......................... .............................

6. Conclusions, Appendix. Answers to Questions about MYCIN's Consultation in Section 1.1 General References. . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

203 204 205 207 210 210

1. Introduction: What Is an Expert System?

An expert system is a computer program that provides expert-level solutions to important problems and is: 1. Heuristic, i.e., it reasons with judgmental knowledge as well as with formal knowledge of established theories 2. Transparent, i.e., it provides explanations of its line of reasoning and answers to queries about its knowledge 3. Flexible, i.e., it integrates new knowledge incrementally into its existing store of knowledge'

The key ideas have been developed within artificial intelligence (AI) over the last 15 years, but in the last few years more and more applications of these ideas have been made. The purpose of this article is to familiarize readers with the architecture and construction of one important class of expert systems, called rule-based systems. In this overview, many programs and issues are necessarily omitted, but we attempt to provide a framework for understanding this advancing frontier of computer science. Because computers are general symbol-manipulating devices, the nonnumeric and heuristic aspects of problem solving can be encoded in computer programs as well as the mathematical and algorithmic aspects (Newell and Simon, 1976). Artificial intelligence research has focused on just this point. Work on expert systems is, in one sense, the applied side of AI, in which current techniques are applied to problems to provide expertlevel help on those problems. However, there is more to building an expert system than straightforward application of A1 techniques. In this I We use the term expert system more restrictively than many authors. In doing so, we distinguish expert systems from, among others, programs that perform well but which cannot be examined, programs that are examinable but which do not perform well, and programs that solve problems for which no special expertise is required. For other recent reviews of the state of the art of expert systems, see Bonnet (1981, 1982), Davis (1982), Duda and Gaschnig (1981), Feigenbaum (1979), Hayes-Roth et al. (1978), Michie (1979, 1980, 19811, Pinson (1981). Stefik er al. (1982). and Waterman and Hayes-Roth (1978).

PRINCIPLES OF RULE-BASED EXPERT SYSTEMS

165

early stage of development, each new application challenges the current stock of ideas, and many applications force extensions and modifications. We focus on rule-based systems in this survey because they clearly demonstrate the state of the art in building expert systems and illustrate the main issues. In a rule-based system, much of the knowledge is represented as rules, that is, as conditional sentences relating statements of facts with one another. Modus ponens is the primary rule of inference by which a system adds new facts to a growing data base: If B is true B and B implies C, OR B+C C then C is true. Conceptually, the basic framework of a rule-based system is simple; the variations needed to deal with the complexities of real-world problems make the framework interestingly more complex. For example, the rule B + C is often interpreted to mean “B suggests C,” and strict deductive reasoning with rules gives way to plausible reasoning. Other methodologies are also mentioned and briefly discussed. In an expert system, the fundamental assumption is “knowledge is power. Specific knowledge of the task is coupled with general problemsolving knowledge to provide expert-level analyses of dimcult situations. For example, MYCIN (described below) analyzes medical data about a patient with a severe infection, PROSPECTOR (Duda et al., 1979) analyzes geological data to aid in mineral exploration, and PUFF (Kunz et al., 1978) analyzes the medical condition of a person with respiratory problems. In order to provide such analyses, these systems need very specific rules containing the necessary textbook and judgmental knowledge about their domains. The key idea is to separate knowledge of the task area as much as possible from the procedures that manipulate it. This promotes flexibility and transparency because the store of knowledge, called the knowledge base, can then be manipulated and examined as any other data structures. Separation does not guarantee flexibility or transparency, nor does it guarantee high performance. But if a packaged framework of inferences and control procedures can be used, the design of a new system properly focuses on the expertise needed for high performance. For many years, A1 research has focused on heuristic reasoning. Heuristics, or rules of thumb, are an essential key to intelligent problem solving because computationally feasible, mathematically precise methods are known for only a relatively few classes of problems. A large part of what an expert system needs to know is the body of heuristics that specialists use in solving hard problems. Specialists in science, mathemat”

166


ics, medicine, or any discipline do not confine their everyday reasoning to the axiomatic, formal style of stereotyped textbook accounts. An expert system can also benefit from reasoning with informal knowledge. There is no explicit attempt to simulate a specialist’s problem-solving behavior in an expert system; however, the system derives power from integrating the same heuristic knowledge as experts use, with the same informal style of reasoning. The first expert systems, DENDRAL (Lindsay et al., 1980) and MACSYMA (Moses, 1971), emphasized performance, the former in organic chemistry and the latter in symbolic integration. These systems were built in the mid-1960s and were nearly unique in A1 because of their focus on real-world problems and on specialized knowledge. In the 1970s, work on expert systems began to flower, especially in medical problem areas (see, for example, Pople, 1977; Shortliffe, 1976; Szolovits and Pauker, 1978; Weiss et al., 1979). The issues of making the system understandable through explanations (Scott et al., 1977; Swartout, 1981) and of making the system flexible enough to acquire new knowledge (Davis, 1979; Mitchell, 1979) were emphasized in these and later systems. In the early work, the process of constructing each new system was tedious because each was custom crafted. The major difficulty was acquiring the requisite knowledge from experts and reworking it in a form fit for machine consumption, a process that has come to be known as knowledge engineering. One of the most important developments of the late 1970s and early 1980s is the construction of several knowledge engineering frameworks designed to aid in building, debugging, interpreting, and explaining expert systems. Engineering an expert’s knowledge into a usable form for a program is a formidable task. Thus, computer-based aids for system builders are important. Current tools, including EMYCIN (vanMelle, 1980), ROSIE (Fain et al., 1981), KAS (Reboh, 1981), EXPERT (Weiss and Kulikowski, 1979), and OPS (Forgy and McDermott, 1977), provide considerable help. For example, working prototypes of new expert systems with one or two dozen rules have been constructed in a few days using EMYCIN (Bennett, 1981). Considerable effort is then required to refine the knowledge base, but the effort is focused on the knowledge and not on the code. Much research remains, however, before broad, powerful systems can be constructed quickly. 1.1 Example: The MYCIN Program

MYCIN is a rule-based system developed in the mid to late 1970s at Stanford University. Its representation and architecture are described in detail in Davis et al. (1977) and Shortliffe (1976). Although it is now


167

several years old, it is representative of the state of the art of expert systems in its external behavior, which is shown in the following excerpt from a dialogue between the MYCIN program and a physician. It illustrates the interactive nature of most rule-based systems and provides a single example for the rest of the discussion. The task of MYCIN is to help diagnose the likely causes of a patient’s infection and to help recommend the best therapy for that individual patient. The consultation is driven by a knowledge base of about 450 rules and 1000 facts about medicine, mostly about meningitis infections. In the example, we see the system asking questions to obtain a description of a new case, and we see the system providing an explanation of its line of reasoning. The main point of this example is that a simple representation of knowledge as rules and a relatively simple control structure are adequate for constructing a consultation system in a domain as complex (but narrow!) as meningitis diagnosis and therapy. The Appendix shows answers to user’s questions about this consultation and the knowledge base. The following is an edited transcript of a MYCIN consultation session. User’s responses are in upper case and follow double asterisks, and editorial comments are in italics.

- - - - - - - -PATIENT-538-- - - - - - 1) Patient’s name: ** pT538 2)

Age :

3)

Sex: MAEL

** 34 YEARS

**

MYCIN respells the user’s answer.

=MALE 4) Race: ** CAUCASIAN Several questions are omitted. 14) Has Pt538 recently had symptoms of persistent headache or other abnormal neurologic symptoms (dizziness, lethargy, etc.)? ** YES 15) Has Pt538 recently had objective evidence of abnormal neurologic signs (nuchal rigidity, coma, seizures, etc.) documented by physician observation or examination?

168


** YES The CSF cultures will be considered to be associated with MENINGITIS. Note that M YCZN has concluded and informed the user that there is a likely meningitis infection and pursues this line of inquiry. 16) Please give the date on which clinical evidence (symp-

**

toms, signs, or laboratory tests) of the meningitis first appeared. 29-Jan-77 22~15

23) For how many days has Pt538 had abnormal neurologic signs? ** 7.2 HOURS Note that MYClN accepts the different units of time.

28) Do you know the results of a recent CBC of Pt538? ** YES Please enter CBC results in the following table WBC in lOOOs 29** 1.9

Percent PMN UNKNOWN

Percent BANDS UNKNOWN Note that M YClN accepts “ U N K N O W N ’ as a legitimate value. It does not need complete, or even correct, data for all answers.

30) Has Pt538 had any injury or insult to, or defect in the central-nervous-system (excluding neurosurgery)? ** NO

35) Is meningitis a hospital-acquired infection? ** WHY


169

The user asks “WHY” to see an explanation of the line of reasoning MYCIN is following. The program first rephrases the question and then shows the rule whose premise it is trying to establish. [i.e. WHY is it important to determine whether the infection was acquired while the patient was hospitalized?] [2.0] This will aid in determining the organisms (other than those seen on cultures or smears) which might be causing the infection. It has already been established that [2. I] the infection which requires therapy is meningitis, and i2.21 organisms were not seen on the stain of this pending CSF culture, and [2.3] the type of the infection is bacterial, and [2.4] neurosurgery is not one of the relevant items from the history of ptS38, and [2.5] the age of pt538 is greater than 10 days Therefore if [2.6] the infection was acquired while the patient was hospitalized then there is evidence that the organisms (other than those seen on cultures or smears) which might be causing the infection is E.coli (.75) Staphylococcus-coag-pos (.3) Pseudomonasaeruginosa (.3) Klebsiella-pneumoniae (3) Author: Yu [RULE5451 The numbers in parentheses following each conclusion indicate the measure of importance of the conclusion. This single number combines frequency and utility considerations, as discussed in Section 4. [back to question 35...I

** HOW 2.3

170


At this point, the user is asking f o r additional clarification of how clause 2.3 of the above rule has already been determined. MYCIN shows the numbers of the rules used to make conclusions and will print any of them upon request.

[i.e. HOW was it established that the type of the infection is bacterial?] The following rules concluded about: the likelihood that the type of the infection is bacterial Yes RULE 148 RULE500 RULE501 RULE502 RULE526 RULE504 RULE524 Thus it has been established that the type of the infection is bacterial. MYCIN now prints intermediate conclusions to show the user what ir is considering the likely causes of the problem t o be. Further questions are motivated by these conclusions.

I have found evidence (RULE545 RULE557) that treatment should cover for E.coli causing meningitis. I have found evidence (RULE545 RULE557 RULE578) that treatment should cover for Pseudomonas-aeruginosa causing meningitis. I have found evidence (RULE545 RULE557) that treatment should cover for Klebsiella-pneumoniae causing meningitis. I have found evidence (RULE507) that treatment should cover for Diplococcus-pneumoniae causing meningitis.


171

40) Does Pt538 have focal neurological signs? ** UNKNOWN

MYCIN now prints the intermediate conclusions about the possible causes of the problem that are most important to consider in constructing a rational therapy plan.

My therapy recommendations will be designed to treat for organisms that either are very likely or, although less likely, would have a significant effect on therapy selection if they were present. It is important to cover for the following probable infection(s) and associated organism(s): INFECTION-I is MENINGITIS

+ (ITEM-1) + (ITEM-2) + (ITEM-3) + (ITEM-4)

E.COLI [from clinical evidence only] PSEUDOMONAS-AERUGINOSA [from clinical evidence only] KLEBSIELLA-PNEUMONIAE [from clinical evidence only] DIPLOCOCCUS-PNEUMONIAE [from clinical evidence only]

Does Pt538 have a clinically significant allergic reaction to any antimicrobial agent? ** NO [Considering AMPICILLIN-AND-GENTAMICIN for use against INFECTION-1.. .I 42) Do you have reason to suspect that Pt538 may have impaired renal function? ** NO 43 ) Pt538’s weight in kilograms (or (number) POUNDS): ** 70 41)

The final conclusion is about the best therapy to cover for the likely causes of the infection, as determined from the incomplete evidence available.

172


[REC-I] My preferred therapy recommendation is as follows: In order to cover for Items (1 2 3 4): Give the following in combination: 1) AMPICILLIN Dose: 3.5g (28.0 ml) q4h IV [calculated on basis of 50 mg/ kgl 2) GENTAMICIN Dose: 119 mg (3.0 ml, 80 mg/2 ml ampule) q8h IV [calculated on basis of 1.7 mg/kg] plus consider giving 5 mg q24h intrathecal Comments: Monitor serum concentrations Since high concentrations of penicillins can inactivate aminoglycosides, do not mix these two antibiotics in the same IV bottle. Do you wish to see the next choice therapy? ** NO The user may have reasons to prefer another combination of drugs. At this point, MYCIN is prepared to show other therapies, compare its preferred therapy to one the user speciJies, or answer questions about any part of the consultation or about the items in its knowledge base. Appendix A shows some of these explanations.

The level of expertise MYCIN exhibited on meningitis test cases in a double-blind evaluation study equaled the level of expertise of faculty members in infectious diseases (Yu et al., 1979). However, the program was never put into routine use in hospitals. Factors other than the program’s competence, such as human engineering and exportability , were the main barriers to routine use. There are two ways of overcoming these kinds of problems. First, some follow-on research to MYCIN addresses the human engineering problems directly, for example, by integrating high-quality graphics with user-oriented forms and charts for input and output (Shortliffe et al., 1981). Second, some MYCIN-like programs refine many human engineering problems by collecting data from on-line instruments rather than from users (Kunz et al., 1978). Exportability can


173

be gained by rewriting (Carhart, 1979; Kunz et al., 1978) or by designing for export initially (Weiss and Kulikowski, 1979). 1.2 Key Components

The example showed some of the characteristic features of an expert system: the heuristic nature of MYCIN’s rules, an explanation of its line of reasoning, and the modular form of rules in its knowledge base. We postponed a discussion of the general structure of a system until after the example and defer entirely specific questions about implementation. For discussing the general structure, we describe a generalization of MYCIN, called EMYCIN (vanMelle, 1980), for “essential MYCIN.” It is a framework for constructing and running rule-based systems like MYCIN. Generally speaking, an expert system requires a knowledge base and an inference procedure. The knowledge base for MYCIN is a set of rules and facts covering specialized knowledge of meningitis as well as some general knowledge about medicine. The inference procedure in MYCIN (and all systems constructed in EMYCIN) is a large set of INTERLISP functions that control the interaction and update the current state of knowledge about the case at hand. Sections 2 and 3 on representation and inference discuss these two main parts. A system also requires a global data base of facts known or inferred about a specific case, the working data set, in other words. And it requires an interface program that makes the output understandable to users and trgnslates users’ input into internal forms. MYCIN uses a technical subset of English in which there is little ambiguity in the language of communication with users. In addition to these four parts, the EMYCIN system contains an explanation subsystem to answer questions about a consultation or about the static knowledge base. It also contains a knowledge base editor to aid in the construction of new knowledge bases (and thus new systems) and to aid in debugging an emerging knowledge base. All of these components are shown schematically in Fig. 1. We turn now to two basic issues that have become the central foci of work on expert systems: (1) how knowledge of a task area is represented in the computer program, and, (2) how knowledge is used to provide expert-level solutions to problems. 2.

Representation of Knowledge

A representation is a set of conventions for describing the world. In the parlance of AI, the representation of knowledge is the commitment to a

174


Customized

Interaction

b

Problem solver

b

ADVICE FOR THIS CASE

*

EXPLANATIONS

DESCRIPTION

OF NEW CASE Data base of facts about

USER’S -b QUESTIONS

Question-answering routines

Explanation subEvatern

FIG. 1 . Parts of the EMYCIN system showing the separation of the knowledge base from problem-solving procedures and other parts of the system.

vocabulary, data structures, and programs that allow knowledge of a domain to be acquired and used. This has long been a central research topic in A1 (for reviews of relevant work, see Amarel, 1981; Barr and Feigenbaum, 1981; Brachman and Smith, 1980; Cohen and Feigenbaum, 1982).

The results of 25 years of A1 research on representation have been used to establish convenient ways of describing parts of the world. No one

believes the current representation methods are the final word. However, they are well enough developed that they can be used for problem solving in interesting domains. As pointed out above, a central concern is separation of the choice of vocabulary and data structures from the choice of program logic and language. By separating a program’s knowledge base from the inference procedures that work with the knowledge, we have attained some success in building systems that are understandable and extendable. Three basic requirements on a representation scheme in an expert system are extendability, simplicity and explicitness. Extendubiliry. The data structures and access programs must be flexible enough to allow extensions to the knowledge base without forcing substantial revisions. The knowledge base will contain heuristics that are built out of experts’ experience. Not only do the experts fail to remember all relevant heuristics they use, but their experience gives them new heuristics and forces modifications to the old ones. New cases require new distinctions. Moreover, the most effective way we have found for building a knowledge base is by incremental improvement. Experts cannot define a complete knowledge base all at once for interesting problem areas, but


175

they can define a subset and then refine it over many weeks or months of examining its consequences. All this argues for treating the knowledge base of an expert system as an open-ended set of facts and relations, and keeping the items of knowledge as modular as possible. Simplicify. We have all seen data structures that were so baroque as to be incomprehensible and thus unchangeable. The flexibility we argued for just previously requires conceptual simplicity and uniformity so that access routines can be written (and modified occasionally as needed). Once the syntax of the knowledge base is fixed, the access routines can be fixed to a large extent. Knowledge acquisition, for example, can take place with the expert insulated from the data structures by access routines that make the knowledge base appear simple, whether it is or not. However, new reasons will appear for accessing the knowledge base, as in explanation of the contents of the knowledge base, analysis of the links among items, display, or tutoring. With each of these reasons, simple data structures pay large benefits. From the designer’s point of view there are two ways of maintaining conceptual simplicity: keeping the form of knowledge as homogeneous as possible and writing special access functions for nonuniform representations. There is another sense of simplicity that needs mentioning as well. That is the simplicity that comes from using roughly the same terminology as the experts use. Programmers often find ingenious alternative ways of representing and coding what a specialist has requested, a fact that sometimes makes processing more “efficient” but which makes modifying the knowledge base a nightmare. Explicitness. The point of representing much of an expert’s knowledge is to give the system a rich enough knowledge base for high-performance problem solving. But because a knowledge base must be built incrementally, it is necessary to provide means for inspecting and debugging it easily. With items of knowledge represented explicitly, in relatively simple terms, the experts who are building knowledge bases can determine which items are present and (by inference) which are absent.

To achieve these goals, three types of representation framework have been used in expert systems. Although we concentrate on rule-bused systems in Section 2.1, we also mention frame-based and logic-based systems by way of contrast. Such frameworks are often called representation languages because, as with other programming languages, their conventions impose a rigid set of restrictions on how one can express and reason about facts in the world. In all of these languages, one can express conditional expressions and causal dependencies. In rule-based systems, however, one sacrifices the ability to express many other general kinds of

176


relations in favor of the homogeneity and simplicity of conditional rules. These frameworks, also, often include inference and control routines making them even more like languages. 2.1

Rule-Based Representation Frameworks

2.1.1 Production Systems

Rule-based expert systems evolved from a more general class of computational models known as production systems (Newell, 1973). Instead of viewing computation as a prespecified sequence of operations, production systems view computation as the process of applying transformation rules in a sequence determined by the data. Where some rule-based systems (McDermott, 1980) employ the production system formalism very strictly, others such as MYCIN have taken great liberties with However, the production system framework provides concepts that are of great use in understanding all rule-based systems. A classical production system has three major components: (1) a global data base that contains facts or assertions about the particular problem being solved, ( 2 ) a rule base that contains the general knowledge about the problem domain, and (3) a rule interpreter that carries out the problem-solving process. The facts in the global data base can be represented in any convenient formalism, such as arrays, strings of symbols, or list structures. The rules have the form

IF (condition) THEN (action). In general, the left-hand side or condition part of a rule can be any pattern that can be matched against the data base. It is usually allowed to contain variables that might be bound in different ways, depending upon how the match is made. Once a match is made, the right-hand side or action part of the rule can be executed. In general, the action can be any arbitrary procedure employing the bound variables. In particular, it can result in the addition of new facts to the data base, or modification of old facts in the data base.

* The production system viewpoint has been employed in artificial intelligence work in two different ways-as a model of human cognitive processes (Newell and Simon, 1972) and as a framework for pattern-directed inference systems (Waterman and Hayes-Roth, 1978). For a clear comparison of the ditrerent styles of use of production systems, see Davis and King (1976).


177

The rule interpreter has the task of deciding which rules to apply. It decides how the condition of a selected rule should be matched to the data base and monitors the problem-solving process. When it is used in an interactive program, it can turn to the user and ask for information (facts) that might permit the application of a rule. The strategy used by the rule interpreter is called the control strategy. The rule interpreter for a classical production system executes rules in a “recognize-act” cycle. Here the rule interpreter cycles through the condition parts of the rules, looking for one that matches the current data base and executing the associated actions for (some or all) rules that do match. As we point out in Section 3, there are many other ways to control the application of rules, but in all cases the result of executing actions is to change the data base, enabling the application of some rules and disabling others. At this level of generality, production systems are capable of arbitrarily complex behavior. The many ways in which conditions might be matched and variables might be bound, the many factors that might be important for rule selection, and the complicated effects of executing the rule actions can quickly lead to very difficult control problems. As one specific example, in many problem-solving systems the application of one rule can invalidate conditions needed for the application of a previously applied rule. To cope with such possibilities, the rule interpreter may have to employ backtracking strategies or maintain and search detailed records of the interdependencies between facts in the data base. Many of the expert systems construed to date have controlled this complexity by sacrificing the ability to perform general problem-solving tasks. They have achieved their competence by specializing, by exploiting the falfible but effective heuristic methods that human experts bring to a particular class of problems. Many of the high-performance systems (including MYCIN) can be characterized as simple deduction systems, programs in which a fact once entered in the global data base (whether by the user or by the application of a rule) is never subsequently deleted. Their actions are typically limited to the addition of new facts to the data base. In the remainder of this section, we use EMYCIN as a concrete, specific example of a rule-based approach to expert systems. While other rule-based systems have made other design trade-offs, EMYCIN illustrates well the issues that are involved. 2.1.2 EMYClN Viewed as a Production System

To see how EMYCIN uses the production system formalism to represent knowledge, we must see how it represents facts about the current

178


problem in its data base, and how it represents general knowledge in its rules. In all EMYCIN systems, facts are associative triples, that is, attributeobject-value triples, with an associated degree of certainty (called a certainty factor or CF). More generally, the EMYCIN syntax for statements of fact (both within the data base and within rules) is The (attribute) of (object) is (value) with certainty (CF). In the MYCIN dialogue shown above, fact triples are shown in the explanations as individual clauses of rules. For example, after question 35, one fact that has been established is “the type of the infection is bacterial.” It can also be seen that each question is asking for a value to be associated with an attribute of an ~ b j e c tIn . ~question 35, for example, MYCIN is asking whether or not the infection of the patient is hospital acquired.

A rule is a conditional sentence relating several fact statements in a logical relation. The nature of the relation varies from rule to rule. Often rules record mere empirical associations, rules of thumb based on past experience with little theoretical justification. Other rules are statements of theoretical associations, definitions, or causal laws. The condition part of an EMYCIN rule is called its premise. In general, an EMYCIN premise is the conjunction of a number of clauses. Each clause is a simple statement concerning the facts, such as “the age ofthe patient is greater than 10 days,” or “the identity of the infection is unknown.” To enable EMYCIN to query the data base to determine to what degree a clause might be true, each clause includes a matching predicate that specifies how the statement is to be compared against the data base of facts. In particular, a matching predicate is not required to return a binary answer, but may return a number between 0 and 1 indicating how much evidence there is that the predicate is satisfied. About two dozen matching predicates are a standard part of EMYCIN, including is the same as, is not the same as, has already been established, is not known, is greater than (for comparing numerical values), and so forth. The basic EMYCIN syntax for a rule is: These facts carry an associated degree of certainty also. The certainty is assumed to be 1 .O

if the user answers a question without qualifying it. When a fact is concluded by a rule,

its certainty is a function of (1) the certainty of the facts in the premise of the rule and (2) the strength of inference associated with the rule itself. This is described in more detail in Section 4.1 on plausible inference. The important point here is not how the numbers are used, but that they are needed.


179

PREMISE: ($AND (clausel) ... (clause-n)) ACTION: (CONCLUDE (new-fact) (CF)) where the “$” prefix indicates that the premise is not a logical conjunction, but a plausible conjunction that must take account of the certainty factors associated with each of the clauses. The action taken by this rule is merely the addition of a new fact to the data base (or, if the fact were already present, a modification of its certainty). EMYCIN also provide some mechanisms to allow the execution of more complicated actions. For example, in MYCIN we find the following rule (stated in English):

RULE 160 If

1) The time frame of the patient’s headache is acute, 2) The onset of the patient’s headache is abrupt, and 3) The headache severity (using a scale of 0 to 4; maximum is 4) is greater than 3 Then: 1) There is suggestive evidence ( . 6 ) that the patient’s meningitis is bacterial, 2) There is weakly suggestive evidence (.4) that the patient’s meningitis is viral and 3) There is suggestive evidence (.6) that the patient has blood within the subarachnoid space Thus this rule has three conclusions. It is represented internally in LISP as fol10ws:~ PREMISE: ($AND (SAME CNTXT HEADACHE-CHRONICITY ACUTE) (SAME CNTXT HEADACHE-ONSET ABRUPT) (GREATERP* (VAL1 CNTXT HEADACHE-SEVERITY 3))) ACTION: (DO-ALL (CONCLUDE CNTXT MENINGITIS BACTERIAL-MENINGITIS) TALLY 600) (CONCLUDE CNTXT MENINGITIS VIRAL-MENINGITIS) TALLY 400) (CONCLUDE CNTXT SUBARACHNOID-HEMORRHAGE YES TALLY 600))

‘

Since EMYCIN rules have a very regular syntax, a simple procedure can be used to translate rules from their internal format into English. It may be of interest to note that the list structures from the premise and the action are stored as values of the two properties, PREMISE and CONCLUSION, on the property list of the atom RULE160. The variable CNTXT is bound to the current object, in this case the parienr. SAME and GREATERP* are matching predicates used to compare the values of the named attributes of the current object against the named value. The variable TALLY refers to the certainty values, which are given on a - lo00 to lo00 scale for convenience.

180


These examples illustrate the basic techniques for representing facts and knowledge within the EMYCIN framework. Similar examples could be given for each of the several framework systems that have been developed to facilitate the construction of rule-based expert systems, including OPS EMYCIN AWX EXPERT KAS RAINBOW

Carnegie-Mellon University (Forgy and McDermott, 1977) Stanford University (vanMelle, 1980) University of Edinburgh Rutgers University (Weiss and Kulikowski, 1979) SRI International (Reboh, 1981) IBM Scientific Center (Palo Alto) (Hollander and Reinstein, 1979)

These framework systems provide important tools (such as editors) and facilities (such as explanation systems) that are beyond the scope of this paper to discuss. They also vary considerably in the syntax and the rule interpreters they employ. For example, in some of them all attributes must be binary. In some, uncertainty is expressed more formally as probabilities, or less formally as “major” or “minor” indicators, or cannot be expressed at all. And in some, additional structure is imposed on the rules to guide the rule interpreter. Despite these variations, these systems share a commitment to rules as the primary method of knowledge representation. This is at once their greatest strength and their greatest weakness, providing uniformity and modularity at the cost of imposing some very confining constraints.

2.2 Alternatives to Rule-Based Representation of Knowledge

There are alternatives to representing task-specific knowledge in rules. Naturally, it is sometimes advantageous to build a new system in PASCAL, FORTRAN, APL, BASIC, LISP, or some other language, using a variety of data structures and inference procedures as needed for the problem. Coding a new system from scratch, however, does not allow concentrating primarily on the knowledge required for high performance. Rather, one tends to spend more time on debugging the procedures that access and manipulate the knowledge. The nature of the task sometimes requires more flexibility or rigor than the rule-based frameworks provide. In those cases the frameworks mentioned later may provide satisfactory starting points. It should be noted that all of these frameworks easily allow specification of conditional rules.


181

A sign of the times, however, is that the most restrictive frameworks (rule based) are currently the easiest to use. There has been some experimentation with mixed representation as well (Aikins, 1980; Reinstein and Aikins, 1981). The basic idea is to increase the breadth of what one can represent easily while maintaining the advantages of having stylized representations (albeit more than one). 2.2.1 Frame-Based Representation Languages

One approach to representing knowledge that allows rich linkages between facts is a generalization of semantic nets (Brachman, 1977) known asframes (Minsky, 1975). A frame is an encoding of knowledge about an object, including not only properties (often called “slots”) and values, but pointers to other frames and attached procedures for computing values. The pointers indicate semantic links to other concepts, e.g., brother ox and also indicate more general concepts from which properties may be inherited and more specialized concepts to which its properties will be manifested. Programming with this mode of representation is sometimes called objecr-centered programming because knowledge is tied to objects and classes of objects. Some well-known frame-based representation languages are KRL OWL UNITS FRL AIMDS KL-ONE

Xerox PARC (Bobrow and Winograd, 1977) MIT (Szolovits et al., 1977) Stanford University (Stefik, 1979) MIT (Roberts and Goldstein, 1977) Rutgers University (Sridharan, 1980) Bolt, Beranek & Newman (Brachman, 1978)

2.2.2 Logic-Based Representation Languages

A logic-based representation scheme is one in which knowledge about the world is represented as assertions in logic, usually first-order predicate logic or a variant of it. This mode of representation is normally coupled with an inference procedure based on theorem proving. Logicbased languages allow quantified statements and all other well-formed formulas as assertions. The rigor of logic is an advantage in specifying precisely what is known and knowing how the knowledge will be used. A disadvantage has been difficulty in dealing with the imprecision and uncertainty of plausible reasoning. To date there have been few examples of logic-based expert systems as

182

BRUCE G. BUCHANAN AND RICHARD 0.DUDA

we have defined them, in part because of the newness of the languages. Some logic-based representation languages are PLANNER PROLOG ALICE FOL

MIT (Hewitt, 1972) Edinburgh University (Warren et al., 1977) University of Paris (Lauriere, 1978) Stanford University (Weyhrauch, 1980)

2.2.3 Generalized Languages

There is research in progress on general tools for helping a designer construct expert systems of various sorts. Designers specify the kind of representation and control and then add the task-specific knowledge within those constraints. The main advantage of such an approach is freedom. Designers specify their own constraints. The main disadvantage is complexity. Designers must be very knowledgeable about the range of choices and must be very patient and systematic about specifying choices. These tools look even more like high-level programming languages, which they are. The best known are ROSIE AGE RLL HEARSAY-I11 MRS LOOPS

Rand Corp. (Fain et al., 1981) Stanford University (Nii and Aiello, 1979) Stanford University (Greiner and Lenat, 1980) USC/ISI (Erman et al., 1981) Stanford University (Genesereth, 1981a) Xerox PARC

2.3 Knowledge Representation Issues

Regardless of the particular choice of representation language, a number of issues are important in the construction of knowledge bases for expert systems. We mentioned extendability, simplicity, and explicitness as three global criteria. In addition, the issues of consistency, completeness, robustness, and transparency are major design considerations for all systems. For specific problems, it may be essential to represent and reason with temporal relations, spatial models, compound objects, possible worlds, beliefs, and expectations. These are discussed below. Consistency in the knowledge base is obviously desirable. Because much of the knowledge coming from an expert is previously uncodified and much of it comes with uncertainty, however, it is unrealistic to assume that the knowledge base can be sufficiently cleansed to withstand a logician’s scrutiny. In rule-based systems, there are syntactic checks made at the time new rules are entered to see if there is potential conflict


183

between a new rule and existing rules. For example, two rules with the same premises but with different conclusions may be incompatible if the conclusions are mutually exclusive. On the other hand, there are many such pairs of rules that are perfectly acceptable because both conclusions are warranted. In MYCIN, for example, we find the same evidence “pointing to” different causes of an infection, with other rules invoking additional evidence to discriminate among likely causes. Syntactic completeness of the representation language is a logical requirement that many rule-based languages fail to satisfy. There are assertions, e.g., quantified statements, that are difficult or impossible to express. In DENDRAL, for example, it was difficult to express the proposition that if there exists a pair of data points bearing a complex relationship to one another then they constitute evidence for one class of interpretations of the data (Lindsay et al., 1980). Semantic completeness of the knowledge base for a problem area is also desirable. Because of the nature of the knowledge base and the way it is built, however, it will almost certainly fail to cover some interesting (sometimes important) possibilities. In a very narrow problem area, for example, there may be 100 attributes of interest, with an average of four important values for each attribute. (Only in extreme cases will all attributes be binary.) Thus there would be 79,800 possible rules relating two facts (400 items taken two at a time), over 10 million possible rules relating three facts, and so on. While most are semantically implausible, e.g., because of mutually exclusive values, the cost of checking all combinations for completeness is prohibitive. Checking the inferences made by a system in the context of carefully chosen test cases is currently the best way to check the completeness of coverage of the rules. Precision in specialized domains is achievable for many of the facts and rules, but not all. There is a temptation to make overly precise assertions for the knowledge base, even though there is no justification for fine precision. For example, although there are hospital-specific statistics about the incidence of a disease, one has to extrapolate to other (or all) hospitals very carefully. Representing degrees of imprecision is an important part of every representation methodology. Default knowledge is important protection against incompleteness. If you know that devices manufactured by XYZ Corp. are generally very reliable, then you might assume that the XYZ disk drive in your system is not the source of problems if you have no evidence to the contrary. Frame-based methods are designed to use their inheritance mechanisms to propagate default knowledge through parent-daughter links. In a rulebased system, default knowledge can certainly be represented but generally requires explicitly stating defaults for each class of actions.

184


Causal models provide a detailed specification of how a complex device works, whether it be biological or mechanical. For man-made devices, the models can be made more precise. Representing something like a circuit diagram and reasoning with it is difficult, although there have been successful research projects in which causal knowledge is central. Temporal relations, as causal ones, are still difficult to represent and use in generally satisfactory ways. Again, there has been good research, but expert systems generally do not reason well with continuous flows of data, or about continuous processes. Strategies for problem solving are an important part of expertise. However, they are difficult to represent and use efficiently. Strategies are discussed in more detail in Section 3.2 on control. The current state of the art of representing knowledge about technical domains is adequate for many simple problems but requires considerably more research on these major issues, and more. [For comparisons among systems, see Brooks (1981) and Ennis (1982).] Although rule-based frameworks have been used successfully for building several expert systems, the limitations are significant enough that researchers everywhere are looking for extensions or alternatives.

3.

Inference Methods In Expert Systems 3.1 Logical and Plausible inference

Although the performance of most expert systems is determined more by the amount and organization of the knowledge possessed than by the inference strategies employed, every expert system needs inference methods to apply its knowledge. The resulting deductions can be strictly logical or merely plausible. Rules can be used to support either kind of deduction. Thus a rule such as Has(x, feathers) OR (Able(x, fly) & Able(x, lay-eggs)) + Class(x, bird) amounts to a definition, and can be used, together with relevant facts, to deduce logically whether or not an object is a bird. On the other hand, a rule such as State(engine, won’t turn over) & State(headlights, dim) + State(battery, discharged)


185

is a “rule of thumb” whose conclusion, though plausible, is not always correct. Clearly, uncertainty is introduced whenever such a judgmental rule is employed. In addition, the conclusions of a logical rule can also be uncertain if the facts it employs are uncertain. Both kinds of uncertainty are frequently encountered in expert systems applications. However, in either case we are using the rule to draw conclusions from premises, and there are many common or analogous issues. In this section we temporarily ignore the complications introduced by uncertainty, and consider methods for using rules when everything is certain. In terms of the production systems model of rule-based expert systems, this section is concerned with the rule interpreter. As we mentioned in Section 2, the rule interpreter for a production system for unrestricted problem solving may have to employ complicated procedures to handle such things as pattern matching, variable binding, rule selection, and backtracking. To simplify our problem further, we shall restrict our attention to simple deduction systems, programs whose actions are essentially limited to adding new facts to the global data base. Our intent is to describe the general characteristics of the commonly used rule interpreters without becoming entangled in the detailed mechanisms they employ. Control strategies for more general problem solving systems are described in Nilsson (1980).

3.2 Control In this section we describe three commonly used control strategies: (1) data driven, (2) goal driven, and (3) mixed. Since control concerns are procedural, we shall describe these strategies semiformally as if they were programs written in a dialect of PASCAL, a “Pidgin PASCAL.” It is important to note at the outset, however, that these procedures are formal, employing no special knowledge about the problem domain; none of them possesses an intrinsic power to prevent combinatorial explosions. This has led to the notion of incorporating explicitly represented control knowledge in the rule interpreter, an idea that we discuss briefly at the end of this section. 3.2.1 Data-Driven Control

With data-driven control, rules are applied whenever their left-hand side conditions are satisfied. To use this strategy, one must begin by entering information about the current problem as facts in the data base.

186

BRUCE G. BUCHANAN A N D RICHARD 0.DUDA

The following simplified procedure, which we shall call “Respond,” can then be used to execute a basic data-driven strategy: Procedure Respond; Scan the data base for the set S of applicable rules; While S is nonempty and the problem is unsolved do begin Call Select-Rule(S) to select a rule R from S; Apply R and update the data base; Scan the data base for the set S of applicable rules end. Here we assume that a rule is applicable whenever there are facts in the data base that satisfy the conditions in its left-hand side. If there are no applicable rules, there is nothing to be done, except perhaps to return to the user and ask him or her to supply some additional information. (And, of course, if the problem is solved, there is nothing more to do.) If there is only one applicable rule, the obvious thing to do is to apply it. Its application will enter new facts in the data base. While that may either enable or disable previously inapplicable rules, by our assumption it will never disable a previously applicable rule. Lf there is more than one applicable rule, we have the problem of deciding which one to apply. Procedure Select-Rule has the responsibility for making this decision. Different data-driven strategies differ greatly in the amount of problem-solving effort they devote to rule selection. A simple and inexpensive strategy is to select the first rule that is encountered in the scan for S, “doing the first thing that comes to mind.” Unfortunately, unless the rules are favorably ordered, this can result in many useless steps. Elaborations intended to overcome such shortcomings can make data-driven control arbitrarily complex. Data-driven control is very popular, as is evidenced by the fact that it is known by so many different names (bottom up, forward chaining, pattern directed, or antecedent reasoning). R1 is an excellent example of an expert system that employs this strategy (McDermott, 1980). The popularity of data-driven control derives largely from the fact that such a program can respond quickly to input from the user, rather than forcing the user to wait until the program gets around to what the user wants to talk about. We have already mentioned the potential inefficiency of this strategy. Other problems that are often overlooked can arise from programs intended to be used by naive users. For example, as a data-directed program fires off one rule after another, its behavior can appear to be aimless, undermining a user’s confidence in its soundness. Also, since the user must begin by entering a set of facts, some kind of a language is needed to


187

convert facts as expressed in the user’s terms into the appropriate internal representation; menu systems may provide an acceptable solution, but a need for greater flexibility is frequently encountered. Both of these problems can be circumvented by using goal-driven control. 3.2.2 Goal-Driven Control

A goal-driven control strategy focuses its efforts by only considering rules that are applicable to some particular goal. Since we are limiting ourselves to rules that can add simple facts to the data base, achieving a goal G is synonymous with showing that the fact statement corresponding to G is true. In nontrivial problems, achieving a goal requires setting up and achieving subgoals. This can also lead to fruitless wandering if most of the subgoals are unachievable, but at least there is always a path from any subgoal to the original goal. Suppose that the user specifies a goal statement G whose truth is to be determined, typically a fact that the user would like to have present in the data base. Then the following simplified procedure, which we shall call “Achieve,” carries out a basic goal-driven strategy. Procedure Achieve(G);

Scan the knowledge base for the set S of rules that determine G; is empty then ask the user about G else While G is unknown and rules remain in S do

ZS’

begin

Call Choose-Rule(S) to choose a rule R from S; G’ t condition(R); If G’ is unknown then call Achieve(G’); ZfG’ is true then apply R end.

Thus the first step is to gather together all of the rules whose right-hand sides can establish G. If there is more than one relevant rule, procedure Choose-Rule receives the problem of making the choice. Once a rule R is selected, its left-hand-side G’ is examined to see if R is applicable. If there is no information in the data base about G’, the determination of its truth or falsity becomes a new subgoal, and the same procedure Achieve is applied to G’ recursively. The search continues in this fashion, working systematically backward from the original goal, until a subgoal is encountered for which there are no rules. At this point the system turns to the user and asks for the relevant facts. If the user cannot supply the needed information, then the rule the system was working on at the time cannot be used, but other lines

188


of reasoning can still be explored. If the information supplied shows that G’ is true, then R is applied. The process continues in this manner until either G is established to be true or false or no applicable rules remain. Since the left-hand side of the selected rule becomes the next subgoal, the choice of a rule is equivalent to subgoal selection. Different goaldriven strategies differ greatly in the amount of effort they devote to this problem. A simple and inexpensive strategy is to select the first rule that is encountered in the scan for S.s Unfortunately, unless the rules are favorably ordered, this can lead to exploring unpromising subgoals. As in the case of data-driven control, elaborations intended to overcome such shortcomings can make goal-driven control arbitrarily complex. Goal-driven control has also been used in many systems, and is briously known as top-down, backward-chaining, or consequent reasoning. A primary virtue of this strategy is that it does not seek information and does not apply rules that are unrelated to its overall goal. Furthermore, as we have seen in the excerpt from a MYCIN session, a rule-based system using this strategy can provide illuminating explanations of its behavior merely by telling the user what goal it is working on and what rule it is using. Probably the chief disadvantage of a goal-driven strategy is that it does not allow the user to steer it by volunterring relevant information about the problem. This can make goal-driven control unacceptable when rapid, real-time response is required.

3.2.3 Mixed Strategies Data-driven and goal-driven strategies represent two extreme approaches to control. Various mixtures of these approaches have been investigated in an attempt to secure their various advantages while minimizing their disadvantages. The following simple procedure combines the two by alternating between the two modes: Procedure Alternate; Repeat

Let user enter facts into global data base; Call Respond to deduce consequences; Call Select-Goal to select a goal G; Call Achieve(G) to try to establish G until the problem is solved. MYCIN employs a slightly different strategy. Instead of making a selection, MYCIN employs a cautious strategy of applying all of the rules in S. This is because individual MYCIN rules usually do not establish the truth or falsity of G , but merely increase or decrease its certainty; a final certainty assessment cannot be made until all of the rules are used.


189

Here Respond and Achieve are the data-driven and goal-driven procedures described previously. Select-Goal, which we do not attempt to specify, uses the partial conclusions obtained from the data-driven phase to determine a goal for the goal-driven phase. Thus the basic idea is to alternate between these two phases, using information volunteered by the user to determine a goal and then querying the user for more information while working on that goal. A variant of this procedure is used in the PROSPECTOR program (Duda et al., 1979). In this case, Select-Goal uses a heuristic scoring procedure to rank the goals in a prespecified set of “top-level” goals, but the user is allowed to see the results and make the final selection. Furthermore, a modified version of Achieve is used which ceases working on a goal whenever (1) its score drops and (2) it is not the highest-scoring toplevel goal. Thus PROSPECTOR works in a goal-driven mode when it seems to be making progress, but returns to the user for help in goal selection when serious trouble is encountered. 3.3 Explicit Representation of Control Knowledge

The advantages of making the task-specific knowledge modular and explicit extend to control knowledge as well. The strategy by which an expert system reasons about a task depends on the nature of the task and the nature of the knowledge the system can use. Neither data-driven, goal-driven, nor any particular mixed strategy is good for every problem. Different approaches are needed for different problems. Indeed, one kind of knowledge possessed by experts is knowledge of procedures that are effective for their problems. In most expert systems, the control strategy is encoded in procedures much like the ones we exhibited in pseudo-PASCAL. Thus control knowledge is not explicitly represented, but is buried in the code. This means that the system cannot easily explain its problem-solving strategy, nor can the system builder easily modify it. Several interesting attempts have been made to extract this knowledge and represent it explicitly. In his work on TEIRESIAS, Davis included the use of metarules, rules that determined the control strategy (Davis and Buchanan, 1977). TEIRESIAS essentially implements the procedure Select-Rule as a rulebased system; that is, strategic knowledge is used to reason about the most appropriate rules to invoke during problem solving or the most appropriate order in which to invoke them. Because the strategy rules can be context specific, the result is a system that adapts its rule selection strategy to the nature of the problem. Other important work on explicit control of reasoning in expert systems can be found in Aikins (1980),

190


Barnett and Erman (1982), Clancey and Letsinger (1981), deKleer et al. (1977), Genesereth (1981a), and Georgeff (1982). 4.

Reasoning with Uncertalnty

The direct application of these methods of deduction to real-world problems is complicated by the fact that both the data and the expertise are often uncertain. This fact has led the designers of expert systems to abandon the pursuit of logical completeness in favor of developing effective heuristic ways to exploit the fallible but valuable judgmental knowledge that human experts bring to particular classes of problems, Thus we now turn to comparing methods that have been used to accommodate uncertainty in the reasoning. 4.1

Plausible inference

Let A be an assertion about the world, such as an attribute-objectvalue triple. How can one treat the uncertainty that might be associated with this assertion? The classical formalism for quantifying uncertainty is probability theory, but other alternatives have been proposed and used. Among these are certainty theory, possibility theory, and the DempsterShafer theory of evidence. We shall consider all four of these approaches in turn, with emphasis on the first two. 4.2

Bayesian Probability Theory

With probability theory, one assigns a probability value P ( A ) to every assertion A. In expert systems applications, it is usually assumed that P measures the degree to which P is believed to be true, where P = 1 if A is known to be true, and P = 0 if A is known to be false.6 In general, the degree of belief in A will change as new information is obtained. Let P(A) denote our initial or prior belief in A , and let the conditional probability P(A1B) denote our revised belief in A upon learning that B is true. If this Traditionally, the probability P(A) is interpreted to measure the frequency of occurrence of the event A in a series of random trials. With this frequency ratio interpretation, P i s called an objective probability, and the estimation of a numerical value for P is a statistical problem. When P(A) is used to measure degree of belief, it is called a subjective probability, and the estimation of numerical values is done by interviewing experts (Savage, 1971). However, in either case the same calculus is used for combining probabilities, and a frequency ratio interpretation is often used in practice to estimate subjective probabilities and vice versa. The distinctions between objective and subjective probability theory are treated in depth in Fine (1973).


191

change in probability is due to the application of the rule B -+ A in a rulebased system, then some procedure must be invoked to change the probability of A from P(A) to P(A(B) whenever this rule is applied. In a typical diagnosis situation, we think of A as a “cause” and B as an “effect” and view the computation of P(A(B) as an inference that the cause is present upon observation of the effect. The expert often finds it easier to estimate P(B(A), the probability of observing the effect B when the cause A is active. In medical situations, this is further justified by the argument that the probability of a disease, given a symptom, may vary with time and place, while the probability of a symptom, given a disease, remains invariant (Lusted, 1968). Thus Bayes’ rule is commonly employed to compute P(AIB) from P(B1A). It turns out that the important information that is needed to employ Bayes’ rule is the prior probability P(A) and the likelihood ratio L defined by

where P(B1-A) is the probability of observing effect B when cause A is absent. If we think of the link between B and A as being expressed by a rule of the form B + A, then we can think of the logarithm of the likelihood ratio L as representing the strength or weight of the rule; rules with positive weights increase the probability of A, and rules with negative weights decrease it. Two generalizations are needed to employ this result in practice. First, we need a way to combine the effects of several rules, particularly when they are in conflict. Second, we need a way to employ uncertain evidence. While a thorough treatment of these topics is beyond the scope of this article, it is useful to explore this topic sufficiently to reveal the problems that are encountered and the general nature of their solutions. 4.2.1

Combining Rules

Suppose that we have n plausible rules of the form Bi+A,B*-*A,

..., B , + A ,

each with its own weight. Formally, the generalization of Bayes’ rule is simple. We merely consider B to be the conjunction B = BI & Bz & ..-& B, , and use the likelihood ratio L =

P(B1 & B2 & & B,(A) P(BI & Bt & * * . & B,I-A)

192


The problem with this solution is that it implies that we not only have weights for the individual rules connecting the Bj to A, but that we also have weights for the pairs Bi & Bj , triples Bi & B, & Bk , and so on, not to mention combinations involving negations when the evidence is known to be absent. This not only leads to extensive, nonintuitive computations, not directly related to the rules, but also requires forcing the expert to estimate a very large number of probabilities. A major simplification can be achieved if it is possible to assume that the Bi are statistically independent, both when A is present (true) and when A is not present (false). In that case, the conditional probabilities shown factor, and L simplifies to the product of the separate likelihood ratios. In other words, under that assumption, we need only have one weight associated with each rule, and we can combine the effects of several pieces of evidence by adding the separate weights. In general, of course, this assumption is not justified. In fact, it can be shown that the assumption cannot be made repeatedly when there are multiple assertion^.^ In its extreme form, this approach suggests designing an expert system by gathering all the evidence that bears in any way on a final conclusion A and going in one step from the observations to the conclusion. Such an approach typically founders on the complexity of the interactions among observations that are not even approximately independent. The pragmatic solution to this problem places the responsibility on the knowledge engineer to see that the rules are properly structured. Many problems caused by interactions can be solved by employing a hierarchical structure, with several levels of assertions between the direct observations and the final conclusions. The goal is to localize and limit the interactions and to have a relatively small number of clauses in a condition and a relatively small number of rules sharing a common conclusion. Note that this limitation on the number of rules does not reduce the amount of evidence considered in reaching a conclusion but, rather, controls the ways in which the observations are allowed to interact. A hierarchical structure is typically employed by the experts themselves to reduce the complexity of a problem. Wherever the remaining interactions still prevent the assumption of local independence, the rules have to be reformulated to achieve the desired behavior. For example, in the strongly interacting situation where BI suggests A and Bz suggests A, but the simulta-

’

To be more specific, if A, , A 2 , ..., A,,, are m mutually exclusive and exhaustive assertions, then the 2m assumptions that the Bi are independent under A, and -Aj are inconsistent with the laws of probability theory if rn > 2 (see Pednault ei al., 1981). Computations based on such assumptions will fail to show that A, must be true if A2 through A, have been ruled out.


193

neous presence of both B1 and Bz rules out A, one may have to augment the rule set {(B, + A with weight LI) (B2 + A with weight L2)) with the rule B, & B2 + A with weight --OCI. Thus, rather than viewing probability theory as a paradigm that prescribes how information should be processed, the knowledge engineer employs it as a tool to obtain the desired behavior. 4.2.2

Uncertain Evidence

There are two reasons why an assertion B might be uncertain: (1) The user might have said that B is uncertain, or (2) the program might have deduced B using a plausible rule. If we want to use B in a rule to compute P(AIB), the question then arises as to how to discount the conclusion because of the uncertainty associated with B. Let E symbolize whatever evidence was used to determine B,‘and let P(B(E) denote the current probability of B. Then our problem is to compute P(AIE), the current probability of A given this same evidence. It is shown in Duda et al. (1976) that under certain reasonable assumptions we should be able to compute P(A1E) by the formula P(A1E) = P(A1B) * P(B1E)

+ P(A1-B) * [ I

-

P(BIE)].

This formula certainly works in the extreme cases of complete certainty; that is, if we know that B is true, we obtain P(AJB),and if we know that B is false, we obtain P(A1-B). Unfortunately, a serious problem arises in intermediate cases. In particular, suppose that E actually supplies no information about B, so that P(B1E) is the same as the prior probability P(B). While the previous formula promises to yield the prior probability P(A), when the computation is based on numerical values obtained from the expert, the resulting value for P(A1E) will usually not agree with the expert’s estimate for the prior probability P(A). That is, the four quantities P(A), P(B), P(AIB), and P(A1-B) are not independent, and the expert’s subjective estimates for them are almost surely numerically inconsistent. In this particular case, the problem can be solved in various ways, such as by not asking the expert for P(A), but by computing it from P(A) = P(AJB)P(B)+ P(A1-B)P(-B). However, that only makes the parameters for one rule consistent, and the solution is not at all obvious when there is a network of rules that have inconsistent values for probability parameters. The solution adopted in the PROSPECTOR system was to replace

194


the preceding expression for P(AJE)by a piecewise linear function of P(BJE)that yields the expert’s estimate for P(A) when P(B1E) is numerically equal to the expert’s estimate for P(B) (see Duda et al., 1976). Interestingly, the resulting formulas turn out to be closely related to the certainty measure used in MYCIN, which we consider next. 4.3 Certainty Theory

We have seen several problems that arise in using traditional probability theory to quantify uncertainty in expert systems. Not the least of these is the need to specify numerical values for prior probabilities. While an expert may be willing to say how certain he or she feels about a conclusion A when evidence B is present, he or she may be most reluctant to specify a probability for A in the absence of any evidence, particularly when rare but important events are involved. Indeed, some of the problems that are encountered in obtaining consistent estimates of subjective probabilities may well be due to the fact that the expert is not able to separate probability from utility or significance and is really expressing some unspecified measure of importance. To accommodate this reality, the designers of the MYCIN system developed a theory of certainty that provides a simple and effective alternative approach (Shortliffe and Buchanan, 1975).The central notion is that we can associate a certainty measure C(A) with every assertion A, where C = 1 if A is known to be true, C = - 1 ifA is known to be false, and C = 0 if nothing is known about A.* A similar certainty factor CF is associated with every rule. The theory consists of procedures for updating certainties as rules are applied, and an analysis of the properties of these procedures. The procedures for updating certainties are easily stated. Initially, the certainty associated with any assertion is 0. If a rule says that B --* A with a certainty factor CF, then the certainty of A is changed to CF when B is observed to be true. Only two things remain to be specified: (1) the procedure for combining evidence when more than one rule concludes A and (2) the treatment of uncertain evidence. We consider each of these in turn. If we also assume that we can associate probabilities with assertions, then it would appear that C = 1 corresponds to P = 1, C = - 1 corresponds to P = 0, and C = 0 corresponds to P being at its prior value. The original MYCIN definitions of certainty were in terms of piecewise-linear functions of probability. However, the EMYCIN view, which we present in this section, is that the calculus of certainty factors is a heuristic approach that allows rule-based systems to cope with uncertainty in a simple and straightforward way, judging it more by its usefulness than by its theoretical properties (vanMelle, 1980). It is a one-numbercalculus that combines subjective estimates of probability and risk in a measure of importance, which bears no simple relationship to probability alone.


195

4.3.1 Combining Evidence

Suppose that (1) the present certainty of an event A is CA (which may be nonzero because of the previous application of rules that conclude A), (2) there is an unused rule of the form B + A with a certainty factor CF, and (3) B is observed to be true. Then the EMYCIN formula for updating C(A) to C(A1B) is

+ CF - CA * CF CA + CF + CA * CF CA + CF

(CA C(A(B) =

1

-

min[(CAI,ICFI]

if CA and CF > 0 if CA and CF < 0 otherwise.

This is the EMYCIN analog of the procedure of multiplying likelihood ratios to combine “independent” evidence. By applying it repeatedly, one can combine the conclusions of any number of rules BI + A, Bz+ A, ..., B, + A. Aside from being easy to compute, it has several other desirable properties. First, the resulting certainty C(AIB) always lies between -1 and 1, being + I if CA or CF is +1, and -1 if CA or CF is -1. When contradictory conclusions are combined (so that CA = -CF), the resulting certainty is 0. Except at the singular points (1, - 1) and (- 1, I ) , C(AIB) is a continuous function of CA and CF, increasing monotonically in each variable. The formula is symmetric in CA and CF, and the results it yields when more than two pieces of evidence are combined are independent of the order in which they are considered. Of course, none of these properties prove that this is the “correct” way to combine the conclusions of rules. Indeed, the results will be wrong in strongly interacting cases, such as our previous example in which B1 suggests A and B2 suggests A but the simultaneous presence of BI and B2 rules out A. As with the use of Bayesian methods, the knowledge engineer should view certainty theory as a tool to be employed to produce the desired behavior. 4.3.2 Uncertain Evidence

When the evidence B for a rule B + A is itself uncertain, it is clear that the strength of the conclusion must be reduced. The EMYCIN procedure is to multiply the certainty factor CF for the rule by the certainty of B, provided that the certainty of B is positive. If the certainty of B is negative, the rule is considered to be inapplicable and is not used.9 EMYCIN If the absence of B has a significant effect on A, we could add a rule of the form -B -A with some certainty factor CF.

+

196


assumes that a rule cannot be employed unless the certainty of its antecedent is greater than a threshold value of 0.2. This heuristic, which implies that the certainty of a conclusion is not a strictly continuous function of the certainty of the evidence, saves time by inhibiting the application of many marginally effective rules, and saves confusion by making explanations provided by the system more understandable. One more effect of uncertain evidence remains to be mentioned. In general, the antecedent B of a rule is a logical function of predicates or relations on associative triples. Since these functions or relations can return certainty values rather than truth values, there is a question as to how the Certainty of theiriogical combination is determined. The answer is that it is computed through the recursive application of the following formulas: C(Bl OR B2) = max[C(Bl), C(Bd1 C(B1 8~B2)

=

min[C(Bd, C(B2)I

C(-B) = - C(B). These formulas are essentially the same as the corresponding formulas of possibility theory, which is discussed briefly in Section 4.4. 4.4 Possibility Theory

Probability theory captures only some of the important aspects of uncertainty, and a variety of alternative approaches, such as certainty theory, have been developed to overcome its limitations. One of the most interesting of the recent alternatives is Zadeh’s theory of possibility (Zadeh, 1978). It is based on his earlier development of the theory of fuzzy sets (Zadeh, 1965), much as probability theory is based on measure theory. Zadeh asserted that although the random model of probability theory may be appropriate for problems involving the measure of information, it is inappropriate for problems concerning the meaning of information. In particular, much of the uncertainty surrounding the use of English terms and expressions concerns vagueness rather than randomness. Possibility theory provides a formalism for treating vagueness that is analogous to probability theory as a formalism for treating randomness. The theory of fuzzy sets expresses this kind of imprecision quantitatively by introducing characteristic or membership functions that can assume values between 0 and 1. Thus if S is a set and if s is an element of S, a fuzzy subset F of S is defined by a membership function pF(s) that measures the degree to which s belongs to F. To use a standard example,

PRINCIPLES OF RULE-BASE0 EXPERT SYSTEMS

197

if S is the set of positive integers and F is the fuzzy subset of small integers, then we might have @( 1) = I , pF(2) = 1, pF(3) = .8, ...,pF(20) = .01, and so on. Let X be a variable that can take values in S. The statement “X is F” (for example, the statement “X is a small integer”), induces a possibility distribution on X, and the possibility that X = s is taken to be pF(s). Now probability theory is not concerned with how the numerical values of probabilities are determined but, rather, with the rules for computing the probability of expressions involving random variables. Similarly, possibility theory is not concerned with how the numerical values of the possibility distributions are obtained but, rather, with the rules for computing the possibilities of expressions involving fuzzy variables. In particular, if Poss(X = s) is the possibility that the fuzzy variable X is equal to s, then the formulas for disjunction, conjunction, and negation are

Poss(X = s OR Y =

t) =

Poss(X = s & Y = t ) Poss(X # s)

=

max[Poss(X = s), Poss(Y = t ) ] min[Poss(X = s), Poss( Y = t ) ]

= 1 -

Poss(X = s).

For most of the concepts of probability theory there is a corresponding concept in possibility theory. For example, it is possible to define multivariate possibility distributions, marginal possibility distributions, and conditional possibility distributions (see Zadeh, 1978). Thus, in principle one can use fuzzy possibility theory much like probability theory to quantify the uncertainty introduced by vagueness, whether the vagueness comes from the data or from the rules. Although possibility theory is a subject of great interest, it has yet to be exploited in work on expert systems. This is partly due to the fact that most of the problems that limit probability theory also arise in possibility theory, such as the problem of prior possibilities and that of dependence in multivariate possibility distributions. Furthermore, as with certainty theory, possibility theory suffers from the problem that the semantics of its measure are not objectively defined. However, the distinction between uncertainty due to randomness and uncertainty due to vagueness is both valid and important and should play a role in work in expert systems. 4.5 The Dernpster-Shafer Theory of Evidence

We conclude this overview of formalisms for treating uncertainty with a brief consideration of a generalization of probability theory created by Dempster and developed by Shafer that has come to be known as the Dempster-Shafer theory of evidence (Shafer, 1976; Barnett, 1981).

198


Dempster and Shafer insisted that one must make a fundamental distinction between uncertainty and ignorance. In probability theory, one is forced to express the extent of one’s knowledge about or belief in an assertion A through a single number P ( A ) . Dempster and Shafer pointed out that the classical Bayesian agony concerning prior probabilities is often due to the fact that one often simply does not know the values of prior probabilities, and this ignorance may make any particular choice arbitrary and unjustifiable. The Dempster-Shafer theory of evidence recognizes the distinction between uncertainty and ignorance by introducing belief functions that satisfy axioms that are weaker than those of probability functions. Thus probability functions are a subclass of belief functions, and the theory of evidence reduces to probability theory when the probability values are known. Roughly speaking, the belief functions allow us to use our knowledge to put constraints or bounds on the assignment of probabilities to events without having to specify the probabilities themselves. In addition, the theory of evidence provides appropriate methods for computing belief functions for combinations of evidence. As one might expect, a theory that includes probability theory as a special case suffers from many of the same problems that plague probability theory. The greater complexity results in an increase in computational problems as well, and the conclusions that can be reached are necessarily weaker. However, when available knowledge does not justify stronger conclusions, this latter fact has to be accepted. Whether or not the theory of evidence will provide the basis for computationally effective procedures for treating uncertainty, it deserves attention for exposing the effects of lack of knowledge on probabilistic reasoning.

5. Key Concepts

In Sections 2-3 we focused on three central issues in the ,design of expert systems, with special attention to rule-based systems. The representation, inference methods, and methods for reasoning under uncertainty are the elements of the design of rule-based systems that give them power. We turn now to a broader look at several less technical aspects of building an expert system. These are observations derived from our own experience and constitute suggestions for designing an expert system. They also reflect the current state of the art. In this section we first list several of these considerations, with very little explanation. Then we look at the nature of the problem (the first of the considerations) in more detail from three different perspectives: the


199

types of problems for which expert systems have been developed (Section 5 . I), the nature of the data encountered in these problems (Section 5.2), and the nature of the expertise (Section 5.3). We spoke earlier of the importance of separating task-specific knowledge from a system’s inference methods, and we discussed the representation and inference methods by which we can realize the truth in the assumption that knowledge is power. We list here some of those and other key ideas in putting together an expert system. 5.1 Nature of the Problem

Narrow scope. The task for the system must be carefully chosen to be narrow enough so that the relevant expertise can be encoded, and yet complex enough so that expertise i s required. This limitation is more because of the time it takes to engineer the knowledge into a system including refinement and debugging, than because of space required for the knowledge base. lo Existence of an expert. There are problems so new or so complex that no one ranks as an expert in the problem area. Generally speaking, it is unwise to expect to be able to construct an expert system in areas where there are no experts. Agreement among experfs. If current problem-solving expertise in a task area leaves room for frequent and substantial disagreements among experts, then the task is not appropriate for an expert system. Data auailable. Not only must the expertise be available, but test data must be available (preferably on-line). Since an expert system is built incrementally, with knowledge added in response to observed difficulties, it is necessary to have enough test cases to be able to explore the boundaries of what the system knows. Milestones dejnabfe. A task that can be broken into subtasks, with measurable milestones, is better than one that cannot be demonstrated until all the parts are working. 5.2 Representat ion

Separation of task-specific knowledge from the rest of the program. This separation is essential to maintain the flexibility and understandability required in expert systems. Attention to detail. Inclusion of very specific items of knowledge about the domain, as well as general facts, is the only way to capture the expertise that experience adds to textbook knowledge. la

This contrasts with early work in A1 in which space was at least as much an issue.

200


Uniform data structures. A homogeneous representation of knowledge makes it much easier for the system builder to develop acquisition and explanation packages. 5.3 Inference

Symbolic reasoning. It is commonplace in AI, but not elsewhere, to regard symbolic, nonnumeric reasoning as a powerful method for problem solving by computers. In applications areas where mathematical methods are absent or computationally intractable, symbolic reasoning offers an attractive alternative. Combination of deductive logic and plausible reasoning. Although deductive reasoning is the standard by which we measure correctness, not all reasoning-even in science and mathematics-is accomplished by deductive logic. Much of the world’s expertise is in heuristics, and programs that attempt to capture expert-level knowledge need to combine methods for deductive and plausible reasoning. Explicit problem-solving strategy. Just as it is useful to separate the domain-specific knowledge from the inference method, it is also useful to separate the problem-solving strategy from both. In debugging the system it helps to remember that the same knowledge base and inference method can produce radically different behaviors with different strategies. For example, consider the difference between “find the best” and “find the first over threshold.” Interactive user interfaces. Drawing the user into the problem-solving process is important for tasks in which the user is responsible for the actions recommended by the expert system, as in medicine. For such tasks, the inference method must support an interactive style in which the user contributes specific facts of the case and the program combines them in a coherent analysis. 5.4 Explanation

Static queries of the knowledge base. The process of constructing a large knowledge base requires understanding what is (and is not) in it at any moment. Similarly, using a system effectively depends on assessing what it does and does not know. Dynamic queries about the line of reasoning. As an expert system gathers data and makes intermediate conclusions, users (as well as system builders) need to be able to ask enough questions to follow the line of reasoning. Otherwise the system’s advice appears as an oracle from a black box and is less likely to be acceptable.


201

5.5 Knowledge Acquisition

Band width. An expert’s ability to communicate his or her expertise within the framework of an expert system is limited by the restrictions of the framework, the degree to which the knowledge is already well codified, and the speed with which the expert can create and modify data structures in the knowledge base. Knowledge engineer. One way of providing help to experts during construction of the knowledge base is to let the expert communicate with someone who understands the syntax of the framework, the rule interpreter, the process of knowledge base construction, and the practical psychology of interacting with world-class experts. This person is called a “knowledge engineer. ”

5.6 Validation

Level ofperformance. Empirical measures of adequacy are still the best indicators of performance, even though they are not sufficient for complete validation by any means. As with testing new drugs by the pharmaceutical industry, testing expert systems may best be accomplished by randomized studies and double-blind experiments. Static evaluation. Because the knowledge base may contain judgmental rules as well as axiomatic truths, logical analysis of its completeness and consistency will be inadequate. However, static checks can reveal potential problems, such as one rule subsuming another and one rule possibly contradicting another. Areas of weakness in a knowledge base can sometimes be found by analysis as well.

5.7 Classes of Problems for Expert Systems

The first of the key concepts listed previously was the nature of the problem. We examine this issue in somewhat more detail in this section and Sections 5.8 and 5.9. While there are many activities an expert performs, the activities for which expert systems have been built fall into three categories: analysis, synthesis, and interface problems. Analysis problems have been the most successfully solved with the knowledge engineering approach to date. Many applications programs that have the characteristics of expert systems have been developed for analysis problems in a diversity of areas, including chemistry (Buchanan and Feigenbaum, 1978; Carhart, 1979); genetics (Stefik, 1978), protein crystallography (Engelmore and Terry, 1979), physics (Bundy et al.,

202


1979; Larkin e l al., 1980; Novak and Araya, 19801, interpretation of oil well logs (Barstow, 1979b; Davis et al., 19811, electronics troubleshooting (Addis, 1980; Bennett and Hollander, 1981; Brown et al., 1982; Davis et al., 1982; Genesereth, 1981b; Kandt and Newlon, 1981; Stallman and Sussman, 1977), materials engineering (Basden and Kelly, 1982; Ishizuka et al., 1981), mathematics (Brown and Burton, 1978; Moses, 1971), medical diagnosis (Chandrasekaran et al., 1980; Fagan, 1980; Gorry et al., 1978; Heiser et al., 1978; Horn et al., 1981; Kaihara et al., 1978; Lindberg et at., 1981; Patil et al., 1981; Pople, 1977; Reggia, 1978; Shortliffe, 1976; Shortliffe et al., 1981; Swartout, 1977; Szolovits and Pauker, 1978; Tsotsos, 1981; Weiss et al., 1979), mineral exploration (Duda et al., 1979), aircraft identification and mission planning (Engelman et al., 1979), military situation assessment (McColl et al., 1979; Nii et al., 1982), and process control (Mamdani, 1982). Within these and other disciplines, analysis problems are described using many different terms, including Data interpretation Explanation of empirical data Understanding a complex of data (e.g., signal understanding) Classification Situation assessment Diagnosis (of diseases, equipment failures, etc.) Troubleshooting Fault isolation Debugging Crisis management (diagnosis component) An expert system working on one of these problems analyzes a description of a situation and provides plausible interpretations of what the data seem to indicate. The data may come from a variety of sources, ranging from subjective opinion to precise readings of instruments. Synthesis problems have the character of constructing a solution to satisfy a goal within stated constraints. In many cases, solutions to small, local problems need to be synthesized into a coherent solution that satisfies global constraints. Synthesis problems arise in many fields, including planning experiments in molecular genetics (Friedland, 1979; Stefik, 1980), configuring the components of a computer system (McDermott, 1980; 1981), scheduling (Fox et al., 1982; Goldstein and Roberts, 1979; Lauriere, 19781, automatic programming (Barstow, 1979a;McCune, 1977), electronics design (deKleer and Sussman, 1980; Dincbas, 1980; Sussman, 19781, and chemical synthesis (Gelernter et al., 1977; Wipke et al., 1977). These problems have been called


203

Planning (or constructing a plan of action) Fault repair Process specification Design (of complex devices or of experiments) Configuration Therapy (or therapy planning) Automatic programming Computer-aided chemical synthesis planning In addition to analysis and synthesis problems, expert systems have been built to provide advice on how to use a complex system (Anderson and Gillogly, 1976; Bennett and Engelmore, 1979; Genesereth, 1978; Hewitt and Smith, 1975; Krueger et al., 1981; Rivlin et al., 1980; Waterman, 1979) or to tutor a novice in the use or understanding of a body of knowledge (Brown et al., 1982; Clancey, 1979; O’Shea, 1979). These problems are partly analytic, since the advice or tutorial must be guided by an analysis of the context, and partly synthetic since the advice must be tailored to the user and the problem at hand. 5.8 The Data

One of the central concerns in choosing a task for an expert system is the nature of the data. In problems of analysis, the data require interpretation by means of some model or theory; yet in many interesting problems, the data are not as complete or “clean” as the theory seems to require. In applying a theory to individual cases, the data are not always available to “plug into” formulas and principles. In the absence of perfect data, however, experts can still provide good suggestions, when a novice can not. We have identified several important concerns, briefly discussed below: incompleteness, noise, and nonindependence. Incompleteness of the data is a common difficulty. In medical diagnosis, for example, a physician usually must act before all possibly relevant tests have been made. Uncertainty of the data compounds the difficulty. Decision makers know that their sources of information are fallible, whether the sources are instruments or other persons. Some tests are notoriously unreliable, some items of information are so incongruous with other data that something must be wrong. Yet in the face of these uncertainties in the data, expert decision makers can still integrate the results of many tests better than novices. Noise in the data can be confusing. Spurious data points, or “red herrings,’’ can throw the best problem solvers off the track. However, ex-

204


perts have had more experience in sorting out good and bad data and are less likely to remain confused than novices. The data given to a decision maker can be noisy for a variety of reasons, including electronic noise, misreading dials and indicators, and transcription errors. By the time the decision maker sees the data, it is often too late to check the validity of any single data point. Nonindependence in the data is often a difficulty, particularly for statistical methods that rely on assumptions of independence to combine pieces of evidence. In most interesting problems, though, there are processes linking many parts of complex systems, so that evidence about one part of the system is richly linked with other pieces of evidence. If the data were known to be error free, then avoiding redundancy would simplify the decision-making process. However, in the face of possibly unreliable data, redundancy is beneficial in helping reduce the effects of spurious data. The data are often of uneven grain size, combining gross descriptive reports with minute, precise statements of fact. Qualitative and quantitative information are mixed. Subjective reports are mixed with objective statements. There is no uniform theoretical framework in which information at all these levels can be combined; yet decision makers faced with less than perfect data often welcome more information, regardless of how heterogeneous it is. The volume of data, however, can get to be confusing. The combinatorics of considering meaningful clusters of data quickly swamp a person’s ability to consider combinations of data points systematically. One of the primary advantages of an expert system in coping with all of this ambiguity is its ability to exploit redundancy. Multiple pieces of data can indicate more or less the same interpretation, some more strongly than others, while others indicate mutually exclusive interpretations. An expert system will work with the data available, using the overlapping contributions to help make up for missing data and incomplete interpretation rules. 5.9 The Expertise

The proficiency of an expert system is dependent on the amount of domain-specific expertise it contains. But expertise about interesting problems is not always neatly codified and waiting for transliteration into a program’s internal representation. Expertise exists in many forms and in many places, and the task of knowledge engineering includes bringing together what is known about a problem, as well as transforming (not merely transcribing) it into the system.


205

We have already said that much of the expertise is symbolic, heuristic, and not well formalized. That implies that an expert’s knowledge is not always certain, that it is provisional, without guarantees of correctness. Because it is not well formalized (e.g., in neat theoretical formulations in textbooks), a specialists’s knowledge is not always easily accessible. In addition, we have to assume that it is incomplete, since the facts and heuristics change with increased experience. Because of the multitude of sources of expertise, an expert articulates what he or she knows in a mixture of frameworks, using terminology ranging from broad notions of common sense to precisely defined theoretical terms. As with the data, there is a mixture in concepts that the knowledge engineer must help the expert map into the system. Moreover, these frameworks are richly intertwined and not neatly separated into distinct subspecialty areas. Woven into the facts and relations are many examples, exceptions, links to other specialty areas. They appear to be well indexed, for experts seem to have no difficulty in citing examples for every principle and two exceptions for every example. Finally, an expert’s store of knowledge is large. Regardless of how one measures the size of an expert’s knowledge base, it is almost a truism to say that the more a problem solver knows, the better its advice will be. As with the data available for solving interesting problems, the expertise that is available may be redundant, with rich interdependencies in the reasoning network. In the case of the expertise, as well, the redundancy can be exploited as protection against being led into blind alleys by spurious data or inappropriate heuristics. 6.

Conclusions

Expert systems represent an important set of applications of artificial intelligence to problems of commercial as well as scientific importance. There appear to be three main motivations for building an expert system, apart from research purposes: (1) Replication of expertise, providing many (electronic) copies of an expert’s knowledge so it can be consulted even when the expert is not personally available. Geographical distance and retirement are two important reasons for unavailability. ( 2 ) Union of expertise, providing in one place the union of what several different experts known about different specialties. This has been realized to some extent in PROSPECTOR (Reboh, 1981) and CASNET (Weiss et al., 1979) which show the potential benefits of achieving such a superset of knowledge bases.

206


( 3 ) Documentation, providing a clear record of the best knowledge available for handling a specific problem. An important use of this record is for training, although this possibility is just beginning to be exploited (Brown et al., 1982; Clancey, 1979).

Rule-based systems are currently the most advanced in their systembuilding environments and explanation capabilities, and have been used to build many demonstration programs. Most of the programs work on analysis tasks such as medical diagnosis, electronic troubleshooting, or data interpretation. The capability of current systems is difficult to define. It is clear, however, that they are specialists in very narrow areas and have very limited (but not toally missing) abilities to acquire new knowledge or explain their reasoning. One of the important concepts of this style of programming is the throw-away program. The existence of a framework system in which to construct a new program allows the expert and knowledge engineer to focus on the knowledge needed for problem solving. Without a framework, they spend more time on syntactic considerations than on semantic ones. Because the framework is already in place, however, they can readily scrap bad conceptualizations of the problem-solving knowledge. For programs that are built incrementally, as expert systems are, throwing away false starts is important. Technological innovations will be incorporated into expert systems as the conceptual difficulties of representation and inference in complex domains yield to standardized techniques. These will be most noticeable in the size of the computer and in the input/output of the system. A portable device for troubleshooting, with voice input/output for example, is not out of the question in the near future. Systems will use much larger knowledge bases than the few hundred to few thousand rules now used. They will be linked electronically to large data bases to facilitate inference and avoid asking questions whose answers are matters of record. “Smart” interpretation systems will be directly linked to data collection devices, as PUFF is linked to a spirometer, to avoid asking about the data for the case at hand. For the time being, the major difficulty in constructing an expert system will remain engineering the knowledge that experts use into a form that is usable by the system. Every problem area and every expert is unique. Nevertheless, many common features have been identified and built into knowledge acquisition packages of the major frameworks. Future systems will integrate several modes of knowledge acqusition: Some rules can be extracted from an expert, some from large data bases, and some from experience.


207

Finally, more powerful system-building frameworks will be developed to reduce the time it takes to iterate on the build-test-refine cycle and to increase the range of applications. There is considerable research in A1 of interest to designers of expert systems (Buchanan, 1981, 19821, much of it relating to the two central issues of representation and inference, some of it relating to improving the interface between system builders and the emerging system. As this work is integrated into more powerful frameworks, the breadth and depth of applications will increase.

Appendix. Answers to Questions about MYCIN’S Consultation in Section 1.1

Please ask any simple English questions regarding the above consultation session or the overall knowledge of the consultation system. When discussing the consultation above, please refer to organisms, cultures, etc. by their numeric names (e.g., ORGANISM-I). Questions which were asked during the consultation may be specified by the numbers that preceded them (e.g., “WHY DID YOU ASK QUESTION I?”). Control-0 will abort type out, or “cancel,” the current question. Type HELP for a list of the options available. Enter the word STOP when you have no further questions. Type HELP for the list of question options.

** WHY DID YOU TREAT E.COLI? ITEM-1 is E.coli associated with meningitis. I decided to treat ITEM-1 on the basis of clinical evidence only. The following rules concluded about: the likelihood that E.coli is one of the organisms (other than those seen on cultures or smears) which might be causing the infection RULE545 was used to conclude that E.coli is one of the organisms (other than those seen on cultures or smears) which might be causing the infection (.74). The last question asked before the conclusion was made was 35. RULE557 was used to conclude that E.coli is one of the organisms (other than those seen on cultures or smears) which might be causing the infection (.93). The last question asked before the conclusion was made was 37.

208


The parameters most frequently used in these rules were: the infection which requires therapy whether a smear of the culture was examined whether organisms were seen on the stain of the culture the type of the infection Would you like a summary of ALL the parameters used by these rules? ** YES Parameter (rules with this parameter) the white count from the patient’s peripheral CBC (in thousands) (557)

whether the results of a recent CBC of the patient are available (557) whether the infection was acquired while the patient was hospitalized (545)

the age of the patient (545) the relevant items from the history of the patient (545) the type of the infection (545, 557) whether organisms were seen on the stain of the culture (545, 557) whether a smear of the culture was examined (545, 557) the infection which requires therapy (545, 557) ** WHY DIDN’T YOU GIVE TETRACYCLINE FOR E.COL1 IN REC-1? Shall I describe the therapy algorithm for you? ** YES DESCRIPTION OF THERAPY SELECTION [Type t 0 to abort printout] MYCIN is designed to provide optimal therapy recommendations. The program uses the following three steps: The first step ranks the drugs for each organism that requires therapy. Rules for doing this take into account in vitro (lab) sensitivities and the patient’s response to prior and current therapies. In the absence of other information, default lab sensitivities are used. Preference is given to current therapies if certain criteria are satisfied. Next, 1 or 2 drugs are selected to cover for the organisms which are most likely to be causing the infection. As many of the “less likely” organisms as possible are treated, but they are disregarded altogether if 3 or more drugs are necessary. Throughout, preference is given to drugs that appear higher in the treatment lists. (These lists can be examined via the RX option.)


209

Finally, the set of drugs is tested for patient factors that preclude their administration, such as allergies, age, and renal function. TETRACYCLINE was not prescribed for ITEM-1 in RECOMMENDATION- 1 because:

TETRACYCLINE was discounted for ITEM- 1 because there is evidence that this E.coli is not sensitive to it. Would you like to see some details?

** NO

** HOW DO YOU TREAT PNEUMOCOCCAL PNEUMONIA? The MYCIN system currently can treat the following infections: BACTEREMIA MENINGITIS CYSTITIS. Because the knowledge base is incomplete, all other infections will be treated like bacteremia.

For treatment of DIPLOCOCCUS-PNEUMONIAE in BACTEREMIA The drugs of choice (ranked by relative efficacy, assuming sensitivity) are : I st

choice: PENICILLIN AMPICILLIN 2nd choice: NAFCILLIN CEPHALOTHIN ERYTHROMYCIN CLINDAMYCIN 3rd choice: TETRACYCLINE CHLORAMPHENICOL VANCOMYCIN Duration of therapy: 28 days

** WHAT DOSE OF ERYTHROMYCIN DO YOU RECOMMEND? When ERYTHROMYCIN is given for BACTEREMIA, the recommended dosage is: if age is > 12 years then 12 mg/kg q6h IV

21 0


else if age is > .Of3 years then 5 mg/kg q6h IV else if age is > .02 years then 5 mg/kg q12h IV [Different dosages are recommended for different infections.]

**

HOW DO YOU TREAT CYSTITIS?

In general, treatment for an infection depends on the causative organism. Please include in your question the organism of interest.

** STOP GENERAL REFERENCES Davis, R., and King, J. (1976). An overview of production systems. In “Machine Intelligence” (E.W. Elcock and D. Michie, eds.), Vol. 8, pp. 300-332. Wiley, New York. Feigenbaum, E.A. (1979). Themes and case studies of knowledge engineering. In “Expert Systems in the Micro-Electronic Age” (D. Michie, ed.), pp. 3-25. Edinburgh Univ. Press, Edinburgh. Michie, D., ed. (1981). “Machine Intelligence,’’ Infotech State.of the Art Rep. Ser. 9, No. 3. Pergamon Infotech Ltd., Maidenhead, England.

REFERENCES Addis, T.R. (1980). Towards an “expert” diagnostic system. ICL Tech. J. pp. 79-105. Aikins, J.S. (1980). Prototypes and production rules: A knowledge representation for computer consultation. Ph.D. Dissertation, STAN-CS-80-814. Computer Science Department, Stanford University, Stanford, California. Amarel, S. (1981). “Problems of Representation in Heuristic Problem Solving: Related Issues in the Development of Expert Systems,” Tech. Rep. No. CBM-TR-I 18. Laboratory for Computer Science Research, Rutgers University, New Brunswick, New Jersey. Anderson, R.H., and Gillogly, J.J. (1976). The Rand Intelligent Terminal (RITA) as a network access aid. AFIP Proc. 45, 501-509. Barnett, J.A. (1981). Computation methods for a mathematical theory of evidence. Proc. IJCAI-79 pp. 868-875. Barnett, J.A., and Erman, L. (1982). “Making Control Decisions in an Expert System Is a Problem-Solving Task,” USC/ISI Tech. Rep. Barr, A., and Feigenbaum, E.A., eds. (1981). “The Handbook of Artificial Intelligence,” Vol. I. Wm. Kaufmann, Los Altos, California. Barstow, D. (1979a). An experiment in knowledge-based automatic programming. Artif. Intell. U,73-1 19. Barstow, D. (1979b). Knowledge engineering in nuclear physics. Proc. IJCAI-79 pp. 34-36. Basden. A., and Kelly, B.A. (1982). An application of expert systems techniques in materials engineering. Proc. Colloq. Appl. Knowledge Bused (or Expert) Syst., 1982. Bennett, J., and Engelmore, R. (1979). SACON: A knowledge-based consultant for structural analysis. Proc. IJCAI-79 pp. 47-49.


21 1

Bennett, J.S. (1981). On the structure of the acquisition process for rule based systems. “Machine Intelligence” (D. Michie, ed.), lnfotech State of the Art Rep. Ser. 9, No. 3. Pergamon Infotech Ltd., Maidenhead, England. Bennett, J.S., and Hollander, C.R. (1981). DART: An expert system for computer fault diagnosis. Proc. IJCAI-81 pp. 843-845. Bobrow, D., and Winograd, T. (1977). An overview of KRL, a knowledge representation language. Cognit. Sci. 1(1),3-46. Bonnet, A. (1981). Applications de I’intelligence artificielle: Les systtmes experts. RAIRO I n f K o m p u t . Sci. l5,(4), 325-341. Brachman, R.J. (1977). What’s in a concept: Structural foundations for semantic networks. Int. J . Man-Mach. Stud. 9, 127-152. Brachman, R.J. (1978). “A Structural Paradigm for Representing Knowledge,” Tech. Rep. No. 3605. Bolt, Beranek & Newman, Cambridge, Massachusetts. Brachman, R.J., and Smith, B. (1980). Special issue on knowledge representation. SIGART 50. Brooks, R. (1981). A comparison among four packages for knowledge-based systems. Proc. Int. Conf. Cybernet. S o c . , 1981 pp. 279-283. Brown, J.S., and Burton, R. (1978). Diagnostic models of procedural bugs in basic mathematical skills. Cognit. Sci. 2(2), 155-192. Brown, J.S., Burton, R.R., and deKleer, J . (1982). Knowledge engineering and pedagogical techniques in SOPHIE I, I1 and 111. In “Intelligent Tutoring Systems” (D. Sleeman and J.S. Brown, eds.), pp. 227-282. Academic Press, New York. Buchanan, B.G. (1982). Research on expert systems. In “Machine Intelligence 10” (J.E. Hayes, D. Michie, and Y-H. Pao, eds.), pp. 269-299. Wiley, New York. Buchanan, B.G., and Feigenbaum, E.A. (1978). DENDRAL and Meta-DENDRAL: Their applications dimension. Art$. Intell. 11, 5-24. Bundy, A., Byrd, L., Luger, G., Mellish, C., and Palmer, M. (1979). Solving mechanics problems using meta-level inference. I n ”Expert Systems in the Micro-Electronic Age” (D. Michie, ed.), pp. 50-64. Edinburgh Univ. Press, Edinburgh. Carhart, R.E. (1979). CONGEN: An expert system aiding the structural chemist. In “Expert Systems in the Micro-Electronic Age” (D. Michie, ed.), pp. 65-82. Edinburgh Univ. Press, Edinburgh. Chandrasekaran, B., Mittal, S., and Smith, J.W. (1980). RADEX-Towards a computerbased radiology consultant. In “Pattern Recognition in Practice” (Gelsema and Kanal, eds.), pp. 463-474. North-Holland Pub]., Amsterdam. Clancey, W.J.(1979). Tutoring rules for guiding a case method dialogue. Int. J . Man-Mach. Stud. 11, 25-49. Clancey, W.J., and Letsinger, R. (1981). NEOMYCIN: Reconfiguring a rule-based expert system for application to teaching. Proc. IJCAI-81 pp. 829-836. Cohen, P., and Feigenbaum, E.A., eds. (1982). “The Handbook of Artificial Intelligence,” Vols. 11 and Ill. Wm. Kaufmann, Los Altos. California. Davis, R. (1979). Interactive transfer of expertise: Acquisition of new inference rules. Art$. Intell. 12, 121-157. Davis, R. (1982). Expert systems: Where are we? and where do we go from here? A1 M a g . 3(2), 3-22. Davis, R., and Buchanan, B.G. (1977). Meta-level knowledge: Overview and applications. Proc. IJCIA-77 pp. 920-928. Davis, R., and King, J . (1976). An overview of production systems. In “Machine Intelligence” (I.W. Elcock and D. Michie, eds.), pp. 300-332. Wiley, New York.

212


Davis, R., Buchanan, B.G., and Shortliffe, E.H. (1977). Production rules as a representation of a knowledge-based consultation program. Art$. Intell. 8, 15-45. Davis, R., Austin, H., Carlbom, I., Frawley, B., h c h n i k , P., Sneiderman, R., Gilbreth, J.A. (1981). The DIPMETER ADVISOR: Interpretation of geologic signs. Proc. IJCAI-81 pp. 846-849. Davis, R., et al. (1982). Diagnosis based on description of structure and function. Proc. 2nd. Natl. Conf. Art$. Intell., pp. 137-142. deKleer, J., and Sussman, G.L. (1980). Propagation of constraints applied to circuit synthesis. Circuit Theory Appl. 8, 1982. deKleer, J., Doyle, J . , Steele, G., and Sussman, G . (1977). AMORD: Explicit control of reasoning. SIGART News. 64, 116-125. Dincbas, M. (1980). A knowledge-based expert system for automatic analysis and synthesis in CAD. Proc. IFIP Congr. pp. 705-710. Duda, R.O., and Gaschnig, J.G. (1981). Knowledge-based expert systems come of age. Byte 6(9), 238-281. Duda, R.O., Hart, P.E., and Nilsson, N.J. (1976). Subjective Bayesian methods for rulebased inference systems. AFIPS Proc. 45, 1075-1082. Duda, R.O., et al. (1978). Semantic network representations in rule-based inference systems. I n “Pattern Directed Inference Systems” (D.A. Waterman and F. Hayes-Roth, eds.), pp. 203-221. Academic Press, New York. Duda, R.O., Gaschnig, J.G., and Hart, P. (1979). Model design in the Prospector consultant system for mineral exploration. In “Expert Systems in the Micro-Electronic Age” (D. Michie, ed.), pp. 153-167. Edinburgh Univ. Press, Edinburgh. Engelman, C., Berg, C.H., and Bischoff, M. (1979). KNOBS: An experimental knowledge based tactical air mission planning system. Proc. IJCAI-79 pp. 247-249. Engelmore, R.S., and Terry, A. (1979). Structure and function of the CRYSALIS system. Proc. IJCAI-79 pp. 250-256. Ennis, S.P. (1982). Expert systems: A user’s perspective of some current tools. Proc. 2nd Natl. Conf. Art$. Intell. p. 319-321. Erman, L.D., London, P.E., and Fickas, S.F. (1981). The design and an example use of HEARSAY-111. Proc. IJCAI-81 pp. 409-415. Fagan, L. (1980). VM: Representing time-dependent relations in a clinical setting. Ph.D. Dissertation, Computer Science Dept., Stanford University, Stanford, California. Fain, J., Hayes-Roth, F., Sowizral, H., and Waterman, D. (1981). “Programming Examples in ROSIE,” Tech. Rep. N-1646-ARPA. Rand Corporation. Feigenbaum, E.A. (1977). The art of artificial intelligence: Themes and case studies in knowledge engineering. Proc. IJCAI-77 pp. 1014-1029. Feigenbaum, E.A. (1979). Themes and case studies of knowledge engineering. In “Expert Systems in the Micro-Electronic Age” (D. Michie, ed.), pp. 3-25. Edinburgh Univ. Press, Edinburgh. Fine, T.L. (1973). “Theories of Probability: An Examination of Foundations.” Academic Press, New York. Forgy, C., and McDermott, J. (1977). OPS, A domain-independent production system language. Proc. IJCAI-77 pp. 933-939. Fox, M.S., Allen, B., and Strohm, G. (1982). Job-shop scheduling: An investigation in constraint-directed reasoning. Proc. 2nd Natl. Conf. Art$. Inrell. pp. 155-158. Friedland, P. (1979). Knowledge-based experiment design in molecular genetics. Ph.D. Dissertation, STAN-CS-79-771. Computer Science Dept., Stanford University, Stanford, California.


213

Gelernter, H.L., Sanders, A.F., Larsen, D.L., Agarival, K.K., Boivie, R.H., Spritzer, G.A., and Searleman, J.E. (1977). Empirical explorations of SYNCHEM. Science 197, 1041-1049. Genesereth, M.R. (1978). Automated consultation for complex computer systems. Ph.D. Dissertation, Harvard University, Cambridge, Massachusetts. Genesereth, M.R. (1981a). “The Architecture of a Multiple Representation System,” Memo HPP-81-6. Computer Science Dept., Stanford University, Stanford, California. Genesereth, M. (1981b). “The Use of Hierarchical Models in the Automated Diagnosis of Computer Systems,” Stanford Memo HPP-81-20. Stanford University, Stanford, California. Georgeff, M.P. (1982). Procedural control in production systems. Artif. Intefl. 18, 175-201. Goldstein, I.P., and Roberts, B. (1979). Using frames in scheduling. In “Artificial Intelligence: An MIT Perspective” (P. Winston and D. Brown, eds.), Vol. I , MIT Press, Cambridge, Massachusetts. Gorry, G.A., Silverman, H., and Pauker, S.G. (1978). Capturing clinical expertise: A computer program that considers clinical responses to digitalis. Am. J . Med. 64,452-460. Greiner, R., and Lenat, D. (1980). A representation language’s language. Proc. Ist Narl. Con$ Artif. Intell., 1980 pp. 165-169. Hayes-Roth, F., Waterman, D.A., and Lenat, D.B. (1978). Principles of pattern-directed inference systems. I n “Pattern Directed Inference Systems” (D.A. Waterman and F. Hayes-Roth, eds.), pp. 577-601. Academic Press, New York. Heiser, J.F., Brooks, R.E., and Ballard, J.P. (1978). Progress report: A computerized psychopharmacology advisor. Proc. Colleg. Inr. Neuro-Psychopharmacol., I 1 th, 1978. Hewitt, C. (1972). Description and theoretical analysis [using schemata] of PLANNER: A language for proving theorems and manipulating models in a robot. Ph.D. Thesis, Department of Mathematics, Massachusetts Institute of Technology, Cambridge. Hewitt, C., and Smith, B. (1975). Towards a programming apprentice, IEEE Trans. So& ware Eng. SE-1(1), 26-45. Hollander, C.R., and Reinstein, H.C. (1979). A knowledge-based application definition system. Proc. IJCAI-79 pp. 397-399. Horn, W., Buchstaller, W., and Trapp, R. (1981). Knowledge structure definition for an expert system in primary medical care. Proc. IJCAI-1981 pp. 850-852. Ishizuka, M., Fu, K.-S., and Yao, J.T.P. (1981). Inexact inference for rule-based damage assessment of existing structures. Proc. IJCAI-81 pp. 837-842. Kaihara, S., Koyama, T., Minamikawa, T., and Yasaka, T. (1978).A rule-based physicians’ consultation system for cardiovascular diseases. Proc. Int. Con$ Cybernet. Soc. 1978. pp. 85-88. Kandt, R.K., and Newlon, R. (1981). Self-improvingdiagnostics for automatic testing equipment. Proc. 8th Semi-annu. Semin.lExhibit. Krueger, M.W., Cullingford, R.E., and Bellavance, D.A. (1981). Control issues in a multiprocess computer-aided design system containing expert knowledge. Proc. Int. Conf. Cybernet. Soc. 1981 pp, 139-143. Kunz, J.C., Fallat, R.J., McClung, D.H., Osborn, J.J., Votteri, B.A., Nii, H.P., Aikins, J.S., Fagan, L.M., and Feigenbaum, E.A. (1978). ‘*A Physiological Rule-Based System for Interpreting Pulmonary function Test Results,” Tech. Rep. HPP-78-19, Cornputer Science Dept., Stanford University, Stanford, California. Larkin, J., McDermott, J., Simon, D.P., and Simon, H.A. (1980). Expert and novice performance in solving physics problems. Science 208, 20.

214


Lauriere, J.L. (1978). A language and a program for stating and solving combinatonal problems. Artv. Intell. 10, 29-127. Lindberg, D.A.B., Gaston, L.W., Kingsland, L.C., and Vanker, A.D. (1981). AIICOAG, a knowledge-based system for consultation about human hemostasis disorders: Progress report. Proc. 5th Annu. Symp. Comput. Appl. Med. Care pp. 253-257. Lindsay, R., Buchanan, B.G., Feigenbaum, E.A., and Lederberg, J. (1980). “Applications of Artificial Intelligence for Organic Chemistry: The DENDRAL Project.” McGrawHill, New York. Lusted, L.B. (1968). “Introduction to Medical Decision Making.’’ Thomas, Springfield, Illinois. McColl, D.C., Moms, P.H., Kibler, D.F., and Bechtel, R.J. (1979). “STAMMER2 Production System for Tactical Situation Assessment,” Tech. Rep. TD 298. Naval Ocean Systems Center, San Diego, California. McCune, B.P. (1977). The PSI program model builder synthesis of very high-level programs. SIGPLAN Not. 12, No. 8, 130-139. McDermott, J. (1980). RI: An expert in the computer systems domain. Proc. 1st Natl. Conf. Art$ Intell. pp. 269-271. McDermott, J. (1981). XSEL: A computer salesperson’s assistant. I n “Machine Intelligence 10” (J. Hayes, D. Michie, and Y.-H. Pao, eds.), pp. 325-337. Wiley, New York. Mamdani, E.H. (1982). Rule-based methods for designing industrial process controllers. Proc. Colloq. Appl. Knowledge Based (or Expert) Syst. Michie, D., ed. (1979). “Expert Systems in the Micro-Electronic Age.” Edinburgh Univ. Press, Edinburgh. Michie, D. (1980). Expert systems. Comput. J. 23(4), 369-376. Michie, D., ed. (1981). “Machine Intelligence,” Infotech State of the Art Rep. Ser. 9, No. 3. Pergamon Infotech Ltd., Maidenhead, England. Minsky, M. (1975). A framework for representing knowledge. In “The Psychology of Computer Vision” (P. Winston, ed.), McGraw-Hill, New York. Mitchell, T.M. (1979). Version spaces: An approach to concept learning. Ph.D. Dissertation, Tech. Rep. HPP-79-2. Computer Science Dept., Stanford University, Stanford, California. Moses, J. (1971). Symbolic integration: The stormy decade. Commun. ACM 8, 548-560. Newell, A. (1973). Production systems: Models of contol structures. I n “Visual Information Processing” (W. Chase, ed.), Academic Press, New York. Newell, A., and Simon, H.A. (1972). “Human Problem Solving.” Prentice-Hall, Englewood Cliffs, New Jersey. Newell, A., and Simon, H.A. (1976). Computer science as empirical inquiry: Symbols and search. The 1976 ACM Turing lecture. Commun. ACM 19, 113-126. Nii, H.P., and Aiello, N. (1979). AGE (attempt to generalize): A knowledge-based program for building knowledge-based programs. Proc. IJCAI-79 pp. 645-655. Nii, H.P., Feigenbaum, E.A., Anton, J.J., and Rockmore, A.J. (1982). Signal-to-symbol transformation: HASPEIAP case study. A1 Mag. 3(2), 23-35. Nilsson, N.J.(1980). “Principles of Artificial Intelligence.” Tioga Press, Palo Alto, California. Novak, G., and Araya, A.A. (1980). Research on expert problem solving in physics. Proc. 1st Con$ Art$. Intell. 1980 pp. 178-180. O‘Shea, T. (1979). Rule-based computer tutors. Proc. 1979 AISB Summer Sch. pp. 226-232. Patil, R.S., Szolovits, P., and Schwartz, W.B. (1981). Causal understanding of patient illness in medical diagnosis. Proc. IJCAI-81 pp. 893-899.


215

Pednault, E.P.D., Zucker, S.W., and Muresan, L.V. (1981). On the independence assumption underlying subjective Bayesian updating. Art$. Intell. 16, 213-222. Pinson, S. (1981). Representation des connaissances dans les systemes experts. RAIRA If./ Compur. Sci. 15(4), 343-367. Pople, H.E. (1977). The formation of composite hypotheses in diagnostic problem solvingan exercise in synthetic reasoning. Proc. IJCAI-77 pp. 1030- 1037. Reboh, R. (1981). “Knowledge Engineering Techniques and Tools in the Prospector Environment,” Tech. Note 243. Artificial Intelligence Center, SRI International, Menlo Park, California. Reggia, J.A. (1978). A production rule system for neurological localization. Proc. 2ndAnnu. Symp. Cornput. Appl. Med. Care pp. 254-260. Reinstein, H.C., and Aikins, J.S. (1981). Application design: Issues in expert system architecture. Proc. IJCAI-81 pp. 888-892. Rivlin, J.M., Hsu, M.B., and Marcal, P.V. (1980). “Knowledge Based Consultation for Finite Element Structural Analysis,” Tech. Rep. MARC Analysis Research Corp. Palo Alto, California. Roberts, R.B., and Goldstein, I.P. (1977). “The FRL Primer,” MIT A1 Lab Memo No. 408. Massachusetts Institute of Technology. Cambridge. Savage, L.J. (1971). Elicitation of personal probabilities and expectations. J . A m . Star. ASSOC. pp. 783-801. Scott, A.C., Clancey, W., Davis, R., and Shortliffe, E.H. (1977). Explanation capabilities of knowledge-based production systems. Am. J . Cornput. Ling. Microfiche 62. Shafer, G. (1976). “A Mathematical Theory of Evidence.” Princeton Univ. Press. Princeton, New Jersey. Shortliffe, E.H. (1976). “Computer Based Medical Consultations: MYCIN.” Am. Elsevier, New York. Shortliffe, E.H., and Buchanan, B.G. (1975). A model of inexact reasoning in medicine. Marh. Biosci. 23, 351-379. Shortliffe, E.H., Scott, A.C., Bischoff, M.B., Campbell, A.B.. van Melle, W . , and Jacobs, C.D. (1981). ONCOCIN: An expert system for oncology protocol management. Proc. IJCAI-81 pp. 876-881. Sridharan, N.S. (1980). “Representational Facilities of AIMDS: A Sampling,” Tech. Rep. No. CBM-TM-86. Dept. of Computer Science, Rutgers University, New Brunswick, New Jersey. Stallman, R.M., and Sussman, G.J. (1977). Forward reasoning and dependency-directed backtracking in a system for computer-aided circuit analysis. ArriJ Intell. 9, 135-196. Stefik, M. (1978). Infemng DNA structures from segmentation data. Arf$. Infell. 11, 85114. Stefik, M. (1979). An examination of a frame-structured representation system. Proc. IJCAI-79 pp. 845-852. Stefik, M. (1980). Planning with constraints. Ph.D. Dissertation, STAN-CS-80-784. Computer Science Dept., Stanford University, Stanford, California. Stefik, M. et al. (1982). The organization of expert systems, a tutorial. Arf$. Intell. 18, 135173. Sussman, G.A. (1978). SLICES: At the boundary between analysis and synthesis. I n “Artificial Intelligence and Pattern Recognition in Computer-Aided Design” (J.C. Latombe, ed.), North-Holland Publ., Amsterdam. Swartout, W. (1977). A digitalis therapy advisor with explanations. Proc. IJCAI-77 pp. 819823.

216


Swartout, W. (1981). Explaining and justifying expert consulting programs. Proc. IJCAI-81 pp. 815-822. Szolovits, P., and Pauker, S.G. (1978). Categorical and probabilistic reasoning in medical diagnosis. Artif. Intell. 11, 115-144. Szolovits, P., Hawkinson, L.B., and Martin, W.A. (1977). “An Overview of OWL, a Language for Knowledge Representation,” LCS TM 86. Massachusetts Institute of Technology, Cambridge. Tsotsos, J.K. (1981). On classifying time-varying events. IEEE Comput. Soc. Con$. Pattern Recognition Image Process., 1981 pp. 193-199. van Melle, W. (1980). A domain independent system that aids in constructing knowledge based consultation programs. Ph.D. Dissertation, STAN-CS-80-820. Computer Science Dept., Stanford University, Standford, California. Warren, D., er al. (1977). PROLOG: The language and its implementation compared with LISP. Proc. SICARTISIGPLAN Symp. Programm. Lung. 1977. Waterman, D.A. (1979). User-oriented systems for capturing expertise: A rule-based a p proach. In “Expert Sy5tems in the Micro-Electronic Age” C.D. Michie, ed.), pp. 26-34. Edinburgh Univ. Press, Edinburgh. Waterman, D.A., and Hayes-Roth, F., eds. (1978). “Pattern-Directed Inference Systems.” Academic Press, New York. Weiss, S., and Kulikowski, C. (1979). EXPERT: A system for developing consultation models. Proc. IJCAI-79 pp. 942-947. Weiss, S., Kukikowski, C., Amarel, S., and Safir, A. (1979). A model-based method for computer-aided medical decision making. Arrif. Intell. 11, 145-172. Weyhrauch, R.W. (1980). Prolegomena to a theory of mechanized formal reasoning. Arrif. Intell.

U,1-2.

Wipke, W.T., Braun, H., Smith, G., Choplin, F., and Sieber, W. (1977). SECS-Simulation and evaluation of chemical synthesis: Strategy and planning. In “Computer Assisted Organic Synthesis” (W.T. Wipke and W.J. House, eds.), pp. 97-127. Am. Chem. Soc., Washington, D.C. Yu, V.L.,Fagan, L., Wraith, S.M., Clancey, W.J., Scott, A.C., Hannigan, J., Blum, R., Buchanan, B.G., Cohen, S.N., Davis, R., Aikins, J.S., vanMelle. W., Shortliffe, E.H., and Axline, S. (1979). Antimicrobial selection for meningitis by a computerized consultant: A blinded evaluation by infectious disease experts. J . Am. Med. Assoc. 241(12), 1279- 1282. Zadeh, L.A. (1%5). Fuzzy sets. I d . Control pp. 338-353. Zadeh, L.A. (1978). Fuzzy sets as a basis for a theory of possibility. “Fuzzy Sets and Systems.’’ North-Holland Publ., Amsterdam.

Conceptual Representation of Medical Knowledge for Diagnosis by Computer: MDX and Related Systems B. CHANDRASEKARAN AND SANJAY MIlTAL* Department of Computer and Information Science The Ohio State University Columbus. Ohio

I . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I .I Aims of This Article . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Diagnosis by Computer: Why? . . . . . . . . . . . . . . . . . . . 1.3 The Issues to Be Addressed . . . . . . . . . . . . . . . . . . . . 1.4 The Nature of the Cognitive Activity of the Physician . . . . . . . . . 2. Overview of the Conceptual Structure Methodology . . . . . . . . . . . . 2.1 Diagnosis as Distributed, Classificatory Problem Solving . . . . . . . . 2.2 Relationship to the Organization of the Medical Community . . . . . . 2.3 Embedding of Problem Solving . . . . . . . . . . . . . . . . . . . 2.4 MDX Has a Compiled Knowledge Structure . . . . . . . . . . . . . 2.5 “Intelligent” Data Bases . . . . . . . . . . . . . . . . . . . . . . 3. Diagnostic Problem Solving . . . . . . . . . . . . . . . . . . . . . . . 3.1 The Domain of MDX: Cholestasis . . . . . . . . . . . . . . . . . . 3.2 Problem-Solving Strategy . . . . . . . . . . . . . . . . . . . . . . 4. Auxiliary Systems: Intelligent Access to Medical Data . . . . . . . . . . . 4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 PATREC: An Intelligent Data Base for Medical Data . . . . . . . . . 4.3 RADEX: Radiology Consultant . . . . . . . . . . . . . . . . . . . 4.4 Organization of Temporal Information . . . . . . . . . . . . . . . . 5 . Evaluation of Diagnostic Performance . . . . . . . . . . . . . . . . . . 5.1 Issues in Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Preliminary Evaluation of MDX . . . . . . . . . . . . . . . . . . . 6. Extensions to Diagnostic Problem Solving . . . . . . . . . . . . . . . . 6.1 Parallel Problem Solving . . . . . . . . . . . . . . . . . . . . . . 6.2 The Blackboard . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 The Role of the Blackboard in Multiple Diseases . . . . . . . . . . . 7. Comparative Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 8. Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Summary of the Methodology . . . . . . . . . . . . . . . . . . . .

218 218 219 219 220 221 221 224 225 226 227 230 230 234 240 240 241 249 259 266 266 269 271 271 271 272 272 274 275 275

* Present address: Xerox Palo Alto Research Center. 3333 Coyote Hill Road. Palo Alto. California 94304. 217 ADVANCES IN COMPUTERS. VOL. 22

.

Copyright 0 1983 by Academic Press Inc. All rights of reproduction in any form reserved . lSBN 0-12-012122-0

21 8

B. CHANDRASEKARAN AND SANJAY MITTAL

8.2 A Sampler of Further Research Issues. . . . . . . . . . . . . . . 8.3 What Is in Store? . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A. Performance of MDX on an Example Case . . . . . . . . Appendix B. Detailed Example of Query Evaluation in PATREC . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

.

.

276 278 278 289 292

1. Introduction 1.1 Aims of This Article

Within a decade since the beginning of the modern electronic computer age, many attempts to use the power of the computer in the difficult task of medical decision making began to be made. Early attempts combined elementary statistical and logical techniques (Lipkin and Hardy, 1958), but soon a large proportion of work in computer-aided medical decision making began to be devoted to the application of Bayesian or related statistical classification techniques (typical but by no means exhaustive references are Ledley and Lusted, 1959;Gorry, 1973; McNeil et af., 1975; Schwartz et af., 1973; Bjerregaard et af., 1976) or clustering algorithms (Kulikowski, 1970; Patrick et ai., 1977). A major line of work began in the mid- 1970s, when researchers in artificial intelligence (AI) began to consider problems of medical reasoning. The development in artificial intelligence of a subarea called “knowledge-based systems” has given a great deal of impetus to the design of computer programs which solve a variety of clinical problems: diagnosis, therapy selection, explanation of a medical state of affairs, etc. Kulikowski (1980) ably traced the development of computer aids for medical reasoning from its early roots through its Bayesian and pattern recognition eras to the current emphasis on the problem-solving techniques of AI. Szolovits and Pauker (1978) provided a comprehensive evaluation of many of the early A1 approaches to clinical reasoning. The purpose of this article is not to provide another survey of the field but, rather, to describe and discuss an approach to the design of medical decision-making systems based on the notion of conceptual structures for knowledge representation. A collection of related systems that have been under development in our laboratory exemplifies this approach, but the ideas are more general than the particular systems to be described. In Section 2, we provide an overview, from a theoretical viewpoint, of the conceptual structure methodology. In later sections we describe the functioning of the systems that we have been developing to give concreteness

CONCEPTUAL STRUCTURES FOR MEDICAL DIAGNOSIS

219

to the theoretical ideas. The central system in this group of systems is called MDX, which is a diagnostic system, i.e., it attempts to classify a given case as an element of a disease taxonomy. This system interacts with two other systems during its problem solving, PATREC and RADEX, the former a knowledge-based patient data base system that answers MDX’s queries about patient data, and the latter a radiological consultant which helps MDX in the interpretation of various kinds of imaging data. Both PATREC and RADEX are invoked by MDX as needed, but MDX is in control of the overall diagnostic process. Since the major aim here is the presentation of ideas that constitute a new approach, we forgo an extensive discussion of the body of earlier work on medical reasoning systems, except as necessary for comparative remarks. In any case, a number of other articles, in particular the excellent overviews mentioned earlier (Kulikowski, 1980; Szolovits and Pauker, 1978), are available for this purpose. The latter article also contains an insightful discussion of the inadequacies of medical reasoning approaches based on Bayesian approaches. 1.2 Diagnosis by Computer: Why?

Medical knowledge is rapidly growing and changing, and often, especially at the primary care level, only a subset of available medical knowledge is put to actual use. Computer-based consultation would make available the most advanced and complete clinical knowledge, especially in areas where experts are difficult to find. A related advantage of research on computer-based consultation is that the processes of reasoning and use of medical knowledge will be made explicit, and thus aid in the training of clinicians. Thus both medical care and medical education will be aided by successful research in this area. 1.3 The Issues to Be Addressed

Just what is a “knowledge-based artificial intelligence” approach to medical reasoning? Research in A1 over the years has concerned itself with the issues of understanding intelligence from a computational viewpoint, i.e., with the question of what sort of structures and processes in a digital computer would be able to produce behavior that would be deemed intelligent. The term knowledge bused emphasizes the growing perception among A1 researchers that successful problem solving in many areas such as medicine depends not only on having powerful methods of solving

220


problems, but also on possessing a great deal of knowledge in the domain in which the problem is being posed. In the case of medicine, this knowledge will take the form of medical facts, concepts, rules of inference, and rules of thumb. Artificial intelligence research seeks ways in which this sort of knowledge can be represented in the computer in a symbolic form and used during the problem-solving process. Thus any knowledge-based system for solving problems must address the following issues: How is the knowledge in the system organized and represented? What sort of problem-solving method is used by the program? What is the relationship between the problem-solving method and the knowledge; i.e., how are appropriate pieces of knowledge made available and used during problem solving? The approach that we will be discussing provides a methodology for addressing these issues. In its response to these issues the work to be described differs in important ways from most of the other AI systems that have been built in the medical area. A comparative discussion of these systems u i s - h i s the conceptual approach will be undertaken (Section 7) after a detailed exposition on how MDX and its satellite systems work.

1.4 The Nature of the Cognitive Activity of the Physician

Let us consider briefly the nature of the task or tasks that confront a physician as a problem solver in a case involving a patient. We use the term problem solver to delimit the concern to certain purely cognitive activities involving expertise in the medical domain. Vision and tactile manipulation play important roles both during diagnosis and therapy, but our systems do not attempt to include these abilities. Similarly, a doctor's personal warmth and involvement in a patient's well-being, for example, are generally conceded to be important ot his role as a physician-again, A1 systems generally do not attempt to reproduce these aspects of the physician. The cognitive, problem-solving tasks that are faced by a physician when a patient arrives with some presenting symptoms are many and varied. In the initial stage the task is one of diagnosis, i.e., classifying the case as an element of a disease taxonomy. (There may be rare instances in which a physician discovers a new disease, i.e., contributes to the disease taxonomy itself, but this is a much more creative task and is not part of the normal problem-solving activity of the physician.) The diagnostic process might involve decisions about ordering tests to confirm or reject


22 1

various hypotheses; physicians have to be selective for reasons of cost, the effects of tests, and time. Once a tentative or complete diagnosis is made, patient management is the next task. This would involve therapy selection and patient monitoring. The tentative diagnostic decision may be modified during this process. Therapy selection itself may often call for a certain kind of predictive problem solving in that it requires deducing the possible consequences of actions taken in a given state of the organism. Underlying all these is the task of coherently comprehending the patient’s situation, perhaps at a physiological or anatomical level. There is some question as to whether the latter task is essential for proper performance as diagnostician or therapist, but it appears that physician training emphasizes the need for that level of understanding for providing acceptable explanations if needed. Without making any commitments at this stage about the relationships between the mentioned list of tasks and the organization of A1 problemsolving systems, we can use this list as a means of characterizing the capabilities of medical consultation systems. For example, INTERNIST (Pople, 1977) is mainly a diagnostic system, whereas MYCIN (Shortliffe, 1976) performs both diagnostic and therapy recommendation tasks. The MDX system that will be discussed in this article is also a purely diagnostic system; however, PATREC, one of its satellite systems, performs reasoning corresponding to “intelligent” patient data retrieval (kte Sections 2.5 and 4.2), whereas RADEX, another auxiliary system, does some of the problem solving that a radiologist performs (see Section 4.3). We are currently in the process of designing extensions to MDX for test ordering and therapy selection, but these aspects will not be discussed here.

2. Overview of the Conceptual Structure Methodology 2.1 Diagnosis as Distributed, Classificatory Problem Solving

In this section we shall outline how MDX is organized, with special emphasis on the methodological principles underlying the work. A later section will discuss the workings of MDX in greater detail. It bears repetition here that diagnosis is viewed as a process of class$cation, much like a biologist classifies a plant whose identity is at first unknown to him by systematically seeking specific characteristics until the plant is identified with a node in the botanical taxonomy. In the

222


organization of MDX, each node in the diagnostic hierarchy’ can be viewed as a diagnostic concept, concentrating in it much knowledge that is specific to that concept. The nodes higher in the hierarchy are more general, while those lower are more specific. At the top of the hierarchical structure is the node “internist,” followed by general disease concepts such as “liver,” “heart,” etc. Examples of specific concepts lower in the hierarchy are “infectious hepatitis,” “cholestatis due to biliary stones,” etc. Corresponding to each concept is a package of “how-to” knowledge, which can often be conveniently represented in the form of a cluster of diagnostic rules. These rules typically relate various kinds of findings, i.e., signs, symptoms, lab data, historical information, etc., to the diagnostic concepts. The function of this packet of knowledge is generally to establish whether the disease or causal condition corresponding to this concept is present in the given case, and if so, to decide which of the subconcepts may be more likely,2 so control may be transferred to that subconcept for further consideration. The problem solving is top-down; for example, every case will first be examined by the topmost node, “internist.” “Internist” is always established, and it will use its packet of rules to decide which of the successors, ‘‘liver,” “heart,” etc., is most likely to be relevant. Control will be transferred to that concept, say, in a particular case, the liver concept. Now a similar packet of diagnostic knowledge in the liver concept will be used to decide if the liver concept can be established with some degree of confidence or rejected. In making this determination, the liver concept will use rules of the form, “IF (finding) is present, addhubtract (units) of evidence fodagainst (disease concept). ” (In the actual implementation, the establishlreject package uses a more complex evidence combination mechanism much like the signature tables idea used in early checker-playing programs. This is discussed in some detail in Section 3.2.2.) When all the I Here an assumption is made that the disease taxonomy is always a hierarchy. It has often been suggested that a hierarchical structure cannot always be imposed on the disease space, especially when complex interactions between diseases may be taking place. Our view is that the disease space can be decomposed as a collection of interacting heirarchies, such that the MDX methodology we are describing can be applied. In our opinion, the apparent lack of hierachy is often a consequence of an insufficiently deep analysis of the domain. Of couse, this is only a working hypothesis, and no claim is being made that we have analyzed all of clinical medicine to establish the hypothesis. On the other hand, we have successfully analyzed several complex subdomains of medicine, and designed satsifactory interacting hierarchies for them. * Gomez and Chandrasekaran (1981) have extended the MDX approach so that a specialist no longer has the explicit knowledge to decide which of the subconcepts is relevant. In this extension, the subconcepts are invoked in parallel, and the appropriate ones establish themselves. This extension is described in detail in Section 6.


223

evidence for establishing and rejecting a concept is considered together, the net degree of confidence can be translated into a symbolic measure of confirmation or rejection in a discrete scale ranging from “definitely” to “definitely not.” If a concept rejects itself, then control reverts back to its superior, which will then try other successors (siblings of the rejected concept). If a concept establishes itself, then a similar process of choosing a likely successor and transferring control will be repeated. The entire process goes on until one or more of the tip node concepts are established. Every node in the tree should be accounted for by either being established, rejected, or having an ancestor rejected. Notice that the rules that were actually used in the establishheject process, i.e., those rules whose findings were in fact present in a particular case, can be directly used to justify the establishheject decision. Thus when the entire process of diagnosis is over, the system may print out some thing like, “Liver disease established due to (finding 1). (finding 2), ...;Cholestasis established due to ( ), ( ); Hepatitis rejected due to ( ), ( ); ‘Stones in the bile duct causing cholestasis’ established due to ( ), ( ).” In addition to the role played in justifying the decision, the list of abnormal findings that can be made up from this as having been “explained” can be checked against patient data to see if the case has been completely solved, i.e., all abnormal findings have been explained. Since each concept in this problem-solving process is a package of active, how-to knowledge, the concepts can be termed “specialists” in that diagnostic notion. Also, the concepts directly play the role of “hypotheses.” In the preceding account several issues have been simplified. In particular, the problem of multiple diseases needs more attention. The problem solving of MDX can be extended to account for this complication. Gomez and Chandrasekaran (1981), referenced earlier, provided a framework in which interacting multiple diseases, in particular diseases secondary to another disease, can be handled (see Section 6). Criteria for Distribution of Knowledge. It can be seen that in MDX the totality of diagnostic knowledge is distributed among various specialists. This distribution modularizes the knowledge base in a natural way. Perhaps a discussion of the criteria for distribution, i.e., deciding which piece of diagnostic knowledge should go in what specialist, may be useful at this stage. Let us consider two concepts, one somewhat high in the hierarchy, say, “liver,” and another which is a tip node in the hierarchy, say, “cholestasis due to biliary stone.” Clearly evidence for the latter is also evidence for the former, since the latter is a special case of the former. However, at

224


the level of “liver,” only those bits of knowledge satisfying the following criteria need to be placed:

(1) Evidence that is common to many liver diseases, rather than to particular ones. For example, “SGOT3 in blood is abnormally high” is general evidence in favor of improper liver function and would thus cover a number of liver diseases. On the other hand, “obstruction in the biliary duct in x-ray,” while definitely indicative of liver disease, is too specific to one liver disease to be placed at the “liver” level. (2) Evidence that is specific to very common liver diseases or to diseases that require immediate attention. For example, hepatitis is a common liver disease, and a findingthat would support it may be placed in the “liver” node. The rationale is that in a large number of cases liver disease can be established with relatively little work. Similarly, rules about very specific diseases which are highly dangerous and thus require immediate attention should go higher in the hierarchy, since waiting until the appropriate lower level concept is accessed may waste valuable time. (3) Evidence that can be quickly, easily, or inexpensively obtained. This evidence should be placed in the higher nodes, while knowledge about evidence that requires invasive or expensive procedures should be relegated to the more specialized concepts. 2.2 Relationship to the Organization of the Medical Community

The approach that we are proposing is very similar to how the medical community itself is organized for diagnostic problem solving and how knowledge is distributed among the specialists in the community. For example, the transfer of control of a case from an internist to a cardiologist is akin to that of the “internist” node in our approach transferring to the “heart” node. In both cases, the intent is to access a body of more detailed knowledge for the purpose of diagnostic problem solving. Principles similar to those mentioned previously for MDX also control the abstraction and distribution of knowledge. This similarity between MDXs organization and that of the medical community is not accidental, In our view the medical community as a whole evolves and adapts itself to create a distributed, cooperative problem-solving regime in response to the same set of computational problems that are intrinsic to the task. These are the following.

(1) When the total amount of knowledge becomes large, it needs to be appropriately modularized into chunks of manageable complexity. The Serum glutamic-oxdocacetic transaminase.


225

concepts of MDX are such chunks, and the specialties in the medical community play a similar role; the only difference is in the “grain size.’’ (2) In order to guarantee a focused access to the different chunks of knowledge, well-defined problem-solving regimes that match the task need to be established. The regime that is appropriate to the diagnostic task is that of establishlrefine, invoked by the specialists in the diagnostic hierarchy in a top-down fashion. Again, the medical community’s control transfer processes display this behavior. (3) The hierarchical, modular nature of the community of specialists makes it relatively easy to implement changes in diagnostic knowledge, since such changes can be confined to a few appropriate specialists, with the rest of the specialist structure unaffected. The computational task of learning and change mandates such a structure. Chandrasekaran (1981) has given an analysis of how certain kinds of distributed problem solving tasks arise in relation to the demands of the task environment. 2.3 Embedding of Problem Solving

The currently dominant paradigm in the design of knowledge base systems is based on the separation of knowledge and its use; that is, normally a “knowledge base” of facts, relations (networks), frames, production rules, or a combination thereof is assumed, independent of a problem solver. This central knowledge base is then accessed by different problem solvers, which themselves are devoid of the domain knowledge that is resident in the knowledge base. The situation is diagrammatically shown in Fig. 1. The idea is that the same knowledge base can be used by different problem-solving algorithms. In MYCIN, for example, the knowledge base is a collection of production rules, while the problem solver is a backward-chaining algorithm. The important thing to emphasize is that the problem solver is itselfa sort of “pure” algorithm, without any domain knowledge.

Problem Solver

Knowledge Ease 1

I

FIG. I . Dominant paradigm in knowledge-based problem solving.

226

B. CHANDRASEKARAN AND SANJAY MllTAL

On the other hand, in the MDX approach that has been outlined here there is no separation between the knowledge base and the problem solver. The problem-solving technique of establishhefine (which, incidentally, is appropriate only for the classification task) can be said to be embedded in the specialists; that is, each specialist’s knowledge is directly in the form of the kinds of things to do to establish or reject a hypothesis or transfer control to a subspecialist. There is no knowledge in the specialists that is not meant for this purpose, nor is there a notion of knowledge in a general form. The classical paradigm illustrated in the Fig. 1 may appear at first to have significant advantages. “Knowledge” can be added without concern as to how it is to be used, and several different problem solvers can use the same knowledge base without redundant representation of knowledge, etc. However, how to use knowledge is itself a part of knowledge, and this “control” knowledge, as it is often called in the literature, is itself highly domain and task dependent. In order to effectively make use of the knowledge in the knowledge base, the control knowledge has to be introduced either at the problem solver level, increasingly making it another domain-dependent knowledge base, or at the knowledge base level. In the latter case, since the control knowledge is task dependent (i.e., different depending upon whether one wants to do diagnosis, predictive problem solving, or some other task), the main knowledge base will become increasingly fragmented into separate knowledge bases for different tasks. In the MDX approach, on the other hand, all the knowledge is geared only to the diagnostic task. The control knowledge necessary for focused performance, i.e., a problem-solving behavior that is well defined and consistent over time with respect to the pursuit of goals, is actually implicit in the conceptual structure organization and the embedded problem solving. We think that there is a very basic trade-off between keeping knowledge in a general form for multiple purposes with the attendant problems of lack of focus and control in problem solving, and organizing knowledge for efficient use for well-defined problem-solving tasks. The idea that separate knowledge structures should be created for different types of problem solving and that the problem solving should be embedded in the specialists in the structure is discussed in further detail in Chandrasekaran (1983). 2.4 MDX Has a Compiled Knowledge Structure

The MDX knowledge structure can be viewed as a highly compiled problem-solving structure; that is, each of the diagnostic rules as well as the conceptual structure is derived from more basic knowledge structures


227

such as anatomical, physiological, and biochemical ones. The bits of knowledge from these other structures that are relevant for the diagnostic task in that domain are compiled directly to perform the diagnostic role. An example might make this clear. Consider the diagnostic rule “high SGOT in blood +- liver disease.” This rule may be generated from a knowledge of the biochemistry of SGOT and its relation to liver function. In principle, a diagnostic problem solver without this rule in the liver concept, but with the underlying biochemical and other knowledge structures and a knowledge of how to use that knowledge could hypothesize liver disease when faced with a case involving high SGOT in blood; but once such a problem-solving task with the deeper structures is carried out, there is no need to repeat it. In principle, given a body of underlying medical knowledge, all of it that plays a role in diagnosis can be compiled, and there is no need to explicitly represent and manipulate these deeper structures for the purpose of producing expert diagnostic performance. The role of commonsense knowledge is similar. If a doctor knows that a patient who has liver trouble is a farmer or works in a chemical plant, the general world knowledge that the doctor has about farmers or chemical factories may help him to look for evidence of ingestion of contaminated water or exposure to certain chemicals. The question is whether it is necessary in an automated diagnosis system to represent general world knowledge of this type. Again the relationship between occupations and exposures can be compiled in the appropriate concepts without a need to access general world knowledge structures. Of course physicians-even expert ones-often resort during diagnosis to these underlying knowledge structures, especially when confronted with difficult problems; but this is because in a given individual the diagnostic structures, being continually in a state of construction by the learning processes, are never complete. Experts are experts because they have more of the diagnostic structure complete and ready to go. In an automated system it is worth striving to obtain as complete a structure as possible right at the design stage, especially because of the poor current state of understanding in A1 about how to represent and manipulate the deeper knowledge structures. This issue has recently been debated in the A1 community in the context of the differences between so-called “deep” and “surface” systems (Hart, 1982; Michie, 1982; Chandrasekaran and Mittal, 1982). 2.5 “Intelligent” Data Bases

In the preceding description of how MDX works, knowledge in each specialist was of a form that related findings to diagnostic hypotheses. We termed knowledge of this kind diagnostic knowledge. Now consider a piece

228

8. CHANDRASEKARANAND SANJAY MlUAL

of diagnostic knowledge: “If exposure to anesthetics, consider hepatitis.” Here the “if” part of the rule is a finding and the “then” part is a diagnostic hypothesis. Let us assume that a patient’s chart had no record of exposure to anesthetics, but one of major surgery. We would expect that the preceding rule should be considered instantiated (or “fired” in the parlance of AI) in view of our medical knowledge about the relation between major surgery and anesthetics. But this reasoning is not diagnostic, i.e., it does not relate a finding to a diagnostic hypothesis but, rather, a finding to another finding. Clearly reasoning of this type is often needed during diagnosis, but to make the diagnostic structure responsible for this reasoning would result in the number of diagnostic rules needed proliferating in an unbounded manner. To handle this particular example, one would have to have another diagnostic rule: “If major surgery, consider hepatitis.” Proliferation of diagnostic rules in this manner is neither desirable nor always possible. The essence of the situation is that the reasoning is of a different kind than diagnosis, and thus calls for a different structure geared to that particular problem-solving task. One can imagine an expert diagnostician turning, in the course of diagnostic reasoning, to the nurse in charge of the patient’s records and asking if there was evidence of anesthetic exposure, and the nurse answering affirmatively without being trained in diagnosis at all. The reasoning needed for information retrieval of this kind is mainly one of having access to the appropriate concepts and executing the inference rules associated with them. For instance, in the case involving anesthetics, the concept “anesthetics” will have a rule of the form, “If no direct evidence of exposure to anesthetics, check if major surgery performed.” The data retrieval problem solver would access the anesthetics concept, be directed to look in the data base for direct evidence, would fail, would then be directed by the preceding rule to access the surgery concept to see if it was performed, succeed in this, and finally conclude that anesthetics exposure was likely. Thus the technical issue in such knowledge-based reasoning for data retrieval activity becomes one of discovering the appropriate organization of “data concepts” and the appropriate way to embed inference rules in them. The PATREC data base system that will be described later specializes in reasoning of this type. PATREC is a collection of data concept specialists engaged in data retrieval reasoning. Again, as in the diagnostic task and MDX,the problem solving task is not separate from the knowledge base but, instead, embedded in it. RADEX, the other subsystem, is also used by MDX for consultation during its reasoning. The function of RADEX is to provide help in relating imaging information (such as from x-rays, ultrasonograms, etc .) to diag-


229

nostic conclusions. For instance, a finding in a case may be, “In the xray, a conical, narrow obstruction was seen in the biliary duct.” To relate this to the cause of the obstruction, a subspecialist in MDX, say, “stone as cause of cholestasis,” will engage in a consultation with RADEX to decide if this finding can be taken as evidence of stones. In the form of its reasoning, RADEX is similar to PATREC, but in terms of the nature of its concepts, RADEX differs in its concentration on data which are highly perceptual in nature. RADEX also performs some interesting types of reasoning based on anatomical models. The overall organization of MDX and its subsystems is shown in Fig. 2.

I

Cholestasis

JyiExtra0 0

,yc\

Conceptual Model of Organs, Deformities, and Morphological Tests

intra-

0

Diagnosis System

0 0

Patient Model of Radiological Information

Radiology Consultant (RADEX)

FIG.2. Overview of MDX organization.

230

B. CHANDRASEKARANAND SANJAY MITTAL

In Section 3 we will turn to a detailed description of how MDX is actually implemented in the domain of a liver syndrome called cholestasis.

3. Diagnostic Problem Solvlng 3.1 The Domain of MDX: Cholestasis 3.7.7

What Is Cholestasis?

Before we discuss the details of the diagnostic system in MDX, it would be instructive to describe the medical domain of MDX and show how it is analyzed to form the diagnostic knowledge structure. Currently, MDX is capable of diagnosing the many different causes of cholestasis, a major liver disease. Cholestasis is a syndrome caused by the disruption of the bile flow from the liver to the duodenum (see Fig. 3 for a sketch of the relevant anatomy of the human body). Physiologically, this disruption can be characterized in two ways: secretionary and excretionary. In the former, the disease is caused by disruption of bile secretion in the liver; in the latter, the disease is caused if excretion of bile through the biliary system is prevented.

- Porta Hepatis \/x?? -\

-1'

Cystic Duct

Pancreas \

-

Hepatic Commo'n Blle Dub1

-.

u

FIG.3. Schematic anatomy of the liver and the biliary system.


231

Cholestasis can also be characterized anatomically as either extrahepatic (literally, outside the liver) or intrahepatic (literally, inside the liver). In other words, depending on whether the cause of bile disruption is inside or outside the liver, the cholestasis may be described as extra- or intrahepatic. Notice that the two characterizations are not isomorphic; in particular, not all intrahepatic conditions are secretionary in nature, though extrahepatic conditions are always excretionary. There are other alternate classifications of cholestasis. For example, a temporal characterization would be in chronic terms, acute terms, or neither of these. This description is almost orthogonal to the anatomical and physiological characterizations: excretionary-secretionary or intrahepatic-extrahepatic cholestasis can be chronic or acute in nature, though some types more so than others. 3.1.2 Conceptual Structure of Cholestasis

Whether a patient has cholestasis or not can be determined in a fairly straightforward manner and largely independently of other problems the patient might have. Concepts with this property can be viewed as anchoring concepts because they allow the problem-solving process to be anchored at those nodes (Fig. 4). Cholestasis can be divided into two main types: extrahepatic and intrahepatic. This division is made not because of its anatomical nature, but because there are three important factors which are crucial from a diagnostic viewpoint. First, there are diagnostic tests which can help make this distinction. The implications of this are that in many cases it would be possible to focus attention on one or the other type, thus reducing the problem size. Second, there is a good therapeutic reason for making this distinction: The treatments for the two types of cholestasis are very different. The former usually requires surgical procedures and the latter can often be alleviated with medicinal treatment. Thus it would be essential to make this distinction and to do so quickly. Finally, there is a diagnostic Internist

FIG.4a. Diagnostic hierarchy: top levels.

232

B. CHANDRASEKARAN AND SANJAY MITTAL Cholestasis LExtrahepatic LInflammation

physical Llntrahepatic

-Mechanical Causes LNonmechanical Causes

FIG.4b. Diagnostic hierarchy: cholestasis.

reason for this classification, namely, that the decision regarding intrahepatic cholestatic diseases is often less certain than the extrahepatic ones, but becomes easier once the extrahepatic possibilities have been ruled out. It is not surprising that this is indeed the classification used by many diagnosticians (Wintrobe et al., 1972; Sherlock, 1975). Let us now refine the concept of extrahepatic cholestasis. The movement of bile through the biliary tree can be obstructed by a physical entity such as a stone, stricture, or cancers of various kinds or by narrowing of the ducts caused by inflammation due to various diseases. Similarly, intrahepatic diseases may be due to mechanical obstruction (or obliteration) of ducts within the liver or to nonmechanical causes which usually result in a dysfunction of the secretion of bile within the liver. The conceptual structure of cholestasis is shown, as a hierarchy, in Fig. 4(b-d). Associated with each node in this hierarchy is a procedure (the terms expert or specialist are also used for these procedures for reasons discussed in Section 2. I). These procedures contain knowledge about establishing or rejecting the concept which they embody. They also contain knowledge for calling other specialists, refining their concept into applicable subconcepts, building an explanation of their decisions, deciding if the problem is solved, etc. This analysis has indicated the need for different kinds of specialists. Let us briefly summarize them. Extrahepatic Llnflammatlon LCholangitis LSclerosing cholangitis LPancreatitls LAlcoholic pancreatitls LPhysical LStone LStricture LGall bladder Cancer LBile duct Cancer L Ampullary Cancer LAmpulla of vater cancer LHead of pancreas cancer LBodyltail of pancreas cancer

FIG.4c. Diagnostic hierarchy: extrahepatic cholestasis.


233

lntrahepatic LNonmechanical -Early Primary Biliary Cirrhosis -Chronic hepatitis LPregnancy related

recurrent -Benign

rotor Syndrome LDubin-Johnson

Syndrome

LPostoperative

LAnesthetics L Acute LAbscess

LCongestion LHemolysis LSeptic

LHodgkin’s

disease (nonhepatic) cancer Primary LSecondary LAlcohol related LVirus LDrug-toxicity L Mechanical causes LHodgkin’s disease (hepatic) LSecondary liver cancer Llntraductal carcinoma LSclerosing cholangitis L Biliary Atresia L Late Primary Biliary Cirrhosis LLiver

FIG.4d. 3.1.3

Diagnostic hierarchy: intrahepatic cholestasis.

Types of Specialists

(a) Anchoring Specialists. Anchor specialists contain two kinds of knowledge: One kind helps them in establishing (or rejecting) the concept they embody; the second kind is used in the refinement of their concept. More specifically, this latter kind of knowledge enables specialists to call their subspecialists, mediate between competing advice, decide when the problem is solved, etc. CHOLESTASIS is an example of such a specialist. (b) Control Specialists. These are the nontip specialists in the hierarchy and contain only the knowledge required to control the refinement process. Some examples of such specialists are EXTRAHEPATIC, INTRAHEPATIC and INTRAMECHANICAL. (c) Primary Specialists. These are the specialists corresponding to the tip concepts in the hierarchy. Their knowledge is limited to making decisions as to whether the concepts they embody can be established or not. Such decisions are often made by combining different pieces of evidence and rarely require any complex search through a space of possibili-

234


ties. Ideally, the solution of a diagnostic problem should be in terms of these primary concepts. Concepts such as STONE, BILE DUCT CANCER, or PRIMARY BILIARY CIRRHOSIS are examples of primary specialists. (d) Auxiliary Specialists. These specialists do not represent any concept in the hierarchy, or in other words, they do not comprise the solution refinement of the problem. As a result, these specialists do not exert a direct control over problem solving; rather, they act as resources (or consultants), which are called upon to render specific advice to other specialists in the hierarchy. The radiology specialist (discussed in some detail in Section 4.3) is a good example of such a specialist. Some specialists, such as CHOLANGITIS, act as auxiliary as well as primary specialists, depending on the context. In the former case, CHOLANGITIS merely decides if the patient has cholangitic disease. For this purpose, a decision can be made from simple signs, symptoms, and lab data, though cultures of bile or blood would be more definitive. As a primary specialist in the conceptual structure of cholestasis, CHOLANGITIS has to additionally decide if cholangitis is indeed causing cholestasis. This decision involves a more detailed processing. Strictly speaking, one should view these multifaceted specialists as many different experts. However, as these facets often have much knowledge in common, it is convenient to view them as a single expert which can be asked to make different decisions. (Also, see Section 7 for more on this issue.) 3.2 Problern-Solving Strategy 3.2.1

Overview: Establish, Refine, and Explain

The basic problem-solving strategy is a three-part process, applied recursively, as determined by each specialist. The first stage is the establishment of one or more anchor concepts. Once these are established, such anchor specialists try to refine their concept by considering their subconcepts. As the “ideal” refinement is in terms of primary concepts, which may be many levels removed from such anchor concepts, the intervening control specialists may be given selective control to refine their part of the hierarchy. Usually, there may be knowledge available which enables one or more such subconcepts to be rejected or “suspended.” In such cases, there may be no need to consider any possibility under those control specialists. The final stage in the processing of each specialist is an attempt to build an “explanation” of why a certain decision was made,


235

what data was explained, how conflicting decisions were resolved, and otherwise determine if the task they were asked to solve was indeed solved or not. This issue of generating an explanation (or determining if the problem was solved) is, in general, a very hard problem and we will discuss it in more detail in Section 6. Before we discuss the details of the problem-solving process, it is important to emphasize one aspect of this process, namely, that there is no uniform mechanism which operates on some description of the knowledge within each specialist but, rather, each specialist has different kinds of knowledge and different mechanisms for carrying out these three major steps, This point is important because of the danger in forcing diverse knowledge and strategies into ad hoc formalisms. The richness in expert problem solving and the consequent performance derives from this diversity and the applicability of different mechanisms under different circumstances. In our approach, this sort of localization is made possible by the embedding of problem solving in the different specialists. 3.2.2 Establishing a Concept

Establishing a concept essentially involves a decision as to whether such concepts “fit” the data and thereby refine their controlling concept. For example, refemng to Fig. 4(c), establishing the concept of “stone” is a decision about “stones as a cause of extrahepatic cholestasis based on available evidence” and not just as to whether there are stones in the biliary tree. This distinction is fundamental to understanding the implicit context created by the conceptual structure. (a) Two Types of Rules. Let us examine the kinds of knowledge which help in making such establishments possible. There are two kinds of rules which are used in making such decisions: ones which help establish the concept and ones which reject that concept. For example, the concept of stone can be established by the following single rule: If any cholangiogram shows a stone blocking the biliary tree, then establish stone as a cause of extrahepatic cholestasis. Similarly, the concept of gall bladder cancer can be rejected by the rule If the gall bladder was removed by some prior surgery, then reject gall bladder cancer. (b) Grouping the Rules. Rules which can by themselves (or in simple combinations with other data) establish or reject a concept are called pathognomonic rules (see Fig. 5, for example). The process of establishing a concept would be a rather straightforward job if all knowledge were

236


I. PATHOGNOMONIC RULE GROUP IF Cholangiogram available THEN IF Stones seen in Biliary-tree THEN XEV + 3 ELSE XEV + - 1 ELSE I F Plain film xray available THEN IF Stones seen in Biliary-tree THEN XEV + 3 ELSE XEV + 0 ELSE XEV + 0 IF XEV > 2 RETURN XEV as value of Stone Establishment 11. RULE GROUP FOR HISTORY EVIDENCE

This decision is made by asking about stone-related diseases: hernolyticdisease, cholecystitis, gall-stones etc. 111. RULE GROUP FOR CLINICAL EVIDENCE

The rules in this group combine evidence for cholangitis, colicky-pain, vornitting and nausea in a weighted-sum logic. IV. RULE GROUP FOR SUMMARY EVIDENCE The evidence from I, 11, and 111 is combined to obtain the consistency evaluation for Stone.

FIG.5 . Partial specialist for stone causing cholestasis.

pathognomonic in nature. Unfortunately, most concepts have very few, if any, pathognomonic rules and even these are applicable in only a few cases (in the statistical sense). Most rules represent knowledge which by itself is insufficient to make a decision one way or the other. An important issue is how to combine these individual pieces of evidence to make a more definite decision. Our method of combining these uncertain rules is similar to the “signature table” idea proposed by Samuel (1967) and is described in more detail by Chandrasekaran et al. (1982). Here we will illustrate the main aspects of our scheme using the stone specialist as the example. The evidence for or against diagnosis of biliary stones is some cornbination of the following contributory pieces of evidence: clinical evidence, evidence from imaging data, and evidence from historical data. The clinical evidence is itself put together from further constituents: cholangitis, colicky pain, vomiting, and nausea. Information about these is directly available from the patient data base. In the case of clinical evidence for stones, these data have ternary values: present, absent, and unknown; however, in general, the data base may need to make a conversion from numerical values to a small set of discrete values. For example, the datum “bilirubin” may be represented as “highly elevated,” “moderately elevated,” “normal,” etc., while the raw value for it may be stored as so many units per cubic centimeter. Let us assume for the purposes of cur-

237


TABLEI. COMBINATIONS FOR CLINICAL EVIDENCE ~~

Cholangitis

Colicky pain

Vomiting

T

T

T

...

Nausea

T

. .

T

T

...

...

Clinical evidence for stone

F

...

+3

...

. . F

i-2

...

...

rent discussion that the system has simple thresholds to make these conversions. Each stage in this abstraction process, that is, in going from data in the data base to deciding on clinical and historical evidence, etc., and then proceeding to an estimate of summary evidence, can be represented by means of a table (or a rule group). Table I gives a fragment of such a table for obtaining “clinical evidence for stone” from knowledge of its constituent pieces of evidence, and Table I1 gives a similar table for “summary evidence for stone” from clinical and historical evidence. In Table I the numbers in the clinical evidence column are not constructed by any formula, global or local; instead they are filled in by debriefing human experts, i.e., by presenting the expert with the following: “Given reasonable evidence for cholestasis, in particular extrahepatic obstruction, and given that the patient has cholangitis and colicky pain, but no vomiting or nausea, how much weight, in a scale of + 3 to -3, would you give for clinical evidence for biliary stone?” Once the values for the clinical and historical evidence are synthesized, the process is repeated at the next level, as shown in Table 11.

3.2.3 Refining a Concept Refining a concept involves calling on its subconcepts in order to decide which ones are applicable. The conceptual hierarchy provides a systematic way to call on the subconcepts in a top-down fashion. However, TABLE11. COMBINATIONS FOR S U M M A R Y EVIDENCE Clinical evidence

Historical evidence

Summary evidence for stone

+3

T

+3

...

...

...

+I

T

..

+2

...

238


when the subtree is large (as happens at nodes higher up in the diagnostic hierarchy), it is often useful to have further knowledge which can help in ranking the order of priority in which these subconcepts are called. In Section 7, we describe a parallel problem-solving regime in which the subconcepts select themselves and a parent concept does not have to rank them. In the current version of MDX,the refinement process is a combination of a systematic search mechanism and heuristic rules which allow the problem to be solved quickly in many cases. We will use the cholestasis specialist as an example in the rest of this section. (a) Systematic Search. Systematic search is basically an attempt to select the most promising candidates for further refinement from among all the nontip subconcepts. This is done using local selection rules which can help select or reject these subconcepts (or by calling them and using the results of their evaluation). The rationale for such a selection is obvious. If most of these subconcepts can be quickly eliminated, then the primary concepts lying in that part of the hierarchy may never need to be considered. This will substantially reduce the search. For example, rules of the following kind, located at “cholestasis,” help in selecting between its two subconcepts, “extra-hep” and “intra-hep”:

IF Intrahepatic ducts are dilated (as seen on the Ultrasonogram or CT-scan) AND Extrahepatic ducts are normal (as seen on the ERCP or Percutaneous Cholangiograms) THEN Select Intra-hep IF Extrahepatic ducts are abnormal (as seen on the ERCP or Percutaneous Cholangiograms) THEN Select Extra-hep The selected subconcepts are then prioritized, given control according to this priority, and asked to refine themselves. Additional knowledge, based on other kinds of criteria, may also be used in this prioritization. For example, in the case of cholestasis, it is sometimes possible to select one of its subconcepts: extra- or intrahepatic. But when the information does not permit such a selection, additional criteria such as “extrahepatic diseases must be considered before intrahepatic ones because the former are easier to establish,” enables “extrahepatic” to be given a higher priority. In the case of controlling concepts whose subconcepts are all primary, the refinement process simply becomes one of calling each of them and asking them to establish their concept.


239

(b) Suggestion Rules. The systematic strategy outlined thus far is largely a goal-directed scheme. Suggestion rules complement it with a data-directed component. These rules are located at all levels (i.e., each controlling specialist has some rules making diagnostic suggestions). When the rule is “fired” (i.e., its conditions are satisifed), it suggests some possibilities which are then considered by the specialist. It is important to emphasize that the suggestions in themselves do not add weight to or establish the suggested concept; rather, they indicate that the specialist must be called and asked to establish itself. Of course, it sometimes happens that the knowledge in the rule may also play a role in establishing the suggested concept; if the rule happens to be a pathognomonic one, then the concept may be quickly established. These suggestion rules largely provide a means of “recognizing” commonly occurring diseases and handling them quickly. If the suggestions turn out to be adequate in solving the problem, then the refinement process terminates and the systematic search can be aborted. However, and this is important, even if the suggestion rules do not solve the problem, the systematic procedure will. Thus the major importance of the suggestion rules lies in reducing (sometimesdramatically)the number of concepts needed to be considered in solving the problem. Furthermore, as the number of rules at each concept is not very large, and many groups of rules are never even tried, solving a typical case may require only a small fraction of such suggestion rules to be tried. Here again, in contrast with global rule organizations, the distributed organization of knowledge cuts down on the search for finding the relevant knowledge to apply in solving a problem. Some of the suggestion rules located at “cholestasis” are the following:

IF History of Ulcerative colitis SUGGEST Scl-cholangitis and BD-cancer IF Colicky pain in the abdomen SUGGEST Stone

IF History of Biliary-surgery SUGGEST Stricture IF Patient is Female AND Alkaline-Phosphatase is between 3 and 20 times Normal AND SGOT is no more than 2 times Normal SUGGEST Early Primary Biliary Cirrhosis

240

B. CHANDRASEKARAN AND SANJAY MllTAL

3.2.4 Explaining the Data

The final action by each specialist is to decide if the established concepts collectively “explain” the important data relevant at the concept being refined. For example, if the concept of cholestasis is being refined and there are data {dl, d2, ...,dn} to be explained, then the refinement will be complete when one or more primary concepts, such as stone-, stricture-, or drug-related intrahepatic diesease, are established which together account for all the data in this set. The notion of things to explain, per se, is not new. It has been used by Pople (1977) and Pauker et al. (1976), among others. However, unlike their approaches, where there was a uniform global weighting scheme, we view the process of explaining (or accounting for) important data as a local one. Each specialist, as part of its expertise, knows what is important for it to account for and accordingly decides when it has completed its refinement process. In Section 8.2, we will discuss how such a process might be accomplished by means of overview critics located at each specialist. 4. Auxiliary Systems: Intelligent Access to Medical Data 4.1 Motivation

In Sections 2 and 3 we described the problem-solving approach in MDX. However, as pointed out in the beginning, the diagnostic system contains only those pieces of knowledge which have a direct bearing on the diagnoses to be made. In order for those diagnostic rules to be “satisfied,” a wide variety of nondiagnostic medical knowledge is needed. Let us consider a typical diagnostic rule to examine this issue in some detail:

IF Jaundice onset within a week after Surgery AND Pruritus developed after Jaundice THEN Consider Post-Operative Cholestasis caused by Anesthetics The medical data typically available may be as follows: Halothane was administered at Cholecystectomy Biliribuin was 12.2 three days after surgery Patient had pruruitus a week later One can see that it is not possible to say that the preceding rule is applicable or not based on the data alone. Other medical knowledge is needed to make the necessary inferences and calculations to satisfy the rule. Some of these inferences with the associated knowledge are shown here:


24 1

Cholecystectomy +-surgery (Cholecystectomy is a type of surgery.) (2) Bilirubin 12.2 + bilirubin elevated (There are rules for calculating the norm of a lab test from its value.) (3) Bilirubin elevated + Jaundice present (Inference is based on domain knowledge.)

(1)

In addition, the following temporal relationships have to be established: Jaundice occurred within a week of surgery (3 days < 1 week; time of jaundice was same as for bilirubin.) (2) Pruritus developed after jaundice (a week after the time when bilirubin was elevated)

(1)

Thus one can see that what appeared to be a simple diagnostic rule nevertheless required a wide variety of medical knowledge to decide if it was applicable for a particular patient in the current diagnostic situation. In this section we will discuss three subsystems which were implemented to provide intelligent access to data about patients.

4.2 PATREC: An Intelligent Data Base for Medical Data

PATREC was designed to manage all the data (except those involving advanced procedures such as x-rays or biopsies) about patients and answer questions about this data from the diagnostic system. However, PATREC is more than just a data base management system: It was intended to have a wide variety of medical knowledge which can be used to make inferences, interpret lab tests, make assumptions, and handle imprecisely phrased queries. Functionally, it consists of the following: (1) A conceptual model of medical data, i.e., a model that applies to any patient (2) A patient model for each patient in the data base (3) A data acquisition system for acquiring and interpreting new data about a patient (4) A query language processor to facilitate asking a variety of questions about individual patients

4.2.1

Conceptual Model of Medical Data

Given that PATREC should provide intelligent access to medical data, it must contain the sort of knowledge about medical data such as signs,

242

8. CHANDRASEKARAN AND SANJAY MllTAL

symptoms, and lab data that one would expect a competent nurse to have. We were led, inevitably, to modeling the data entities as different concepts. Each such concept contains knowledge about a single medical data entity. Some of this knowledge is as follows:

(1) The classification of these concepts. For example, some of the basic classes are PHYSICAL, LABDATA, and DRUG. (2) The different attributes of a concept. For example, some of the attributes of pain are location, quality, intensity, trend, and response to drugs. (3) Constraints on the values of these attributes. For example, the possible values of quality of pain are “colicky” and “intermittent.” In some cases there are dependencies between these values. (4) Sometimes default assumptions can be made about some of the attributes of a concept which allow important inferences to be made in answering questions. For example, often the lab test values may be given without mentioning any units. In such cases, a default unit can be assumed-one suitable for the clinical setting. (5) Some data entities are not just single-level concepts, but refine into more detailed concepts. One example arises in the case of some lab tests which have components. In such cases, the concept has to be represented at different levels because both data and questions pertain to all the levels. (6) An important aspect of modeling these concepts is the representation of inferential knowledge, i.e., knowledge which would enable the system to infer something about a concept based on other concepts or other attributes of the same concept. Some of these inferences are inherent in the knowledge outlined so far, such as classification, defaults, and constraints. There are other inferences which cannot be captured in the structures outlined above and have to be explicitly represented. An example is the relationship between jaundice and bilirubin, which allows one to make inferences about one based on knowing something about the other. (7) There are other kinds of knowledge which are important from the point of view of reasoning about concepts in the data base context. For example, in order to provide a user-friendly interface for asking questions, often assumptions and inferences have to be made about the concepts being queried about. We will not go into detail about these aspects in this article and interested readers are referred to Mittal and Chandrasekaran (1981a). (a) How Much to Represent in a Conceptual Model. We have enumerated a long list of knowledge about data concepts that we represent in PATREC. However, one of the problems in representing knowl-


243

edge about “real-world” entities is, “How much detail should be represented?” Let us illustrate this problem with an example. Consider the phrase, “SGOT was 26 and remained steady.” In order to “understand” this, the model of SGOT must contain the following information: (1) The SGOT level is a type of lab data (Fig. 6). The creation of a more abstract entity called LABDATA would enable similar entities to share a common description. (2) It has a VALUE attribute, whose value in this case is 26. From the medical domain we know that lab data VALUEs typically have units assigned to them. This requires additional information. First, the valid unit for each individual type of lab data, such as SGOT, must be represented. Second, there may be some default units which can be assumed if none are provided by the user, as happened in this case. Finally, there should be a procedure that ensures that all values have assigned units and which checks that the units are valid for that entity. (3) It has a TREND attribute, whose value in this case is STEADY. Again, from the medical domain we know that TREND can be calculated from the VALUEs obtained over some time span. Therefore one can have a procedure which will calculate the TREND from VALUEs.

In addition, there are other kinds of information associated with lab data which will be represented. However, there is still a large amount of information about different lab data, and SGOT in particular, which will not be represented in this model. For example, the details about the biochemistry of SGOT need not be represented, because no diagnostician will ask for that information from a data base containing patient records. It seems, therefore, that some criteria are needed for deciding on what entities from the domain and how much detail about them should be represented. The intuitive criteria adopted by us are based on the needs of the diagnostic system which is the user of the data base. A different system such as an instructional program would require different kinds of information to be represented. Thus the representation of a concept is (SGOT (TY’PE(SVALUE(LABDATA))) (VALUE($VALUE . . .) ($DEFUNIT(IU))) (NORM(SRANGE(((21.0 52.5 85.0 (8 U) ? IU) . . .))) ($VALUE(. . .)) (SDEFAULT(N)) (FROM(VALUE))) (TREND(SVALUE . . .) (FROM(VALUE)))

FIG.6. Conceptual frame for SGOT.

244

B. CHANDRASEKARANAND SANJAY MllTAL

geared to the classes of uses to which the data base is expected to be put, namely, to questions about medical data entities that apply to individual patients and not about these entities in general. (b) Representation of the Conceptual Model. The conceptual models in both PATREC and RADEX are represented using frames, in a modified version of the FRL frame language (Roberts and Goldstein, 1977), because the frame representation allows the different kinds of knowledge indicated earlier to be conveniently represented in the machine. Each concept is represented by a frame. For example, Fig. 6 shows the frame for SGOT and captures the knowledge discussed in Section 3. These frames are linked in a classification hierarchy using the pair of attributes TYPE and INSTANCES to implement this link. Thus SGOT is described as a type of LABDATA in Fig. 6. Part of this conceptual hierarchy is shown in Fig. 7. The major property of the TYPE link is that it allows attributes, constraints, defaults, and procedurally attached knowledge to be inherited. A more detailed description of the conceptual model is provided in Mittal (1980).

MEDATA -HISTORY LMED CON LBO

TUMOR

LEXTRAHEP LMORPH

TEST LX-RAY

LPLAIN FILM

L ULTRASONOGRAM -ORGAN

PART L MICRO PART L WALL

L JUNCTION LORGAN L LIVER

L GALL BLADDER LPHYSICAL LPAIN

LJAUNDICE

\LAB DATA LWBC L BILIRUBIN

LSGOT

FIG.7. Hierarchy of medical data concepts.


245

4.2.2 Patient Model

Actual data for each patient are stored in instances of the corresponding generic frame. The data are organized in a hierarchy mirroring the conceptual hierarchy, but the highest level frame is now a unique patient record frame. The data are divided into general information and history and by each clinical episode (which is identifiable by some temporal description, e.g., at admission or 3 rnonlhs after halothane was given). Figure 8 shows the organization of frames in the patient model. Let us briefly look at how new data gets stored as part of this patient model. Given that “at admission, the patient had intermittent pain in the abdomen, unrelieved by drugs,” the system would first create a new episode for “at admission” (if one did not already exist) and then instantiate the PAIN frame, link it to the episode, and store the information in that frame. The frame corresponding to this input data is shown in Fig. 9. As each individual frame containing data for a particular patient is actually an instance of the generic frame for that particular data concept, all questions about this data for the patient are automatically mediated by the generic frame (directly via the process of inheritance and indirectly by examination). This allows the general knowledge about a concept to interpret the actual data instead of just retrieving it. Later, when we consider a detailed example of answering a question, this process of interpretation will become clearer. Patient record -General info LHlStory LDrugs -Surgery LFamily history LEpisodal record

L“At

Admission“ -Clinical LAlk-Phos LPhysical

.-. ..

LPain

LMorphological tests

LUltrasonogram L“episode

0

i”

L o . .

L“episode n” LOther information

FIG.8. Hierarchical representation of the patient model.

246

6.CHANDRASEKARAN AND SANJAY MllTAL (PAIN-1 CTYPE(SVALUE(PA1N))) (PATI ENT-EPlSODE($VALUE(PDOOl))) (LOCATION(SVALUE(ABD0MEN))) (TREND(SVALUE(INTERM1TTENT))) (DRUG-RESP(SVALUE(F))) (PRESENT(SVALUE(T (INFER-FROM: TREND))))

FIG.9. An instance of the PAIN frame,

We mentioned that these data frames are grouped episodically. In Section 4.4, we will discuss how the temporal information is “understood” to form these episodes and also how some of the temporal questions are answered. 4.2.3 Data Acquisition System

The data acquisition system is responsible for acquiring new data about a patient and storing it in the patient model. The input data are processed by a collection of procedures which are attached to the generic frames of the entity being described. For example, there is one procedure in the LABDATA frame which is inherited by all the individual lab data. However, those lab data which have more complex descriptions have separate procedures “tailored” for them and do not use the general procedure. For example, some lab tests have components, which calls for recognizing subtests and different ways of describing their values. Data about these lab tests is processed by a joint effort of the general lab data procedure and a special procedure for subtests. This allows knowledge about input descriptions of entities to be localized in separate procedures. Currently, new data is input to the system in a language which, while not a free-form natural language, nevertheless allows considerable flexibility in entering data. Let us consider an x-ray description example to show this: Plain-film x-ray showed that the intrahepatic ducts were dilated; there was no neovasculature or calcification in the liver; the left and right lobes were enlarged; there was a defect in the diaphragmatic area; and the gall bladder was normal.

The following are two ways of entering this information, though other ways are also possible. Parentheses are used as the only syntactic marker to group information about one entity: [l] (PLAIN-FILM (GALL-BLADDER NORMAL) (LIVER (IHD DILATE) (NEOVASCULATURE F) (CALCIFICATION F) (LIV-LLOBE LARGE) (LIV-RLOBE LARGE)(LIV-DIAPH DEFECT)))


247

[2] (PLAIN-FILM (IHD DILATE) (LIV-LLOBE (SIZE LARGE)) (LIV-DIAPH DEFECT) (LIVER (NEOVASCULATURE F) (CALCIFICATION F) (LIV-RLOBE LARGE)) (GALL-BLADDER NORMAL))

Initially, this input is given to the data acquisition “specialist” for PLAIN-FILM (the specialist may actually be inherited from the MORPHTEST frame). It checks if the entities described can actually be seen on plain film or not and then sends them over to the specialist for each entity. (The actual transfer is done via the frame, so each specialist does not directly have to know whom to call.) In step [ I ] above, this involves sending the respective inputs to the specialists for gall bladder and liver. In [2], however, entities such as IHD or LIV-LLOBE are also called. These later ones are components of LIVER and first create the appropriate links and then process the input. This process happens recursively until we reach descriptions which are attributes or values of some attributes of the entity being described. Each specialist uses the conceptual model to relate descriptions to the entities and remove ambiguities if necessary. For example, if LARGE could be a value of more than one attribute of lobes, then the specialist would ask for clarification. Sometimes there is strategy knowledge which can be located in the specialists for removing the ambiguities from particular situations. For example, the description (GALLBLADDER NORMAL) is actually ambiguous because it could mean either “the gall bladder is normal sized” or “the gall bladder is normal overall.” The latter interpretation is the preferred one and this is done at the level of the specialists which process the input. 4.2.4 Query Language

The query language was originally designed to enable MDX to phrase complex questions to PATREC as logical predicates, which are answered as true, false, or unknown. One of the motivations was that MDX should be able to view questions as predicates, without worrying about the many situations in which the answer is not directly available. The query predicates encoded strategy knowledge for deciding when to answer false or unknown when no data was available or inferred. They also provided a means for localizing syntactic information about a class of questions freeing MDX from having to specify such questions in great detail. We currently have about 10 query predicates which suffice in phrasing all questions (as needed by MDX) about the entities in the conceptual model,.

248

8. CHANDRASEKARAN AND SANJAY MllTAL

Data: The patient was given halothane and was already on cortesones. QI: Was any anesthetic given? Phrased-as: IS? ANESTHETIC GIVEN Answer: YES Explanation: Upward inheritance based on anesthetic-halothane Type hierarchy.

42: Did he have surgery? Phrased-as: IS? (SURGERY (LOC ?)) PERFORMED Answer: YES Explanation: Inferred from the fact that anesthetics were given.

Q3: Did he have liver surgery? Answer: UNKNOWN Explanation: The system can infer that surgery was performed somewhere, but not knowing the exact location cannot ruleout liver surgery. This unknown is in the sense of “I am not sure”, as opposed to “I don’t know”. Data: Bilirubin was 12.2 IU and the direct component was 70%. Q4: Is Indirect component above normal? Phrased-as: IS? INDIRECT AN Answer: YES Explanation: The information about Indirect is not directly available. However, its compof relationship to Bilirubin allows its percentage value to be calculated. From this value NORM can be calculated using the normal ranges for Indirect. QS: Did he have jaundice? Answer: YES Explanation: Inferred from the fact that if Bilirubin is above normal, then jaundice is present. The NORM of Bilirubin can again be calculated from the normal ranges. Data: PMN count is above normal.

Q6: Is WBC normal? Answer: UNKNOWN Explanation: Often one can inherit upwards along the comp-of hierarchy. However, in this case such an inference is invalid. The default values cannot be used either because some information is available about a component. Hence the best answer is Unknown, of the “I am not sure” variety. Data: A liver-scan showed a filling-defect in the left hepatic lobe. The liver was normal on physical exam. Q7: Is liver normal? Phrased-as: MORPH? LIVER ANY-PROCEDURE NORMAL Answer: NO Explanation: The liver was normal on the physical exam. On the liver-scan data, the following chain of inference took place: ( I ) filling-defect in lobe + lobe is NOT normal; (2) if lobe (or any component of liver) is abnormal, then liver is abnormal too. The query specialist resolved the conflict by choosing the non-default value. FIG. 10.

Sample session with PATREC.


249

Figure 10 shows a typical set of questions and how they are phrased using the query predicates. From the point of view of general interaction with PATREC, we find that this functional query approach is still useful. One can view each query function as a “specialist” in answering a class of questions. We have already discussed the inferential knowledge attached to each concept frame. There are other bits of knowledge-strategies about answering questions, when to use defaults, and knowledge about resolving conflicting information-which are not dependent on any particular entity being questioned about. Rather, they are related to the kind of question being asked. Clearly, such knowledge should be localized in these specialists. The specialists can then direct the search for the information, control inferences, parse the syntax, make comparisons, and do anything else needed to answer the question. They can also provide explanations showing the basis for the answers returned. Furthermore, these specialists can interact, allowing for a taxonomy of query specialists to be created. Let us illustrate these ideas by considering two of the query specialists that we currently have. What is-type questions can be asked using the “WHAT?” function. Similarly, questions of the type, “Is something of this form?,” can be asked using the “IS?” function. Figure 10 shows some examples of this type of question. The WHAT? function looks for the specified data, causes it to be inferred, uses defaults if needed, and returns the data along with an explanation. The IS? function calls WHAT? to get the data, performs the comparison implied in the question, resolves conflicts, and returns “true,” “false,” or “unknown” along with an explanation. In Appendix B we discuss how PATREC answers a question by considering an example in some detail.

4.3 RADEX: Radiology Consultant 4.3.1

Role of the Radiology Consultant in Diagnosis

The radiology expert in the medical community is a perceptual expert who can analyze complex images and interpret these in the context of a set of relevant diseases. The physician, lacking the perceptual training of the radiologist, often relies on a verbal description of the images in order to perform the diagnosis. The interaction between the two can be viewed as essentially a question-answer process. In simpler cases the physician may ask the radiologist for evidence or lack of it for specific hypotheses.

250

8. CHANDRASEKARANAND SANJAY MITTAL

In complex cases the interaction is more dynamic. The disease context the physician establishes may become more specific as a result of the radiological information provided by the radiologist. In other cases, the radiologist may suggest new possibilities, which are then verified (confirmed or rejected) by the physician on the basis of other information available to him. The final decision is usually made by the physician, who nevertheless may have to rely, in many cases almost completely, on the information provided by the radiology expert. Shared Representation. The radiologist and the physician share some medical knowledge which permits interaction between them. For example, both have a model of the human anatomy and physiology. They also share a model of the abnormalities of different organs or ducts which may be seen in imaging procedures; however, these shared models are embedded in different representation structures. The knowledge structure of the radiologist contains a refinement of the shared model. These refinements aid in the perceptual task of image analysis and interpretation and may not be needed by the physician for his diagnosis and therapy task. The perceptual level task requires a model which explicitly represents the variations that occur from one type of image to another. For example, the liver and the biliary tree appear differently on the percutaneous cholangiogram and ultrasonogram, in terms of size, orientation, and spatial layout. However, such variations are often unimportant from a diagnostic viewpoint. There are many levels of detail at which medical images can be described. A radiologist can describe images from the level of edges and regions to the level of organs and ducts, from “reduced uptake of dye” through “filling defects,” to stone and tumor. A physician, however, is usually not concerned with all such descriptions; he is interested in the identification of the body organs and parts of these organs. He is also interested in the description of abnormalities of these organs which would aid in diagnosing the diseases causing those abnormalities (or rejecting those diseases whose characteristic abnormalities were not seen on the images). We have discussed elsewhere (Chandrasekaran et al., 1980) the nature of this interaction in greater detail. 4.3.2 What RADEX Does

RADEX is a knowledge-based radiology consultant for MDX and performs some of the functions described above for a human radiologist. One of the things it does not do is actually interpret radiographic images; rather, it works from linguistic descriptions of these images. It currently performs two major functions, which we will discuss in this section.


251

(a) Managing the Radiology Data Base. One of the main functions of RADEX is the management of the wide variety of data obtained from the different imaging and morphological procedures (such as x-rays, cholangiograms, and ultrasonograms). It performs similar kinds of functions for radiology data as was discussed for the patient data base of signs, symptoms, etc., in Section 4.2. As the techniques and procedures for data entry and processing employed by RADEX are similar to those discussed in Section 4.2, we will not dwell on them here. The primary reason for viewing radiology data management as separate from the rest of the patient data base (though there are many similarities in detail) is the close relationship between such data and the anatomical-physiological model needed to provide radiological consultation. It should be pointed out here that RADEX is nor a complete information management system for the radiology unit of a hospital. It does not perform such functions as billing, scheduling, long-term storage and retrieval of radiographic reports, and provides only limited facilities for report entry. The MARS system (Lodwick, 1974) developed by Lodwick and his colleagues is a more comprehensive information management system. Their focus, on the other hand, was not on the development of a radiology consultant and their anatomical model is essentially keyword based. (b) Reasoning with an Anatomical- Physiological Model. As has been pointed out, communication between a radiologist and a physician is facilitated by means of a model of the human anatomy, its physiological systems, and a model of abnormalities which can be seen or inferred from medical images. Such a model in the MDX system is currently maintained by the RADEX expert. We have already mentioned one use of such a model, namely, acquisition and processing of image descriptions to build a data base of radiographic information. Toward this purpose, the model is used to “understand” the descriptions and appropriately store them. The anatomical model is also used to answer questions from the other experts in the MDX system. These questions may ask about certain kinds of radiographic information; inquire about the typical descriptions of organs, their components, or their relation to other organs; or may require some problem solving on the part of RADEX, based on this model. 4.3.3

Conceptual Model in RADEX

The conceptual model of the human anatomy (“structure”) and physiology (‘‘function”) is represented using frames similar to those described in Section 4.2. Currently, four kinds of entities are represented in the model:

252

6.CHANDRASEKARAN AND SANJAY MITTAL

Morphological procedures such as x-rays, cholangiograms, and ultrasonograms (2) Organs such as the liver, gall bladder, and the spleen (3) Organ parts such as ducts (which can also be organs), walls, and lobes (4) Deformities such as “obstruction” and “narrowing” which help describe abnormalities in organs (1)

(a) Morphological Procedures. The hierarchy of morphological procedures is shown in Fig. 11. Two major pieces of information are represented for each such procedure: first, the organs for which information may be obtained from a test or class of tests. This information is useful in building a “search set” (i.e., a set of morphological procedures from which data may be obtained about a particular organ) for answering questions about radiographic data. For example, given the question, “Was the liver large?,” RADEX will use this kind of knowledge to find out which imaging procedures can provide answers to questions about the liver. Second, there are certain stereotypical situations which carry important information about the organs typically visualized on these procedures, not directly but by inference. For example, if the ERCP (endoscopic retrograde cholangiography) was described as normal, then it is possible to infer that certain organs appeared “normal” on the image. This may be enough to answer no for most questions about the abnormalities in these organs (or parts of these organs). This knowledge is often distinct from the previous kind, because the inference of normalcy is often made for only a subset of organs which can be visualized.

LMORPH

TEST LCT SCAN LX-RAY LLIVER SCAN LCHEST~X-RAK LBARIUM MEAL LPLAIN FILM LCHOLANGIOGRAM LERCP -ORAL CHOL LINTRACHOL LOPER CHOL LPOSTOP CHOL LPERC TRANS CHOL LLAPAROTOMY LS P U N AGRAM L BIOPSY LULTRASONOGRAM

FIG.11. Hierarchy of morphological tests.


253

(6) Anatomical-Physiological Model. Organs are the principal entities around which the anatomical-physiological model is organized. Currently, two kinds of information are represented for each organ entity. First, the structure of an organ in terms of other organs or component parts is clearly represented. The structural description is what is typically meant by the “anatomical model.” Second, the feature descriptions, deformities, and other information obtained from various images are also decribed in the same entity frame. The frame describing the class of organs is shown in Fig. 12. The typical components of organs may be surfaces and ducts. How these components combine to form the organ may also be represented, though this kind of information is currently modeled only for ducts. In addition to the components, an organ also has structural features such as size, edge description, consistency on physical examination, etc. More details are provided elsewhere (Mittal, 1980; Chandrasekaran ef a / . , 1980). Finally, there are a variety of descriptive and diagnostic features, ranging from physiological ones like “functionality,” to descriptions about abnormalities such as “fibrosis” and “necrosis.” The descriptions at the level of organs apply to the organ as a whole; however, often some feature (ORGAN (COMPONENT(1NFER-FROM(DUCT)(SURFACE) (ARTERIES)(VEINS))) (CONSISTENCY(POSSIBLE-VALUES(HARD)(SOFT)(NODULAR) ...)) (EDGE(P0SSIBLE-VALUES(SMOOTH)(ROUND) ...)) (SlZE(P0SSlBLE-VALUES(LARGE)(NORMAL)...)) (DEFORMITY($VALUE(“Pointerto deformity description”)) ($IF-ADDED(1. IF any deformity THEN set organ’s NORMAL = F))) (FISTULA($VALUE(“Pointerto fistula description”))) (FIBROSIS(P0SSIBLE-VALUES(T)(F)(U))) (FUNCTION) (NECROSIS) (NORMAL($IF-NEEDED(1. IF enclosing organ’s NORMAL = T THEN set organ’s NORMAL = T)) ($IF-ADDED(1. IFNORMAL=F THEN set the enclosing organ’s NORMAL = F))) (INSTANCES($VALUE(ABDOMEN)(PANCREAS) (ARTERY)(DUCT) (DUODENUM)(LIVER) (SPLEEN)(GALL-BLADDER))) (USER-INP($IF-ADDED((#BTREE-I” FRAME: VALUE:) (COMM: “Process deformities”))) ((#ORG-INP FRAME: VALUE:) (COMM: “Process rest of desc.”))) (NAME($LAST-NAM(OSOO0))) /Seed name for naming instances of Organ frame/ (PAIN($VALUE(“Pointerto pain description”))) (TYPE($VALUE(MEDATA))))

FIG. 12. Conceptual frame for a class of organs.

254


applies to only a part of the organ. In such cases, the feature values are stored only under the respective organ-part frame. For example, if the liver were described as “fibrous,” then the liver frame would contain the description4 of fibrosis; however, if fibrosis were observed on only a part of the liver (say, the right lobe), then only the frame for the right lobe would contain the fibrosis information. Of couse, the “component o f ’ relationship between the liver and its right lobe would enable the inference that there was fibrosis in the liver. At the same time, questions about fibrosis in other parts of the liver would be answered in the negative (unless they had some specific information). We will discuss the questionanswering process in Section 4.3.5. There are different kinds of procedural knowledge which may be attached to organ descriptions. We have talked about some, namely, rules which allow inferences to be made about organs from descriptions about tests from which information is obtained about these organs. Some others are shown in the organ frame (see Fig. 12). The two procedures attached to the NORMAL attribute, respectively, infer that an organ is normal if its enclosing organ (i.e., the organ of which it is a component of) is normal and assert the abnormality of the enclosing organ if this one is abnormal. The procedure attached to the DEFORMITY link asserts the abnormality of an organ if there is any deformity. Similarly, there are other kinds of rules and procedures for inferring features from lower level descriptions, relating descriptions across the components of organs, and inferring and establishing relationships among different attributes. (c) Organ Parts. Unlike organs, which have both anatomical as well as physiological descriptions, organ parts are entities which are primarily used in describing the anatomical structure of organs. Many of the organ parts may not even be objects in the sense of having a mass and volume. Often they are logical entities which serve to describe an organ from different perspectives. For example, the lobes of a liver represent a division of the mass of the liver, and the “areas” a division of the surface area of the liver. The importance of organ parts as entities in the anatomical model lies in the fact that these are the entities most often seen in images, biopsies, or dye contrast studies. Therefore a complete model of these organs for a particular patient can only be built from the description of the organ parts as obtained from different morphological tests.

‘In the frame shown in Fig. 12, fibrosis is represented as a feature which may or may not be true for an organ. This is because, from a diagnostic viewpoint, knowing about the presence of that feature is enough. However, the histology expert who would provide this information from biopsy reports has a more detailed representation for the concept of *‘ fibrosis.”


Lumen

Wall on iJtu ,/

Pancreatic

255

)urface,

...

Portahepatic

FIG.13. Hierarchy of parts of organs.

The different types of organ parts used in the descriptions of organs in this model are shown in Fig. 13. For more details about these, the reader is referred to Mittal (1980). (d) Deformities. The abnormalities and deformities that can be described and queried about for the various parts of the anatomical model are also represented at various levels, as shown in Fig. 14. A duct may be obstructed, narrowed, or dilated. Going a level deeper, one might know the type of obstruction or narrowing. Descriptions at different levels have different diagnostic values. For example, knowing that the biliary tree is obstructed is enough to strongly suspect extrahepatic cholestasis, but it is not enough to indicate the cause of obstruction. Similarly, if a liver scan shows a filling defect at the porta hepatis region of the liver, one would suspect something like a cyst or tumor, but a tumor would be established by a more refined description of a cyst.

, ,, Deformity in duct

Obslctio\,

Mass

Tumor

0 Let Istack = ( . . . x,...) Invariant 0 5 Length(1stack) 5 n Initially lsrack = Nullseq Function Push(s:lstack,x:lnteger) Pre 0 5 Lengths(s‘) < n Post s = s’ -x Pop(s:lstack) Pre 0 < Length(s’) 5 n Post s = Leader(s’) Read(s:lsrack) Returns (x:lnreger) Pre 0 < Length(s‘) 5 n Post x = Lasr(s’) Empty(s:lsrack) Returns (&:Boolean) Post b = (s’=Nullseq)

(the implementation part follows)

Endform

304

ALFS T. BERZTISS AND SATISH THAlTE

prototypical sequence). It assumes that sequences are well known and well understood. The behavior of the operations comprising the type is specified in terms of state transitions on an underlying sequence representation. Thus this “specification” may be thought of as an abstract implementation in the world of sequences. The functions Length, Leader, Last, -,and = ,and the constant Nullseq have been borrowed from the world of sequences. The behavior of Push, Pop, Read, etc. is specified using these functions in conventional Pre and Post conditions. The syntax of the specification is more or less self-explanatory, except that - stands for concatenation, and in the conditions, x’ stands for the value of the formal parameter x at the beginning of the operation and x stands for the value at the end. Note that the operations are given as conventional procedures with “reference” parameters. The reader should be warned that part of the complexity of this specification stems from the fact that bounded stacks are being specified here. In particular, all the Pre conditions for the operations would be simpler otherwise. Further examples of operational specifications can be found in a compendium of Alphard papers (Shaw, 1981) and in Appendix 3 of the important article “Proof rules for the programming language Euclid“ (London et al., 1978). An expository introduction to the Euclid example has been provided by Guttag (1980). 2.3 The Algebraic Approach

Our first algebraic example, in Table 11, is taken from Guttag et al. (1978a). This example, in contrast to Alphard, specifies the unbounded stack type. The Declare part is the syntactic specification. It gives the names of the operators comprising the type and specifies their “functionality” (names of domains and ranges) using standard mathematical notation. Note that the constants of a type (in our case New, which is the empty stack) are usually given as domainless functions, for uniformity and theoretical simplicity. This part is syntactic in the sense that it may be viewed as a context-free grammar of the well-formed terms (WFTs) of various types that can be constructed using the function symbols in the type definition. For example, Push(Pop(Push(New,-5)),3) is a WET of type Istack, and Read(Push(New,100))

ABSTRACTDATATYPES

305

TABLE I1 ALGEBRAIC STACK SPECIFICATION Type Istack Dednre New : + Istack Push : Istack x Integer -+ Istack --* Istack Pop : Istack Read : Istack -+ Integer u {error) 4 Boolean Empty : Istack For All s E Istack, i E Integer Let Empty(New) = True Empty(Push(s,i)) = False Pop(New) = New Pop(Push(s,G) = s Read(New) = error Read(Push(s,i)) = i End Istack

is a WFT of type Integer (a type we have assumed to be previously defined). The function Pop differs from the way we have been accustomed to viewing it. Conventionally, a pop procedure returns a value, but also changes the stack by deleting the value from it. In other words, the result is a pair having the range Istack x Integer. Such complex ranges would make functional composition very difficult to deal with. Consequently the two tasks that the conventional procedure performs have been assigned in Istack to the separate functions Read and Pop. The For All ... Let part is the so-called semantic specification. It would be more appropriate to call it the semantic restrictions part. This part consists of axioms that specify certain relations that the actual operations of the type must satisfy in any implementation. Each axiom is afurmal equation between two WITS of the same type. Note that the variables in the WETS are not free. They are universally quantified in the For AU Let clause. Also, each variable used in the right-hand side of an equation must have been used in the left-hand side. The formal equality is to be interpreted as “permission to substitute,” Le., interchangeability, regardless of context. Let us now look again at

.--

Push(Pop(Push(New,-5)),3)

and

Read(Push(New,100))

These WFTs are grounded in the sense that they do not involve any variable symbols of any type, unlike the WFTs used in the axioms.

306

ALFS T. BERZTISS AND SATISH THATTE

WFTs which involve variables may be called generalized or parametrized WFTs. Each such parametrized WFT can be grounded by substituting a grounded WFT of the appropriate type in place of each variable symbol. The axioms are, in general, allowed to be conditional equations. This means that any part of the right-hand side of the equation may be of the form

If w I Then w2 Else w3 where w 1is a WFT of Boolean type, and w2 and w3 are WFTs of the same type, which is the type of the conditional term as a whole. Our next example contains conditional equations. Suppose we needed a queue instead of a stack. Intuitively, both stacks and queues are lists with restricted access: For a stack it is at the same end for adding in a new integer, reading data, and popping the stack; in a queue the integer is added to the end opposite to that at which reading and popping take place. Table I11 is the algebraic specification of a queue of integers (Iqueue). Except for name changes, syntactic specification of lqueue is the same as that of Zstack, but there are important differences in the semantics. The Pop and Push of Zstack are somewhat like “inverses” of each other, which helps in their specification. Not so with Qpop and Qpush, which access different ends of the queue. As a result, two of the

TABLE111 ALGEBRAIC SPECIFICATION OF A QUEUE

Type lqueue Declare Qnew : Iqueue Qpush : Iqueue x Integer + lqueue + Iqueue Qpop : Iqueue Qread : Iqueue -+ Integer U {error} Qempty : Iqueue Boolean For AN q E Iqueue, i E Integer Let Qempry(Qnew) = True Qempty(Qpush(q,i))= False Opop(Qnew) = Qnew Q p w ( Q p u . N q , ~ N= If Qempry(q) Then Qnew Else Qpush(Qpop(q),i) Qread(Qnew) = error Qread(Qpush(q,i))= If Qernpry(q) Then i Else Qread(q) End Iqueue -+

-+

ABSTRACTDATATYPES

307

TABLEIV ALGEBRAIC SPECIFICATION

OF A

BINARY TREE

Type Ibin Declare New : + Ibin Make : Ibin x Integer x Ibin + Ihin Left : Ibin + Ihin Data : Ibin + Integer U {error} Right : Ibin + Ibin Empty : Ibin + Boolean For All s,d E Ibin, i E Integer Let E m p t y ( N e w ) = True Empty(Make(s,i,d))= False Lefr(Make(s,i,d))= s Lefr(New) = N e w Da/a(Make(s,i,d))= i Dara(New) = error Right(Make(s,i,d)) = d Righr(New) = N e w End Ibin

axioms of Iqueue have to be written as conditional equations, and these conditional equations are recursive. We finish with an example of hierarchical development: A specification in which use will be made of a previously specified data type. This is the binary tree, for which an initial specification is Table IV. Let us now add the three classicial traversals of binary trees to our specification. Following Guttag et af. (1978b), we interpret a traversal as a function that builds up a queue of the integers stored in the binary tree. The functions that return the queues of integers corresponding to preorder, postorder, and inorder will be called Preord, Postord, and tnord, respectively. Let us look at tnord in some detail. Under inorder traversal, for any node in the binary tree, the left subtree of the node is traversed under inorder, next the node itself is processed, and finally the right subtree of the node is traversed. Such a process lends itself well to recursive specification. Given the binary tree Make(s,i,d),one would create the queue Inord(s), append to it i by means of Qpush(Inord(s),i),create the queue Inord(d), but then be in possession of two queues and no way of combining them. The way out is to add a new function to data type Iqueue: a o i n : Iqueue

X

Iqueue + Iqueue

308


with axioms

For All q,r E Iqueue, i E Integer Let Qjoin(q,Qnew) = q Qjoin(q ,Qpush(r,i)) = Qpush(Qjoin(q,r),i) We are now finally in a position to augment the specification of Ibin with the declarations Preord ; lbin Postord : lbin Inord : Ibin

+ Iqueue + Iqueue

+ Iqueue

and the axioms

For MI s,d E Ibin, i E Integer Let Preord(New) = Qnew Preord(Make(s,i,d)) = Qjoin(Qjoin(Qpush( Qnew ,i),Preord(s)), Preord(d)) Postord(New) = Qnew Postord(Make(s,i,d))= Qjoin(Postord(s),Qjoin(Postord(d), Qpush(Qnew,i))) Inord(New) = Qnew Znord(Make(s,i,d)) = Qjoin(Qpush(Inord(s),i),Inord(d)) We conclude this section with a useful classification of the functions in an algebraic specification-the type being defined is referred to as the type of interest (TOI) following Guttag and Horning (1978): Generators, which are domainless functions that generate new starting values of the TO1 (example: New of Istack) Constructors, which are the functions, other than the generators, to have the TO1 as range, i.e., those functions which construct new values of the TO1 from old ones (examples: Push and Pop of Istack) Extractors, which are functions with non-TO1 range, i.e., those functions which extract the non-TO1 values associated with an instance of the TO1 (examples: Read and Empty of Istack) 2.4

Evaluation of the Approaches

There is a clear difference between Alphard-like and algebraic specifications. The Alphard specification has an operational, implementationlike character. It is dependent on the underlying representation domain (i.e., sequences in our example). The kind of information to be retained in any actual implementation is already rigidly specified. The only leeway

ABSTRACTDATATYPES

309

offered to the implementor is in the actual algorithms to be used for implementing the various operations of the underlying representation domain. Algebraic specifications, on the other hand, are self-sufficient. The only non-TO1 domain that the specification of Table I1 relies on is the domain of integers, and only because stacks of integers are being defined. In general, the only underlying domain that algebraic specifications need is the domain of Booleans, which is itself specified algebraically in a completely self-sufficient way (see, for example, Guttag et al., 1978a). Moreover, the semantic restrictions on the actual operations of the TO1 do not specify their behavior operationally. They merely state certain required relationships. As we will see later, this permits a range of essentially different implementations. For those familiar with elementary mathematical logic (see, for example, Mendelson, 1964), an algebraic specification is a formal theory, and an abstract implementation is a model for the theory. Just as for any (incomplete) theory there are several (nonisomorphicj models, so for each (incomplete) algebraic specification there are several (nonisomorphic) abstract implementations. The notion of completeness in our context of algebraic specifications will be discussed in an intuitive way in what follows. To conclude, it appears that the Alphard-like specifications may be useful as abstract implementations, where the type being defined is closely related to some well-known abstract domain. However, in general it is too rigid and may not be of much use in describing the essential requirements for types based on loosely defined techniques such as file directories and priority queues, and may constrain implementations to too narrow a range. Standish (1978) has described another axiom-based approach which falls between the two described here. He has essentially tried to characterize a single algebra for all data types or, in other words, a universal data type.

3. The Meaning of Algebraic Specifications

3.1 Algebras The reason why algebraic specifications are called “algebraic” is that each such specification axiomatizes the essential properties of a class of algebras. This is based on the notion that data types are heterogeneous algebras, which were introduced by Birkhoff and Lipson (1970). This

31 0


approach is no longer novel. It is already some time since Morris (1973) argued that “data types are not sets,” and the pioneering work of Zilles (1974), Guttag (1975), and Goguen et al. (1975), among others, has become quite influential. Here we shall attempt to outline the intuitions behind the idea of algebraic specification. The data type “natural numbers” consists of the set of objects called “natural numbers” along with the collection of the characteristic operations on natural numbers such as successor, addition, etc. As is obvious, this “type” is the same as the algebraic structure studied in elementary number theory. The type “stacks of integers” also consists of a set of objects, which are called “stacks of integers,” along with a collection of characteristic operations such as Push, Pop, Read, etc. The only difference is that the operations on natural numbers act within a single domain, whereas the operations on stacks of integers act within two distinct and disjoint domains: “stacks of integers” and “integers.” Hence the natural number type is called a homogeneous algebra, and the type “stacks of integers” a heterogeneous algebra. Conceptually, the various types in any problem domain are hierarchically related to each other. For example, the type “stack of integers” is obviously dependent on the type “integer.” At the bottom of this hierarchy are the basic types, such as natural numbers and Booleans, which are, by definition, self-sufficient and thus are implemented as homogeneous algebras. The types “higher up” are, again by definition, dependent on types “lower down” and are implemented as heterogeneous algebras. At this point, at the risk of being repetitive, let us emphasize that the distinction between an algebraic specification and an algebra must be clearly borne in mind. A specification is apurely formal system consisting of some domain and function symbols, rules for forming well-formed terms (the syntax), and formal equations between well-formed terms (the axioms). Algebras, on the other hand, are fully defined “working” systems (like abstract implementations), consisting of sets of objects and actual, well-defined functional mappings from objects to objects. Properties of the function symbols can be formally proved in the world of specifications, so that they must hold for the corresponding functions in any algebra that satisfies the specifications. And as we will see, there are, in general, several distinct algebras (abstract implementations) for each specification. 3.2 The Algebra of Natural Numbers

Before going on to a discussion of the meaning of specifications for classes of heterogeneous algebras, it might be helpful to see an example of

31 1

ABSTRACTDATATYPES

a specification for a class of homogeneous algebras. We will use the familiar algebra of the natural numbers with the successor and addition operations, and give a specification for it. Remember that we are dealing with the “abstract” algebra of natural numbers, in the sense that it is immaterial whether the number three is represented as I11 or 3 or (binary) l l . Table V gives an algebraic specification for the natural numbers. The conventional algebra of natural numbers obviously satisfies this specification. However, it is not the only algebra that satisfies the specification. All that the specification directly requires is that the domain of any algebra satisfying it must contain at least one object, corresponding to the “result” of the “0” function symbol. Note that we will use the same character 0 ambiguously to refer to the function symbol in the specification and the corresponding actual funcrion in the model algebra, and similarly for the other function symbols and functions. The ambiguity should be easily resolvable from the context. For an illustration of this consider the trivial algebra in which the domain consists of a single object “0” corresponding to the result of the function 0.This algebra has two trivial functions Suc and Add such that Suc(0) = 0

and Add(0, 0)

=

0

and it satisfies the specification. Another unusual algebra that satisfies the specification corresponds to the nonstandard (or unnatural) models of arithmetic, where we have “an object at infinity” which then has its successors, etc. This kind of unnatural algebra can be excluded from our notion of meaning by stipulating that each object in the TO1 domain of any admissible algebra must correspond to some finite WFT of the TOI. What this means is that each object in the domain can be constructed by a finite number of applications of the constructor function(@to the constant(s) generated by the generator(s). In our example, any finite natural number n TABLEV SPECIFICATION FOR NATURAL NUMBERS ~~

~

Type Nut Declare 0: Suc : Nut

+ Nat

+ Nut Add : N a t x Nat ---* N a t For All i j E Nut Let Add(i,O) = i Add(i,Suc(J)= Suc(Add(i,j)) End N a t

312

ALFS T. BERZTISS AND SATISH THAITE

can be constructed as the result of n applications of Suc to the constant 0 generated by the generator 0. The “number at infinity,” however, cannot be so constructed. By the exclusion of “unnatural” algebras, we restrict ourselves to the so-called finitely generated algebras. The ADJ group (Goguen et af., 1975) and Wand (1978) do not use this restriction. Hence their treatment is a little more complex. We will defer the problem of excluding the trivial algebra for the moment, and come back to it later. Note that, as pointed out by Birkhoff and Lipson (1970), all our general remarks are equally valid for homogeneous as well as for heterogeneous algebras. Going back to the specification for natural numbers, it can be seen that any particular property of particular natural numbers can be formally proved using the specification, which gives us a basis for calling it a specification. For example, using the notation that Sucn(i) stands for n applications of the Suc function symbol to the WFT i, we can prove that

sucyo)) = suC3(0)

A ~ ~ ( s ~ C ( O ) ,

The proof is trivial. We have Add(Suc(0),Suc2(0)) = Suc(Add(Suc(O),Suc(O))) = Suc2(Add(Suc(0),0)) = suc3(0)

by axiom 2 by axiom 2 by axiom 1

The axioms are used here as rewrite rules to produce a sequence of inferences very similar to the recursive computation of addition using the successor function, a fact which will become of significance later. In the conventional algebra of natural numbers, Suc”(0) is modeled by n. Hence the formal proofs in terms of Suc”(0) establish the properties of interest for the corresponding natural number objects in the algebra. Although these axioms are sufficiently powerful to prove such particular properties, they are not powerful enough to prove general properties of the natural numbers, such as commutativity of addition, unless that property is added as an axiom. For proving general properties, we need to add the induction principle to the axioms. The induction principle, being based on the natural numbers, cannot be defined as a single axiom. This is so because one cannot express general statements about Sucn(0) as axioms. Therefore, the induction principle has to be stated as an axiom scheme generating a countably infinite number of axioms. Intuitively, there is one for every natural number n. Moreover, this scheme involves the use of the propositional calculus. Further discussion of this would take us too far afield, so we terminate our discussion of the algebra of natural numbers here.

ABSTRACTDATATYPES

313

3.3 Pinning down the Meaning of a Specification

The parallel between (1) the relationship of an algebraic specification to the class of admissible algebras (in the sense of Section 3.2) which satisfy it and (2) the relationship of a purely recursive program (equation) to its fixed points in denotational semantics appears to be deep and fruitful. We will use this analogy to explicate relationship (I), something that, to the best of our knowledge, constitutes a new approach. Stoy (1977, p. 79) points out (using slightly different notation) that given the following recursive function definition fix)

=

E x = 0 then 1

else if x = 1 thenf(3) elsef(x - 2) both 1 if x is even undefined if x is odd

fi(x)

=

I for all x

satisfy the recursive equation. Note that fi and fi are given as concrete mappings without any computational machinery. They are objects from the appropriate function space (i.e., [ N + Nl) which are offering themselves as candidates to fill the post of the “meaning” of the function symbol in the recursive equation, or, in other words, as “solutions” of that equation. They have the minimum qualification that they abide by the constraint specified in the equation. Hence they are called “fixed points” of the equation. Denotational semantics chooses the “least defined” fixed point (i.e., the fixed point defined for the least number of argument values) as the appropriate solution of such an equation. In our case, the least fixed point is the function fi. It can be shown (see Stoy, 1977) that a unique least fixed point exists for every recursive equation. Note the technical detail that here we are only considering equations involving “continuous functionals.” All the usual kinds of recursive equations fall in this category. For more information on continuity of functionals, etc., see Stoy (1977). The method for obtaining the least fixed point is lucidly explained by Manna et af. (1973), and we will not go into its details here. For our purposes, the following slightly different view of the least fixed point is more appropriate. Let us think of the given recursive equation as an axiom. Then, given the properties of the natural numbers, the value y offi (i.e., the least fixed point) at any x for whichfi is defined can be inferred

314


from the recursive equation as a “theorem” of the form f(x) = y

(wherefis the function symbol of the equation)

For example, f(0) = 1 f(2) = f(2 f(4) = f(4

-

2) = 1 2) = 1

can be inferred in one step can be inferred in two steps can be inferred in three steps, etc.

However, the proposition ‘ t f ( 3 ) = 1” (correspondingto the fact thatfi(3) = 1) cannot be inferred in this manner in a finite number of steps, though it is consistent with the axiom (Le., this proposition is independent of the axiom). In this sense, the least fixed point is the fixed point which contains only the information that is “warranted,” i.e., directly implied by the given recursive equation. When we come to algebraic specifications, the situation is similar, though somewhat more complex. An algebraic specification is analogous to a recursive equation in that both are formal systems for which we must seek a “solution.” The appropriate function space for a recursive equation is simply the space of all functions with the appropriate domain and range. In analogy, we must consider an appropriate space of algebras for an algebraic specification. An appropriate algebra consists of the following: (1) An abstract set of objects, corresponding to the TO1 domain (2) A set of functions, with the appropriate domains and ranges, with one function for each function symbol used in the specification

In this, the non-TO1 types involved are, of course, assumed to be already well defined, and their algebras (implementations) are assumed to be available. The space of appropriate algebras consists of all such appropriate algebras. All that the appropriateness condition requires is that the algebra have a correct form syntactically. Hence only the syntactic part of the specification needs to be considered here. Next we have to identify something corresponding to the fixed points of a recursive equation. For this, we observe that the semantic restrictions part is a set of equations rather similar to a set of mutually recursive equations. We then demand that all qualified candidates satisfy the set of constraints specified by this set of equations, this in addition to the caveat we made about finite generation, which excludes “unnatural” algebras

too. The next step, that of finding something corresponding to a least fixed point, is not so straightforward. If all domains were well defined, then we

ABSTRACTDATATYPES

315

could apply an approach virtually the same as the least fixed point principle. However, the TO1 domain is not, in general, uniquely determined, even up to isomorphism. We deal with this, and related questions, in Section 3.4.

3.4 Initial and Terminal Algebras 3.4.1 Initial Algebras

We first introduce the (=) relation. It is defined as a generalization of the = relation used in the axioms: If a and b are WFTs of the same type, then a (=) b if and only if there is a finite sequence a l , a 2 , ..., a, of WFTs of the same type, such that each of

a = a1 a l = a2 a, = b

can be inferred in one step from some axiom. Informally, a (=) b iff a = b can be inferred from the axioms in afinite number of steps. An axiom of the form a = b implies that the parametrized WFTs a and b are interchangeable. Therefore, in any algebra satisfying the axioms, grounded instances of a and b must correspond to the same object. When we say that a WFT “corresponds to an object,” we are talking of the object which results from the application, in the given order, of the actual functions corresponding to the function symbols in the WFT, including the generator symbols. For example. an axiom of the type Istack says that Pop(Push(s,i))= s

Therefore, Pop(Push(New,3))and New must correspond to the same object. The same argument carries over to WFTs a and b, where a (=) b, since b can be substituted for a and vice versa in a finite number of steps. Thus if we partition the set of all grounded WFTs of each type into equivalence classes defined by the (type specific) relation (=), then all WFTs in any particular equivalence class must correspond to the same object in any algebra satisfying the axioms. This is particularly of significance for WFTs of the type being defined (i.e., the “TOI”) because the domain of objects of this type is not yet well defined. An algebra in which two grounded WFTs correspond to the same object if and only i f they

316


belong to the same equivalence class is the “finest grained” algebra (the algebra with the greatest number of objects in the TO1 domain) consistent with the axioms. The ADJ group (Goguen et al., 1975,1978) have popularized this class of algebras under the name of “initial” algebras. They have shown that all such algebras are necessarily isomorphic, i.e., essentially the same except for the representation of the objects and functions. They have also shown that an initial algebra exists for every consistent specscation. These two properties (uniqueness and existence) led them to say that the initial algebra (we will talk of the initial algebra to refer to the isomorphic class of such algebras) is the most desirable choice for the “meaning” of an algebraic specification. Initial algebras have some other desirable properties, but also some undesirable ones, which we will discuss briefly later. To illustrate their methods, consider the demonstration of the existence of an initial algebra for every consistent specification. The essential construction is to “represent” each WFT by a string in the obvious way to obtain the “free” algebra correspondingto the syntactic part of the specification. The strings which correspond to TO1 WFTs belonging to the same equivalence class under (=) are then grouped together and each such group represents a single object in the TO1 domain. This gives the quotient algebra under (=), which can be easily shown to be initial. 3.4.2 Terminal Algebras

The diametrically opposite approach is to choose “terminal” algebras (Wand, 1978; Wirsing et al., 1980; Kamin, 1980). While the initial algebra is the finest grained algebra consistent with the axioms, the terminal algebra is the “coarsest grained” one. Informally, in the initial algebra, two WFTs correspond to different objects unless they can be proved to be equivalent. In the terminal algebra, two WFTs correspond to the same object unless they can be proved to be “not equivalent.” To formalize this notion, we need a formal relation (#), which is the opposite of (not the negation of) (=). Note that the relation = of the axioms served as the basis for defining (=). There is no corresponding relation in our system of specification to serve as the basis of (#). In other words, in our system of specification, based on conditional equality relations, there is no way to prove two WFTs “not equivalent.” There are two (related) ways to get out of this cul de sac. The formal discussion is beyond the scope of this article, so we will try to intuitively outline the ideas behind them. Suppose we are defining some type T which is dependent on some “lower” types c, t Z , ..., t,. A WFT of type ti (for some 1 5 i 5 n) is a

ABSTRACTDATATYPES

317

primitive WFT if function symbols of type T do not occur in it. For example, in defining stacks of natural numbers, Suc(Add(Suc(O),O))fits this requirement but Suc(Read(Push(New,O)))does not, even though the latter is also of type “natural number.” Note that in our previous examples, we have used object representations such as - 5 and 3 as parts of “higher” WFTs, instead of using the corresponding primitive WFTs. Though this aids clarity, in a purely formal sense it is not quite proper. We can now define the relation # on primitive WFTs of type ti under the assumption that a particular algebra has been chosen as the meaning of the specification for t i . Very simply, given primitive grounded WFTs a and b, a f b i f f O ( a )# O(b),where O(x) stands for the object which, in the algebra, corresponds to the WFT x. Next we define the relation (#) for grounded WFTs of type T , using the extractor (i.e., non-TO1 range) function symbols of T and the (=) relations for WFTs of types ti (1 Ii 5 n), where the (=) relation “native” to each t i has been augmented with the consequences of the equations (axioms) between WFTs of type ti in the specification for T. This allows us to include the WFTs involving function symbols of T into the equivalence classes as long as they are of the right type. For example, the axiom Read(Push(s,i))= i in the specification for stacks of integers includes Read(Push(s,i))in the same equivalence class as i for each i. The augmented (=) relations thus allow us to reduce many, though in general not all, WFTs of each lower type to primitive WFTs of the same type; i.e., they allow us to get rid of the function symbols of T while preserving equivalence. Let W(x) stand for a parametrized WFT of some type t i , such that it contains only one variable symbol x, where x is of type T . For grounded WFTs a and b of type T , a (#) b if and only if there is a WFT W ( x ) such that

W ( a )(=>rr

W b ) (=>s

where r and s are primitive WFTs and r # s . Note that W ( x ) cannot itself be a (parametrized) primitive WFT since, by definition, a primitive WFT cannot contain TO1 variables. Thus W(x) consists of a composition of various functions belonging to type T; with an extractor at the outermost level, since W(x) is of a lower type ti. The reason for restricting W ( x ) to contain only one variable is that we want the # relation to be applicable to the primitive WFT equivalent to W ( x ) after a grounded WFT is substituted for x, and the # relation is applicable only to grounded WFTs. We call W ( x ) a “composite extractor” operation, which can extract from x information which a single application of an extractor may be unable to do. For example, W ( s ) = Read(Pop(Pop(s)))

318


extracts the third element from the top of stack s, something which the simple extractors Read and Empty cannot accomplish in a single application. Thus a composite extractor may be able to “distinguish between” two TO1 objects when a simple extractor cannot. For convenience, we may think of a simple extractor as a trivial case of a composite extractor. Thus, informally, a (#) b if and only if some (possibly composite) extractor can provably distinguish between a and b. An example should make this clearer. Suppose we are defining stacks of integers, and we assume the conventional algebra of integers to be the meaning of the integer type. Now we have Push(New,3) and Push(New,4) as grounded WFTs of type T (where T = “stacks of integers”). We have the extractor Read and by the (=) relation on integers, augmented by the stack axioms, Read(Push(New,3)) (=) 3 Read(Push(New,4))(=) 4 where 3 # 4. Thus we conclude that Push(New,3) (#) Push(New,4) This final result may appear anticlimactic, something in the nature of the mountain and the mouse, but that is only because we have been using an example which has earned some (undeserved) contempt by familiarity. Now we may use the negation of (#) as an equivalence relation; that is, we may say that a and b are in the same class if and only if “a not (#) b.” This finally lets us say that WFTs a and b correspond to the same object if and only if they belong to the same class in this sense, and gives us the terminal algebra. Note that “not (#)” is not the same as “(=).” The relation “not (#)” is much coarser, in that even if you are unable to prove a (=) b, this does not mean you have proved a (#) b. If neither a (=) b nor a (#) b can be proved, then the (=) relation puts a and b into different classes, but the “not (#)” relation puts them into the same class. Consequently, O(a) # O(b)in the initial algebra, but O(a) = O(b)in the terminal algebra. Note that this construction cannot be used for a homogeneous type, since such a type does not depend on any lower types. In the trivial algebra of our natural numbers example all WFTs correspond to the same object, and this algebra is consequently the terminal algebra for the given specification. We certainly do not think of natural numbers in the way suggested by this algebra, which means that in practice we cannot make the terminal algebra the invariable choice for the meaning of a specification in our system. Note, however, that there are other, more general

319

ABSTRACTDATATYPES

systems for algebraic specification in which the terminal algebra is not always trivial (Wirsing et al., 1980). The other way out is to axiomatize the equality relation in at least some of the lower types ti when defining the higher type T which is dependent on them. At the bottom of the hierarchy this usually turns homogeneous types into heterogeneous ones. Consider the augmented version of the specification for the natural numbers given in Table VI. The axioms of this specification closely parallel the first-order formulation given in Mendelson (1964; p. 103): Our axioms labeled A3, AS, and A6 restate axioms S3, S5, and S6 there, A2 stands for axioms S2 and S4 there, and axiom S1 there is a consequence of our A l , A2, and A3 along with induction. Now suppose we were defining “stacks of natural numbers” and we treat the Eq function symbol as a distinguished one; we can say for two WFTs a and b of type natural number that a f b iff Eq(a,b)(=) False. The rest of the construction for the (#) relation and the terminal algebra proceeds as before. The availability of the special Eq function for a (usually lower) type makes it simple to check for the equality of two instances of that type, as we do in the specifications for Sets and Arrays in what follows. We prefer to use a function name like Eq to signify the equivalence operator when it is included explicity in the specification. The alternative of using the “=” or the “(=)” symbols is confusing and raises the same questions about equivalence that we discussed earlier in this section. The axiomatization of equality has some interesting consequences the discussion of which is outside our scope (see Musser, 1980b, for details). As mentioned above, the initial algebra always exists, given a consisTABLEVI AUGMENTED NAT EXAMPLE

Type Nai Declare Nai Nai Add : Nat x N a i - , Nai Eq : Nat x Nai -+ Boolean For All iJ E Nai Let Eq(O.0) = True Eq(Suc(i),Suc(.A)= Eq(iJ3 Eq(O,Suc(r]) = False Add(i,O) = i Add(i,Suc(J?)= Suc(Add(iJ3) End Nat 0:

Suc : Nai

-+

-+

(Al)

(A3 (A3) (A3 (A6)

320


tent specification, but, unfortunately, the terminal algebra may not exist. This is more clearly seen with regard to our first construction for the terminal algebra. Suppose for some composite extractor W ( x )of type t i , and for some WFTs a and b of type T, W(a) (=) s,

W ( b ) (=>t

where s and c are primitive WFTs of type ti and s # 1 . Then, according to our definition of (#), a (#) b. Therefore, in the terminal algebra, O(a) # O(b). Now suppose there is a WFT c of type T and suppose there is no composite extractor W(x) such that both W ( c ) and W ( a )can be simultaneously reduced to primitive WFTs. In other words, suppose that c and a are “not comparable,” and further, that c and b are also “not comparable.” Then there is no way to prove either c (#) b or c (#) a. Therefore “c not ( Z ) a” and “c not ( # ) b.” Therefore, in the terminal algebra, O(a) = O(c) = O(b)

which contradicts our assumption that O(a) # O(b).This suggests that the terminal algebra is guaranteed to exist only if for every composite extractor W(x) of type ti, and every grounded WFT a of type T, W ( a ) can be reduced to a primitive WFT of type ti using the axioms. In other words, the extractor functions in any algebra (i.e., in any implementation) of the specification are forced to be total for the algebra to be consistent with the axioms. This condition is called “sufficient completeness” and we will say something more about it in Section 3.5.

3.5 Choosing the Best Model

The question as to what is the “best” choice for the meaning of an algebraic specification is still a subject of controversy. In fact, the initial algebra and the terminal algebra (when it exists) are not the only possible models of implementation for a given specification. There is a whole spectrum of algebras between these two extremes, which form a partial order under homomorphism and a lattice if the terminal algebra exists (Wirsing and Broy, 1980). The crucial determining factor is the “granularity” of the TO1 domain. Once that is determined, the rest is relatively straightforward. The constructor functions are usually taken to be total: The application of a constructor symbol results in a WFT of the TOI, which corresponds to some object in the TO1 domain, etc. The exception is, of course, error situations. For example, some authors prefer to treat Pop(New) as an error, others as a nonoperation. The treatment of error situations is a

ABSTRACT DATA TYPES

321

whole subject in itself (see, for instance, Goguen, 1977). The least fixed point principle can be applied to the extractor symbols; i.e., given a WFT F(x) where F is an extractor symbol and x is an appropriate list of parameter WFTs, the actual functionfcorresponding to the symbol F is defined at O(x) (i.e., the list of objects corresponding to the parameter WFTs) if and only i f F ( x )(=) s, where s is a primitive WFT of the appropriate type. And, of course,f(O(x)) = O(s).Informally,fis defined at O(x) if and only if the corresponding WFT can be reduced to a “value” in a finite number of inference steps, exactly as in our characterization of least fixed points. Finally, we look at some advantages and disadvantages of the initial and terminal algebras. Most importantly, the terminal algebra does not always exist, and when it does, it sometimes gives a trivial “implementation.” On the other hand, the initial algebra always exists and (almost) never gives a trivial implementation. For example, the initial algebra implementation of our original specification for natural numbers gives us the conventional algebra of natural numbers. However, we shall see that the initial algebra requires us to remember all possible information about the history of the construction of an object, except for the information that the axioms force us to forget. The terminal algebra, on the other hand, requires all information about the history of the construction of an object to be forgotten, except for the information that the axioms force us to remember. Each has its advantages and disailvantages, as the example in Table VII illustrates. This example also illustrates a new feature of our system of specification: It defines a parametrized type Set, which may be instantiated with any member type. This is TABLEVII PARAMETRIZED SETTYPE

Type Set[Item] DedPre Empryser : + Set Isempty : Ser + Boolean Insert : Set X Item 4 Set Has : Set X Item + Boolean For All s E Set, ij E Item Let Isempty(Emptyset) = True Isempty(lnsert(s,i]) = False Has(Emptyser,i] = False ffas(lnserr(s,i],j] = ff Eq(i,j] Then True Elm?Has(s,j] (and optionally the commutativity axiom for Insert) lnserr(lnserf(s,i],j~ = Insert(lnsert(s,j],i] End Set

322


exactly like the Set facility of Pascal, which is clearly a single parametrized type. Also note that we have assumed the availability of the special equivalence operator Eq for the type Item. In the absence of the commutativity axiom, the initial algebra remembers the order of insertion of the elements into the set. The terminal algebra does not, since the extractors (i.e., the membership test) cannot distinguish the “seniority” of one member from that of another. Thus the terminal algebra is more economical in the sense of retaining less information. For example, an implementation of sets (without the commutativity axiom), according to the terminal algebra semantics, could be a hash table, which does not remember “seniority.” The addition of the commutativity axiom changes the initial algebra (which is now required not to retain “seniority” information), but leaves the terminal algebra unchanged. Let us now extend the set specification, without the commutativity axiom, by adding a new operation Find, with syntax Find : Set

X

Nut + Item U {error}

and the laws

For All n E Nut Let Find(Emptyset,n) = error Find(Znsert(s,i),n)= If Eq(n,O) Then i Else Find(s,n - I) The terminal algebra is unable to accommodate this new operation, since Find depends on the order of insertion. But the initial algebra has no trouble, since it had already retained the necessary information. Thus the initial algebra is sensitive to changes in the equations between WFTs of the type being defined and the terminal algebra is sensitive to changes in the equations between WFTs of “lower” types (i.e., those with an extractor symbol at the outermost level). The terminal algebra is the most “economical” and the initial algebra the most “conservative” in retaining information. Some authors (Guttag et al., 1978a; Wirsing et al., 1980) prefer to leave the choice of model to the implementor and view the multiplicity of the possible (nonisomorphic) models as an asset in terms of flexibility. They feel that any restriction the specifier wants to see in the implementation should be explicitly axiomatized. This is perhaps the most practical view, though it may not be theoretically so neat. Note that in the axioms for Find we have used an undefined operation for Nut. The Find(s,n - 1) assumes subtraction or at least the predecessor function for type Nut. The latter can be specified as follows: Pred(Suc(i))= i, which is to hold for all members of Nat except 0.

ABSTRACTDATATYPES

4.

323

Consistency and Cornpteteness 4.1

Basic Concerns

At this point we turn to another major issue concerning algebraic specifications, namely, adequacy, which comprises consistency and completeness. The classical notion of consistency is that a theory is consistent if and only if it is impossible to derive a contradiction as one of its consequences. The classical form of a contradiction is

PA-P where P is any predicate. The traditional notion of completeness is that for any predicate P, either P or -P should be a consequence of the theory. In our system of specification, by restricting the axioms to be conditional equations, the only predicate symbol we have allowed is =. That is why we had trouble dealing with the notion a (#) b in defining terminal algebras. The other predicate symbols, such as A, v, -, etc., can be viewed as functions of the Boolean type, which produce values of the Boolean type as their result. To obtain something like the first-order predicate calculus, the Boolean type must be treated as a special or distinguished type, as in fact we do in allowing conditional equations, as well as in the construction involving the axiomatization of the equality predicate ( E q ) in Section 3. Of course, any such predicate has to ensure that for any WFTs a and b of the given type, CI (=) b implies Eq(a,b)(=) True. Once we treat the Boolean type as special (and given that we already have quantifiers, as in the For All -.-Let clause), we have the full range of firstorder predicates, and we can speak of consistency and completeness in the usual sense.

4.2 Consistency

When we speak of “implementing a specification,” what we mean is the implementation of an algebra that will pass muster as a meaning of the specification. Since an inconsistent specification would give rise to a “nondeterministic” implementation, it is obviously important to ensure that any specification we make is consistent. A consistency check would also be valuable as a means of catching inadvertent errors, somewhat in the same manner type checking does in programming languages. Unfortunately, the general problem of proving the consistency (or otherwise) of a

324

ALFS 1.BERZTISS AND SATISH THATTE

specificationis undecidable. Even in particular cases it is no easy matter, especially when the inconsistency is due to more than a typographical error. Take a simple case of inconsistency. Suppose we included both the commutativity axiom for Insert, and the Find operation in our specification for Set[Itern].It is not immediately obvious that the specification is now inconsistent, as in fact it is. Intuitively, the reason is that the commutativity axiom says that order of insertion does not matter, whereas the Find operation depends on this order. A simple formal derivation of inconsistent results from the same WFT follows: Find(Insert(Insert(Emptyset,1),2),0) = Find(Insert(Insert(Empty~ef,2), l),O) by commutativity of insert = 1 by the Find axiom

But, Find(Znsert(Znsert(Emptyset,1),2),0) =2

by the Find axiom

It is quite possible that in writing a specification for a new type, one makes contradictory demands on it in a way that is not immediately obvious. The most tractable method for proving the consistency of a theory is to demonstrate that there is a model for it. In our context, a specification is consistent if there is a “correct” implementation for it. Both Guttag and Homing (1978) and the ADJ group (Goguen et al., 1978) take this approach. The ADJ group do so rather indirectly. They talk of proving the ‘‘correctness of a specification” by demonstrating that the specification conforms to an already existing operative model or implementation. Their view is that a specification does not spring full blown from the head of the specifier, like Athena from the head of Zeus. It usually corresponds to the abstraction of the essential properties of an already existing and useful working system. Such a verification would not only show that the specification does not contradict itself, but also that it does not contradict the expected behavior of the system. Of course, this does not mean that the specification demands the expected behavior. For that we need the notion of completeness, to which we turn next. 4.3 Completeness 4.3.1

Logical Completeness

The concept of completeness has gained fame in particular since Goedel proved his “Incompleteness Theorem.” This theorem, in settling

ABSTRACTDATATYPES

325

accounts with the “Entscheidungsproblem,” showed how difficult (if not impossible) it is to give a complete axiomatization for a “sufficiently powerful” operative system. In our context, a complete specification would mean that all predicates, and in particular the equality predicate, concerning the TO1 are decidable or recursive. Any two WFTs a and b of the TO1 must therefore satisfy either a(=)b

or

a(#)b

which means that “(=)” and “not (#)” are the same equivalence relation, and consequently the initial and terminal algebras coincide. In other words, only one algebra (up to isomorphism) satisfies the specification, and this algebra may not even be finitely generated! Apart from the fact that it is difficult to give such a rigorous specification, it rhay not even be desirable. If the implementation is to be constrained to so narrow a range, we might as well give an abstract implementation in the Alphard manner, and save ourselves some trouble. One of the virtues of algebraic specifications is that they capture the essential properties of the type and still allow different implementations. Hence nobody really demands that a specification be complete. But, on the other hand, we would like to ensure that the specification does capture the essential properties of a type. In other words, a specification must be sufficiently restrictive to ensure that any implementation will exhibit the required behavior. There are many types (such as stack and queue, for example), which are “syntactically isomorphic,” i.e., their syntactic specifications are identical if we rename the domains and functions appropriately. We would certainly not like to specify stacks and accept queues as a valid implementation. 4.3.2 Sufficient Completeness

A weak form of completeness that is important for data type specifications has been popularized under the name “sufficient completeness” by Guttag and Homing (1978). As we mentioned before, sufficient completeness demands that the behavior of all extractors of the TO1 be completely specified, or, in other words, the axioms must ensure that the extractors in any implementation will be total functions. Formally, suppose we are defining a new type T. Let W be a grounded WFT of T with some extractor symbol F a t the outermost level. WFT W is then of type t i , where ti is the range of F. The specification for T is sufficiently complete if and only if for any such W, we can show that W (=) a, where a is a primitive grounded WFT of type ti . The a may, of course, be an error value, as in the range of the extractor Read in the specification for Istack. The use of

326


error values allows explicit specification of undefined situations, thus allowing us to speak of “total” functions. Guttag and Horning (1978) have thoroughly discussed the problem of deciding whether a particular specification is sufficiently complete. As they pointed out, the general problem is not decidable. In fact they showed a stronger result. Suppose S is a specification for some type T. A function R is called a semirecognition procedure if R(S)=True implies that S is sufficiently complete. (The qualifier “semi” is used because R(S)=False does not necessarily imply that S is not a sufficiently complete specification.) When R(S)=True, we say that S is “recognizably sufficiently complete” with respect to R. Suppose A is a data type algebra and we are trying to specify its behavior. If all the extractors of A are total functions, then we say that A is “acceptable.” Guttag and Homing showed that there is no recursive procedure R such that each acceptable A has at least one recognizably sufficiently complete specification with respect to R. This has to do with the fact that the problem of determining whether a function is total, given a finite description, is unsolvable. However, as one may expect, if all functions of A are primitive recursive, then the required semirecognition procedure R exists. Guttag and Homing went on to give syntactic constraints on the form of the axioms (essentially concerning levels of nesting of function applications) which guarantee sufficient completeness. For further details on the decision problem, the reader is referred to their article. Apart from (though related to) the aspect of the “adequacy” of a specification, sufficient completeness has some other theoretical implications. As we suggested in Section 4.3.2, sufficient completeness guarantees the existence of a terminal algebra. There is another subtler implication. Suppose there is a particular hierarchy of specifications, such as, say, the specificationfor stacks of natural numbers (Nstuck) along with the specifications for the necessary lower types, natural numbers (Nut) and Booleans. We have to ensure that the specification for Nstuck does not alter the range of possible implementations for, say, Nut. Consider the specification for Nstuck obtained by replacing Znteger by Nut in the specification of Istuck in Table 11. If the set {error}in the range of Read and the axiom Reud(New) = error were omitted, then Reud(New) would be a WFT of type Nut, which, however, could not be reduced to a primitive WFT of type Nut. It would be possible to “build on” this WFT and get Suc(Read(New))etc. One can have a hierarchy of algebras as a model (implementation)for the hierarchy of specifications,and in one such model Reud(New) may be regarded

ABSTRACTDATATYPES

327

as a “new” natural number, distinct from the usual ones. This “new” number would have its successors, etc. This model would not be valid given just the specification for Nut, because we insist on finitely generated models. The point is that sufficient completeness ensures that this kind of violence cannot be done ‘‘legally’’to the lower types, since all WFTs of each lower type can be reduced to primitive ones. 4.4 Adequacy and Nondeterminism

Enforcing the requirement that all specifications must be sufficiently complete, or even consistent, may not be as unmixed a blessing as one may suppose at first glance. These requirements may force overspecification in the “don’t care” or nondeterministic situations one encounters not infrequently in practice. Take, for example, the specification for arrays, given in Table VIII. This specification has been taken from Guttag et al. (1978a), with some changes in notation for uniformity. The axiom Fetch(Newarr,n,)= error

forces any implementation of arrays to perform initialization checking at run time. That is, the usual sort of “quick and dirty” implementation, which is based on the premise that if you fetch from an uninitialized location, then you deserve whatever you may happen to get, just will not do. An error message, corresponding to the “error” value, must be produced. To allow for a “don’t care” attitude, one must drop this axiom; but then the specification is not sufficiently complete! In this case, it is clear that we are not out to create new instances of the Item type. We merely want to say that any instance of Item will do as the “value” of TABLEVllI ARRAYSPECIFICATION

Type Array[lfem] Declare Newarr : -+ Array Srore : Array x Nat X Ifem Array Fefch : Array X N a f -+ Item u {error} For AU a E Array, nl ,n2 E N a f , i E Item Let Fefch(Newarr,nl)= error Fetch(Store(a,nI,i),n*) = If Eq(nl ,nz) Tben i Else Fetch(a,n2) End Array -+

328


Fetch(Newarr,i). We can circumvent the problem of “new” values of lower types by stipulating that the correctness of a model for any type t will be judged only in relation to the specification for I (and those for the types t depends on), and not in relation to a whole hierarchy in which I plays a subordinate role. This is in the spirit of the principle of referential transparency, which demands context independence. Since we do not insist on the terminal algebra as the only correct implementation, we need not bother about the nonexistence of the terminal algebra in this case. This sort of nondeterminism can arise even in situations which do not involve an error condition, as we see in the next example. Consider the specification for sets, given in Table V, including the commutativity axiom for insertion, which expresses a basic property of sets. Now suppose we wanted to add the operation Some(s) which returns any arbitrary member of the set s. The most natural way to specify the operation would be to add Some :Set + Item Some(Insert(s,i))= i

It is easy to see that Some(s) is the same operation as Find(s,O), where Find is the operation by which the specificationfor sets was augmented in Section 3.5. Just as the addition of Find makes the specification(including the commutativity axiom) inconsistent, the addition of Some also makes it inconsistent, because we can prove that any member of a set s is a valid “value” for Some(s). But, unlike in the case of Find, this is just what we intended ! This sort of situation is not far fetched, even from a practical point of view. Consider, for example, the suggestion of Shaw et al. (1977), that one should have an iterative construct for sets in which one can say, for a set s, For x in s do Z

The intended semantics for this construct is that one must process each item in s exactly once, but in any order, which is a nondeterministic situation rather similar to our example. Another example concerns the “random polling” of several input ports, as in the case of a terminal controller, say. The collection of input ports may be thought of as a set s, and the operation Sorne(s) randomly selects a member for polling. How one can handle this sort of nondeterministic situation within the framework of algebraic specifications is not clear. One idea may be to flag the nondeterministic operators and leave them out of the specification when determining consistency. Some issues along these lines have been discussed by Subrahmanyam (1979).

ABSTRACTDATATYPES

5.

329

Implementation and Verification

Most of the material in this section is based on the work of Guttag et al. (1978a). This article remains an invaluable source reference. 5.1 An Implementation Example

Consider the implementation for Istack given in Table IX.The implementation uses two objects, an array of integers and a natural number, to represent an Istack object. Note that the dependence of Istack on the types Array and Nut in the implementation is different from the dependence of Istack on the integer type in the specification. The type Israck cannot be conceived of without the integer type. It can, however, be implemented without the array type, using a type List for example. The function Stak is a representation function which converts an array of integers and a natural number into a stack of integers. Apart from acting as an interface to make the implementation conform to the syntactic part of the specification, it helps us get around difficulties in proofs, as we will see in the discussion on verification. In general, when implementing a type T, using objects of types tl , t 2 , ..., t , ,one needs a representation function, say, Rep, with the functionality Rep: t l

X

t2 x

x t , -+ T

It is noteworthy that if Srak is added to the list of functions in the syntactic specification for Istack, then the function definitions given in the implementation can serve as a clumsy sort of semantic restrictions part of a “specification” of Israck. The point is that the language of specification and the language of implementation are the same. This gives rise to the TABLEIX ISTACK IMPLEMENTATION Implementation For Israck Representation Sfak(Array[lnfeger],Nar) F M C ~ Definitions ~O~ For AU a E Array, n E Naf, i E Integer Let New = Stak(Newarr,O) Push(Slak(a,n),i) = Stak(Srore(a,n+ I A n + 1) Pop(Sfak(a,n)) = If Eq(n,O) Then Stak(a.0) Else Sfak(a,n- 1) Read(Srak(a,n)) = Fetchfa+) Empty(Stak(a,n)) = Eq(n.0) End Implementation For Istack

330


question, Why can we not use the axioms in the specification itself as function definitions directly, to obtain an “automatic” implementation? In fact, given some restrictions on the form of the axioms, this is possible, as the work reported by Guttag et al. (1978a) and Moitra (1979) shows. The basic problem is that the kind of axioms we allow are too general to be used as simple rewriting rules. For example, the arbitrary level of nesting on both the left- and right-hand sides of the axioms necessitates complex pattern matching to determine applicability. Commutativity axioms can be used for rewriting only under heuristic guidance for fear of looping indefinitely, etc. The “axioms” in the implementation have the virtue that they have a very restricted form. If we ignore the syntactic device of the representation function Stak and treat Stak(a,n) as just an arbitrary instance of Zsrack, then there are no nested function applications on the left-hand sides of these axioms. In other words, the axiom implementing a function F can be used as a rewrite rule in any situation where the WFT concerned has F at the outermost level. Moreover, there is exactly one “axiom” relating to the implementation of F, and it is therefore necessarily the only rule that can be applied. In fact, the function definitions in this type of implementation are exactly like the function definitions (recursive equations without side effects) used in pure applicative languages. The extensive discussion on substitution (rewriting) strategies in Manna et al. (1973) is therefore directly applicable to our function definitions. To verify that such an implementation is “correct,” one must show that the function definitions in the implementation, together with the specifications of the representation types, imply the validity of the axioms in the specification for the type being implemented. Note that the implementations of the representation types are not involved at all, which makes for nice modularity in the proofs. We shall now take a relatively cursory look at the verification of the implementation given here. 5.2 Verifying the Implementation

Even the verification of an implementation for a simple type such as Zstack brings out many of the problems one confronts in the general case. The verification of the axioms for the functions Read and Empty is easy enough, as the following shows.

To verify Read(New) = error: Read(Ne w ) = Read(Stak(Newarr,O)) by New implementation

ABSTRACT DATA TYPES = =

331

by Read implementation by axiom for Array

Fetch(Newarr,O) error

To verify Read(Push(s,i)) = i: Read(Push(s, i)) = Read(Push( Stak( a ,n ),i)) =

Read(Stuk(Store(a,n+1,i),n+ I))

=

Fetch(Store(a,n+ 1 ,i),n+ I )

= I

by Istack representation invariant (see later) by Push implementation by Read implementation by axiom for Array

To verify Empty(New) = True: Empty( New) = Empty(Stak(Newarr,O)) by New implementation = Eq(0,O) by Empty implementation by axiom for Nut = True

To verify Empty(Push(s,i))= False: Empty(Push(s,i)) = Emp ty (Push(Sta k( a ,n ),i))

by Istack representation invariant (see later) = Empty(Stak(Store(a,n+I ,i),n+ 1)) by Push implementation = Eq(n+ 1,O) by Empty implementation as can be easily shown = False from Nut specification

The verification of Pop(New) = Pop(New) = Pop(Stak(Newarr,O)) = Stuk(Newarr,O) = New

New is also straightforward. We have

by New implementation by Pop implementation by New implementation

However, when we try our hand at verifying Pop(Push(s,i))= s, we run into difficulties. The following steps are straightforward: Pop(Push(s,i)) = Pop(Push(Stak(a,n),i))

by Istack representation invariant (see later)

332


= Pop(Stak(Store(a,n+1,i),n+ 1)) = Stak(Store(a,n+1,i),n)

by Push implementation by Pop implementation using the fact that Eq(n+ 1,O) (=) False

The last term, Stak(Store(a,n+ 1,i),n), cannot be immediately reduced to Stuk(a,n) for the obvious reason that the array Store(a,n+ 1,i) is not the same as the array a. The problem arises because the axiom Pop(Push(s,i)) = s

wants Pop to “undo” the previous Push (if any) completely. The implementation, for efficiency, makes only the essential change by lowering the stack pointer one notch. This is the only essential change, because store operations at locations greater than the stack pointer can never be detected by the extractors of Istuck in any way. Hence, though we can never prove that Store(a,n+l,i)= a , we may contrive to prove that Stak(Store(a,n+1A n ) = Stak(a,n)

because the presence of Stak at the outermost level implies that the argument pair of Stak is not a pair of arbitrary objects, but a representation of an Istack object, whose present state has been built up as a result of successive applications of Istuck constructors on the Istack generator. Moreover, information from the pair can be extracted only by Istuck extractors. Apart from its “cosmetic” role, therefore, Stak also allows us to make stronger statements about its argument pair than we could make in general. Before proceeding to indicate the method of proof we need, a conceptual point needs to be cleared up regarding what we are trying to prove. We said that we are trying to demonstrate the validity of the Istack axioms. The axiom Pop(Push(s,i)) = s

states an equivalence relationship between Istack objects, i.e., between objects of the type being specified. Since we do not insist on the initial algebra or the terminal algebra semantics, all we are concerned with is that the implementation should be at least “as good as” the terminal algebra. Recalling our characterization of terminal algebras, this means that any parametrized WFT W(x),of type Znteger or Boolean, must have the same value when either of two supposedly equivalent Zstack objects sI and s2 is substituted for x. Therefore what we have to show is that Stak(Store(a,n+1,g,n) and Stak(a,n) are equivalent objects in this sense. At this point, a digression is necessary to discuss a very basic proof

ABSTRACTDATATYPES

333

technique, which we will use to show the above result. This technique has been variously called generator induction (Spitzen and Wegbreit, 1975; Wegbreit and Spitzen, 1976) and data type inducrion (Guttag et al., 1978a). Following Burstall (19691, we prefer to call it structural induction. The technique is a generalization of the familiar induction principle for natural numbers. Given a specification for any type T, with the function symbols classified into generators, constructors, and extractors in the usual manner, the method provides a way to prove the validity of a predicate P for all grounded WFTs of type T. The basis step is to prove P(G) for each generator G. Suppose the arguments of each constructor F are partitioned into arguments XI , xz ,..., xnFof type T, and arguments y l , y2 ,..., ymF of lower types. The induction step requires us to prove that P ( x , )A P ( x , )A

..- A P(x,,)

implies P M x i ,..., x R F , Y I,..., y m F ) )

Informally, the induction step requires that if P is true for the TO1 arguments of any constructor F , then it is true for the new TO1 value constructed by F. Thus the basis step ensures that P is true for the starting values of T, and the induction step ensures that it is true for any value of T that corresponds to a finite number of applications of constructor functions to the starting values. Of course, the principle applies only if one restricts oneself to finitely generated models, as we do. Actually, the set of constructors can be further subdivided into essential and nonessential constructors. Recall the (=) equivalence relation we defined when discussing initial algebras. This relation partitions the set of all TO1 WFTs into classes of prouably equivalent WFTs. Since the WFTs of a class are all provably equivalent, any one of them can act as a representative way of building up the (single)TO1 object correspondingto the class. The set of such representatives may be called the set of canonical WFTs. If we can contrive to choose these representatives in such a way that only some of the constructors (the smallest possible number) are present in the set of canonical WFTs, then it is clear that any TO1 object can be built up using these essential constructors alone, and any TO1 WFT involving any other constructor can be shown to be provably equivalent to a WFT involving only the essential constructors. For example, Push is the only essential constructor in the type Istack. Note that the set of essential constructors need not be unique. If we can obtain one such set, then the induction step in a proof by structural induction needs to be verified only for the essential constructors, which may save a lot of effort.

334


As an aside, notice that according to this criterion, Suc is the only essential constructor for Nut. Thus the application of the structural induction principle to the type Nut gives precisely the conventional induction principle for natural numbers. Returning to our problem, we want to show that Stak(Store(a,n+ l,i),n) and Stak(a,n) are equivalent. This is easier if we first show a somewhat stronger result. Paraphrasing Guttag et al. (1978a), we define an equivalence condition for Istack representations as follows: Stak(a,n) == Stak(b,rn) iff Eq(n,rn) Zeq(Fetch(a,i),Fetch(b,i)), 1 Ii 5 n. Here == stands for “is equivalent to,” and Eq and leq are equality operators for the Nut and Integer data types, respectively. Similarly, Be4 (used later on) is the equality operator for Booleans. Now suppose that sI and s2 are representations of two arbitrary Istack objects. If we can show that SI = = s2 implies Push(sl,i) = = Push(s2,i) (1) implies Pop(s1) == Pop(s2) (2) then, using a slightly modified version of the structural induction principle, we may claim that SI

== s2

implies W ( s l ) == W(s2) (3) for any parameterized WFT W ( x ) ,where both Wand x are of type Istack, and x is a direct argument of a constructor. We may make this claim because we have now shown that all the constructors of Istack preserve the = = relation. Note here that the basis step may involve nonessential constructors, which is the reason why we could not restrict ourselves to the essential constructor Push alone. If we further prove that 31 == s2

s1 = = s2

implies leq(Read(s1),Read(s2))

(4)

implies Beq(Ernpty(s1 ),Empty(s2)) (5) then we have established that sI = = s2 implies the equality of U(sl) and Ll(s2)for any parametrized WET U of type Integer or Boolean. This is so because such a WFT U(si)must consist of the application of an extractor F to a WFT W(si),where W is a WFT relating to Zstack of the kind used in (3). It is easy to show that (l), (2), (4), and (5) are valid, and we will not go into the details here. To obtain our final result, it only remains to show that Stak(Store(a,n+ 1,i),n) = = Stak(a,n) S I = = s2

ABSTRACTDATATYPES

335

which is obvious. Thus we have completed the verification of the axiom Push(s,i)) = s . Finally, we shall say a word about representation invariants. The proposition that, given a WFT s of type Istack, there exist an array a and a natural number n such that Stak(a,n)is a representation of s is assumed in all our verifications. This proposition is the representation inuariant. Actually, the representation invariant cannot be assumed. It has to be proved, usually by means of structural induction. In our example, as the reader can easily see, the proof is trivial. All these details may seem rather tedious for such a simple result, and it may appear that these proof techniques are in danger of making themselves inaccessible for normal use. But it should be remembered that many of the steps in the verification process can be easily automated, given their similarity to recursive computation, as described by, for example, Guttag et al. (1978a). The only “creative” step was the definition of the equivalence condition. Given the range of implementations permitted, the definition of such a condition for a particular implementation is bound to require a certain amount of ingenuity. 6.

Problems Associated with Data Abstraction 6.1 The Traversible Stack Syndrome

In the back of every formal specification there is an informal specification, i.e., a more or less vague expression of the programmer’s intent. The question of whether it is always possible to translate an intuitively clearly understood intended behavior of a data type into an algebraic specification is crucial. Unfortunately it has been shown that algebraic specification presents difficulties for some rather simple data types. A well-known example of a difficult data type is Majster’s traversible stack (Majster, 1977). It is similar to Istack of Table 11, except that it is provided with a hidden pointer. There are two pointer operations: Down, which moves the pointer down one element in the stack, and Return, which restores the pointer to the top of the stack. Both these operations have the stack for domain and range (hence we speak of a hidden pointer). When the pointer is at the top of the stack, then the nonpointer operations behave exactly as Israck operations. When it is not, then Read returns the element that is being pointed to, and Push and Pop both result in exceptional conditions (errors). Actually the algebraic approach can be used to specify the traversible stack, but one either has to use hidden auxiliary functions or infinitely

336


many axioms. Neither solution is esthetically pleasing. The traversible stack generated a lively correspondence in ACM SZGPLAN Notices [see, for example, Kapur (1979)l. The importance of this example lies in showing that the writing of algebraic specifications can be very difficult. It seems that algebraic specifications are convenient only as long as one is content with access to data at exposed boundary points of a data structure. The specification problem has been discussed in highly theoretical terms by a number of authors. Majster (1979) has surveyed the relevant literature and carried out a very general investigation of this problem.

6.2 Functional a n d Procedural Programming The algebraic specification technique goes better with functional than with procedural programming, but in the near future functional programming will probably not achieve much wider popularity than it already has. Another factor of relevance is whether we are better attuned to thinking in terms of equations or computational procedures. Two studies suggest the latter (Clement er al., 1980; Welty and Stemple, 1981). Whether this is a matter of training or a fundamental trait remains to be seen. Functional programs are not necessarily all that transparent. Consider a program that accepts a binary tree T of integers as specified in Section 2.3 (Type Zbin of Table IV augmented with Inord). It is assumed that the tree is a search tree, i.e., that for every node in the tree, the value held at the node is greater than that held by its left successor (ifit exists) and smaller than that held by its right successor (if it exists). It is required to scale the data that the tree carries by subtracting the smallest integer from every integer in the tree, including itself, so making the smallest value zero. Using a rather self-explanatory syntax, the program is Scale(b,T)is if not Empty(b) then Make (Scale(Left(b),T), Da?a(b) - Qread(Znord(T)), Scale(Right(b),T ) ) else b; The confusing part about this example is that the function specification describes what looks like inorder traversal, but we cannot use Znord because that would produce a queue from which one could not reconstruct the binary tree T . This is why we have to recursively dismantle the tree only to build it up again. On the other hand, Znord is invoked every time one needs the scaling value. Of course, this does not mean that every time the Qread(Znord(T))is reached, the entire inorder sequence is com-

ABSTRACTDATATYPES

337

puted. Lazy evaluation (Henderson and Morris, 1976)allows this problem to be dealt with reasonably efficiently. Nevertheless, it seems unnatural to have to provide Scale with two arguments, only because fnord gets invoked. A really difficult problem for which to write a purely functional program is one in which the nodes of a binary tree are visited under inorder and scaled as follows: Subtract 1 at the first node visited, subtract 2 at the second node, 3 at the third, and so forth. A procedural program that is still well protected from programmer errors would be much easier to comprehend. The straining for purify has been motivated by the need to have a sound base for automatization of program proofs. It now appears that the initial optimism regarding automatic program proving may have been premature. Difficulties are arising from an unexpected quarter, namely, complexity studies. Suppose we regard a functional program as a specification and derive from it an “efficient” procedural program by means of correctness-preserving transformations. It is not at all clear how far one can go with such transformations; the NP-completeness of program equivalence in a very simple programming language investigated by Gurari and Ibarra (1981) suggests need for caution. The use of the first-order predicate calculus as a specification language is not without problems either. London (1977) pointed out that assertions in this language may be more complex than the program. This is not in itself a bad thing, because under approaches that use the first-order calculus the formal specification and the working program are developed from an informal specification independently, and one then has the safeguard that an error in either formulation should show up in the subsequent consistency check. By contrast, the transformational approach generates a correct program, but this may well be the correct program for the wrong task. The complexity of the first-order specification would seem an acceptable price for the added protection it provides. Unfortunately some results in data base theory imply that the first-order predicate calculus is insufficient for the specification of fairly straightforward programs (Aho and Ullman, 1979). Also, Jones and Muchnick (1981) have showed that the length of a proof may be exponential in the size of the program, i.e., that program proof is NP-hard. It is not our aim to discuss program verification in detail. For that we refer the reader to surveys by London (1977, 1979). We have merely put down a few jottings, but they still serve an important purpose. They indicate that one or another.specific approach to program proving will probably not become dominant, and that, at least in the short term, a realistic goal of programming language development will be to combine

338


functional and procedural approaches. The work of Sokolowski (1980) is in this vein. Also, Manna and Waldinger (1981) have devised a new methodology for dealing with assignments, which lends a firm base for such a combined approach.

6.3 Synchronization Problems

If an attempt is to be made to combine functional and procedural programming, it seems that the best start would be to look at the introduction of algebraic specifications into a procedural language. The fundamental question is how much is gained by doing so. We shall return to this general question in Section 8. For the time being, however, let us look at one particular issue, the synchronization of processes. This is far from straightforward, as the following examples show. One algorithm for the strong components of a digraph takes a tree representation of the digraph and subjects the tree to intermeshed preorder and postorder traversals (Berztiss, 1980a). To start, preorder traversal is carried out until a terminal node is reached, then postorder is taken up and continued until a node is reached that has not yet been visited under preorder, at which point preorder traversal is resumed and carried out until again a terminal node is visited, and so forth. Taking an abstract view, we can regard the algorithm as three processes: the preorder traversal, the postorder traversal, and a computational process. The latter also determines the points at which to switch from one traversal to the other. In the interests of modularity this last process should be independent of the first two. The approach of Guttag et al. (1978b) is to interpret traversals as queues. In this instance such an interpretation would create considerable difficulties for the programmer in both functional and procedural settings. One solution is to regard the intermeshed traversals as a single traversal under which each node is visited twice, and to incorporate the composite traversal in the data type of tree. This is rather unsatisfactory, as we have argued before (Berztiss, 1980b).The places at which a switch is made from one to the other traversal are determined by one specific algorithm, and it seems only proper that we should disallow problem-specific operations as basic functions of our abstract data types. Let us look at another example in which simultaneous traversals are made of two structures. This can arise when the words of a sentence are stored at the terminal nodes of a derivation tree, and it is required to compare this sentence against another, which is stored in a second derivation tree. Although the two trees might differ in appearance, they could

ABSTRACTDATATYPES

339

still represent the same (ambiguous) sentence. Traversals of the two trees would be called for, and we should break off the traversals as soon as dissimilarity is established. In a functional setting one would build queues of the words of the two sentences, and under lazy evaluation the queues would contain only as much data as is required by the comparison process, just one word in each queue. If the words from the two queues were found to differ, then the process would be broken off without any superfluous computational effort having been expended. Suppose now that the comparison program were rewritten as a procedure with the two queues as inputs. The advantage given by lazy evaluation would be lost. Consequently it is essential to find some other means for synchronizing the access to the data in the trees with the comparison process. 7. A Practical Approach to Data Abstraction

In Section 6 we established that one of the inconveniences of algebraic specification is the need to completely dismantle and rebuild a complicated data structure merely to gain access to a datum that is not on the “boundary” of the data structure. We also found that traversals of data structures are very important, and that the synchronization of traversals with other computing activities may be difficult. However, our greatest worry is the difficulty of writing algebraic specifications in general. Sections 7.1-7.3 deal with these three concerns. 7.1 Data Primitives and Data Structures

We contend that there is a significant difference between primitive data types, such as integers o r complex numbers, and composite data structures. The primitive data types exist in and for themselves. A composite data structure, on the other hand, has two distinct guises: It acts as a carrier of data and it represents by its structure a relationship between its constituent elements. By dealing with the two in the same way, conventional algebraic approaches to data abstraction fail to provide a natural means of access to the data being carried, access being more or less restricted to some exposed or boundary points (e.g., the head and tail in the case of a linear list). Hence in order to reach a datum that is not on the boundary, the structure has to be reduced until the location of the datum becomes a boundary point. Afterward the structure has to be restored back to its original form. Such wholesale restructuring, even if it be only at a conceptual level, appears unnatural.

340


Another unpleasant consequence of the total integration of the datacarrying function with the structure is the loss of some viewpoints that are essential for the full appreciation of a data structure. For example, the binary tree is well understood in graph theory, and in that context the concept of a node has fundamental importance. Yet the specification of Zbin (Table IV, Section 2.3) makes no mention of nodes. Indeed, the only way we can have binary trees in the sense of the specification of Table IV is as carriers of data, which in the case of Zbin are integers. To ensure homogeneity, function Make permanently associates an object of the type Znteger with the root of the binary tree that it creates; more precisely, the association is with the whole binary tree. In Section 7.2 we shall consider traversals of structures by means of iterators. Any practicable data structuring facility must provide traversals, and we want traversals implemented as iterators. We see as the main purpose of an iterator the accessing of data carried by a composite structure that are not on the boundary of this structure, but we want to go further and allow changes to values of the data elements so accessed. Of course, much of the worth of algebraic specifications derives precisely from the fact that the binary trees before and after a change in a data value are totally different objects, and that the only way of deriving one from the other is by application of appropriate functions combined by functional composition in a well-understood way. It seems, however, that the only hope for the introduction of algebraically specified data structures into procedural languages lies in a departure from this orthodoxy. Structural concerns and the data-carrying capability can indeed be kept separate if any changes made to a data object subsequent to its introduction into the binary tree by means of a Make are assumed not to affect the identity of the binary tree with which the data object is associated. This means that changes to the data associated with the nodes of a binary tree visited under a traversal do not alter the tree with respect to its algebraic specification; i.e., we would say that we are traversing the same binary tree even when the data carried by the tree change during the traversal. We emphasize strongly that such an interpretation loses all validity if structural changes are made as well.

7.2 Iterators Iterators are provided by a number of modern procedural programming languages (Hanson et al., 1979). Execution of a procedure that invokes an iterator is intermeshed with execution of the iterator. When the invoking procedure requests the iterator to deliver a datum, the latter proceeds to generate this datum and suspends operation at the place at which it is

ABSTRACTDATATYPES

341

ready to deliver the datum. Operation is resumed from this same place on the next invocation of the iterator. Iterators are therefore a simple type of coroutine (Atkinson et al., 1978). Let us define an iterator, using Ada-like syntax (suitably extended). The iterator makes use of functions of type Ibin (Table IV), but not in an applicative manner. We leave the justification of such an approach for later. Our iterator is called Inorder, and the place at which Inorder suspends operation is the deliver statement:

iterator Inorder(b) delivers Integer; begin if not Empty(Left(b))then Inorder(left(b))end if; deliver Data@); if not Empty(Right(b))then Inorder(Right(b))end if; end; In terms of Inorder, the code equivalent to function Scale of Section 6.2 is

if not Empty(T) then for x (Val: Integer) in TJnorder loop adjustment: = x; exit; end loop; for x (Val: Integer) in T h o r d e r loop x : = x - adjustment; end loop; end i c Although this program is longer and not as graceful as function Scale, it is in better accord with present-day realities. The most important difference is that the iterator provides access to the nodes of which the structure T is composed. It is these nodes, which here are of type Integer, that the iteration variable x returns. We want to emphasize very strongly that we permit changes to these integer values, but that we would not permit the structure of T to be changed during the iteration. The feature of procedural languages that is most detrimental to program verification is assignment, and here, by interpreting T to be the same object before and after the second traversal, we avoid the adverse effects of assignment. Moreover, with iterators expressed in terms of operations for which algebraic specifications are given, the specification of the iterators is precise, and their independence from implementation concerns is fully preserved. The remaining problem is how to synchronize iterators, a necessity for the strong component algorithm described in Section 6.3. We have pro-

342


posed an extension of the conventional for loop for this purpose (Berztiss, 1980b), which we call controlled iteration. A sketch of its features follows. Under controlled iteration the loop statement generally identifies several iteration processes, some of which may be controlled and some uncontrolled. All these processes are initialized on entering the loop. However, at the start of a subsequent iteration through the loop, only those controlled iteration sequences are advanced for which an explicit authorization to this effect has been issued during the current iteration. An uncontrolled iterator is advanced automatically. Exit from the loop occurs when all iteration processes have completed or it is effected by an explicit exit. In the strong component algorithm both the preorder and postorder traversals would be controlled, and in each iteration through the loop an authorization for advancement would be issued for precisely one of the two iterators. 7.3 Standard Data Structures 7.3.1 Bases

for Selection

Although at first it was believed that applications programmers would write algebraic specifications as the need arose, the current attitude is that the writing of such specifications is best left to experts (Guttag, 1980). Sizable libraries of data structures could thus be built up. The library idea would work well were it not for the practical matter that potential users of the library could be rather unsophisticated. An investigation of the consequences of this has led us to propose a standardized data model comprising a limited set of predefined data structures that can be easily manipulated (and extended) by a variety of users (Berztiss and Ford, 1982). Arguments can be advanced for the same kind of restraint in the creation of data structures that is readily accepted in the case of control structures. First, construction of algebraic specifications is a very difficult task. Therefore the creation of a new specification for each variety of a particular main type, say, the linear list or the binary tree, would be an inefficient use of the expert’s time. Moreover, such proliferation would lead to a library that would overwhelm the applications programmer with its size. Second, the abstraction should provide a buffer between the programmer and implementations of the structures, which means that mappings from abstractions to implementations must be provided. Although, as we noted in Section 5.1, in some instances it is possible to go automatically from the axioms of the specification to an implementation,

ABSTRACTDATATYPES

343

automatization of the implementation selection is not feasible, at least for the near future, when the implementation is to be in a procedural setting. Hence the implementations have to be handcrafted, which can be a difficult and lengthy task. Third, the rise of distributed systems is making the transmission of data structures an important concern. It is being argued (Liskov, 1980) that transmission of storage representations is unsuitable, that “abstract values” should be transmitted instead, and that the transmission of the abstract values should be under user control. Again, many programmers would find the exercise of such control a knotty proposition. Standard transmission protocols should be provided as part of the specifications, and again this is practicable only when the set of basic data structures is limited. Although the selection of the standard set involves somewhat arbitrary decisions, a few selection principles can nevertheless be established. First we appeal to mathematics. Set theory includes three objects of fundamental importance: the set, the sequence (an n-tuple, which is a member of an n-ary relation), and the binary relation in a set (which is best represented by a digraph). Data structures must be provided to cater for these objects. Second is accessing, where we distinguish four basic mechanisms: access in arbitrary order, access to an element identified by a key, ordered access specified by an iterator, and access to elements in a restricted sense, as in a queue or a stack. Third, there is data organization. Consideration of this indicates need of a structure in which objects of different types are brought togethzr. This is the record. The selection of our standard classes of data structures has been based on these principles. In addition, we have examined numerous algorithms drawn from many different application areas and found our model adequate for their implementation; nevertheless, we regard our selection as neither exhaustive nor final. 7.3.2 A Representative Set

The first step in the development of a standardized data model is the selection of the structures that are to be included in the model and the definition of the operations that are to define the structures. An abstraction model that we have found convenient for most purposes comprises five classes of data structures: collection, record, array, linear list, and tree. Collections. The essential property of the set as a mathematical object is absence of order, and this makes the set an ideal structure for contexts

344


in which potential for parallelism is to be indicated. On the other hand, efficient implementation of set operations, such as union and intersection, requires ordering of elements. This basic contradiction suggests that the mathematical object set should have two realizations by data structures, depending on whether order is or is not wanted. The corresponding data structures are the linear list and a structure we have called collection. The collection allows for nondeterministic access to its elements, and an example of how this can be used to allow for parallelism of execution will be found in (Berztiss, 1981). The operations of a collection are addition and deletion of an element, membership determination, and iteration over the elements of the collection. Note the absence of the usual set operations, such as union and intersection. Our intent is that these operations be built up using the basic operations of the linear list. Records. Records are provided as a basic type in many programming languages. Hence they are well known and there is no need for a discussion of this class here. Arrays. An array is a structure whose most prominent operation is immediate access to an element through a key, which is a k-tuple in the case of a k-dimensional array. The components of the k-tuple are conventionally called subscripts. The generality of our interpretation of subscripts identifies an array as a special case of content-addressable memory and allows the class of arrays to subsume direct access files, hash tables, and the like. All of these structures provide direct access to their elements; their differences are matters of implementation or interpretation. Linear lists. Though it might seem that linear arrays provide all the facilities desired of a linear structure, this is not so because the linear array must generalize to a multidimensional array. Therefore features based specifically on linearity cannot be supported. For example, a restricted access device such as the stack or queue is meaningless in a multidimensional setting. Trees. This is a parametrized general class in which a parameter indicates the possible successors of a node in the tree. This parameter is a set of n labels used to distinguish n possible sucessors for each node in the tree, e.g., {left, right} for the binary tree. Note that trees suffice to represent digraphs (Berztiss, 1980a). To summarize, our five classes of standard abstract data structures fall quite naturally into three categories, characterized by a prevalent access mode for each category. Collections alone belong to the first category, distinguished by nondeterminism of element access. For the second category, which contains records and arrays, access is mainly by specification

ABSTRACTDATATYPES

345

of a key. Records differ from arrays by having heterogeneous elements (fields), the only class for which nonhomogeneity is supported. The third category comprises lists and trees, and structural traversal is the primary access mode for this class. We have not listed the operations that we believe should be provided for each of the five types; these details can be found elsewhere (Berztiss and Ford, 1982). 7.3.3 Variants of the Linear List

The methodology for algebraic specification of data structures that we describe in this section has been developed by Ford (1980; Berztiss and Ford, 1982). It consists of three stages, which we explain by considering a specific data structure, the linear list. A linear list must minimally provide operations for addition and deletion of elements at each end, for retrieving these elements, for iterations over the list in both forward and backward directions, and for addition, retrieval, or deletion of an element at a point in the middle of the list. The first stage of Ford’s methodology is to provide algebraic specifications for a basic list model that is powerful enough to permit definition of the operations we require, but it need not necessarily provide any of them as primitives. The purpose of keeping the basic model as concise as possible is to simplify the formal determination of its consistency and completeness. This determination is the second stage. Finally user operation are predefined in terms of the primitives. Now if this data-structuring facility is embedded in an existing programming language, such as Pascal or Fortran, the data structure operations are already encapsulated as supplied functions, and their semantics are rooted in the primitive abstract operations. A verification system for a particular host language could then be built by combining assertions about the operations with aspects of the host language (see, e.g., Guttag and Horning, 1980). Ford’s primitive specification of the linear list is given in Table X. What distinguishes it from other specifications of linear lists is its sparseness. Indeed, comparison of Tables X and I1 will show that the specification is that of a stack, and an abbreviated one at that. Most notable is the absence of any high-level operations, such as insertion in the middle of a list. Such operations are difficult to specify, as we saw in the relatively simple case of the traversible stack (Section 6. l), and the resulting specifications are difficult to understand and to reason about. Besides aiding us in reasoning about the specifications, the sparseness of the model has two other advantages. First, it permits an incremental development of user operations. These operations, which are defined in terms of the primitives, can be added as required at different times without affecting the

346


TABLEX

PRIMITIVELISTSPECIFICATION ~~

Type Lisf(ltem1 Declare Lnew ; +List Lbuild; List x Item+List Lnext ;List +List Lread: List +Item +Boolean Lempty :List For All L E List, i E Item Let Lempty(Lnew) = True Lempty(Lbuild(L,i)) = False Lnext(Lbuild(L,i))= L Lread(Lbuild(L,i))= i End List

soundness of the primitive specification. Second, in contrast to the approach of Table 11, the functions in the specification of List are partial. Exceptions corresponding to the Pop(New) and Read(New) of Table I1 have been disregarded in the specification of List. The view taken here is that exceptions should be trapped by proper checks built into the definitions of the user operations. Since the latter are predefined, full user protection is still being provided. Ford (1980) also provided a novel approach to the proof of sufficient completeness and consistency of the specification. Rather than attempting to show that the axioms describe every WFT allowed by the syntax, sufficient completeness is established for a limited subset of such WFTs, which is regarded as the set of canonical WFTs of the larger set of possibilities. The canonical WFTs comprise set C*, which is a subset of the set of all WFTs U*.In defining C*, we start with the set L*, which contains compositions of Lnew and Lbuild alone. Three other sets of canonical WFTs can now be described in terms of L*: (I) E*, consisting of all Lempty(L) such that L E L* (2) R * , consisting of all Lread(L) such that L E L*, and LemptyfL) = False ( 3 ) N*, consisting of all Lnext(L) such that L E L", and Lempty(L) = False

Restricted language C* is the union of L*, E*, R*, and N*, and Ford X are sufficiently complete and consistent with respect to language C*.In addition, it is required that the (1980) proved that the axioms of Table

ABSTRACTDATATYPES

347

axioms be safe with respect to the WFTs in N*,in the sense that the object corresponding to a WFT in N* belongs to the set described by L* itself. Safeness is related to closure, and it must be satisfied in order to have a sound underlying model for the definition of user operations. Table XI contains the user operations that are being provided for the data type List. The symbol L indicates an arbitrary member of L*, and i (or i followed by a symbol from { 1 , 2 , ...,n)) indicates an arbitrary element of Item. The 0 shown as the value of Dara(L)when Lempty(L) = True is a special null value associated with type Item. Iterators Forward and Backward are defined as sequences, where () stands for the empty sequence,

TABLEXI

HIGH-LEVEL USERLISTOPERATIONS Firsr(L) LasW

= =

L

if Lempfy(L)then L elsi Lempfy(Lnext(L))then L else Last(Lnexf(L)) Next(L) = if Lempty(L) then L else Lnexr(L) Nrh(L,k) = if (k= 1 or Lempfy(L))then L else Nth(Lnexf(L),k-1) Dafa(L) = if Lempty(L)then 0 else Lread(L) Length@) = if Lernpfy(L) then 0 else Lengfh(Lnext(L))+1 Create(i1,i2 ,...,in) = Lbuild*(n,il ,i2....,in) Push(L,i) = Lbui/d(L,i) Pop(L) = if Lempty(L) then L else Lnext(L) Append(L,i) = if Lempty(L) then Lbuild(L,i) else Lbui/d(Append(Lnext(L),i),Lread(L)) Chop(L) = if Lempfy(L)then L elsif Lempty(Lnexf(L))then Lnext(L) else Lbuild(Chop(Lnexr(L),Lreud(L))) Inserf(L,i,n) = if (n= 1 or L e m p f y ( L ) )then Lbuild(L,i) else Lbuild(lnsert(Lnext(L),i,n- l),Lread(L)) De/efe(L,n) = if Lempty(L) then L e lsi n= 1 then Lnexf(L) else Lbuild(De/ete(Lnext(L),n- I ),Lrend(L))

if Lempfy(L)then () else (L;Forward(Lnexr(L))) Backward(L) = if Lempty(L) then () else (Backward(Lnexr(L));L) Furward(L)

=

348


and iterators deliver the elements of a structure in the order defined by the corresponding sequences in the specifications of the user operations. Note that the iterators are defined in fully applicative terms here. Membership in L* can be expressed by the following notation, which is made use of in Table XI: Lbuild*(n,i1 ,i2,. ..,in) = if n > 0 then Lbuild(il,Lbuild*(n - l,i2,i3,...,in)) else Lnew Otherwise the definitions of Table XI should be self-explanatory, and the reader should have no difficulty in understanding what each operation attempts to achieve.

8. Conclusions and Future Trends -

It is safe to say that formal specifications are here to stay. In Section 2.3 we argued that algebraic specifications have advantages over operational specification, but we would hesitate to predict whether one or the other of the two will ultimately win out. Very likely both approaches will coexist, and our emphasis on algebraic specifications here has been primarily motivated by the lesser familiarity of this approach. Algebraic specifications are now finding their way into functional languages, but much remains to be done before they become a useful tool for designers and programmers of procedural languages. We have sketched in Section 7 one approach that may ease the latter process, but this outline is to be regarded merely as a beginning. The fundamental question is whether anything is in fact gained by taking the algebraic rather than the operational approach outside a functional setting. We believe that there is. The dependence of an operational specification on a representation domain (as in the example of Table I) is too rigid a constraint. Algebraic specifications, on the other hand, provide clear descriptions of our computational objects in a self-contained way. However, we still lack a clear understanding regarding the nature of iterators. Even strictly functional algebraic specifications cease to be selfcontained when traversals are introduced (recall the dependence of the traversals of Zbin on Zqueue). In Section 7 we introduced two ways of defining iterators. In the more conventional approach of Table XI there is dependence on the sequence, which again is an extraneous representation domain. Iterator Inorder does not exhibit such dependence, but the interpretation of the objects to which it provides access is somewhat unortho-

ABSTRACTDATATYPES

349

dux. Iterators Forward and Backward of Table XI produce sequences of lists, and the corresponding iterators for the binary tree would produce sequences of binary trees. Our iterator Inorder, on the other hand, accesses data nodes. We permit value changes to be effected at these data nodes, and we even use the assignment statement for this purpose. It remains to be seen how these two viewpoints are to be reconciled. A very difficult problem relates to structural changes that one might wish to make at a node reached by means of a procedural traversal. A functional description of the traversal would have to coexist with the procedural description in order to allow a structural change to be made in such a way that the result of the change would be properly understood in the formal semantics sense. Let us look again at changes that are strictly limited to data values. Since we regard isomorphism as equality, data changes alone do not change a data structure. But we might wish to assert some invariant to which the data are to be subject. Following Lockemann et al. (1979), we consider constraints imposed on the data. For example, if we had a binary tree of numbers that was to function as a binary search tree, then we would impose on each node the following constraints:

If not Ernpty(Left(s))then Data(s) > Data(Left(s)) If not Ernpty(Righr(s))then Data(s) < Data(Right(s)) Here we have something very much like the Pre-Post conditions of an Alphard specification, which suggests that data changes might be best dealt with in terms of operational specifications, and structural changes in terms of algebraic specifications. Access to the data would be provided by iterators expressed in terms of operations to which meaning would be assigned by the algebraic component. Once these immediate problems have been resolved, the really important task will be the integration of the abstract data-structuring facility with concurrency. Again, because one is not hampered there by explicit sequencing, this will be easier to deal with in a functional setting. Beyond that we will have to come to grips with the effects of hardware failures on data structures; the design of fault-tolerant data structures is a challenge that will be with us for many years to come. ACKNOWLEDGMENTS We would like to express our sincere appreciation for the many stimulating discussions with Ray Ford, Prateek Mishra, and S. S. Ravi, who also made helpful comments on earlier versions of this survey.

350


REFERENCES Aho, A.V., and Ullman, J.D. (1979). Universality of data retrieval languages. Proc. ACM Symp. Princ. Prog. Lang., 6rh, 1979 pp. 110-117. Atkinson, R.R., Liskov, B.H., and Scheifler. R.W. (1978). Aspects of implementing CLU. Proc. Ann. Conf., Assoc. Comput. Mach., 33rd, 1978 pp. 123-129. Backus, J. (1978). Can programming be liberated from the von Neumann style? A functional style and its algebra of programs. Cornmun. ACM 21, 613-641. Bauer, F.L., and Broy, M., eds. (1978). “Program Construction,” Lect. Notes Comput. Sci. No. 69. Springer-Veriag, Berlin. Berzins, V.A. (1979). Abstract model specifications for data abstraction. Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge. Berztiss, A.T. (1980a). Depth-first K-trees and critical path analysis. Acta ZnJ 13, 325-346. Berztiss, A.T. (1980b). Data abstraction, controlled iteration, and communicating processes. Proc. Annu. Conf.,Assoc. Comput. Mach., 1980 pp. 197-203. Berztiss, A.T. (1981). Iterators and concurrency. Proc. IEEE In?. Conf.Parallel Proc., 1981 pp. 168-169. Berztiss, A.T., and Ford, R. (1982). Data abstraction and standard data structures. (Submitted for publication.) Birkhoff, G., and Lipson, J.D. (1970). Heterogeneous algebras. J. Combin. Theory, Ser A 8, 115-133. Broy, M., and Pepper, P. (1981). Program development as a formal activity. IEEE Trans. Sofrware Eng. SE-7, 14-22. Burstall, R.M. (1969). Proving properties of programs by structural induction. Cornput. J. l2,41-48. Burstall, R.M., and Darlington, J. (1977). A transformation system for developing recursive programs. J . Assoc. Comput. Mach. 24, 44-67. Burstall, R.M., and Goguen, J.A. (1980). The semantics of CLEAR, a specification language. In “Proceedings of the Advanced Course on Software Specifications,” Lect. Notes Comput. Sci. No. 86, pp. 292-332. Springer-Verlag, Berlin. Burstall, R. M., MacQueen, D.B., and Sannella, D.T. (1980). HOPE, an experimental applicative language. Proc. LZSP Conf.,1980 pp. 136-143. Clement, J., Lochhead, J., and Soloway, E. (1980). Positive effects of computer programming on students’ understanding of variables and equations. Proc. Annu. Conf.,AsS O C . Comput. Mach., 1980 pp. 467-474. Dahl, 0.-J., Myhrhaug, B., and Nygaard, K. (1968). “The SimuIa67 Common Base Language,” Report. Norwegian Computing Centre, Oslo. DeMillo, R.A., Lipton, R.J., and Perlis, A.J. (1979). Social processes and proofs of theorems and programs. Commun. ACM 22, 271-280. Dungan, D.M. (1979). Bibliography on data types. ACM SIGPLAN Nor. 14, No. 11, 31-59. Ford, R. (1979). “A Survey of the Development and Implementation of Data Abstraction,” Tech. Rep. 79-3. Dept. Comput. Sci., University of Pittsburgh, Pittsburgh, Pennsylvania. Ford, R.F. (1980). Design of abstract data structures to facilitate storage structure selection. Ph.D. Thesis, University of Pittsburgh, Pittsburgh, Pennsylvania. Gannon, J., McMullin, P., and Hamlet, R. (1981). Data-abstraction implementation, specification, and testing. ACM Trans. Prog. Lang. Sysr. 3, 211-223. Geschke, C., and Mitchell, J. (1975). The problem of uniform reference t o data structures. IEEE Trans. Software Eng. SE-1,207-219.

ABSTRACTDATATYPES

351

Geschke, C.M., Moms, J.M., and Satterthwaite, E.H. (1977). Early experience with Mesa. Commun. ACM 20, 540-553. Goguen, J.A. (1977). Abstract errors for abstract data types. Proc. IFlP Work. Conf. Formal Description Prog. Concepts, 1977 Paper 21. Goguen, J.A.. and Tardo, J.J. (1979). An introduction to OBJ: A language for writing and testing formal algebraic specifications. Proc. SpeciJications Reliuble Software, 1979 pp. 170-189. Goguen, J.A., Thatcher, J.W., Wagner, E.G., and Wright, J.B. (1975). Abstract data types as initial algebras and correctness of data representations. Proc. Conf. Comput. Graphics, Pattern Recog. Data Struct., 1975 pp. 89-93. Goguen, J.A., Thatcher, J.W., and Wagner, E.G. (1978). An initial algebra approach to the specifications, correctness, and implementation of abstract data types. In “Current Trends in Programming Methodology” (R.T. Yeh, ed.), Vol. 4, pp. 80-149. PrenticeHall, Englewood Cliffs, New Jersey. Guarino, L. (1978). “Evolution of Abstraction in Programming Languages,” Tech. Rep. CMU-CS-78-120. Dept. Cornput. Sci., Camegie-Mellon University, Pittsburgh, Pennsylvania. Gurari, E.M., and Ibarra, O.H. (1981). The complexity of the equivalence problem for simple programs. J. Assoc. Comput. Mach. 28, 535-560. Guttag, J. (1975). Specification and application to programming of abstract data types. Ph.D. Thesis. University of Toronto. Guttag, J. (1980). Notes on type abstraction (Version 2). fEEE Trans. Soffware Eng. SE-6, 13-22. Guttag, J.V., and Homing, J.J. (1978). The algebraic specification of abstract data types. Acta lnf. 10, 27-52. Guttag, J.V., and Homing, J.J. (1980). Formal specifications as a design tool. Proc. ACM Symp. Princ. Prog., Lang., 7th. 1980 pp. 251-259. Guttag, J.V., Horowitz, E., and Musser, D.R. (1978a). Abstract data types and software validation. Commun. ACM 21, 1048-1064. Guttag, J.V., Horowitz, E., and Musser, D.R. (1978b). The design of data type specifications. In “Current Trends in Programming Methodology” (R.T. Yeh, ed.), Vol. 4, pp. 60-79. Prentice-Hall, Englewood Cliffs, New Jersey. Guttag, J., Homing, J., and Williams, J. (1981). F P with data abstraction and strong typing. Proc. ( A C M ) Conf. Funct. Prog. Lang. Comput. Arch., 1981 pp. 11-24. Hanson, S., Jullig, R., Jackson, P., Levy, P.. and Pittman, T. (1979). Summary of the characteristics of several “modem” programming languages. ACM SIGPLAN N o t . 14, NO. 5, 28-45. Henderson, P. (1980). “Functional Programming: Application and Implementation.” Prentice-Hall, Englewood Cliffs, New Jersey. Henderson, P., and M o m s , J . (1976). A lazy evaluator. Proc. ACM Symp. Princ. Prog. Lang., 3rd. 1976 pp. 95-103. Jones, N.D., and Muchnick, S.S. (1981). Complexity of flow analysis, inductive assertion synthesis, and a language due to Dijkstra. In “Program Flow Analysis: Theory and Applications” (S.S. Muchnick and N.D. Jones, eds.), pp. 380-393. Prentice-Hall, Englewood Cliffs, New Jersey. Kamin, S. (1980). Final data type specifications: A new data type specification method. Proc. ACM Symp. Princ. Prog. Lung., 7th, 1980 pp. 131-138. Kapur, D. (1979). Specifications of Majster’s traversable stack and Veloso’s traversable stack. ACM SIGPLAN Not. 14, No. 5 , 46-53.

352


Lampson, B.W., Homing, J.J., London, R.L., Mitchell, J.G., and Popek, G.L. (1977). Report on the Programming Language Euclid. ACM SIGPLAN Not. U ,No. 2. Liskov, B. (1980). “Linguistic Support for Distributed Programs: A Status Report,” Comput. Struct. Group Memo 201. Lab. Comput. Sci., Massachusetts Institute of Technology, Cambridge. Liskov, B.H., and Berzins, V. (1979). An appraisal of program specification. In “Research Directions in Software Technology” (P. Wegner, ed.), pp. 276-301 (see also discussion on pp. 364-380). MIT Press, Cambridge, Massachusetts. Liskov, B., and Zilles, S. (1975). Specification techniques for data abstractions. ZEEE Trans. Software Eng. SE-1, 7-18. Liskov, B.H., and Zilles, S.N. (1977). An introduction to formal specifications of data abstractions. In “Current Trends in Programming Methodology” (R.T. Yeh, ed.), Vol. I , pp. 1-32. Prentice-Hall, Englewood Cliffs, New Jersey. Liskov, B., Atkinson, R., Bloom, T., Moss, E., SchafTert, J.C., Scheifler, R., and Snyder, A. (1981). “CLU Reference Manual,” Lect. Notes Comput. Sci. No. 114. SpnngerVerlag, Berlin. Lockemann, P.C., Mayr, H.C., Weil, W.H., and Wohlleber, W.H. (1979). Dataabstractions for database systems. ACM Trans. Database Syst. 4, 60-75. London, R.L. (1977). Perspectives on program verification. In “Current Trends in Programming Methodology” (R.T. Yeh, ed.), Vol. 2, pp. 151-172. Prentice-Hall, Englewood Cliffs, New Jersey. London, R.L. (1979). Program verification. In “Research Directions in Software Technology” (P. Wegner, ed.), pp. 302-315 (see also discussion on pp. 380-404). MIT Press, Cambridge, Massachusetts. London, R.L., Guttag, J.V., Homing, J.J., Lampson, B.W., Mitchell, J.G., and Popek, G.J. (1978). Proof rules for the programming language Euclid. Acta 1 4 . 10, 1-26. McCarthy, J. (1960). Recursive functions of symbolic expressions and their computation by machine. Commun. ACM 3, 184-195. Majster, M.E. (1977). Limits of the algebraic specification of data types. ACM SIGPLAN Not. U ,NO. 10, 37-42, Majster. M.E. (1979). Data types, abstract data types and their specification problem. Theor. Compui. Sci. 8, 89-127. Manna, Z., and Waldinger, R. (1981). Problematic features of programming languages: A situational-calculus approach. Acia lnf. 16, 371-426. Manna, Z., Ness, S . , and Vuillemin, J. (1973). Inductive methods for proving properties of programs. Commun. ACM 16, 491-502. Mendelson, E. (1964). “Introduction to Mathematical Logic.” Van Nostrand-Reinhold, Princeton, New Jersey. Moitra, A. (3979). “Direct Implementation of Algebraic Specification of Abstract Data Types,” Tech. Rep. 48. NCSDCT, TIFR, Bombay. Moms, J.H. (1973). Types are not sets. Proc. ACM Sump. Princ. Prog. Lung., lst, 1973 pp. 120- 124. Musser, D.R. (1980a). Abstract data type specification in the AFFIRM system. ZEEE Trans. Sofrware Eng. SE-6,24-31. Musser, D.R. (1980b). On proving inductive properties of abstract data types. Proc. ACM Symp. Princ. Prog. Lung., 7th, 1980 pp. 154-162. Palme, J. (1976). New feature for module protection in Simula. ACM SIGPLAN Not. 11, NO. 5, 59-62. Parnas, D.L. (1972a). A technique for software module specification with examples. Commun. ACM 15. 330-336.

ABSTRACTDATATYPES

353

Parnas, D.L. (1972b). On the criteria to be used in decomposing systems into modules. Commun. ACM 15, 1053-1058. Shaw, M.,ed. (1981). “Alphard: Form and Content.” Springer-Verlag, New York. Shaw, M.,Wulf, W. A., and London, R. L. (1977). Abstraction and verification in Alphard: Defining and specifying iteration and generators. Commun. ACM 20, 553-564. Sokolowski, S. (1980). A uniform approach to applicative and imperative features in programming languages. In “Mathematical Foundations of Computer Science,” Lect. Notes Comput. Sci. No. 88, pp. 612-626. Springer-Verlag, Berlin. Spitzen, J., and Wegbreit, B. (1975). The verification and synthesis of data structures. Acfa lnf. 4, 127-144. Standish, T.A. (1978). Data structures-An axiomatic approach. I n “Current Trends in Programming Methodology” (R.T. Yeh, ed.), Vol. 4, pp. 30-59. Prentice-Hall, Englewood Cliffs, New Jersey. Stoy, J.E. (1977). “Denotational Semantics: The Scott-Strachey Approach.” MIT Press, Cambridge, Massachusetts. Subrahrnanyam, P.A. (1979). “Nondeterminism in Abstract Data Types,” Tech. Rep. Dept. Comput. Sci., University of Utah, Salt Lake City. U.S. Dept. of Defense (1980). “Reference Manual for the Ada Programming Language.” U.S. Dept. of Defense, Washington, D.C. Wand, M.(1978). “Final Algebra Semantics and Data Type Extensions,” Tech. Rep. 65. Dept. Comput. Sci., Indiana University, Bloomington. Wegbreit, B., and Spitzen, J. (1976). Proving properties of complex data structures. J . Assoc. Comput. Mach. 23, 389-3%. Welty, C., and Stemple, D.W. (1981). Human factors comparison of a procedural and a nonprocedural query language. ACM Trans. Database Sysr. 6,626-649. Wirsing, M.,and Broy, M. (1980). Abstract data types as lattices of finitely generated models. In “Mathematical Foundations of Computer Science,” Lect. Notes Comput. Sci. No. 88, pp. 673-685. Springer-Verlag, Berlin. Wirsing, M., Pepper, P., Partsch, H., Dosch, W., and Broy, M. (1980). “On Hierarchies of Abstract Data Types,” Tech. Rep. TUM-18007.Technical University of Munich. Wirth. N. (1977). MODULA: A language for modular multiprogramming. Software-Pract. EXP. 7 , 3-36. Wulf, W.A. (1977). Languages and structured programs. In “Current Trends in Programming Methodology” (R.T. Yeh, ed.), Vol. I , pp. 33-60. Prentice-Hall, Englewood Cliffs, New Jersey. Wulf, W.A. (1980). Abstract data types: A retrospective and prospective view. In “Mathematical Foundations of Computer Science,” Lect. Notes Comput. Sci. No. 88, pp. 94-1 12. Springer-Verlag, Berlin. Zilles, S.N. (1974). “Algebraic Specification of Data Types,” Proj. MAC Prog. Rep. 11. MIT,Cambridge, Massachusetts.


Author Index Numbers in italics indicate the pages on which the complete references are listed.

A Adams, R. D., 232,293 Addis, T.R., 202, 210 Adelstein. J., 218, 293 Adleman, L.M., 102 Adlman, L., 67, 68, 69, 80, 85, 86, 88, 106 Adnon, W.R., 149, 159 Agarival, K.K., 202, 213 Aho, A.V. 62, 63, 88, 103, 337, 350 Aiello, N., 182, 214 Aikins, J.S., 165, 172, 173, 181, 189, 210, 213, 215, 216 Allen, B., 202, 212 Amarel, S., 166, 174, 202, 205, 210, 216, 274,293 Anderson, R.H., 203,210 Anton, J.J., 202,214 Araya, A.A.. 202, 214 Arazi, B., 91, 103 Ash, R.B.. 49, 103 Asmuth, C.A., 60,103 Atkinson, R., 114, 160 Atkinson, R.R., 341.350 Austin, H., 202, 212 Axline, S., 172, 216 I

B Backus, J., 301, 302, 350 Ballard, J.P., 202, 213 Barnett, G.O., 125, 159 Barnett, J.A., 190, 197,210 Barr, A., 174,210 Barstow, D., 202, 210 Basden, A., 202,210 Basili, V., 117, 159 Bauer, F.L., 300. 350 Bechtel, R.J., 202, 214 Bellavance, D.A., 203,213 Bennett, C.H., 102,103 Bennett, J., 203, 210 Bennett, J.S., 166, 202, 210, 211

Berg, C.H., 202, 212 Berkovits, S., 84, 103 Berlekamp, E.R., 49, 64, 81, 82, 93, 103 Berry, D.M., 136, 160 Bersoff, E.H., 119, 159 Berzins, V.A., 298, 302, 350, 352 Berztiss, A.T., 338, 342, 344, 345, 350 Birkhoff, G., 309, 312,350 Bischoff, M., 202, 212 Bischoff, M. B., 172, 202, 215 Bjerregaard, B.. 218, 292 Blakley, B., 87, 88, I03 Blakley, G.R., 60, 87, 88, 103 Blois Marsden, S., 278, 292 Bloom, T., 114, 160 Blum, R., 172, 216 Bobrow, D., 181,211 Boehm, B.W., 117, 152, 159 Boggs, D.R., 156, 160 Boivie, R.H., 202, 213 Bonnet. A,, 164,211 Booth, K.S., I03 Bordley, J., 111, 277, 292 Borosh, I., 87, 88, 103 Brachman, R.J., 174, 181, 211 Branstad, C.K., 96,I03 Branstad, M., 149, I59 Brassard, G., 101, 102, 103 Braun, H., 202, 216 Braunwald, E., 232, 293 Brinch Hansen, P., 114, 159 Brooks, D., 43 Brooks, F.W., Jr., 150, 159 Brooks, R., 184,211 Brooks, R.E., 202,213 Brown, J.S., 202, 203. 206, 211 Broy, M., 300, 316, 319, 320, 322, 350, 353 Brynitz, S., 218, 292 Buchanan, B., 267, 293 Buchanan, B.G., 166, 172, 183, 189, 194, 201, 207, 211, 212, 214, 215, 216 Buchstaller, W., 202, 213 Bundy, A., 201, 202,211 Burstall, R.M., 300, 302, 333, 350

AUTHOR INDEX

Burton, R., 202,211 Burton, R.R., 202, 203, 206,21 I Byrd, L., 201, 202, 211

Dosch, W., 316, 319, 322,353 Doyle, J., 190, 212 Duda, R.O., 164, 165, 189, 193, 194, 202, 212 Dungan, D.M., 296,350

C Caine, S.H.,132, 159 Campbell, A.B., 172, 202,215 Campbell, C.M., 103 Carhart, R.E., 173, 201,211 Carlbom, I., 202,212 Carter, J.L., 91, 94, 95, 107 Chandrasekaran, B., 202, 211, 222. 223, 225,226, 227, 236, 242, 250, 253, 259, 268, 271, 275, 277, 285, 289, 292, 293 Chaum, D.L., 100, 103 Chen, P.P.-S., 135, 159 Choplin, F., 202, 216 Clancey, W., 166,215 Clancey, W.J., 172, 190, 203, 206, 211, 216 Clement, J., 336,350 Codd, E.F., 135, 159 Cohen, P., 174, 211 Cohen, S.N., 172, 216, 267, 293 Colton, K.W., I05 Constantine, L.L., 132, 161 Cryer, P.E., 278, 292 Cullingford, R.E., 203, 213 Curtis, B., 117, 159

D

E Ehrsam, W.F., 60,83, 96, I04 Elwell, J.F., 144, 159 Engelman, C., 202,212 Engelmore, R.,203, 210 Engelmore, R.S., 201,212 Ennis, S.P., 184, 212 Erdos, P., 104 Erman, L., 190,210 Erman, L.D., 182,212, 271,292 Essig, A., 218,293 Evans, A., Jr., 91, 95, 104

F Fagan, L., 172, 202,212, 216 Fagan, L.M., 165, 172, 173, 213 Fagan, M., 118, 159 Fain, J., 166, 182, 212 Fairley, R.E., 152, 160 Fallat, R.J., 165. 172, 173, 213 Feigenbaum, E.A., 164, 165, 166, 172, 173, 174, 183, 201. 202, 210, 211, 212, 213, 214 Feistal, H.,59, 94, 104 Feldman, S.I., 131, 159 Fickas, S.F., 182, 212 Fine, T.L., 190,212 Ford, R.,296, 342, 345, 350 Ford, R.F., 345, 346,350 Forgy, C., 166, 180,212 Fox, M.S., 202,212 Frawley, B., 202,212 Friedland, P., 202,212 202, 213 Fu, K.-S.,

Dahl, 0.-J., 300, 350 Darlington, J., 300, 350 Davida, G.I., 100, 101, 102, 103 Davis, R.. 164, 166, 172, 176, 189, 202, 210, 211. 212, 215. 216, 267, 293 Davis, R.M., 60,I03 Deavours, C.A., 51, 56, 102, 103 de-Bombal, F.T., 218, 292 DeKleer, J., 190, 202, 203, 206, 211, 212 DeMarco, T., 135, 159 DeMillo, R., 115, 159 DeMillo, R.A., 298, 350 Denning, D.E., 96, 103 Denning, P.J., 102, 103 G Diffie, W., 46, 53, 60,64, 82, 96, 104 Dincbas, M., 202,212 Gane, C., 135, 159 Dippe, M.D., 114, 161 Gannon, J., 302,350

357

AUTHOR INDEX

Gardner, M., 104 Garey, M.R., 62, 88, 89, 93, 104 Gasaway, L., 43 Gaschnig, J.G.,164, 165, 189, 202, 212 Gaston, L.W., 202,214 Gelernter, H.L., 202, 213 Gemignani, M., 43 Genesereth, M., 202, 213 Genesereth, M.R., 182, 190, 203, 213 Georgeff, M.P., 190, 213 Gersho, A., 104 Geschke. C., 300, 350 Geschke, C.M., 300, 351 Gilbreth, J.A., 202, 212 Gill, J., 102, 103 Gillogly, J.J., 203, 210 Goguen, J.A., 302, 310, 312, 316, 321, 324, 350, 351 Goldkuhl, G., 119, 160 Goldstein, I.P., 181, 202, 213, 215, 244, 293 Gomez, F., 222, 223, 271 277, 285, 289, 292 Gordon, E.K., 132, 159 Gorry, G.A., 202,213, 218, 240, 274, 292, 293 Greenes, R.A., 125, 159 Greiner, R., 182, 213 Guarino, L., 298,351 Gudes, E., 100, 104 Gurari, E.M., 337,351 Guttag, J . , 304, 310, 342, 351 Guttag, J.V., 302, 304, 307, 308, 309, 322, 324, 325, 326, 327, 329, 330, 333, 334, 335, 338, 345, 351, 352 Gutz, S., 149, 153, 159, 161 Guy, R.C., 80, 104 ~

H Habermann, A.N., 131, 160 Hall, D.E., 132, 160 Hamlet, R., 302, 350 Hannigan, J., 172, 216 Hanson, S., 300, 301, 340,351 Hardy, J.D., 218, 292 Hart, P., 165, 189, 202, 212 Hart,P.E., 193, 194,212. 227,292 Harvey, A.M., 277, 292 Hawkinson, L.B., 181, 216

Hayes-Roth, F., 164, 166, 176, 182, 212, 213, 216 Heaslet, M.A., 64,65, 79, 107 Heiser, J.F., 202, 213 Hellman, M.E., 46, 53, 60,64,74, 82, 84, 89, 90, 91, 92, 96, 104, 105, 106 Henderson, P., 301, 337, 351 Henderson, V.D., 119, 159 Herlestam, T., 86, 92, 104 Hershey, E.A., 152, 161 Hewitt, C., 182, 203,213 Hilden, J., 269, 292 Hill, S., 104 Hoffman, L.J., 104 Hollander, C.R., 180, 202, 211, 213 Holst-Christensen, J., 218, 292 Hopcroft, J.E., 62. 63, 88, I03 Horn, W.,202, 213 Homing, J . , 302, 351 Homing, J.J., 114, 160. 298, 304, 308, 324, 325. 326, 345,351 352 Horowitz, E., 304, 307, 309, 322, 327, 329, 330, 333, 334, 335, 338, 351 Horrocks, H.D., 218, 292 Hsiao, D.K., 100, 104 Hsu, M.B., 203, 215 Hyvarinen, L.P., 49, 104 ~

I Ibarra, O.H., 337, 351 Ichbiah, J.D., 114, 160 Ingemarsson, I., 91, 92, 96,99, 104, 105 Ishizuka, M., 202, 213 Isselbacher, K.J., 232, 293 h i e , E.L., 130, 160

J Jackson, P., 300, 301, 340, 351 Jacobs, C.D., 172, 202,215 Johnson, D.S., 62, 88, 89, 93, 104 Jones, N.D., 298, 337,351 Jullig, R., 300, 301, 340, 351

K Kahn, D., 46, 51, 53, 102, 105 Kaihara, S., 202, 213

358

AUTHOR INDEX

Kalaja, E., 218, 292 Ramin, S., 316,351 Kandt, R.K., 202,213 Kantrowitz, W., 91, 95, 104 Kapur, D., 366,351 Karp, R.M., 89, 105 Kassirer, J.P., 218, 240, 274, 293 Katzen, H., Jr., 105 Kay, A., 126, 160 Keeler, E., 218, 293 Kelly, B.A., 202, 210 Kernighan, B.W., 129, 152, 160 Kerr, D.S., 100, 104 Kersten, M.L., 114, 136, 160, 161 Kibler, D.F., 202, 214 King, J., 176, 210, 211 Kingsland, L.C., 202,214 Kissane, J.M., 278, 292 Kline, C.S., 96, 105, 106 Knuth, D.E., 66, 71, 74, 80, 105 Koch, H.S.,100, 104 Kolata, G.B., 102, 105, 107 Konheim, A.G., 56, 57, 60, 100, 105 Kowalchuk, J., 84, 103 Koyarna, T., 202,213 Kraemer, K.L., 105 Krueger, M.W., 203, 213 Kulikowski, C., 166, 173, 180, 202, 205, 216 Kulikowski, C.A., 218, 274,292, 293 Kunz, J.C., 165, 172, 173, 213

L Lampel, A., 102, 105 Lampson, B.W., 114, 160, 298, 304, !52 Larkin, J., 202, 213 Larsen, D.L., 202, 213 Lauriere, J.L., 182, 202,214 Lederberg, J., 166, 183,214 Ledley, R.S., 218, 292 Lehmer, D.H., I05 Lenat, D., 182, 213 Lenat, D.B., 164,213 Lennon, R.E., 105 Lesser, V.R., 271,292 Letsinger, R., 190, 211 LeVeque, W.J., 64, 67, 70, 73, 77, 78, 79, 105

Leveson, N.G., 136, 160 Levy, P., 300, 301, 340,351 Lieberherr, K., 98, 105 Lindberg, D.A.B., 202,214 Lindsay, R.,166, 183,214 Lipkin, M., 218,292 Lipson, J.D., 309, 312, 350 Lipton, R.J., 115, 159. 298,350 Liskov, B., 114, 160, 300, 302, 343,352 Liskov, B.H., 298, 302, 341, 350, 352 Lochhead, J., 336,350 Lockemann, P.C., 349,352 Lodwick, G.S., 251,292 London, P.E., 182,212 London, R.L., 114, 136, 160, 161, 298, 304, 328, 337,352, 353 Luccarelli, P., 9,43 Luger, G., 201, 202, 211 Lundeberg, M., 119, 160 Lund-Kristensen, J., 218,292 Lusted, L.B., 191,214, 218,292

M Mack, M.H., 100, 105 MacQueen, D.B., 302,350 Madnik, S.E., 100, 104 Majster, M.E., 335, 336,352 Mamdani, E.H., 202,214 Manna, Z., 313, 330, 338,352 Marble, C.W., 125, 159 Marcal, P.V., 203,215 Martin, J., 46, 105 Martin, W.A., 181,216 Mashey, J.R., 129, 160 Masinter, L., 127, 161 Matyas, S.M., 60,83, 96, 104, I05 Mayr, H.C., 349,352 McCarthy, J., 301,352 McClung, D.H., 165, 172, 173, 213 McColl, D.C., 202, 214 McCue, G.M., 144, 160 McCune, B.P., 202,214 McDermott, J., 166, 176, 180, 186, 202, 212,213, 214 McEliece, R.J., 49, 64,93, 97, 105 McEliece, R.M., 49, 93, I03 McIlroy, M.D., 154, 160 McMullin, P., 302, 350

AUTHOR INDEX

McNeil, B.J., 218, 293 McNeill, R.K., 100, 105 Mellish, C., 201, 202, 211 Mendelson, E., 309, 319, 352 Merkle, R.C., 60, 82, 83, 89, 90. 91, 92, 96,105. 106 Metcalfe, R.M.,156, 160 Meyer, C.H., 60, 83, 96, 104, 105 Michie, D., 164, 210, 214, 227, 293 Miller, E.F., Jr., 152, 160 Miller, G.I.,86, 106 Minamikawa, T., 202, 213 Minsky, M., 181, 214 Mitchell, J., 300, 350 Mitchell, J.G., 114, 160, 298, 304, 352 Mitchell, T.M., 166, 214 Mittal, S., 202, 211, 227, 236, 242, 244, 250, 253, 255, 258, 259, 262, 268, 275, 285, 288, 289, 292, 293 Moitra, A., 330, 352 Moms, J., 337, 351 Moms, J.H., 310,352 Moms, J.M, 300,351 Moms, P.H., 202, 214 Moms, R., 60, 106 Moses, J., 166, 202, 214 Moss, E., 114, 160 Muchnick, S.S., 298, 337, 351 Muresan, L.V., 192,215 Murphy, M., 43 Musser, D.R., 302, 304, 307, 309, 319, 322. 327, 329, 330. 333, 334, 335, 338, 351. 352 Myers, G.J., 115, 118, 160 Myhrhaug, B., 300,350

N Needham, R.M., 95, 106 Ness, S., 313, 330, 352 Newell, A., 164. 176. 214 Newlon, R., 202, 213 Nii, H.P., 165, 172, 173, 182, 202, 213, 214 Nilsson, A.. 119, 160 Nilsson, N.J., 185, 193, 194, 212, 214 Noms, M.J., 86, 107 Notkin, D.S., 131, 160 Notz, W.A., 94, 104 Novak, G,, 202,214

359

Novick, M., 43 Nygaard, K., 300, 350

0 O’Neill, D., 119, 160 Osborn, J., 165. 172, 173, 213 O’Shea, T., 203,214

P Palme, J., 301, 352 Palmer, M., 201, 202, 211 Pappalardo. A. N., 125, 159 Parnas, D.L., 301, 352, 353 Partsch, H., 316, 319, 322, 353 Patil, R.S., 202, 214 Patrick, E.A., 218,293 Patterson, N., 49, 93, 106 Pauker, S.G., 166, 202,213, 216, 218, 219, 240, 274,293 Pednault, E.P.D., 192, 215 Pepper, P., 300, 316, 319, 322,350. 353 Perlis, A., 115, 159 Perlis, A.J., 298, 350 Perry, D.E., 131, 160 Petersdorf, R.G., 232, 293 Peterson, W. W., 49, 64, 8 I, 93, 106 Pinson, E.N., 154, 160 Pinson, S., 164, 215 Pittman, T., 300. 301, 340,351 Plauger, P.J., 152, 160 Pohlig, S.C., 74, 84, 106 Pollard, J.M., 80, 106 Popek, G.J., 96, 105. 106. 304,352 Popek, G.L., 114, 160, 298,352 Pople, H.E., 166, 202, 215, 221, 240, 274, 293 Potenza, J., 43 Pressman, C., 43 Pruchnik, P., 202, 212 Purdy, G.B.,91, 106

R Rabin, M.O., 87, 97, 98, 101, 106 Reboh, R., 166, 180, 205, 215

360

AUTHOR INDEX

Reggia, J.A., 202,215 Reinstein, H.C., 180, 181,213, 215 Riddle, W.E., 152, 160 Ritchie, D.M., 129, 160 Rivest, R.L., 67,68, 69, 80, 85, 86, 88, 102, 106 Rivlin, J.M., 203, 215 Roberts, B., 202, 213 Roberts, R.B., 181,215, 244, 293 Rochkind, M.J., 131, 160 Rockmore, A.J., 202, 214 Rose, G., 43 Ross, D.T., 118, 161

S Sacco, G.M., %, 103 Safir, A., 166, 202,205,216, 274,293 Samuel, A., 236, 293 Sanders, A.F., 202, 213 Sandewell, E., 127, 161 Sannella, D.T., 302,350 Sarson, T., 135, 159 Satterthwaite, E.H., 300,351 Savage, L.J., 190,215 SchaRert, C., 114, 160 Schanning, B., 84, I03 Scheifler, R.W., 341, 350 Scherrer, D.K., 132, 160 Schiefler, B., 114, 160 Schmid, B., 86, 107 Schmidt, W., 43 Schoman, K.E., Jr., 118, 161 Schroeder, M.D., 95, 106 Schroeppel, R., 89, 106 Schwartz, W.B., 202,214, 218, 240, 274, 293 Scott, A.C., 166, 172,202,215,216, 267, 293 Searleman, J.E., 202, 213 Sendrow, M.,96, 106 Shafer, G., 197,215 Shamir, A., 67, 68, 69, 80, 85, 86, 88, 89, 91, 92, 102, 106 Shannon, C.E., 50, 51, 53, 56, 60,106 Shapley, D., 102, 106, 107 Shaw, J.C., 125, 161 Shaw, M., 136, 161, 298, 300, 301, 303, 304, 328,353

Sherertz, D.D., 114, 161 Sherlock, S., 232,293 Shewmake, D.T., 137, 161 Shortliffe, E.H., 166, 172, 194, 202,212, 215, 216, 221, 267, 274,293 Sieber, W.,202,216 Siegel, S.G., 119, 159 Silverman, H., 202,213 Simmons, G.J., 46,47, 84, 86, 95, 98, 100, 107 Simon, D.P., 202, 213 Simon, H.A., 164, 176, 202,213,214 Sinkov, G.J., 56, 57, 107 Sloane, J.J.A., 60,106 Smith, B., 174, 203,211, 213 Smith, D.C.P., 135, 161 Smith, G., 202, 216 Smith, J.L., 59, 94, 104, 107 Smith, J.W., 202,211, 236, 250, 253, 259, 268, 275, 285, 289,292, 293 Smith, J.W., Jr., 269, 293 Sneiderman, R., 202,212 Sokolowski, S., 338,353 Solovay, R., 80, 107 Soloway, E., 336,350 Sowizral, H., 166, 182, 212 Spier, M.J.,153, 159 Spitzen, J., 333, 353 Spritzer, G.A., 202,213 Squires, J., 30, 43 Sridharan, N.S., 181, 215 Stahl, F.A., 100, 104 Stallman, R.M., 202,215 Standish, T.A., 309,353 Steele, G., 190, 212 Stetik, M., 164, 181, 201, 202, 215 Stemple, D.W., 336, 353 Stem, R., 30, 43 Stoy, J.E., 313, 353 Strohm, G., 202,212 Strssen, V., 80, 107 Subrahmanyam, P.A., 328,353 Sugraman, R.,60,102, 107 Sussman, G . , 190,212 Sussman, G.A., 202, 215 Sussman, G.J., 202,215 Sussman, G.L., 202,212 Sventek, J.S., 132, 160 Swartout, W.,166, 202,215, 216 Synder, A., 114, 160

361

AUTHOR INDEX

Szolovits, P., 166, 181, 202, 214, 216, 218, 219, 293

T Tague, B.A., 154, 160 Tang, D.T., 99, 105 Tardo, J.J., 302, 351 Tatman, J.L., 269, 293 Teichroew, D., 152, 161 Teitelman, W., 127, 161 Terry, A., 201,212 Tesler, L., 126, 161 Thatcher. J.W., 310, 312, 316, 324, 351 Thompson, K., 129, 160 Thorn, G.W., 232, 293 Trapp, R., 202, 213 Tsotsos, J.K., 202, 216 Tuchman, W.L., 60, 83, 96,104 Tuckerman, B., 100, 105

U Ullman, J.D., 62, 63, 88, 103, 337, 350 Uspensky, J.V., 64,65, 79, 107

V van de Riet, R.P., 114, 161 Vanker, A.D., 202,214 van MeIIe, W., 166, 172, 173, 180, 194, 202, 215, 216, 278, 293 Van Tilbor, H.C.A., 49, 93, 103 Vinogradov, I.M., 64,67, 73, 77, 78, 107 Votteri, B.A., 165, 172, 173, 213 Vuillemin, J., 313, 330, 352

W Wagner, E.G., 310, 312, 316, 324,351 Waldinger, R., 338,352 Wallenstein, H., 43 Walsh. J., 102, 107

Wand, M., 312, 316,353 Warren, D., 182, 216 Wasserman, A.I., 114, 119, 131, 136, 137, 149, 153, 159, 160, 161 Waterman, D., 166, 182, 212 Waterman, D.A., 164, 176, 203, 213, 216 Wegbreit, B., 333, 353 Wegman, M.N., 91, 94, 95, 107 Weil, W.H., 349, 352 Weingarten, F.W., 102, 107 Weiss, E., 91, 95. 104 Weiss, S., 166, 173, 180, 202, 205, 216, 274, 293 Wells, D.L., 100, 101, 103 Welty, C., 336, 353 Weyhrauch, R.W., 182,216 Willet, M., 107 Williams, H.C., 80, 86, 87, 101, 107 Williams, J., 302, 351 Winograd, T., 181, 211 Wintrobe, M.M., 232,293 Wipke, W.T., 202, 216 Wirsing, M., 316, 319, 320, 322, 353 Wirth, N., 114, 161, 298, 353 Wohlleber, W.H., 349, 352 Wong, C.K., 91, 96,99, 104, I05 Wraith, S.M., 172,216, 267,293 Wright, J.B., 310, 312, 316, 351 Wulf, W.A., 136, 161, 297, 298, 328, 353 Wyner, A.D., 60, 106

Y Yao, J.T.P., 202, 213 Yasaka, T., 202, 213 Yourdon, E., 118, 119, 132, 161 Yu, V.L., 172, 216, 267, 293

Z

Zadeh, L.A., 196, 197,216 Zilles, S.N., 298, 302, 310,352, 353 Zippel, R.E., 91, I06 Zucker, S.W., 192,215

Subject Index A Abstract data types, see also Data abstraction adequacy and nondeterminism in, 327328 algebraic approach to, 304-308 axiomatic specifications for, 302-308 consistency and completeness in, 323328 functional and procedural programming with, 336-338 future trends in, 348-349 high-level user list operations in, 347 implementation and verification in, 329334 logical completeness in, 324 operational approach to, 303-304 specification and implementation in, 295-349 Ada-like syntax, iterator and, 341 Ada Programming Support Environments, 131-138, 300 multilevel structure of, 133 tools for supporting of, 131-133 ADSS, see Automated development support system ADT, see Abstract data types AGE language, 182 AIMDS language, 181 AIfred Bell & C o . v. Catalda Fine Arts, Inc., 3, I I Algebraic coding theory, in public key cryptosystems, 93-94 Algebraic specifications of binary tree, 307 meaning of, 309-322 of queue, 306 Algebras heterogeneous, 309-310 homogeneous vs. heterogeneous, 310 initial, 315-316 of natural numbers, 3 10-3 12 properties of class of, 309 terminal, 316-320 Algorithms, for public key cryptosystems, 45- 102

ALICE language, 182 Alphard-like specifications, vs. algebraic, 308-309 ALIX system, 180 American Association for the Advancement of Science, 102 Anchor specialists, in CHOLESTASIS program, 233 ANESTHETIC frame, 289 APL language, 180 APSE, see Ada Programming Support Environments Arrays, data structures and, 344 ASCII code, 49 Associations of Computing Machinery, 102 Associative triples, in EMYCIN systems, 178

Automated development support system, software tools and, 153 Avco Corp. v. Precision Air Parts, Inc., 9

B BASIC language, in software development environment, 125, 130, 180 Bayesian probability theory, 190-198 Binary tree, algebraic specification for, 307 Blackboard, in MDX problem solving, 27 1-274 Board of Patent Appeals, 16 C

California, University of, 133 CASNET program, 205 CCPA, see Court of Customs and Patent Appeals Certainty theory, in expert systems, 1941%

Chinese remainder theorem, 70 CHOLANGITIS program, 234 Cholestasis anchor specialists and, 233 conceptual structure of, 231-233 defined, 230 diagnostic hierarchy in, 231 as domain of MDX, 230-234

362

363

SUBJECT INDEX

extrahepatic, 232-233 intrahepatic, 231-233 two types of, 231 CHOLESTASIS program, 233-234 Chosen plain text attack, 51 Cipher text only attack, 51 Classical cryptosystems. examples of, 5360,see also Cryptography; Cryptosystem; Public key cryptosystems Cochrane v. Deener, 11, 34 Collections, data structures and, 343344 Commission on New Technological Uses of Copyrighted Works, 7 Compilation unit, module as, 300-301 Completeness concept, in abstract data types, 324-326 Complete residue systems, in public key cryptosystems, 69 Composite extractor operation, 317 Computer, medical diagnosis by, 167-170, 207-210, 219 Computer programs copyrighting of, 6 expert system as, 164 as trade secret, 25-26 Computer Service Corporation, 1 Computer terminals, in software engineering environments, 147-149 Conceptual structure representation languages, 277-278 Conference key distribution systems, in public key cryptosystems, 99-100 Congruences, in public key cryptosystems, 68-7 I Consistency, in knowledge representation, 182 Control knowledge, explicit representation of, 189-190 CONTU, see Commission on New Technological Uses of Copyrighted Works Copyright, trade secrecy law and, 27-28 Copyright Act (1909), 3, 7, 29 Copyright Act (19761, 4, 7-8 trade secrecy and, 26-29 Copyrighting, of software, 3-9 Copyright law history and current state of, 6-9 preemption in, 8-9 primer of, 3-6 Copyright notice, form of, 27

Court of Customs and Patent Appeals, 1218, 22-24, 28-29 Cryptanalysis, 50-5 1 Cryptographic research, growth in, I02 Cryptography vs. coding, 47 computational complexity theory in, 53 computationally secure system in, 52 vs. decoding, 47 defined, 46 vs. encoding, 47 factorization in, 101 monoalphabetic substitution in, 56 nature and scope of, 45-49 one-time pad system in, 101 polyalphabetic substitution system in, 56-57 product cipher system in, 59-60 recent advances in, 46 security as basic premise of, 101 simple substitution system in, 55-57 transposition system in, 57-59 unconditional or perfect security in, 5253 Cry ptosystem classical, 47, 53-60 defined, 46 Goppa ?odes and, 49, 93 public key, see Public key cryptosystem CSRL, see Conceptual structure representation languages

D Dann v. Johnston, 14 Data abstraction defined, 296 encapsulation in, 301 history of, 300-302 motivation for, 296-298 nature of, 298-300 practical approach to, 339-342 problems associated with, 335-339 synchronization problems and, 338-339 Data base security, in public key cryptosystems, 100-101 Data Cash Systems, Inc. v. J S & A Group Inc., 3 Data completeness, in expert systems, 203-204

364

SUBJECT INDEX

Data-driven control, in expert systems, 185-187 Data nonindependence, in expert systems, 204 Data primitives, 339-340 Data structures collections of, 343-344 linear list variations and, 345-348 representative set of, 343-345 selection of, 342-343 standard, 342-348 Deduction systems, 177 Default knowledge, in representation language, 183 Dempster-Shafer theory of evidence, 197198 DENDRAL expert system, 166 Denotational semantics, 313 DES (Data Encryption Standard) cipher, 60 Diagnosis, see Medical diagnosis Diagnostic problem solving conceptual structure in, 235-237 MDX extension to, 271-274 medical community organization and, 224-225 rules in, 235-236 strategy in, 234-240 Diamond v. Bradley, 22 Diamond v. Chakrabarry, 20,33 Diamond v. Diehr, 20-21, 32-37 Digital signatures, 46 Discrete logarithms to primitive base, 76 in public key cryptosystems, 71-75, 8285

Disease taxonomy, as hierarchy, 222 n. DRUG basic class, 242 DRUG frame, 289-290 DWIM, in software development environment, 128

E EMYCIN framework, fact representation in, 180 EMYCIN languages, MDX-like systems and, 278 EMYCIN rule, premise in, 178 EMYCIN syntax, 178-179

EMYCIN system, 166, 173 certainty theory and, 195 facts as associative triples in, 178 parts of, 174 as production system, 177-180 Encapsdadon, in data abstraction, 301 Enciphering, defined, 46 Encryption, defined, 46 English alphabet, useful representation of, 54 Entscheidungsproblem, 325 Essential MYCIN, see EMYCIN Ethernet-like media, networking capability and, 156 Euclid’s algorithm or criterion, 66, 70, 77, 80 EXPERT SYSTEM, 166,180 Expert systems agreement among experts in, 199 answers to user’s questions about, 207210 artificial intelligence research and, 164 basic framework and characteristics of, 164-173 Bayesian probability theory in, 190-194 classes of problems for, 201-203 control strategies in, 185 data-driven control in, 185-187 defined, 164 expertise in, 204-205 explanation in, 200 goal-driven control in, 187-188 inference methods in, 184-190, 200 key concepts in, 198-205 knowledge acquisition in, 201 knowledge engineering in, 204-205 mixed strategies in, 188-189 nature of data in, 203-204 nature of problem in, 199 plausible inference in, 190 possibility theory in, 196-197 problem-solving strategies in, 184, 200 proficiency of, 204-205 reasoning with uncertainty in, 190-198 replication of expertise in, 205 representation of knowledge in, 173-184, 199-200 rule-based, see Rule-based expert systems symbolic reasoning in, 200 synthesis problems in, 202-203

SUBJECT INDEX

throw-away program in, 206 union of expertise in, 205 validation in, 201 Explanation subsystem, 173 Exponentiation, in public key cryptosystems, 82-85 EXTRA-HEP specialist, in MDX system, 204-205

F Fermat’s theorem, 69-70 Finite fields, for public key cryptosystems, 80-82 Firmware microprogramming and, 38 vs. software, 41 FIook cas, see Parker v. FIook FOL language, 182 Formal specifications, future trends in, 348, see also Abstract data types; Data abstraction FORTRAN code, copyrighting of, 5 FORTRAN language, 180 Frame-based representation languages, 181 Frames, defined, 181 FRL language, 181 Functional and procedural programming, 336-338 Functional symbol, in conventional algebra, 311

G Galois field, 80 Generalized languages, 182 Goal-driven control, in expert systems, 187-1 88 Goppa codes, 49, 93 Gorrschalk v. Benson, 4, 1 1 , 13, 16, 33-37 Graham v . John Deer Co., 10 Greatest common divisior, in public key cryptosystems, 67-68

H HALOTHANE frame, 289 HEARSAY411 language, 182 High-level user list operations, in abstract data types, 347

365

Huffman code, 47-49 Huffman decoding tree, 49

I IBM Lucifer, as product cipher system, 58-59 Implementation and verification, in abstract data types, 329-334 Incompleteness theorem (Coedel), 324 Induction principle, 312 Inference logical and plausible, 184-185, 190 rules of, 165 Inference procedure, in expert system, 173, 184-190 Information hiding, encapsulation as, 301 Initial algebras, 315-316 I n re Benson, 13, 24 In re Bernharr, 13 In re Bradley, 16, 22-23, 30, 36-43 In re Chatfield, 42 In re Freeman, IS, 42 I n re Gelnovatch, 42 I n re Musgrave, 13 I n re Noel, 42 In re Prater, 12 In re Toma, 15 In re Waldbaum, 39 Institute of Electrical and Electronic Engineers, 102 INTERLISP, in software development environment, 125-128, 131, 134 INTERLISP functions, in rule-based expert systems, 173 International Business Machines Company, 31 INTERNIST system, 221 Istack implementation, 329 Iterative enhancement, 110 Iterators defined, 341 programming languages and, 340-342 J Jacobi’s symbol, 82 JOSS, in software development environment, 125

366

SUBJECT INDEX

Journal of the American Medical Association, 278

K KASPE, see Kernal KAS program, 166, 180 Kernal, in ASPE program, 132, 138 Key encryption systems, public, see Public key encryption systems KL-ONE language, 181 Knapsack problem algorithm, in public key cryptosystems, 88-92, 97 Knapsack vector, 88 Knowledge expert system and, 165 high performance and, 180 representation of, 173-184 Knowledge acquisition, in expert system, 20 I Knowledge base answers to users’ questions about, 207210 consistence in, 182-183 in expert system, 165, 173 problem solving and, 225-226 Knowledge-based artificial intelligence, in medical reasoning, 219-220 Knowledge base editor, 173 Knowledge distribution, in medical diagnosis, 223-224 Knowledge engineering, defined, 166 Knowledge representation issues, 182-184 Known plain text attack, 51 KRL language, 181

L LABDATA basic class, in medical data conceptual model, 242 Least common multiple, in public key cryptosystems, 67 Linear list variations, algebraic specifications and, 345-348 LISP language, in software development environment, 127, 179-180 Logic-based representation languages, 181-182

M MACSYMA expert system, 166 Management Science America, 1 MAPSE, see Minimal Tool Set, Ada Programming Support Environment MARS system, as information management system, 251 Mathematical algorithm, patentability of, 28, 38, 42 MDX system, 219, see also Medical diagnosis bile duct cancer specialist in, 287-288 blackboard and, 271-274 cholestasis as domain of, 230-234 comparisons with other systems, 274275 diagnostic concept in, 222 end user evaluation of, 266 EXTRA-HEP specialist in, 284-285 “intelligent” data bases in, 227-230 knowledge structure of, 226-227 medical community organization and, 224-225 organization of, 221-224, 229 performance on example case, 278-289 preliminary evaluation of, 269-270 problem-solving embedding in, 225-226 problem-solving strategy and rules in, 234-240 RADEX program and, 251 Medical community organization, diagnostic problem solving and, 224-225 Medical data conceptual model of, 241-244 PATREC intelligent data base for, 241244 temporal organization of, 259-266 Medical diagnosis, see also MDX system auxiliary systems in, 240-266 data acquisition in, 246-247 diagnostic problem solving in, 271-274 evaluation of diagnostic performance in, 266-275 further research in, 276-277 knowledge distribution in, 223, 224 OVERVIEW critic in, 277 patient model in, 245-246 problem solving in, 222, 271-274 query language in, 247-249

SUBJECT INDEX

radiology consultant in, 249-250 temporal information in, 259-266 Medical knowledge, conceptual representation of, 217-291, see also MDX system: Medical diagnosis Medical reasoning, knowledge-based artificial intelligence in, 219 Mental Steps Doctrine, 12-13 Mesa language, 300 Minimal Tool Set, Ada Programming Support Environment, 131-132 Modularization, data abstraction and, 296 Module, as compilation unit, 300-301 Modus ponens inference rule, 165 MRS language, 182 MUMPS, in software development environment, 125 MYCIN consultation session, 167-173 MYCIN system, 165-173, 177, 221 certainty theory and, 194

N National Bureau of Standards, 60 National Science Foundation, 102 National Security Agency, 60 Natural numbers, algebra of, 310-312 Non-TO1 domain, 308, 314-317 0

Object-centered programming, 181 On-board satellite communication systems, public key cryptosystems and, 96 One-time pad system, in cryptography, 57, 101

One-time tape system, 50 One-way function, in public key cryptosystem, 62 OPS framework system, 180 OVERVIEW, in medical diagnosis evaluation, 277 OWL language, 181

P PAIN frame, in medical diagnosis, 246 Parameterized set type, 321 Parker v. Flook, 15, 19-20, 34-35 PASCAL language, 180

367

Patent Act (1793), 33 Patent Office in computer program patenting, 12 Mental Steps Doctrine and, 12-13 Patents and patent law, 10-24 Patient model, episodic organization of, 262-263 PATREC data base system, 219, 222, 228229, 241-248, 275 general interaction with, 249 query evaluation example for, 289-291 representation of, 244 Personal development systems configuration for, 156-157 network of, 155-156 PHYSICAL class, 241 Physical environments, in software engineering environments, 141-145 Physician, cognitive activity of, 220-221 Plain text, defined, 46 PLAIN tool, in User Software Engineering, 137 PLANNER language, 182 Polyalphabetic substitution system, in cryptography, 56-57 Precision, in knowledge representation, I83 Premise, in EMYCIN rule, 178 President’s Commission on the Patent System, 12 Primality testing, in public key cryptosystems, 69, 79-80 Prime factorization, in public key cryptosystems, 79-80, 85-88 Primitive roots, in public key cryptosystems, 71-75 Problem solving knowledge base and, 225 in MDX system, 225-226 strategy used in expert systems, 200 Problem specification, PSL/PSA tools for, I52 Product cipher system, in cryptosystems, 59-60 Production system EMYCIN as, 177-180 in rule-based representation frameworks, 176-177 Program in object code, copyrightability of, 29

368

SUBJECT INDEX

Programmer’s Workbench, Unix environment in. 130-131 Programming systems, in software development environment, 125-129 PROLOG language, 182 PROSPECTOR program, 165, 189, 192, 205 Pseudo-PASCAL dialect, 189 PSLlPSA tools, in problem specification, 152 Public key cryptosystem algorithms, 45102 based on algebraic coding theory, 93-94 based on exponentiation and discrete logarithms, 82-88 based on knapsack problem, 88-92 Public key cryptosystems applications of, 94-101 authentication of, 94-96 binary algorithm for, 71 Chinese remainder theorem in, 70 complete residue systems in, 69 conference key distribution systems in, 99-100

congruences in, 68-71 data-base security in, 100-101 divisibility in, 64-68 examples of, 82-94 finite fields in, 80-82 Galois field in, 81 ground field in, 81 least common multiple and greatest common divisor in, 67 mathematical formalism for, 62-64 mathematical preliminaries for, 64-82 ope-way function in, 62 primality testing in, 69, 79-80 primary motive for development of, 94 prime factorization in, 79-80 primes and composites in, 65 primitive roots and discrete logarithms in, 71-75 quadratic residues in, 75-78 quotient, residue, and remainder in, 65 read-only secure communications in, 98 shared data in, % trapdoor one-way function in, 63 Public key encryption systems, 60-62 PUFF program, 165

Q Quadratic residues, in public key cryptosystems, 75-78 Queue, algebraic specification of, 306 QUIKTRAN programming system, 125

R RADEX program or subsystem, 219,221, 249-259, 275 anatomical-physiological model in, 253254 conceptual model in, 251-259 deformities and abnormalities in, 254 function of, 228-229, 250 interaction with, 258-259 morphological procedures in, 252 organ parts in, 254-255 patient model in, 256-257 radiology data base in, 249-251 representation in, 244 RAINBOW system, 180 RAM (random access machine), 63 RAPID (Rapid Prototypes of Interactive Dialogues), 137-138 Read-only memory, in copyright law, 3 Read-only secure communications, in public key cryptosystems, 98 Reasoning with uncertainty certainty theory and, 194-196 Dempster-Shafer theory of evidence and, 197-198 in expert systems, 190-194 possibility theory and, 196 uncertain evidence and, 193-196 Representation function, 329 Representation invariant, 335 Representation languages frame- and logic-based, 181-182 syntactic and semantic completeness in, 183 Representation of knowledge, 173-184 extendability of, 174-175 framework for, 175-176 rule-based representation frameworks in, 176-180 RLL language, 182 ROM (read-only memory), copyrightability Of, 3-4, 29-30,41

SUBJECT INDEX

ROSIE language, 182 Rule-based expert systems key components of, 173 principles of, 163-210 Rule-based knowledge representation, alternatives to, 180-182 Rule-based representation frameworks, 176- 180

369

Software development environment, 138141

automated support in, 119-120 automation in, 123-133 configuration management in, 124-125 development dictionary in, 125 operating system level support in, 123124

control strategy in, 177 production systems in, 176-177 Rule interpreter, 176

S

programming systems in, 125-129 software tools in, 123-125 static and dynamic analyzers in, 124 tool-kit approach in, 129-131 tool system in, 123 Software development methodology, 110, 119-122

Scrambling matrix, in public key cryptosystems, 93 Security, as basic premise of cryptography, 101 Select-Goal procedure, 189 Select-Rule procedure, 189 Semantic completeness, of knowledge base, 183 Semantic nets, generalization of, 181 Semantic specification, in abstract data types, 305 Shannon-Fano code, 47 Shared data, in public key cryptosystems, %

Smalltalk, as language for Dynabook, 128, 134, 157

Software as “big business,” 1 copyrighting of, 3-9 economics of, 1-2 legal protection of, 1-43 patentability of, 28-29 Software design, system development and, 113-114

Software developer computer connections for, 147 lighting for, 146-147 noise level for, 144-145 office size and furniture for, 145-146 physical environment for, 143-146 privacy for, 144 salary level for, 158 Software development, improved computing support in, 155-157 Software development dictionary, 125

components of, 120 life-cycle coverage in, 120-121 phase transactions in, 121 problem-solving support in, 120 software development organization support in, 121-122 system evolution support and, 122 validation support in, 121 Software development organization developer’s office and, 143-146 support services of, 142-143 Software engineering, defined, 110 Software engineering environments, 109158

and automation in development environment, 123-133 computer terminals and, 147-149 cost and schedule estimations for, 117, 151-152

improved computing support and, 155157

improved methodologies in, 149-152 improved software tools in, 152-155 INTERLISP and Smalltalk in, 127-128 local network configuration in, 157 management procedures in, 115-1 19 parallelism and concurrency improvements in, 150-151 personnel deployment in, 116-1 17 physical environment and, 141-149 problem solving support in, 120 project progress evaluation and, 117-1 18 release control in, 118-1 19 Software life cycle, 111-115 coding and implementation in, 114

370

SUBJECT INDEX

functional specification in, 112 model of, 149-150 problem analysis in, 112 quality testing and verification in, 115 support for, I55 Software process, in real time, 153 Software products, yields of, 139 Software systems, development environment for, 111 Software tools adaptability of, 154 improvement in, 152-155 Software Tools Project, 132 Specification meaning, admissible algebras and, 313 Superincreasing vector, in public key cryptosystems, 89 Supreme court, U.S., 4, 11, 14-24,N Symbolic reasoning, in expert systems, 200 Synchronization problems, data abstraction and, 338-339 Synercom Technology, Inc. v. University Computing Co., 6, 8 Syntactic completeness, in representative language, 183

T Tandy Corporation v. Personal Micro Computers, 30 TDI (Transition Diagnosis Interpreter), USE Specification and, 135-137 TEIRESIAS program, 189 Temporal information importance of, 259-260 issues in, 260-262 organization of, 259-266 questions in, 264-265 Terminal algebras, 316-320 Throw-away program, in expert systems, 206 Time sharing, operational weaknesses of, 156 TO1 domain, see Type of interest domain Tool-kit approach, in software development environment, 129-131 Tool systems, in software development environment, 123-125, see also Software tools Trade secrecy, 24-26 Copyright Act (1976) and, 26-29

Trade secrecy law, primer of, 25-26 Trade secret, defined, 25 Transition Diagram Interpreter, 135-137 Trapdoor ondway function, in public key cryptosystem, 63 Traversible stack syndrome, 335-336 Trees, data structures and, 344 Turing machine, 63 Type of interest domain, 308, 314, 316317, 320, 333

U Uncertain evidence, in expert systems, 195-1% Uncertainty, reasoning with, 190-198 Unconditionally secure cryptosystems, 5253 Unicity.point or distance, 51 Unified Support Environment, 137 United States International Trade Commission, 30 United States Patent Office, 31 UNITS language, 181 Universal Athletic Sales Co. v. Salkeld, 5 Unix operating system, in software development environment, 129-1 31 USE Control System, 137 User Software Engineering design and implementation in, 133, 137138 methodology overview for, 133-135 prototypes in, 136-137 specification format for, 135-136

V Valmont Industries, Inc. v. Yuma Manufacturing Co., 10 VALUE attribute, in medial diagnosis, 243

W Warrington Associates, Inc. v. Real-Time Engineering Systems, 9 Well-formed terms in algebraic approach, 311, 315-316, 318-320,333 context-free grammar of, 304-306 primitive list specification and, 346 Works of authorship, defined, 3

Contents of Previous Volumes Volume 1 General-Purpose Programming for Business Applications CALVIN c. GOTLIEB Numerical Weather Prediction NORMAN A. PHlLLrPs The Present Status of Automatic Translation of Languages YEHOSHUA BAR-HILLEL Programming Computers to Play Games L. SAMUEL ARTHUR Machine Recognition of Spoken Words RICHARDFATEHCHAND Binary Arithmetic GEORGEw. REITWIESNER

Volume 2 A Survey of Numerical Methods for Parabolic Differential Equations JIM DOUGLAS, JR. Advances in Orthonormalizing Computation PHILIP J. DAVIS A N D P H I L I P RABINOWITZ Microelectronics Using Electron-Beam-Activated Machining Techniques KENNETH R. SHOULDERS Recent Developments in Linear Programming SAULI. GLASS The Theory of Automata: A Survey ROBERTMCNAUGHTON

Volume 3 The Computation of Satellite Orbit Trajectories SAMUEL D. CONTE Multiprogramming E. F. CODD Recent Developments of Nonlinear Programming PHILIP WOLFE Alternating Direction Implicit Methods GARRETBIRKHOFF, RICHARD s. V A R G A , A N D DAVID YOUNG Combined Analog-Digital Techniques in Simulation HAROLDF. SKRAMSTAD Information Technology and the Law REEDC. LAWLOR

Volume 4 The Formulation of Data Processing Problems for Computers WILLIAM c . MCGEE

371

372

CONTENTS OF PREVIOUS VOLUMES

All-Magnetic Circuit Techniques DAVIDR. BENNIONA N D HEWITTD. CRANE Computer Education HOWARDE. TOMPKINS Digital Fluid Logic Elements H. H. GLAETTLI Multiple Computer Systems WILLIAMA. CURTIN Volume 5

The Role of Computers in Electron Night Broadcasting JACKMOSHMAN Some Results of Research on Automatic Programming in Eastern Europe WLADYSLAW TURKSI A Discussion of Artificial Intelligence and Self-organization GORDONPASK Automatic Optical Design ORESTESN . STAVROUDIS Computing Problems and Methods in X-Ray Crystallography L. COULTER CHARLES Digital Computers in Nuclear Reactor Design ELIZABETH CUTHILL An Introduction to Procedure-Oriented Languages HARRYD. HUSKEY Volume 6

Information Retrieval CLAUDEE. WALSTON Speculations Concerning the First Ultraintelligent Machine IRVING JOHNGOOD Digital Training Devices CHARLES R. WICKMAN Number Systems and Arithmetic HARVEY L. GARDER Considerations on Man versus Machine for Space Probing P. L. BARGELLINI Data Collection and Reduction for Nuclear Particle Trace Detectors HERBERT GELERNTER Volume 7

Highly Parallel Information Processing Systems JOHNC. MURTHA Programming Language Processors RUTHM. DAVIS The Man -Machine Combination for Computer-Assisted Copy Editing WAYNEA. DANIELSON Computer-Aided Typesetting WILLIAMR. BOZMAN


Programming Languages for Computational Linguistics ARNOLDC. SATTERTHWAIT Computer Driven Displays and Their Use in Man-Machine Interaction ANDRIESV A N DAM Volume 8 Time-shared Computer Systems THOMAS N. PIKE,JR. Formula Manipulation by Computer JEANE. SAMMET Standards for Computers and Information Processing T. B. STEEL, JR. Syntactic Analysis of Natural Language NAOMISAGER Programming Languages and Computers: A Unified Metatheory R. NARASIMHAN Incremental Computation LIONELLO A. LOMBARDI Volume 9 What Next in Computer Technology W. J. POPELBAUM Advances in Simulation JOHNMCLEOD Symbol Manipulation Languages PAULW. ABRAHAMS Legal Information Retrieval AVIEZRIS. FRAENKEL Large-Scale Integration- An Appraisal L. M. SPANDORFER Aerospace Computers A. S. BUCHMAN The Distributed Processor Organization L. J . KOCZELA Volume 10 Humanism, Technology, and Language CHARLESDECARLO Three Computer Cultures: Computer Technology, Computer Mathematics, and Computer Science PETERWEGNER Mathematics in 1984-The Impact of Computers BRYANTHWAITES Computing from the Communication Point of View E. E. DAVID,JR. Computer-Man Communication: Using Graphics in the Instructional Process FREDERICK P. BROOKS,JR.

373

374


Computers and Publishing: Writing, Editing, and Printing ANDRIESVAN DAMAND DAVIDE. RICE A Unified Approach to Pattern Analysis ULF GRENANDER Use of Computers in Biomedical Pattern Recognition ROBERTS. LEDLEY Numerical Methods of Stress Analysis PRAGER WILLIAM Spline Approximation and Computer-Aided Design J. H. AHLBERG Logic per Track Devices D. L. SLOTNICK

Volume 11 Automatic Translation of Languages Since 1960: A Linguist’s View HARRYH. JOSSELSON Classification, Relevance, and Information Retrieval D. M. JACKSON Approaches to the Machine Recognition of Conversational Speech KLAUSW. OTTEN Man- Machine Interaction Using Speech DAVIDR. HILL Balanced Magnetic Circuits for Logic and Memory Devices R. B. KIEBURTZ AND E. E. NEWHALL Command and Control: Technology and Social Impact ANTHONYDEBONS

Volume 12 Information Security in a Multi-User Computer Environment JAMESP. ANDERSON Managers, Deterministic Models, and Computers G. M. FERRERO DIROCCAFERRERA Uses of the Computer in Music Composition and Research HARRYB. LINCOLN File Organization Techniques DAVIDC. ROBERTS Systems Programming Languages R. D. BERGERON, J. D. GANNON, D. P. SHECHTER, F. W. TOMPA, A N D A. VAN DAM Parametric and Nonparametric Recogination by Computer: An Application to Leukocyte Image Processing JUDITHM. s. h E W t T T

Volume 13 Programmed Control of Asynchronous Program Interrupts RICHARD L. WEXELBLAT Poetry Generation and Analysis JAMES JOYCE


Mapping and Computers PATRICIA FULTON Practical Natural Language Processing: The REL System as Prototype FREDERtCK B. THOMPSON A N D BOZENA HENISZTHOMPSON Artificial Intelligence-The Past Decade B. CHANDRASEKARAN Volume 14 On the Structure of Feasible Computations J. HARTMANIS A N D J. SIMON A Look at Programming and Programming Systems T . E. CHEATHAM, JR. A N D J U D YA. TOWNELY Parsing of General Context-Free Languages SUSAN L. GRAHAM A N D MICHAELA. HARRISON Statistical Processors W. J. POPPELBAUM Information Secure Systems I. BAUM DAVIDK. HSIAOA N D RICHARD Volume 15 Approaches to Automatic Programming ALANW. BIERMANN The Algorithm Selection Problem JOHNR. RICE Parallel Processing of Ordinary Programs DAVIDJ. KUCK The Computational Study of Language Acquisition LARRY H . REEKER The Wide World of Computer-Based Education DONALDBITZER Volume 16 3-D Computer Animation CHARLES A. CSURI Automatic Generation of Computer Programs NOAHS. PRYWES Perspectives in Clinical Computing A. HALUSKA KEVINC. O’KANEA N D EDWARD The Design and Development of Resource-Sharing Services in Computer Communications Networks: A Survey SANDRA A. MAMRAK Privacy Protection in Information Systems REINT U R N Volume 17 Semantics and Quantification in Natural Language Question Answering W. A. WOODS

375

376


Natural Language Information Formatting: The Automatic Conversion of Texts to a Structured Data Base NAOMISAGER Distributed Loop Computer Networks MINGT . LIU Magnetic Bubble Memory and Logic TIENCHI CHENA N D Hsu CHANG Computers and the Public’s Right of Access to Government Information ALANF. WESTIN Volume 18 Image Processing and Recognition AZRIELROSENFELD Recent Progress in Computer Chess MONROEM. NEWBORN Advances in Software Science M. H. HALSTEAD Current Trends in Computer-Assisted Instruction PATRICKSUPPES Software in the Soviet Union: Progress and Problems S. E. GOODMAN Volume 19 Data Base Computers DAVIDK. HSIAO The Structure of Parallel Algorithms H. T. KUNG Clustering Methodologies in Exploratory Data Analysis RICHARD DUBESA N D A. K. JAIN Numerical Software: Science o r Alchemy? C. W. GEAR Computing as Social Action: The Social Dynamics of Computing in Complex Organizations ROB KLINGA N D WALTSCACCHI Volume 20 Management Information Systems: Evolution and Status GARYW. DICKSON Real-Time Distributed Computer Systems JENSEN,R. Y. KAIN, N D GEORGED. M RSHALL W. R. FRANTA,E. DOUGLAS Architecture and Strategies for Local Networks: Examples and Important Systems K. J. THURBER Vector Computer Architecture and Processing Techniques KAI HWANG,SHUN-PIAOSu, A N D LIONEL M. Nr An Overview of High-Level Languages JEANE. SAMMET

CONTENTS OF PREVIOUS VOLUMES Volume 21

The Web of Computing: Computer Technology as Social Organization ROB KLINCA N D WALTSCACCHI Computer Design and Description Languages SUBRATA DASGUF-TA Microcomputers: Applications, Problems, and Promise C . GAMMII.L ROBERT Query Optimization in Distributed Data Base Systems GIOVANNI MARIASACCO A N D S. BINGYAO Computers in the World of Chemistry Peter LYKOS Library Automation Systems and Networks JAMES E. RUSH

377


Advances in Computers, Volume 22

Advances in Computers, Volume 49

Advances in Computers, Volume 55

Advances in Computers, Volume 56

Advances in Computers, Volume 19

Advances in Computers, Volume 12

Advances in Computers, Volume 43

Advances in Computers, Volume 28

Advances in Computers, Volume 31

Advances in Computers, Volume 9

Advances in Computers, Volume 11

Advances in Computers, Volume 7

Advances in Computers, Volume 3

Advances in Computers, Volume 17

Advances in Computers, Volume 34

Advances in Computers, Volume 47

Advances in Computers, Volume 29

Advances in Computers, Volume 14

Advances in Computers, Volume 45

Advances in Computers, Volume 6

Advances in Computers, Volume 13

Advances in Computers, Volume 5

Advances in Computers, Volume 82

Advances in Computers, Volume 3

Advances in Computers, Volume 18

Advances in Computers, Volume 80

Advances in Computers, Volume 20