Advances in Computers, Volume 28

Advances

in COMPUTERS VOLUME 28

Contributors to This Volume

MUSTAFA A. G . ABUSHAGUR H. JOHN CAULFIELD DASGUPTA SUBRATA M . H . EICH A. R. HURSON ABRAHAM KANDEL MANFRED KOCHEN L. L. MILLER MIRMOJTABA MIRSALEHI S. H. PAKZAD MORDECHAY SCHNEIDER B. SHIRAZI

Advances in

COMPUTERS EDITED BY

MARSHALL C. YOVITS Purdue School of Science Indiana University --Purdue Indianapolis, Indiana

University of Indianapolis

VOLUME 28

ACADEMIC PRESS, INC. Harcourt Brace Jovanovich, Publishers

Boston San Diego New York Berkeley London Sydney Tokyo Toronto

COPYRIGHT @ 1989 BY ACADEMIC PRESS.INC

ALL RIGHTS RESERVED. N O PART OF THIS PUBLICATION MAY BE REPRODUCED OR TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT PERMISSION IN WRITING FROM THE PUBLISHER.

ACADEMIC PRESS, INC. 1250 Sixth Avenue, San Diego, CA 92101

United Kingdom Edition published b y

ACADEMIC PRESS INC. (LONDON) LTD. 24-2XOval Road. London NWI 7DX

LIBRARY OF CONGRESS CATALOG CARDNUMBER:59-15761 ISBN 0-12-012128-X PRINTED IN THE UNITTEV STATE Ok AMERICA

R9 90 91 92

Y H 7 6 5 4 3 2 I

Contents CONTRIBUTORS . . . . . . . . . . . . . . . . . . . PREFACE. . . . . . . . . . . . . . . . . . . . .

vii ix

The Structure of Design Processes Subrata Dasgupta

1. Introduction . . . . . . . . 2. The Basic Characteristics of Design 3 . Design Paradigms . . . . . . 4. Design as Scientific Discovery . . 5. Conclusions . . . . . . . . References . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . .

1

3 29 55 61 62

Fuzzy Sets and Their Applications to Artificial Intelligence Abraham Kandel and Mordechay Schneider

1. 2. 3. 4. 5.

Introduction . . . . . . . . . . . . . . FuzzySets . . . . . . . . . . . . . . . . Fuzziness and Typicality Theory . . . . . . . Applications of Fuzzy Set Theory to Expert Systems Conclusion . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

69

. 71 . . 79 . . 90 . 101 . 103

Parallel Architectures for Database Systems

. .

. .

. .

A R Hurson. L L Miller. S H Pakzad. M H Eich. and B Shirazi

. .

.

1. Introduction . . . . . . . . . 2. Classification of Database Machines . 3. Database Machines . . . . . . . 4. Conclusion and Future Directions . . References . . . . . . . . . . .

. . . .

. . . .

. . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . .

108 110

119 144

146

Optical and Optoelectronic Computing Mir Mojtaba Mirsalehi. Mustafa A G Abushagur. and H John Caulfield

. .

.

1 . Introduction . . . . . . . . . . . . . . . . . . 2. Basicoperationsfor Optical Computations . . . . . . . V

154 155

vi

CONTENTS

3. 4. 5. 6. 7.

Elements of Optical Computers. . . Analog Processors . . . . . . . Digital Processors . . . . . . . Hybrid Processors . . . . . . . Conclusion . . . . . . . . . . . References . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

157 175 196 212 219 221

Management lntelllgence Systems Manfred Kochen

1 . Introduction . . . . . . . . . . . . . . . . . . . 227 2. On the Nature of Intelligence . . . . . . . . . . . . . 234 3. What is a MINTS: Requirements and Uses . . . . . . . . . 242 4. Analysis. Design and Maintenance of MINTSs . . . . . . . 253 5. Managerial Issues . . . . . . . . . . . . . . . . . 267 273 6. Conclusion . . . . . . . . . . . . . . . . . . . . 274 References . . . . . . . . . . . . . . . . . . . . AUTHORINDEX.

. . . . . . . . . . . . . . . . .

279

INDEX. . . . . . . . . . . . . . . . . . SUBJECT

287

OF PREVIOUS VOLUMES . . . . . . . . . . . . CONTENTS

295

Contributors Numbers in parentheses refer to the pages on which the authors' contributions begin.

Mustafa A. G. Abushagur (153), Electrical and Computer Engineering Department. University of Alabama in Huntsville, Huntsville, Alabama 35899 H. John Caulfield (153), Center for Applied Optics, University of Alabama in Huntsville, Huntsville, Alabama 35899 Subrata Dasgupta (l), The Center for Advanced Computer Studies, University of Southwestern Louisiana, Lafayette, Louisiana 70504-4330 M. H. Eich (107), Department of Computer Science, Southern Methodist University, Dallas, Texas 7527.5 A. R. Hurson (107),Computer Engineering Program, Department of Electrical Engineering, Pennsylvania State University, University Park, Pennsylvania 16802 Abraham Kandel (69), Computer Science Department and The Institute for Expert Systems and Robotics, Florida State University, Tallahassee, Florida 32306 - 4019 Manfred Kochen' (227), School of Medicine (Mental Health Research Institute) and School of Business Administration (Computer & Information Systems), University of Michigan, Ann Arbor, Michigan 48109 of Michigan, Ann Arbor, Michigan 48109 L. L. Miller (107), Department of Computer Science, Iowa State University, Ames, Iowa 5001 1 Mir Mojtaba Mirsalehi (153), Electrical and Computer Engineering Department, University of Alabama in Huntsville, Huntsville, Alabama 35899 S. H . Pakzad (107),Computer Engineering Program, Department of Electrical Engineering, Pennsylvania State University, University Park, Pennsylvania 16802 Mordechay Schneider (69), Computer Science Department and The Institute for Expert Systems and Robotics, Florida State University, Tallahassee, Florida 32306-4019 B, Shirazi ( 1 07), Department of Computer Science, Southern Methodist University, Dallas, Texas 7.5275

vii

This Page Intentionally Left Blank

Preface

The publication of Volume 28 of Adaunces in Computers continues the in depth presentation of subjects of both current and continuing interest in computer and information science. Contributions have been solicited from well respected experts in their fields who recognize the importance of writing substantial review and tutorial articles in their areas of expertise. Advunces in Computers permits the publication of survey-type articles written from a relatively leisurely perspective; authors are thus able to treat their subjects both in depth and in breadth. The Advances in Computers series began in 1960 and now continues in its 29th year with Volume 28. During this period, which witnessed great expansion and dynamic change in the computer and information fields, it has played an important role in the development of computers and their applications. The continuation of the series over this lengthy period is a tribute to the reputations and capabilities of the authors who have written for it. Volume 28 includes chapters on design processes, fuzzy sets as related to artificial intelligence, database systems, optical computing, and intelligence systems for management. In the first chapter, Dr. Dasgupta states that design is one of the most ubiquitous of human activities. Anyone who devises a course of action to change an existing state of affairs to a preferred one is involved in the act of design. As such, it is of central concern not only in traditional engineering but also in the generation of symbolic or abstract devices such as plans, organizations, and computer programs. He provides a systematic presentation of current understanding of the structure of design processes. Quite independent of the specific design domain, design problems share a common structure so that it is possible to talk of general theories of design, that is, general, domain-independent, explanatory models of the design process. Dr. Kandel and Dr. Schneider are concerned with fuzzy sets and their applications, particularly to artificial intelligence and to knowledge engineering. They point out that the theory of fuzzy sets has as one of its aims the development of a methodology for formulating and solving problems that are too complex or too ill-defined to be susceptible to analysis by conventional techniques. They believe the main reason for using fuzzy set theory in artificial intelligence and expert systems is that much of the information that is resident in knowledge-based systems is uncertain in nature. Since the early 1970s, the complexity of conventional database management systems has gradually increased by the number and size of databases and the number and type of application programs and on-line users. Professor ix

X

PREFACE

Hurson and his collaborators state that conventional systems using typical software approaches fail to meet the requirements of the various applications, and that since the mid 1970s a great deal of effort has been directed towards the design of special-purpose architectures for efficient handling of large database systems, namely Data Base Machines. The primary goal of their chapter is to examine the impact of current technology on the design of special-purpose database machines. Professors Mirsalehi, Abushagur, and Caulfield tell us that optical computing emerged from the sciences of holography and coherent optical information processing in this decade and developed as an important discipline only in 1983-1984. The authors discuss the fundamentals of optical computing, the elements of optical computers, and different types of optical processors. Optical computing, they maintain, should not duplicate the architectures that have been used for electronic computers, but rather should utilize techniques that take advantage of the strengths of optics and avoid its weaknesses. One of the most important features of optics is its capability of global interconnections. Therefore, areas such as neural networks, which utilize this feature, are the most promising for optical computing. According to Dr. Manfred Kochen, a management intelligence system is intended to scan the environment of the organization it serves, making it possible for management to better assess its position, thus enhancing the value of the organization and its services. Simple versions of such systems, he points out, have existed for a long time. The management intelligence systems required by competing organizations in government and business are the best they can obtain, making use of advanced technology, such as artificial intelligence. The purpose of such man-machine systems is to support professional strategies, planners, and researchers, as would a good semiautomated research assistant. The requirements can also provide needed direction to research in artificial intelligence since both are necessary for a management intelligence system to be effective. The purpose of Kochen’s chapter is to emphasize the importance of management intelligence systems and to encourage studies involving them. I am saddened to learn of the sudden and unexpected loss of my good friend and valued colleague who wrote this chapter on management intelligence systems. Shortly after receipt of the final version of his article, Fred Kochen was suddenly and fatally stricken. He will be missed both as a friend and as a leader in our profession. He and his students have had a major effect on the development of information science, culminating in the innovative and important article in this volume. Fred and I have been close colleagues and friends for many years. I will especially miss his counsel and advice of both a professional and a personal nature. I am pleased to thank the contributors to this volume. They gave extensively of their time and effort to make this book an important and timely

PREFACE

xi

contribution to their profession. Despite the many calls upon their time, they recognized the necessity of writing substantial review and tutorial contributions in their areas of expertise. I t required considerable effort on their part, and their cooperation and assistance are greatly appreciated. Because of their emorts, this volume achieves a high level of excellence and should be of great value and substantial interest for many years to come. It has been a pleasant and rewarding experience for me to edit this volume and to work with those authors. MARSHALL C. YOVITS

This Page Intentionally Left Blank

The Structure of Design Processes SUBRATA DASGUPTA The Center for Advanced Computer Studies University of Southwestern Louisiana Lafayette, Louisiana

1. Introduction. . . . . . . . . . . . . . . . . 2. The Basic Characteristics of Design . . . . . . . . . 2.1 To Design is to Change . . . . . . . . . . . 2.2 Design Begins with Requirements. . . . . . . . 2.3 To Design is to Represent . . . . . . . . . . 2.4 The Satisficing Nature of Design Processes . . . . 2.5 The Evolutionary Nature of Design Processes . . . 2.6 Summary . . . . . . . . . . . . . . . . 3. Design Paradigms . . . . . . . . . . . . . . . 3.1 Some Terminological Clarifications . . . . . . . 3.2 The Analysis-Synthesis-Evaluation Paradigm. . . . 3.3 The Artificial Intelligence Paradigm . . . . . . . 3.4 The Algorithmic Approach . . . . . . . . . . 3.5 The Formal Design Paradigm . . . . . . . . . 3.6 The Theory of Plausible Designs . . . . . . . . 4. Design as Scientific Discovery . . . . . . . . . . . 4.1 The Hypothetico-Deductive (HD) Form of Reasoning. 4.2 Kuhnian Paradigms . . . . . . . . . . . . 4.3 Basis for the DSD Hypothesis . . . . . . . . . 5. Conclusions . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . .

1.

. . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 3 3 5 8 14 11 28 29 29 30 34 38 40 41 55 55 51 51 61 62

Introduction

In its broadest sense, design is one of the most ubiquitous of human activities. As Simon (1981) has pointed out, anyone who devises a course of action to change an existing state of affairs to a preferred one is involved in the act of design. As such, it is of central concern not only in traditional engineering-dealing with such material artifacts as structures, machines, and production plants-but also in the generation of symbolic or abstract devices such as plans, organizations and computer programs. If we extend the sense of the term “engineering” to encompass the generation of all useful artifacfs then, 1 ADVANCES IN COMPUTERS. VOL. 28

Copyright ‘ new Out = In1 and In2 when 2#0100# = > new Out = In1 or In2 when ?#0011# = > new Out = In1 In2

+

end case end module

FIG. 2. Description of the data fragment in a hardware description language.

Consider, however, the usefulness of Fig. 2 when someone is reviewing the design and begins to ask “why” questions-questions of ju.st$cation. For instance, the reviewer may want to know why the functional unit is designed to be active in phase 2 of the clock Clk; or why Clk has the characteristics that it has. As a representation, Fig. 2 is hopelessly inadequate in providing information of this sort-information that is crucial when one views a representation as a medium of experimentation and exploration. For such purposes, a design representation should include a documentation of the justification of design decisions and of the cause-effect relationships between design decisions.

Example 2.8 In the design paradigm called the theory of plausible designs (Aguero and Dasgupta, 1987; see Section 3.6 below), one of the principal issues addressed is that of the plausihiliry of a design-that is, to demonstrate through rhr design represrntation itself, the plausibility (or believability) of a design as a whole or of its specific components. For this purpose, the design of a target system is represented in a number of ways. Firstly, the design can be described in a formal, machine-executable language (e.g., a programming or hardware description language). Such a representation is most appropriate as a blueprint for implementation or for the purpose of experimentation using the description as an input to the simulator.

12

SUBRATA DASGUPTA

s1:

C: R(1) = The interconnection network I is reliable. A: Formal Description of I in S*M R : FD(1) A FT(1) where:

FD(1) = I is fault-diagnosable FT(1) = I is fault-tolerant V : formal and empirical methods

P: uuliduted

FIG.3. A plausibility statement.

In addition, each characteristic or feature appearing in a design is explicitly and completely defined by means of a construct called a plausibility statement which describes the (extent of the) plausibility of the feature and the nature of the evidence used to demonstrate or justify this plausibility. Plausibility statements thus serve to record the questions of justification alluded to earlier. Figure 3 is one such plausibility statement. Here, the property of interest is R(I), the reliability of the interconnection network I. This is a “second order” property taking as an argument a “first order” property, or feature, I. This statement documents the fact that the property R(I), as a feature of the design, will have a high degree of plausibility (more strictly speaking, will be in “plausibility state” validated) if, using a combination of formal and empirical methods, it can be shown there is evidence for “FD(1) and FT(1)” to be true. This statement thus documents why we may have confidence in R(I) as a component of the design. The third type of representation is used to show the interdependencies between the plausibilities of properties. For instance, the plausibility of R(I) depends, in turn, on the plausibilities of FD(1) and FT(1).This state of affairs is described explicitly by a plausibility dependency graph (Fig. 4). In general, a design at any stage can be represented by such a graph. Example 2.9 Liskov and Guttag( 1986)discuss an approach to program development in which a design is represented using a combination of an abstract (but formal) specification notation which expresses the functional characteristics of the target system, dependency diagrams which show dependencies between program modules, and descriptions of performance (or efficiency)constraints to be met by the system. Examples of efficiency constraints are:

worst case time = O(length(n) x length(n)), temporary space used Ilength(n),

THE STRUCTURE OF DESIGN PROCESSES

13

FIG.4. Plausibility dependency graph.

proc A

FIG.5 . Module dependency diagram.

where n is a character string. Figure 5 shows a module dependency diagram. This indicates that procedure A is dependent on (or uses) procedure B and abstract data type D in the sense that A calls B and uses one or more of the operations of D. B is also dependent on D.

14

SUBRATA DASGUPTA

2.4 The Satisficirig Nature of Design Processes

Consider the computer architect who, given a particular set of (initial) requirements, is to design an exo-architecture that meets these requirements’. The principal components of the exo-architecture are the following: (a) Storage organization: the types and organization of programmable storage. (b) Data types: definition and characterization of the data types and their representation in storage. (c) Addressing modes: the different methods of specifying the addresses of storage objects (d) Instruction set: specification of the syntax and semantics of the instructions. (e) Instruction formats: representation and encoding of instructions in storage. (f) Exception conditions: specifications of faults, traps, and interrupts, and their consequences. The problem in designing exo-architectures is that the components interact with one another in the sense that a set of decisions concerning one component will influence the design of, or design choices for, some other components. For instance, ( i ) The choice of a storage organization (including the word length and the unit of addressability) is influenced by the specific data types and their representations. The converse is also true. (ii) The design of instruction formats will be directly affected by the design of the instructions, data types, and addressing modes. (iii) There is an obvious close relationship between data types and the instruction set: the range and composition of the latter will depend on the composition of the former, and vice versa.

Furthermore, these relationships do not follow, or cannot be captured by, a neat mathematical equation such that one can quantitatively predict the effect of variation of one component on another. The computer architect, in other words, can rarely (if at all) make optimum decisions in the course of design. A computer’s rxo-architecture is the total structure and behavior of the computer as seen by the assembly language programmer, operating systems designer or compiler writer (Dasgupta, 1984, 1988a. 1988b). Other terms used to refer to this abstraction level are “instruction set processor architecture”(Siewiorek, Bell, and Newell, 1982) and the “conventional machine level” (Tdnenbaum, 1984).

THE STRUCTURE OF DESIGN PROCESSES

15

The aforementioned issue is typical of most design problems. In general, the process of design involves making decisions that are characteristically of the form 1. Given a number of interacting objectives or goals, how should these

goals be prioritized? That is, how should one order decisions concerning a number of interacting components when the nature of the interactions is known only qualitatively or imprecisely? 2. Given a number of alternative choices (say, for a component of a system) that are equivalent-in terms of function, performance, and/or costwhich of these choices should be made? As noted in Section 2.2.2 there are limits or bounds to the rationality that the designer can bring to a design problem. This is largely due to the complexity of the design problem and the designer’s imperfect knowledge of the long-term consequences of design decisions. The notion of bounded rationality led Simon (1981) to suggest that for design problems of any ressonable complexity, one must be content with “good” rather than “best” solutions. That is, the designer sets some criterion of sarisjactoriness for the design problem and if the design meets the criterion, the design problem is considered (even if only temporarily) to be solved. In Simon’s terms, design problem solving (and many other types of problem solving) are safisjcing procedures in that they produce satisfactory rather than optimal solutions.

2.4.1

The Exponential Nature of Well-Structured Design Problems

It may be remarked that the use of computer-aided design (CAD) or design automation alleviates to some extent the problem of bounded rationality since such tools facilitate the use of systematic or algorithmic means for solving design problems. However, even if the design problem is sufficiently wellstructured as to lend itself to algorithmic procedures, there are inherent limits to the practical attainment of optimal solutions. For, as is well known, most of the interesting optimization problems encountered in design require algorithms that are known to require exponential computation time (or space). That is, the time (or space) required to arrive at an optimal solution is O ( k “ ) where k is a constant and n a parameter characterizing the size of the design problem’. The very high computational cost of arriving at an optimal solution More exactly, such problems are said to be NP-hard or NP-complete (Horowitz and Sahni, 1978).

16

SUBRATA DASGUPTA

Microprogram written in a high level microprogramming language L

Compiler : phase 1

”Vertical” or ”Sequential” Microcode

Compiler : phase 2 (optimizer/compactor)

I

”Horizontal;’ microcode for micromachine H

FIG.6. Structure of an automatic microcode generator.

(especially for large values of n) would be sufficient to discourage the designer (even with the help of automated design tools) to seek solutions that are guaranteed to be optimal.

Example 2.10 One of the most widely studied problems in microprogrammed computer design is that of the automated generation of horizontal microcode from a high-level abstract description of the microprogram (Fig. 6 ) (Dasgupta and Shriver, 1985; Dasgupta, 1988b, Chapter 5; Mueller and Varghese, 1985). Here, the basic problem is, given a sequence of microoperations (“vertical” microcode) S

=

mlm2*9’m,,

possibly produced as the intermediate output of a microcode compiler, to generate a sequence of horizontal micro-instructions H

= I,12...Ik,

where each lj in H encodes or represents a subset of the micro-operations in S

THE STRUCTURE OF DESIGN PROCESSES

17

such that (a) each mi in S appears in exactly one lj in H ; (b) the data dependencies between the operations in S are preserved in H lo; (c) there are no conflicts in the usage of functional units between the micro-operations within the micro-instructions; and (d) the number of micro-instructions in H is minimized. This is a well-structured design automation problem for which an optimal solution (satisfying (d)) demands exponential time. Thus various algorithms have been developed that satisfyconditions(a),(b) and (c)but do not guarantee that (d) is met. All these algorithms run in polynomial time (that is, are of the complexity O(n’) where n is the size of the original microprogram S and k is a low integer constant) but are satisficing algorithms. 2.5

The Evolutionary Nature of Design Processes

To summarize, the main implication of bounded rationality is that the ramifications of a design decision, or a particular chain of such decisions, cannot always be comprehended at the time the decisions are taken. Decisions may also have wholly unintended side effects. Furthermore, since requirements may be imprecise, incomplete or both (see Section 2.2),the designer may not be able to demonstrate exactly that a particular design meets the requirements, or may have to generate requirements as part of the design process itself. Under such circumstances, a design process can be usefully viewed as an evolutionary process and the design itself, at any stage of its development (including the stage at which it is held to be “complete”), as a tentative solution to the problem originally posed; that is, the design is the evolutionary offspring from an earlier design form, and is likely to evolve further in the future. Our use of the term “evolution” in this context is deliberate. Thus it is necessary to establish exactly in what sense the design process is considered to be evolutionary. In biology, “evolution” refers to the unfolding and changing of organisms across generations through natural means. A t the risk of oversimplification, the hallmarks of biological (Darwinian) evolution can be concisely stated as

“’

Let m i . m, be in S such that m, precedes m, in S . Then possible dutu dependencies between m i and m, ure ( i ) m, writes into a register/store which is read by mi ( i i ) m , reads a register/store which is written into by m, ( i i i ) mi,m, both write into a common register/store.

In the literature on pipelined architectures, these are also referred to as hazards and specifically as read-after-write, write-ajier-read, and write-ajier-write hazards respectively (Kogge, 198 I ; Dasgupta, 1988b).

18

SUBRATA DASGUPTA

follows (Ruse, 1986; Bendall, 1983; Maynard Smith, 1975): (a) Within any population of organisms of a given species, there is considerable uariation among individuals largely brought about by genetic factors. (b) In a given environment, the likelihood of an individual surviving to adulthood and reproducing successfully will be influenced by the particular genetic characteristics or traits of that individual. If it does survive sufficiently to reproduce, then the offspring will inherit the traits and increase the frequency of occurrence of these traits in the population. On the other hand, if the traits are such that the individual organism is unable to reach maturity and reproduce, then the traits will be less likely to perpetuate in the population. This process is termed natural selection. (c) By this process organisms are constantly tested against the environment, and those that are genetically endowed to survive and reproduce successfully may be said to be “fit” relative to the environment. If the environment changes then some forms of organisms within the population may become fit relative to that environment while others may die out. Thus, organisms appear to constantly adapt to its surroundings. Clearly, in the context of design, neither the concepts of variation nor natural selection make any sense. What are relevant, however, are the concepts of testing and adaptation in terms of the following features: ( i ) A t any stage of the design process, a design is a conjectural solution to

the design problem. Since the adequacy (or satisfactoriness) of a design is determined solely with respect to the requirements prevailing at that stage, the design must be critically tested against the available requirements. ( i i ) If the design is found to meet the requirements, there is a fit between design and requirements. The former is adapted to the latter. ( i i i ) In testing the design against the requirements, there may be found to be a misfit between the two. The causes of misfit may be many. It may be that the design is incomplete relative to the requirements, or that it is incorrect; the design may be in such a form that it cannot be shown whether or not the design satisfies the requirements-in which case the former may have to be redefined or specified in a different form; or it may be that the requirements are given in such a form that it cannot be shown whether or not the design satisfies the requirements-in which case the latter may have to be reformulated or new requirements may have to be generated.

THE STRUCTURE OF DESIGN PROCESSES

19

Whatever be the cause of the misfit the design (and the requirements) may have to be modified to eliminate or reduce the misfit, thus producing a new (design, requirements) pair. (iu) The design process may be brought to a halt when the design is found to be adapted to the requirements. However, the design process never jbrmally ends since for fixed requirements one can always improve a design to increase the degree of adaptation (i.e., attempt a “more” satisfactory design); or the requirements may change, in which case a misfit between design and requirements is regenerated. Thus, the design process may be depicted according to Fig. 7. The dashed lines indicate the influence of one component on another. For instance, requirements influence design; or the nature of the test is influenced by both the design and the requirements. As Fig. 7 shows, design is a continuous process involving a cycle beginning with a (design, requirements) pair (in which the design component may be the “null” design) and ending with a (design, requirements) pair. The cycles are characterized not only by the fact that the (design, requirements) pair changes; the character of the misfit may differ from one cycle to the next, and this results in differences in the nature of the misfit elimination in successive cycles. This process may come to a halt when adaption has been achieved or it may extend-accompanied by intermittent halts indicating temporary adaptations-over the entire life of the target system.

Misfit Elimination

Misfit Identification

FIG.7. Design as an evolutionary process.

20

SUBRATA DASGUPTA

2.5.1

On the Distinction between lteration and Evolution

A commonly held notion in the design literature is that design is an iterative process where an initial “crude” design is transformed by a successive sequence of cyclic steps into acceptable form (see, e.g., Encarnacao and Schlechtendahl, 1983, Section 3.1 Chapter 3; Dixon, 1986; Mostow, 1985;Rehak, Howard, and Sriram, 1985; Lawson, 1980, Chapter 3, for different versions of this idea). However, in the ordinary sense of the phrase, “to iterate” is “to repeat.” This meaning is carried over more or less intact into computer science, where iteration is depicted by some variants of the schema

while cond do PROCESS repeat PROCESS until cond

The important point to note here is that ( i ) The same condition “cond” is tested in every cycle of the iterative

process, indicating not only that a specific type of misfit is expected, but also that the nature of the misfit is known beforehand. ( i i ) Once the terminating condition is satisfied (i.e., “cond” is not true in the while form or is true in the repeat form) the iteration does actually terminate. However, in the case of the design process consider the following facts: (a) The nature of the cycle itself-specifically the nature of the misfit identification and the misfit elimination stages-may difler from one cycle to the next. (b) The nature of the misfit may be quite unknown in each cycle prior to its actual identification. (c) Both design and requirements may change from one cycle to the next. (d) The fit between some aspect of a design and some aspect of the requirements-a case of adaptation being achieved-may be disrupted at a later stage of the design process, producing a new misfit. Clearly, the notion of iteration (at least in its computational sense) is hopelessly inadequate to describe such a process. Thus, we suggest that the assertion “design is an evolutionary process” is technically a more accurate and insightful description than “design is an iterative process.” 2.5.2

Evidence of Evolution in Design

That the design process is evolutionary in nature (in the sense described above) is an empirical proposition for which evidence must be provided.

THESTRUCTUREOFDESIGNPROCESSES

21

Furthermore, this evidence must be wide ranging and compelling. Unfortunately, limitations of space confine us to only a few examples in this paper, and we must leave it to others to provide further evidence either corroborating or refuting the proposition. We strongly believe, however, that the following examples are sufficiently compelling as to serve as strong corroboration of the proposition. One of the most general and influential principles of program design is that of stepwise rejinement, first proposed as an explicit doctrine in the early 1970s by Dijkstra (1972) and Wirth (1971). This principle can be stated concisely, though informally, as follows: 1 . If the problem is so simple that its solution can be obviously expressed in a few lines of a programming language then the problem is solved. 2. Otherwise, decompose the problem into well-specified subproblems such that it can be shown that if each subproblem is solved correctly, and these are composed together in a specified manner, then the original problem will be solved correctly. 3. For each subproblem return to step 1. A more formal version of stepwise refinement is based on the idea of developing a proof of correctness and the program together, and allowing the proof of correctness to guide stepwise refinement and program development. This version of stepwise refinement, which we may call the design-while-verify approach (or simply DWV), has been widely studied in the programming domain (Mills, 1975; Dijkstra, 1976; Alagic and Arbib, 1978; Gries, 1981; Hoare, 1987) and has also been applied in the domains of computer architecture (Damm and Dohmen, 1987; Dasgupta, 1984), firmware development (Dasgupta and Wagner, 1984, Damn et a!., 1986) and hardware circuits (Gopalakrishnan, Smith and Srivas, 1985; Gopalakrishnan, Srivas and Smith, 1987). We shall demonstrate how DWV conforms to the evolutionary model of design with a trivial problem. The triviality of the example serves two purposes. Firstly, it provides a concise, well-structured problem admitting a concise solution and is thus appropriate as an example in this article. Secondly, it also serves to show how even the most trivial and well defined of programming problems naturally exhibits evolutionary characteristics.

Example 2.11 The problem is to develop a program (call it MAX) ( i ) in Pascal, and ( i i ) which, given two nonnegative integers A and B, sets a variable Z to the larger of the two values. Here ( i ) is an implementation requirement, R i , while ( i i ) is a functional requirement, R,. The total requirement is R = (Ri,R,>.

22

SUBRATA DASGUPTA

Using DWV, we may solve this problem as follows. (a) In DWV, the program design problem is initially formulated in terms of a pair of assertions expressed in predicate calculus, called the precondition (PRE) and postcondition (POST) respectively. The former is a predicate which is satisfied whenever the program begins execution, while the latter states a predicate which must be true when (and if) the program terminates. The relationship between PRE, POST and the program P is notationally expressed as {PRE} P {POST} which states that if PRE is true when P begins execution the POST will be true when P terminates. For the specific problem at hand, this formula becomes (D1)

{PRE: A 2 0

A

B 2 0)

MAX: “place the maximum of A and B in Z ” {POST: z 2 A

A

z2B

A

(z = A

V

z = B)}

(DI) thus constitutes the .first oersion of the design. It is, however, quite conjectural since we have not yet shown that (Dl) satisfies the requirements. (b) In applying DWV, one attempts to prove the correctness of (Dl).Now suppose MAX, i.e., the string

“place the maximum of A and B in 2” is a very high-level machine-executable statement in some programming language. In that case, a proof of correctness of (Dl)can be attempted and, if successful, then the design (D1) would certainly satisfy the requirement R,, but not the implementation requirement R isince the program in this case is not expressed in Pascal. Thus, the design (Dl) fails to meet the given requirements R = (Ri,Rr). The source of the misfit is that the program MAX is not stated in Pascal. (c) Let us assume for convenience that the requirement R is unchangeable. Thus, (Dl) must be modified so as to continue to satisfy R, and, in addition, meet Ri.In DWV one thus attempts to modify the program part of the design with components ( i ) that come closer to the goal of expressing the program in Pascal such that (ii) the resulting design remains functionally correct. This may be attempted by replacing MAX with the assignment statement M A X ‘ : Z : = max ( A , B)

where mux is a function that returns the largest of its two arguments. The

23

THESTRUCTUREOFDESIGNPROCESSES

resulting design is

(D2) [PRE: A 2 0 M A X ' : Z:=

A

B 2 0)

~ U . Y( A , B )

IPOST:Z2 A

Z r B

A

( Z = A v Z = B)}

A

(d) Once more it is found that there is a misfit between the current design (D2)and the implementation requirement R , ;the function mux is not available in the particular implementation of Pascal assumed here. Thus further modification of the design is required. (e) In the next transformation, M A X ' is replaced by M A X " : if A 2 B then Z : = A else Z : = B

thus producing the design

(D3) [ P R E : A ~ O BA 2 O ) M A X " : if A 2 B then Z:= A else Z : = B

(P0ST:Z2 A

A

Z 2B

(Z = A v Z = B))

A

In testing (D3) against the requirements, it is seen that R iis indeed satisfied; furthermore, using the proof rule for the i f . . .then.. .else statement and the axiom of assignment as these are defined for Pascal (Hoare and Wirth, 1973), (D3) can be proved correct. Thus, the design also satisfies Rf,and it may be terminated at this stage. The development of this (quite trivial) program can be schematically described by Fig. 8.

I

Design

D1

Misfit

Requirements

R

=

Design D2



Misfit -Dpzrfji

I

- Design D3

FIG. 8. Evolution of the MAX program development.

Fit between D3 and R

24

SUBRATA DASGUPTA

In Section 2.2.2 we remarked that an often encountered feature of the design process is that requirements are often poorly understood at the early stages of the design and thus the development, expansion or identification of the requirements become integral parts of the design process. This characteristic is explicitly reflected in the evolutionary view depicted in Fig. 7. The following example, based on Aguero’s (1 987) dissertation shows how evolution may work in the very first stages of a design process during which “design” almost entirely involves the development and refinement of the requirements set.

Example 2.12 The objective is the design of an exo-architecture (see footnote 8) such that (Ro.l)The instruction set (“inst-set”) is eficient. (&) The instruction set supports frequent, time consuming operations from a significant sample of high-level language programs (“HLLsample”).

An exo-architecture, it should be recalled from Section 2.4, has several components, notably the instruction set, data types, addressing modes, storage organization, and instruction formats. Let us denote the exo-architecture to be designed, X . The initial design D o is a description of X that says nothing about the components of X except that X as a whole satisfies a particular set of predicates. These predicates are defined as follows. (a) “Eff (inst-set)” is a predicate that is true if the instruction set is eficient. (b) “Oper-supported(inst-set, HLL-sample)” is a set of operations such that op is a member of this set if and only if ( i ) op occurs in at least one member of the HLL-sample; and ( i i ) op can be synthesized from (or generated in terms of) the members of inst-set. (c) “Sig(HLL-sample)” is a predicate that is true if HLL-sample is significant, that is, is taken from a wide variety of applications. (d) “Freq(op)”is a predicate that is true if op is an operation that occurs frequently in HLL-sample. (e) “Time-cons(op)” is a predicate that is true if op is time consuming. The initial design Do of X does not describe the components of X . Rather, it describes X solely in terms of the following property being satisfied: (PO)

Eff (inst-set)

A

Sig(HLL-sample)

A

[Vop E Oper-supported(inst-set, HLL-sample): Freq(op) A Time-cons(op)]

THE STRUCTURE OF DESIGN PROCESSES

25

That is, Do is the statement

X such that Po is true. Clearly, if Po is true then Do would automatically satisfy R , and the design would be complete. Unfortunately, in attempting to test the satisfiability of R , by Do we find that there is no evidence that Po is true, hence Do is a mere conjecture at this stage of the design process. The source of the misfit between Do and R , is, simply, that we have no way of determining whether Q(i) Eff (inst-set) is true or not since we neither have inst-set nor d o we know what it means for inst-set to be “efficient.” Q(ii) Sig(HLL-sample) is true or not since we d o not know how to interpret the phrase “a wide variety of applications.” Q(iii) Oper-supported(inst_set, HLL-sample) is true or not since we d o not have inst-set. Q(iv) Freq(op) is true or not since neither is o p identifiable nor d o we know what is means for an operation to be “frequently 0ccurring”in HLLsample. Q ( u ) Time-cons(op) is true or not since neither is o p identifiable nor d o we know what it means for an operation to be “time consuming.”

Thus, to eliminate the misfit between Do and R , , these questions must all be resolved. Consider the resolution of the first of the above questions, Q(i).We see that this can be factored into two components: Q(i)(a) Defining the predicate “Eff” more precisely. Q(i)(b) Constructing an instruction set “inst-set” such that we can determine whether “Eff (inst-set)” is true or not. Note that the predicate “Eff” is what we desire of the instruction set, hence its precise definition ((a) above) is nothing but the need to modify the original requirements R , ! For this purpose let ( i ) “HLL-benchmarks” denote a set of predetermined benchmark

programs. ( i i ) “Code(p, inst-set)” denote the object code generated when a program p written in a high-level language is compiled into instructions from

inst-set. (iii) LOWSIZE and LOWTIME denote some specific integers.

26

SUBRATA DASGUPTA

We also define ( i u ) Size(Code(p, inst-set)) as the size (in bytes) of Code(p, inst-set). (0) Exec-time(Code(p,

inst-set)) as the (simulated) execution time of

Code(p, inst-set). Using these definitions, the original requirements Ro,l is replaced by the new requirement

(R1.1) V p E HLL-benchmarks:

Size(C:ode(p, inst-set)) ILOWSIZE

A

Exec-time(Code(p, inst-set)) I LOWTIME We have thus resolved Q(i)(a) by modifying the original requirements Ro,,. Similarly, we can resolve Q(ii), Q(iii), Q(iu) and Q ( u ) in part by defining the respective predicates more precisely. This will enable us to replace R , , with a new set of more precise requirements, R,,,, thereby completely replacing R , with R , . This will complete one cycle of design evolution in which we have not made much progress in the “design” itself, but have evolved the requirements (Fig. 9). As a final example of the evidence of design evolution, we refer to the extensive and systematic macrostudies of long-term program euolution conducted over a decade by Lehman and Belady (Lehman, 1980a, 1980b; Lehman and Belady, 1985; Belady and Lehman, 1976, 1979).

Example 2.13 As the empirical basis of their work, Lehman and Belady studied the changes in size and complexity of several large programs across a succession of versions or releases. Specific examples of such programs (Belady

4 DO Initial

4 Misflt 4

Detected

5

Replace RO.l by R1.l

1

Replace R0.2 by R1.2

----L

R1 New

THE STRUCTURE OF DESIGN PROCESSES

27

and Lehman, 1979) were

( i ) The IBM OS/360 operating system, consisting (at its final release) of approximately 3.5 million statements and over 6000 modules, involving 21 releases over 6 years. (ii) The IBM DOS/360 operating system consisting of approximately 900,000 statements and over 2000 modules, and involving 27 releases over 6 years. (iii) A banking system consisting of about 45,000 statements and involving 10 releases over a period of 3 years. In studying the pattern of changes across successive program releases, several global measures of size and complexity were used. These included (a) (b) (c) (d)

the actual number of source statements in the program the number of modules comprising the program the average number of instructions per module the number of modules that required change between successive releases.

The main results of these quantitative studies are summarized by Lehman and Belady in their three qualitative laws qf' proyrum euolution dynamics (Lehman, 1974; Belady and Lehman, 1976)":

I. Law oj Continuing Chunye A system that is used undergoes continuous change until it becomes more economical to replace it by a new or restructured system. 11. Law of Increusing Entropy ( Unstructuredness) The entropy of a system increases with time unless specific work is executed to maintain or reduce it. 111. Luw of Stutisticully Smooth Growth Growth trend measures of global system attributes may appear stochastic locally in time or space but are self-regulating and statistically smooth. 2.5.3 Ontogenic and Phylogenic Evolutions

Note that both Examples 2.1 1 and 2.12 refer to evolutionary processes that are highly loculized in time. Evolution takes place from the time when a ' I Later (Lehman, 1980b) two more laws were added. These are not however significant to our discussion.

28

SUBRATA DASGUPTA

problem is given to the designer to the time when the design is passed on to the manufacturer or implementer. During this period the design, as it were, unfolds from the initial form to the desired form. Borrowing a term from biology, we have previously (Dasgupta, 1988a, Chapter 3) referred to this as ontogenic design eoolution12. In contrast, Example 2.13 describes a form of evolution occurring over much longer time spans and representing the course of change of an implemented design. This form of evolution usually reflects changes in one or more of the various parameters that had dictated the original designe.g., change in technology, emergence of new modes of manufacture, changes in the operating environment of the system in question, or the discovery on the part of the user of different purposes for the system than those previously assumed. Such parameter changes produce new problems-that is, new requirements. Again borrowing a biological term, we refer to this type of evolution as phyloyenic design euolution (Dasgupta, 1988a, Chapter 3)13. Regardless of whether ontogenic or phylogenic evolution takes place, the means of evolution is identical: the use of critical tests, the identification of misfit, and the elimination of misfit (Fig. 7).

2.6 Summary To summarize Section 2, we have identified five fundamental characteristics of design: (a) The act of design originates in the desire to change some particular state of affairs. By implication the design act and its product are valuedependent. (b) Any design process must begin with some requirements. However, the requirements may initially be neither precise nor complete. Thus the development or elaboration of requirements may be an integral component of the design process. (c) The output of a design act is an explicit representation of the target system in some symbolic language. This representation not only provides the basis for implementing the system, it is also the medium of analysis and criticism of, and experimentation with, the design. (d) Design problems are usually complex in the sense that their solutions require many interdependent and intertwined decisions to be made. Consequently, design processes are very often satisficing procedures. Ontogeny: “the life history of an individual, both embryonic and postnatal” (Gould, 1977, p.483). l 3 Phylogeny: “the evolutionary history of a lineage conventionally.. .depicted as a sequence of successive adult stages” (Could, 1977, p.484).

’

THE STRUCTURE OF DESIGN PROCESSES

29

(e) The design process is an evolutionary process in which the design and/or the requirements are continually modified so as to be mutually adapted. When adaptation is achieved, the design process terminates. It may, however, resume if for some reason a state of misfit between the design and the requirements re-surfaces.

At this stage the following two questions may be posed: ( i ) To what extent or in what manner do actual design methods take into

account these fundamental characteristics? ( i i ) Does the value-dependent, satisficing and contingent nature of design

decision making impose a fundamental barrier on attempts to construct ,formal or scientific design methods? We will attempt to address these questions in the rest of this article. 3.

3.1

Design Paradigms

Some Terminological Clarifications

A design method is an explicitly prescribed procedure or set of rules which can be followed by the designer in order to produce a design. There are several reasons why design methods are at all proposed or invented: (a) Such a method when followed by an organization serves as a standard procedure that can be used by all designers within the organization, thereby easing the problems of understanding, checking, criticizing and modifying the design within the organization. Furthermore, the use of an established and familiar design method helps to economize on the cost and time of design. (b) A design method serves as a filter in the following sense: it represents a selection from the characteristics and complexities attending design, a small, specific, and mentally manageable set of concepts that can be used to impose an order on and to guide the process of actual design. All other aspects of the design situation-not stated explicitly in the design method-are effectively suppressed. In other words, a design method is a management tool to cope with the complexities of design. (c) A design method may also serve to embody a particular model or theory of the design process. That is, the method serves as a concrete and practical embodiment of that particular theory. If the method is successful as a practical design tool, the theory may then be said to have been empirically corroborated.

30

SUBRATA DASGUPTA

It is necessary at this stage to make a clear distinction between “method” and “methodology.” They do not mean the same thing. Methodology refers to the study and description of the methods or procedures used in some activity, including the investigation of the broad aims and principles attending such methods. Thus, one talks of the methodology of science; correspondingly, design methodology is the discipline or study of design methods (which happens to be the subject of this article). Regrettably, many writers use “methodology” and “method” s y n o n y m ~ u s l y ~ ~ . Our concern in Section 3 is not to describe specific design methods since these are innumerable. Instead, we shall focus on a small set of significant design p ~ r a d i g m s ’ ~A. design paradigm ( i ) may be viewed as a particular philosophical approach to actual design that ( i i ) serves as a useful abstraction of, or a schema for, a family of similar design methods. In discussing these paradigms we shall also address the first of the questions posed at the end of Section 2, uiz., to what extent and in what manner do these paradigms reflect or embody the characteristics of the design process enunciated in Section 2. 3.2 The Analysis-Synthesis-Evaluation Paradigm

The most widely accepted design paradigm takes the general form shown in Fig. 10. We shall refer to this as the A S E paradigm. Given a set of requirements, the design process involves first a stage of analysis(of the requirements), then one or more stages of synthesis, followed by a stage of eualuation16. As Fig. 10 suggests, on the basis of some decisions made within a stage or problem encountered in that stage, the designer may return to an earlier stage for appropriate modification or clarification. Thus, the ASE paradigm does recognize the evolutionary nature of the design process (see Section 2.5). In very broad terms, the ASE paradigm is not objectionable. It is certainly true that the actiuities of analysis, synthesis and evaluation are performed during design. If, however, one were to accept this paradigm literally, then

l4 Or a s the Fonfuria Dicrionary of M o d r r n Thoughf (Bullock and Stallybrass, 1977) wryly notes, “some scientists use the word [methodology] as a more impressive-sounding synonym for method.” In this article, we shall use the word “paradigm” in two different ways. When we talk of a design paradigm (as in this section) we use the word in its dictionary sense to mean a pattern or archetype exhibiting the characteristics stated in (i) and ( i i ) above. Later, in Section 4, we shall introduce a technically more distinct sense of the word due to Kuhn (1970) for which we shall reserve the special term Kuhnian paradigm (or K-paradigm for short). The word “paradigm” by itself will be used to mean the design or the Kuhnian variety when the context or reference is quite clear. l 6 The most explicit statement of this paradigm is given in Jones (1963). For other versions see Middendorf (1986) and Rehak, Howard and Sriram (1985).

’’

THE STRUCTURE OF DESIGN PROCESSES

f

IDENTIFICATIONOF REQUIREMENTS

I

31

-

P

Analvsis of

Detailed Synthesis

f

Evaluation and Testing

TO IMPLEMENTATION

FIG. 10. The ASE paradigm.

several serious problems arise:

(PI) A n unstated but unmistakable implication of the paradigm is that requirements are ( i ) either well-defined at the beginning of the design or (ii) by virtue of the feedback edges, can be obtained by interrupting a later stage and returning to the requirements identification or analysis stages. But as we have already noted in Section 2.2, in many design problems the initial requirements may be vague, sparse or both, and that consequently the generation of new requirements is an integral and inseparable part of the design process, not a distinct stage as implied in the ASE paradigm”. in their well known paper, Rittel and Webber (1983) summarize this situation with the statement “problem understanding and problem resolution are concomitant to each other”.

32

SUBRATA DASGUPTA

(P2) The ASE paradigm strongly suggests, or is based upon, the notion that synthesis emerges from the requirements and only from the requirements. From a logical perspective this is a faithful rendering of the inductive model of the scientific method which states that “Firsr make observations and gather the data or facts; then construct a theory or law as a generalization of these facts.”

Translated into design methodological terms, we obtain “First accumulate all the requirements that the target system is to satisfy; then synthesize the system to meet the requirements”I8.

We have already noted in Section 2.2 and in (Pl) above, the difficulty of accumulating all the requirements (even for a subsystem) at the start of the design process. We now have the additional problem that the inductive principle implies that the designer’s mind is in an empty state (a tabula rasa) on which requirements impinge like sense impressions and, as a consequence, a design somehow emerges. How this feat of generalization has come aboutthat is, what is the logic of the requirements-to-synthesis step-has never been explained. (P3) The ASE paradigm ignores or leaves unstated the central role that the designer’s weltanschauung or “world view” plays in the design process. By this, we refer to the assumptions, facts, values, theories, heuristics and other rules that the designer possesses and which he or she brings to bear when posed a design problem. When faced with a set of requirements-that is, a design problem-the designer may be influenced by his world view in many ways. For instance, he may perceive an analogy between this particular problem and one for which some design solution is already known to exist, thus suggesting the initial design as an initial tentative solution. Or, based on a few key components of the requirements, the designer may be predisposed towards a particular design style for the system under consideration, which then forms the basis of the overall design (Simon, 1975; Dasgupta 1984). Darke (1979),in the context of building design, called this a “primary generator.” In any case, the designer interprets the requirements according to a particular world viewI9. Is A recent statement of the “engineering method” almost exactly along these lines is given in Middendorf (1986, p.3). l9 It may be noted that this conclusion is similar to and consistent with an aphorism well known to philosophers of science that “all observations are theory laden” (Hanson, 1972; Popper, 1968). We thus see a first connection between the methodology of design and the methodology of science. More will be said about this connection in Section 4.

33

THE STRUCTURE OF DESIGN PROCESSES

(P4) Finally, the ASE paradigm says virtually nothing about redesignthat is, modifying an existing design due to the appearance of a new misfit between the design and requirements. The cause of this misfit may be desired changes in functional or performance requirements, the need to exploit new technology, or the pressure to reduce cost (IEEE, 1987). The principal distinction between what are conventionally termed “design” and “redesign” is that the latter begins with some existing design of the system that has already been implemented and is operational (either as a prototype or “in the field”). Thus, the redesign process is constrained by (some part of) a “current” design (meeting a “current” set of requirements) as well as a “new” set of requirements that must be met. Note that the ASE paradigm, taken literally, does not take account of an existing design as an input to the design process. Example 3.14 In software engineering, the standard (though not the only) model of the software development life cycle is the “waterfall” model (Boehm, 198 1; Wiener and Sincovec, 1984; Sommerville, 1985; Ramamoorthy et al., 1987) which in its essence is depicted in Fig. 11. It will be noted that the nature of software development as conceived here conforms quite well with the ASE paradigm and thus suffers most of the shortcomings of the latter. One exception is that the waterfall model recognizes redesign. Note, however, that redesign appears here as a distinct activity called maintenance as if this is apart from, and possesses characteristics distinct to, the other activities of the life cycle. As we have already noted before, design-including the development of software systems-is a continuous evolutionary activity in which there is no real distinction to be made between what are termed in Fig. 1 1 as “design” and “maintenance.” Rather, keeping in mind Lehman and Belady’s studies and the evolutionary discussions above, software development is fundamentally a

Analysis &

Implementation & Unit Test c L

Conceptual Design

Detailed Design

Integration & System Test c r

s

Maintenance

34

SUBRATA DASGUPTA

Requirements

Requirements

t---4

Requirements

t---4

.....

..... Software

I

Software

Software

FIG. 12. Long term (phylogenic)evolution of software.

process of phylogenic evolution "in the large," the individual stages of which follow the process of ontogenic evolution. Figure 12 schematizes this situation. Over time, requirements change and the software system evolves (through design) to adapt to the new requirements. Each box thus designates a fit between requirements and the software that is brought about by a process (internal to each box) of the short term or ontogenic evolution. 3.3 The Artificial Intelligence Paradigm Within the past decade or so, the ASE paradigm has begun to be influenced and modified by ideas emanating from Artificial Intelligence (AI) (Simon, 1975,1981; Mostow 1985). Perhaps the most concrete embodiment of the A1 perspective is the recent emergence of the so-called expert systems for design (Latombe, 1978; Gero, 1985; Thomas, 1985; Hong, 1986). However, our concern in this section is not such automatic design systems but the design paradigm that the A1 perspective entails. We shall call this, then, the AZ design paradigm. From the A1 perspective, problems are solved by creating a symbolic representation of the problem, called the problem space, such that it can describe the initial problem state, the goal state, the constraints, and all other states that may be reached or considered in attempting to reach the goal state from the initial state. 'Transitions from one state to another are affected by applying one of a finite set of operators (that are also contained in the definition of the problem space). The result of applying a sequence of operators is, in effect, to conduct a search for a solution through the problem space (Langley et al., 1987). The A1 paradigm begins by recognizing that design problems are often, as pointed out in Section 2.2, ill-structured problems. By this it is meant (among other things) that there may be no definite, objective criterion for determining whether a proposed solution meets all the requirements (see Example 2.5). Another characteristic of ill-structured problems is that the problem space

THE STRUCTURE OF DESIGN PROCESSES

35

itself may not be completely definable because of the virtually unbounded range of state-space variables and operators that have to be considered (Simon, 1973). Example 3.15 Consider the problem of designing a uniprocessor computer. A complete definition of the problem space would have to consider (a) All conceivable choices of instruction formats, data types and instruction types. (For example, fixed and variable length instructions, fixed and variable format instructions, the use of expanding opcodes, the choice of frequency-based opcode lengths, etc.) (b) All conceivable types of components and structures for designing the computer’s internal architecture or endo-architecture (Dasgupta, 1984). (For example, single bus, multiple bus, and distributed interconnection structures, alternative forms of pipelining, various cache memory organizations, different styles for microprogrammed and hardwired control units, etc.) (c) All possible technological choices. (For example, TTL, ECL, NMOS, CMOS, customized chips, semi-customized techniques, etc.) Clearly, any attempt to define a comprehensive or complete problem space in this case is doomed to failure. The A1 paradigm for solving such ill-structured problems involves a schema of the type shown in Fig. 13. The designer interacts both with the current design problem space (consisting of goals, constraints, the current state of the design, etc.), and a knowledge base (KB). The latter may be a combination of the designer’s long-term memory, textbooks, a computer data base, etc.

a: Designer

Current Design Space

FIG. 13. Ill-structured (design) problem solving schema (after Simon (1973)).

36

SUBRATA DASGUPTA

Given the initial goals and constraints (which may be sparse, fuzzy or “highlevel”)in the current design space, the designer invokes (from the KB) an initial design. This may be as approximate as a design style (Simon, 1975; Dasgupta, 1984,Chapter 12) or a crude description of the highest-level set of components that the design will have (e.g., in the case of computer design, the initial components may simply be the components “processor,” “memory” and “processor-memory’’ interface). Design then proceeds by successively transforming the “current” design state using, at every stage, some feature or component of the current state to invoke an appropriate action (selected from the KB) thereby refining a part of the design. At the very heart of the A1 paradigm is the means used to converge effectively to an acceptable design. As mentioned previously, the design state transformations are effected by operators. In most design domains these operators take the form ofheuristics which are invoked so as to reduce the amount of search through the design problem space. Heuristics may be very specijic to the particular design or task domain, or they may be general-i.e., independent of a particular design problem and therefore applicable across a range of domains. Generally speaking, during the initial stage of the design process one may expect to use general heuristics; as the design becomes more detailed and the nature of the decisions more specific, domain-specific heuristics may be expected to be invoked. Alternatively, general and domainspecific heuristics may be used in a symbiotic fashion. The so-called (and unhappily named) expert systems are instances of automatic problem solving (including design problem solving) systems that rely very heavily on domainspecific heuristics (“expert-knowledge”).

Example 3.16 A well known instance of a general heuristic is meansends analysis (Newel1and Simon, 1972, p.416). Given a current problem space and a desired (or goal) state, the difference between the two is determined. A specific operator is then invoked to help remoue or reduce this difference. Of course, in a particular design or problem solving domain, some of the operators may themselves be domain-specific. As a specific example, consider a microcode synthesis system of the type described by Mueller and Varghese (1985) which, given a microprogram represented in an abstract form, automatically produces a functionally equivalent executable form (see also Example 2.10 and Fig. 6)”. At any stage of the synthesis system’s operation, the current state will consist of the executable microcode that has already been synthesized (or “designed”) by the system. It is thus a partial design of the executable ’‘Such a microcode generation system is thus an example of a computer-aided or automatic design system.

THE STRUCTURE OF DESIGN PROCESSES

37

microcode. Suppose this current state is such that the execution of the microcode “designed” thus far would leave the micromachine registers with values satisfying (at least) the condition R1

=a A

R2

=

b.

Here, R 1, R 2 are two micromachine registers and a, b are symbolic constants. For the immediate purpose of the synthesis system, (CS,) can be viewed as the current state. Suppose further that the new goal of the synthesis system is to produce more microcode (i.e., further develop the microprogram “design”) which, starting with (CS,) being satisfied, when executed will leave the machine registers satisfying (at least) the condition RI = a + b .

(GSO)

For the immediate purpose of the synthesis system, (GS,) can thus be viewed as the goal state. The synthesis system may use means-ends analysis as a general heuristic to guide the search through its knowledge base. According to (GS,), the desired sum “a + b” must be assigned to register R1. Given that in the particular micromachine under consideration the arithmetic-logic unit performs all additions, taking its inputs from two special registers A I L and A I R and leaving its result in a special register A O U T , the system may attempt to reduce the diflirence between (CS,) and (GS,) by invoking the following rule: If the sum of two registers x and y is to be placed in a register Reg then generate the micro-operation sequence; AOUT Reg

+AIL e

+ AIR;

AOUT

and produce as the new goal: A I L = x A AIR

=y

Applying this operator, (with the arguments R1, a, and b) the system produces the microoperation sequence WSO)

AOUT

e

AIL

+ AIR;

R1+ AOUT

which, when executed with A I L

=

a, AIR = b will produce goal state CS,.

38

SUBRATA DASGUPTA

The resulting new goal state produced is (GS,)

AIL

=a

A AIR = b

The synthesis system would now attempt to reduce or eliminate the difference between (CS,,) and ( G S , ) . In this example, then, the general heuristic used is means-ends analysis. The operator selected to reduce the difference between current and goal states (in this case, by transforming the goal state) is a rule (R)which itself is a heuristic that is quite specific to the task domain-i.e., the domain of microcode generation for a particular micromachine2'. The A1 paradigm provides several insights on the design process. The most important are the following: (a) That design involves (or may involve) a search through some problem space and, as in most search activities, a particular state of design (i.e., a particular state reached within the problem space) may be merely tentutive and liable to be rejected at some later point in the search. (b) The amount of search through a potentially unbounded problem space is reduced to practical levels by the use of general and/or domainspecific heuristics. (c) The role of the designer's world view (in the form of a knowledge base) as an influence on the design process. (d) The fact that the A1 paradigm provides a theory of how the requirements-to-synthesis step (see Section 3.2 and Fig. 10) can be brought about. According to the A1 paradigm, the logic of design is a special case of the logic of human problem solving (Newell and Simon, 1972), and involves the application of heuristics. These heuristics not only help to reduce the amount of search (see (b) above); they are the means to transform the initial problem state into the desired problem state. 3.4 The Algorithmic Approach While the most interesting design problems are largely ill-structured, some of their subproblems or components may, in fact, turn out to be well-structured. This means, essentially, that the requirements and constraints are well-defined, the problem space is bounded and there are definite objective criteria for We are not, by the way, implying that the Mueller-Vdrghese synthesis syslem actually uses the specific operator (R); however, the general nature of their rules is quite similar (Mueller and Varghese, 1985).

THE STRUCTURE OF DESIGN PROCESSES

39

determining whether a design meets the requirements. Indeed, such problems may be so well-structured that one may conceivably construct algorithms for their solutions. Thus, for a relatively privileged class of design problems it make sense to talk of an algorithmic design paradigm.

Example 3.17 While it is incontrovertible that the design of an entire computer poses an ill-structured problem, certain of its subproblems are, at least in principle, candidates for algorithmic solutions. A specific instance is the design of an optimal minimally encoded micro-instruction organization for a computer’s microprogram memory (Dasgupta, 1979). A precise specification of this problem can be formulated as follows: (a) Let M be the set of micro-operations for the computer that is being designed. (b) Let C, denote a subset of M such that any pair of micro-operations in C, cannot be executed in parallel. Call C, a compatible set. (c) Let C = ( C , , C 2 ,. . . , C,} be a set of such compatible sets such that each micro-operation in M appears in exactly one of the compatible sets. In general, there may be many such sets of compatible sets. (d) Let IC,I denote the number of micro-operations in a compatible set C,. Each C, can thus be encoded in a single field F, of the micro-instruction using

bits. (This takes into account the necessity of being able to encode uniquely each of the micro-operations in Ci and also the encoding of the “nooperation” condition.) The total length of the microinstruction would be k

bits. The problem, then, is to determine a set C of compatible sets such that B is a minimum. Clearly, by definition, the algorithmic design paradigm is also a paradigm for automated designz2. Thus, for example, one can construct a program which when executed would “solve” the above design problem. Unfortunately, as previously noted in Section 2.4.1, most of the interesting well-structured design problems require algorithms that are known to consume exponential computational time. The very high cost of solving these problems algorithmically has resulted in more heuristic search-based approaches such as is represented by the A1 paradigm.

2 2 Note, however that it is by no means the only instance of such a paradigm. The A1 paradigm is also quite specifically developed for automation.

40

SUBRATA DASGUPTA

3.5 The Formal Design Paradigm Two decades ago, seminal papers by Floyd (1967)and Hoare (1969)marked the beginning of a formalist school of programming (Dijkstra, 1976; deBakker, 1980; Gries, 1981). The aims of this school are most succinctly captured by the following propositions recently enunciated by Hoare (1986): (a) Computers are mathematical machines. That is, their behavior can be defined with mathematical precision, and every detail can be deduced from this definition by the laws of logic. (b) Programs are mathematical expressions. They describe precisely and in detail the intended (and unintended) behavior of the computer on which they are executed. (c) A programming language is a mathematical theory. It includes notation, concepts, axioms, and theorems which assist the programmer in both developing a program and proving that the program meets its specification. (d) Programming is a mathematical actioity. Its practice requires careful and systematic application of traditional methods of mathematical understanding and proof techniques. The reader has, in fact, already been introduced to the formal approach by way of Example 2.1 1 (Section 2.5.2). According to the formalist, a programmer’s task begins with a pair of assertions stated in a formal language such as the first-order predicate calculus. One of these, the precondition (“PRE”), is a predicate that will always be true when the program P begins execution. This corresponds, then, to the concept of the initial state in the A1 paradigm. The other assertion (corresponding to the goal state in the A1 paradigm), called the postcondition (“POST”), is a predicate that will always be true when P completes execution. The task of the programmer is to design and develop a program P in a particular programming language L such that the formula (HF)

{PRE) P {POST} is true. This formula, which may be termed the Hoare formula, states that if PRE is true to begin with, then the execution of P (providing P terminates) results in POST being true. Furthermore-and this is the key to the formalist’s approach-by proposition (b) above, P i s a mathematical expression (as are PRE and POST, by definition), and by proposition (c), P is expressed in a language L satisfying mathematical laws. Thus the truth of (HF) can be rigorously and logically proved using the laws of L .

41

THE STRUCTURE OF DESIGN PROCESSES

{PREo

I

I I I I I

:

X>O A Y>O}

I

Po :

I I I I I

begin

z :=o; u :=x;

I I I

I I

repeat

z :=Z+Y; U:=U-l

( POST ,:

z

=x

I I I

I I I

x Y}

FIG.14. A Hoare formula for program Po

The idea of programming as a mathematical activity is, for very obvious reasons, strongly appealing-indeed, to such an extent that the influence of the formalist school has reached beyond programming to several other branches of complex systems design, including circuit and logic design (Borrione, 1987),firmware development (Dasgupta and Wagner, 1984; Damm et ul., 1986) and certain aspects of computer architecture design (Dasgupta, 1984). In the general context of computer systems, we shall refer to this approach as the formal design (FD) purudigm.

Example 3.18 Suppose we have constructed a program Po in a Pascallike language L’ that computes the product of two positive integers by repeated addition. Figure 14 is a Hoare formula (call it HF,) for this program. To prove the correctness of this formula requires treating the language L’ as a mathematical theory (proposition (c) above). Figure 15 shows part of such a mathematical theory in the form of a set of axioms and rules of inference for L’ (Hoare and Wirth, 1973). The fundamental axiom is the axiom of assignment. Let P be an assertion and let P [ X j E ] denote P with all free occurrences of X in P replaced by E. Then the axiom of assignment (which itself is a Hoare formula) simply states that if P is the postcondition of the assignment X : = E (where E is an expression) then its precondition will be P [ X / E ] .We can, for instance, use this axiom to prove that the formula ( X 2 0 } X : = X + 1 { X 2 1)

is true by working hackwurd from the postcondition “ X 2 1.”

42

SUBRATA DASGUPTA

Axiom of Assignment :

{P [ X / E ] X } :=E ( P }

Rules of consequence :

(i)

{

(ii) {

P } S { R } ,R > Q IP)S{Q1

P A B } S { Q } ,P A - B > & { P }V B t h e n S { Q )

P A B } s{ P}

Iteration Rules :

{ P }w i l e B do S { P A - B } (ii)

{ P ) S { Q } ,& A - B > P { p ) repeat S until B ( Q AB }

FIG. 15. The basic axiom and proof rules for Pascal.

The remainingconstituents of Fig. 15 are rules qf inference(als0 called proof rules) which enable the designer to infer the correctness of one formula from the correctness of other formulas. Rules of inference are of the general form H , , H , , .... Hn H which states that if the premises (or untecedents) H , , H , , . . . , H,, are true then the conclusion (or cmseyuence) H is also true. Here H , , H , , . . . , H,,are either assertions (in the predicate calculus) or Hoare formulas, while H is a Hoare formula. As an example, consider the Hoare formula (obtained in Example 2.1 1, Section 2.5.2) (HF') (PRE:A 2 0 A B 2 0 ) M A X : if A 2 B then Z : = A else Z:= B { P O S T :Z 2 A A 2 2 B A (2 = A V 2

=

B}

To prove ( H F ' ) we need to apply the axiom of assignment and the proof rule for the if.. .then.. .else statement shown in Fig. 15. More precisely, to prove

THE STRUCTURE OF DESIGN PROCESSES

43

( H F ’ ) requires us to show that the formulas (F1) ( P R E A A 2 B) Z : = A ( P O S T )

(f-2) { P R E A A < B) Z : = B ( P O S T }

are true. Now starting with POST and applying the axiom of assignment backwards to Z:=A produces, as a precondition, A 2 B-that is, proves the correctness of the formula {A2

B )Z : = A ( P O S T ; .

Since PRE A A 2 B implies A 2 B, by the second rule of consequence (Fig. 15), it follows that (FI)is true. Usinga similar argument it can be shown that ( F 2 ) is true. Hence, by the proof rule for the if.. .then.. .else statement, ( H F ’ ) is proved true. Returning to (HF,), the original Hoare formula of interest, in order to apply the rules of Fig. 15 requires inventing new, appropriate assertions as (intermediate) postconditions of statements appearing inside Po. These assertions must be such that, in conjunction with P R E , and POST,, they will allow (HF,) to be proved. The result of inserting such appropriate assertions is a proof’ outline of the form shown in Fig. 16. If we can now prove that

( H F ,1 (X>OA Y>O)

z:= 0 u:=x { ( Z+ U x Y

=

X x Y ) A ( U > 0))

{ ( Z+ U x Y

=

X x Y) A (U > 0))

and (HF,)

repeat

z:= z + Y; U:=

until U {(Z+ U x Y

=

u - 1; =0

X x Y ) A ( U > 0))

44

SUBRATA DASGUPTA

{ ( Z + U x Y = X X Y ) A (U>O)}

I

I

I

I I I I I

!

repeat

{(Z+UxY =XxY)A(U>O))

1

1

I I I

I

!

I

then, by the rule of sequential composition (Fig. 15), we can show that HF, is true. Note that the key to the whole proof in this case is the identification of the assertion (Z+UXY = X X Y)A(U>O).

In general, the most creative and intellectually demanding activity in the formal design paradigm is the construction of such intermediate assertions. The FD paradigm evidently represents the most rigorous and mathematically precise approach to design. However, in light of the characteristics of the design process discussed in Section 2, a number of questions arise. These are discussed below.

3.5.7 Reconciling Formal Design and Design Evolution It will be noted that the Hoare formulas are nothing but theorems which are proved based on the axioms and proof rules of the language and previously

THE STRUCTURE OF DESIGN PROCESSES

45

proved Hoare formulas (theorems). Thus, in the F D paradigm, a design is a theorem.

In view of this, how does one reconcile the F D paradigm with the evolutionary nature of the design process as described in Section 2.5? After all, evolution by its very nature is tentative and contingent, while mathematics is precise and certain. As long as one adheres to a picture of the mathematical process as consisting of an inexorable chain of deductions from axioms and theorems to other theorems, there is indeed a conflict between the evolutionary nature of design and the FD paradigm. However, as has been described by DeMillo, Lipton and Perlis (1979), this picture of the mathematical process is itself a myth. Mathematics is a human activity, and the acceptance of mathematical arguments and proofs is fundamentally a social process within the community. Mathematical proofs are discussed, criticized and worried over by members of the community, and as a result of this sort of a social process a proof may be accepted or it may be rejected. Indeed a proof may be accepted for a considerable period of time before a flaw is detected, at which point it is rejected or modified. More recently, using the four-color theorem as an example, Scherlis and Scott (1983) have also described how a theorem may be only tentatively accepted until extensive critical analysis and discussion either satisfy mathematicians about the theorem’s correctness or reveal flaws in the proof. Thus we see no real contradiction between the fundamental evolutionary nature of design processes and the F D paradigm. In Section 2.5.2 (Example 2. I I ) we illustrated how in a specific instance of the FD paradigm- the desiynwhile-tlerify approach-the development of a formal design was very much within the evolutionary framework. 3.5.2

Limitations of the FD Paradigm

Given the mathematical foundation of the F D paradigm, it would be highly desirable to adopt it as rhe paradigm for design. Unfortunately, there are at least two critical limitations of the formal design approach which prevent its acceptability as the dominant paradigm. Firstly, the FD paradigm ignores or rejects the fact that design may begin with incomplete or imprecise requirements. For instance, a project for designing and building a user-interface (to an operating system, say) could have as a requirement the objective that the interface must have both a “novice usage mode” and an “expert usage mode.” What constitutes such modes or how they are to be characterized may remain quite imprecise or informal for a significant part of the design process. And for this portion of the design process the FD paradigm is simply inapplicable, since the paradigm begins with formally defined specifications. Clearly in designing such a system, other

46

SUBRATA DASGUPTA

paradigms become relevant until the requirements are translatable into formal specifications, at which time the FD paradigm can take over23. The second important limitation, which is closely connected to the first, is the fact that the F D paradigm, by its very nature, does not admit any form of evidence for supporting the validity of a design other than mathematical proofs of correctness. This is one of the reasons why the FD paradigm is virtually useless when the designer is faced with incomplete or imprecise requirements. However, that is not all; even where the requirements are precise or complete, the designer may be forced to invoke other kinds of evidence both as justification for design decisions and during the critical test/analysis of the design (see Fig. 7). Such evidence may be experimental data-gathered by the designer using simulation or test procedures-or it may be the evidence from prior research and analysis conducted by other researchers or designers and documented in the literature. Mathematical proofs are simply one of a number of kinds of evidence. While research or experimental data may not be as certain as proofs, they are capable of providing a very high level of confidence in the design. Besides, they may be the only kinds of evidence that can be used. This latter fact-that is, the necessity of invoking non-formal kinds of evidence (for the validation of a design) under certain circumstancesbecomes clear when we consider the design of a system for which requirements are precise and complete and yet formal techniques are impossible to apply. Example 3.19 Consider the design of a cache memory. In this case, “design” involves identifying the key parameters of the cache. These include (Dasgupta, 1988a; Smith, 1982) ( i ) The placement policy, which determines how main memory words are

mapped onto cache memory words. ( i i ) The size and nature of the blocks to be transferred between main and

cache memory. ( i i i ) The replacement policy, which determines which blocks are to be

removed from the cache to make room for a new, incoming block. (iv) The size of thz cache. Now, in designing a cache, a significant requirement is to establish an upper bound on the cache miss ratio-i.e, the proportion of memory requests that cannot be successfully serviced by the cache. The cache designer may, then, examine the extensive experimental and simulation data involving trade-offs This raises the intriguing possibility of the design process being governed by diflerent paradiynis at diRerent stages. We do not, however, believe that this is really warranted. See Section 3.6 for further discussions of this matter.

THE STRUCTURE OF DESIGN PROCESSES

47

between various cache parameters that have been gathered by researchers (and published in the literature as, for instance, in Smith (1982)).Based on this data he or she may identify a specific set of parameters for the cache memory. In such a situation it is virtually impossible for the designers to formally prove that the selection of these parameters implies satisfaction of the miss ratio requirement. The designer can, however, justify the decision by invoking the evidence in the literature or by simulation.

3.6

The Theory of Plausible Designs

The theory of plausible designs (TPD) is a design paradigm proposed very recently by Aguero and Dasgupta (1987) (see also, Aguero, 1987; Dasgupta and Aguero, 1987)which addresses, directly and explicitly, the following issues in, and characteristics of, design: ( i ) That requirements may initially be neither precise nor complete, and thus the development or elaboration of requirements is an integral component of the design process (Section 2.2). ( i i ) That a design (that is, the product of the design process) is a representation of the target system in some symbolic medium that is appropriate not only as a basis of implementation but also for criticism, analysis, and manipulation (Section 2.3). ( i i i ) That design is an evolutionary process in which the design and/or requirements are continually modified so as to be mutually adapted (Section 2.5). ( i u ) That design processes are very often satisficing procedures (Section 2.4). ( u ) That the evidence invoked or sought by the designer in support of a design decision, or to establish the validity of a design feature, may include formal proofs, experimental evidence, empirical data based on observing previous systems, the results of research and analysis, or sometimes, commonsense reasoning (Section 3.5.2). In addition to the above, the T P D paradigm rests on the following premises: (a) The design of any complex system proceeds on a stage-by-stage basis. This is depicted in Fig. 17. Here each stage denotes the system at a particular abstraction level, and the arrows between the stages indicate that the general direction of design is from the more abstract to the less abstract. (b) The system may be said to be completely described at any one of these

48

SUBRATA DASGUPTA

Abstraction levels

m design at level i

a design at level i

higher

I

-

lower

design at level 1

FIG.17. Stepwise design based on abstraction levels

levels. Equivalently, the description at each stage denotes the design of the system at a particular level of a b ~ t r a c t i o n ~ ~ . (c) Given a pair of adjacent levels i, i + 1 (where level i + 1 is more abstract than level i ) , one may regard the design at level i as a (possibly abstract) implementarion of the design at level i 1. The level i design in turn becomes a spec$cation that is implemented by the design at level i - 1, and so on (Fig. 18). In other words, a design at any particular ab-

+

serves as specification I

I

0 serves as specification

which is implemented by

,

where 0

0

0

C is a specific non-empty set of constraints the plausibility of which is being established. A is the knowledge base (including other constraints) which are collectively used to assist in establishing C’s plausibility. R denoted the relationship (between constraints) or properties (of constraints) that must’be verified in order to establish C’s plausibility.

THE STRUCTURE OF DESIGN PROCESSES

0 0

51

V is the meuns employed for the verification of S (this is explained below). P is the plausibility state in which C will be placed upon successful verification of S.

Example 3.22 Thc following is an example of a plausibility statement for a constraint appearing as part of a processor chip design. ~

~

~~~

(S2): C: K-ary B-bit adder (ADD,,) that is efficient and VLSI-implementable. A : Description of ADD,, in S * M .

R : (a) The asymptotic latency of ADD,, is o(log K B ) . (b) The urea complexity of A D D K , is O ( K B log K B ) . (c) The structure of ADD,, is regulur.

V : Mathematical proof for (a) and (b), and heuristics for (c). P : Vuliduted.

Thus the plausibility state of C will be iididured (or more intuitively, the claim that an efficient and VLSI-implementable adder ADD,, has been designed, is rendered plausible) if it can be shown, using formal proof techniques, that (a) and (b)in R are true and, using heuristic reasoning, that (c)in R is true. In order to assist in establishing the plausibility of C , the formal description of ADD,, in the architecture description language S* M (Dasgupta, Wilsey, and Heinanen, 1986) can be used as stated by the field A .

3.6.3 Means of Verification The means of verification that is used to gather evidence in order to assign a constraint C to a plausibility state may be any combination of the following: (a) Precise: that is, formal, deductive logic. (b) Heuristic: that is, approximate techniques involving the use of heuristics. (c) Experimentul: that is, use of controlled experimental methods. Thus, TPD allows a continuum of verification strategies in order to provide evidence. This ranges from strictly deductive logic through more pragmatic experimental techniques to heuristic (approximate or common sense) reasoning. Note that this is one of the ways in which the T P D paradigm differs from

52

SUBRATA DASGUPTA

the FD paradigm, which admits only deductive logic as the means of verification. 3.6.4

Structure of the Paradigm

Space limitations d o not allow us to describe the formal aspects of T P D in further detail. Such issues as the logic of establishing plausibility states and of deducing one plausibility state from another are developed in Aguero (1987) and Aguero and Dasgupta (1987). Of more immediate interest is the overall structure of the design paradigm. In essence, a design process based on the T P D paradigm has the following characteristics:

(A) A design D, at any stage k is an organized set of constraints. Each constraint may be stated formally (in a description/design language, mathematical notation or logical formulas) or in the semi-informal language of scientific discourse (i.e., a combination of formal notation and natural language). (B) The plausibility of Dk at stage k is captured explicitly by a collection of plausibility statements. Since a design cannot contain any constraint that has been shown to be refuted, the plausibility state of each constraint in Dk will be the assumed, validated or the (default) undetermined state. (C) The constraints in Dk may have dependencies between them in the following sense. Let Si= (Ci, A i , Ri, 4) be the plausibility statement for Ciand let Cj be a constraint appearing in Risuch that in order to show that Ciis in plausibility state 4 requires showing Cj is in some plausibility state. In that case Ciis dependent on C j . Dependencies between constraints within a design Dk can be explicitly depicted by a directed acyclic graph termed a constraint dependency graph (CDG). An example is shown in Fig. 19. The vertices denote constraints, and there is a directed edge from vertex i to vertex j if Cidepends on C j . (D) Given a design Dk at some step k of the design process, the design further evolves by virtue of the fact that the designer attempts to ( i ) change an assumed constraint into a validated constraint, or ( i i ) assign an undetermined constraint into the assumed or validated state. In order to do either, the original constraint C may have to be refined or partitioned into a set of new constraints Ci,Cj, ..., such that the relationship between these constraints can be used to demonstrate the desired plausibility state for C2’.

c,

’’

TPD includes logical rules of partitioning such that when a constraint Cis partitioned into constraints C , , C, ,..., the designer can infer the plausibility of C in terms of the plausibility states of C,, C,, . . . .

THE STRUCTURE OF DESIGN PROCESSES

53

Fic;. 19. A constraint dependency graph.

Example 3.23 The very first constraint C, in Fig. 19 when initially postulated would be placed in the undetermined state. In order for it to be assigned to the cissumed state (say), it is partitioned into three constraints C,, C,, C,; the plausibility statement S, for C, would have its Vl field as assumed and would show in its R , field the relation

C, A C , A C,. If there is no evidence against this relation being true then S, is verified-that is, C, is assigned to the rrssumed state. The original “design” consisting of C, alone has evolved to a design consisting of C,, C, and C, related as shown by the topmost part of Fig. 19.

( E ) A design may evolve i n other ways also. For instance in the process of attempting to change the plausibility state of constraint Ci from the rissunied to the ilnlirfnted state, it may happen that the available evidence actually causes C, to be assigned to the rrlfutedstate. In that case Cimust be removed from the design. As a result, if some other constraint Cj is dependent on C,, then its plausibility state will have to be rcvised in the light of the new state of affairs. In other words, the plausibility states of constraints may well have to be revised on the basis of new evidence at handzh. Example 3.24 Referring to Fig. 19 again, suppose C, could not be assigned to the ussumed state on the basis of the evidence specified by its That IS,the logic of plausibility is a type of fl0n-~10)10t0ni(.logic which is widely used in artilicial intelligence systems (Turner, 1984, Chapter 5: Charniak and McDermott, 1985, Chapter 6 ) .

54

SUBRATA DASGUPTA

plausibility statement S,. By changing its P, field from assumed to refuted, however, S, could be verified-that is, C, is refuted and is thus removed from the design. As a result, both C2 and C 3 ,being dependent on Cg, would have to be reconsidered and their plausibility states revised. Note that as a result of all this, the CDG (Fig. 19) would itself alter. 3.6.5 Some Remarks on the TPD Paradigm

The most important characteristics of the T P D paradigm are (a) The unification of “design” and “requirements” under the single notion of “constraints.” As has been noted several times, during the design process, both design and requirements may evolve. This is recognized in T P D by the notion of a set of constraints evolving. (b) The explicitly evolutionary nature of the paradigm. Furthermore, one can see very clearly what it is that evolves, the direction of evolution, and the mechanism of evolution. What evolves is the set of constraints constituting a design. The direction is towards the attainment of plausibility states that are either assumed or (more preferably) validated. And the mechanism is the attempt to verify the plausibility states of existing constraints. This in turn causes the removal of refuted constraints, possible revision of the plausibility states of other existing constraints, and the generation of new constraints. The evolutionary cycle followed in T P D and its correspondence to the general evolutionary structure of the design process (Fig. 7) are depicted in Fig. 20. (c) The recognition that the evidence that can cause the designer (or anyone else) to acquire confidence in, or reject, a feature in the design can be not

Set of constraints with plausibility statements [Design

D

Verification of plausibility statements based on evidence at hand

+ Requirements] [Critical Test]

-

(a) Removal of rejuted constraints (b) Revision of plausibility


rj. 2. ( S , R ) = S if for every si E S there is rj E R such that si < rj.

NOTE: Obviously, S or R could be single element intervals (i.e. s 1 = s, and r I = r,). , two intervals such Theorem 7 . Let S = { s l . . . s , ) and R = { r l . - . r m ) be that R n S = 0. Then R if ri > s, max(S, R ) = W S if s1 > r,.

Theorem 2. Let S that R n S = 0. Then:

=

{sl...s,} and R = { r I... r,}, be two intervals such

min(S, R ) =

R S

Theorem 3. Let S = {sl ... s,} and R that R n S # @, S 4 R, and R 4 S. Then max(S, R ) =

R S

if r, < s1 if s, < r l . =

{ r l . - . r m ) ,be two intervals such

if r, > s, if s, > r,.

W

Theorem 4. Let S = ( s l . . . s,} and R = { r I ... r,}, be two intervals such S 4 R , R 4 S, and s, > rm. Then that R n S # 0, min(S, R ) =

R S

if r, < s, if s, < r,

W

Definition 7 Let S = {sl...s,} and R = { r l ...r m } , be two intervals such that R c S. Then there is an interval T such that 1. max(S, R, T ) = T, if for every ti E T there is sj E S such that t i 2 sj and for every ti E T there is rk E R such that ti 2 r k . 2. min (S, R, T ) = T, if for every ti E T there is sj E S such that ti I .sj and for every t i E T there is rk E R such that ti I rk.

FUZZY SETS AND THEIR APPLICATIONS TO ARTIFICIAL INTELLIGENCE

Theorem 5. Let S that R E S . Then

=

83

{ s I ~ ~ ~ s and , J . R = { r , ... r,,,),be two intervals such

rn max(S, R ) = r l "'s,,.

Theorem 6. Let S that R c S . Then

=

isI ...s,,f and R = { r l ...r,,,], be two intervals such

rn

min(S, R ) = s 1 "'r,,,. Definition 8

Let c( and /3 be intervals. Then we say that of r is greater then the upper bound of

p if the upper bound Example 4

r

c(

is higher than

p.

Let the characteristic function of the variable OLD be ifx 120.

~ ( x= ) x/IOO

Let the distribution of the population be

30 people are of 20 people are of 30 people are of 20 people are of

the age ranging from the age ranging from the age ranging from the age ranging from

10 to 20

40 to 50 50 to 70 80 to 90.

Using the definition of F E V and the characteristic function, we compute 11 and x: I' 1 .oo

0.70 0.50 0.20

I

?,

0.1 -0.2 0.4-0.5 0.5-0.7 0.8-0.9

Now, the mins of the pairs are min (1.00, [O. 1-0.21)

=

[O. 1-0.23

min (0.70, CO.4-0.51)

=

CO.4-0.51

min (0.50, [0.5-0.71)

=

0.5

min (0.20, [0.8-0.91)

=

0.2.

Thus the ordered list of the MINs computed above is L

=

i[O.lLO.2], 0.2, CO.4-0.5],0.5).

84

ABRAHAM KANDEL AND MORDECHAY SCHNEIDER

Now, according to the algorithm the maximum of L is max(C0.1-0.23, 0.2, CO.4-0.51, 0.5) = 0.5 From the example above, it can be seen that the evaluation of typicality values is possible in spite of vague information. However, we assumed a complete knowledge about the distribution of the population. Next we extend the theory to handle incomplete information about the distribution of the population.

3.2.1

Additions of Fuzzy Intervals

When the information about the distribution of the population is fuzzy, it is necessary to find an interval which will contain the unknown value. UB, the upper bound of pj, is given by j

i=j

UB=, j

max(pi1, Pi,) j- 1

1max(pi,, pi,) + 1min(pi,, Pi,) i= 1

i=j

where pi, is the lower bound of group i and pi, is the upper bound of group i. L B , the lower bound of pj, is given by i n

C.min(pi,, Pi,)

LB = j

i=J

$ min(pi1, Pi,) + c max(pi1, pi,)’ j- 1

3.2.2 Building a Mapping Table The function of the mapping table is to map fuzzy variables into corresponding values. The chosen values are subjectioe and domain dependent. The construction of the mapping table involves four steps: 1. Finding an appropriate range to the domain (it is logical to assume that the range of the variable O L D is between 0 and 120). 2. Finding the possible values for the variables in the range (in the variable PEOPLE we can have only positive integers). 3. Finding the adjectives that may describe objects in the domain (“almost,” “more or less,” etc.).

FUZZY SETS AND THEIR APPLICATIONS TO ARTIFICIAL INTELLIGENCE

85

4. Performing the mappings of the adjectives to the corresponding values without violating the boundaries of the range of the domain and its possible values. Example 5

Assume we have the following information:

almost 20 people more or less 30 people. Using the four steps described above we have 1. The range of the number of people is between 0 and 'm. 2. The possible values are all positive integers. 3. The adjectives are a. almost b. more or less c. over d. much more than e. etc. 4. Perform the mapping of the linguistic adjectives into intervals. Each variable has its own mapping table. The name of the mapping table is the name of the variable associated with it. The mapping table may contain three parts: 1. List of adjectives that may be associated with the variable. 2. The lower bound-the lowest possible value a number associated with the variable and the adjective may accept. 3. The upper bound-the highest possible value a number associated with the variable and the adjective may accept.

Using some subjective judgements we can construct the following mapping table. TAn1.I:

1

EXAMPL.~: OF MAPPING 'rABLE for the variable PEOPLE

Adjective almost more or less over much more than

LH

UB

Y -

10""

x-l

Y -

lo",,

Y

Y + l 2Y

Y

+ lo? 1 of the

-

189

OPTICAL AND OPTOELECTRONIC COMPUTING

-

X-

-

X‘

-c

NL

-

A

Y‘ -

-

-

a

,

NL

Y

matrix limits the relative accuracy of the result vector y. Since x ( A ) > 1 and (16All is at least 0.1, the relative error in y will be greater than 10%. By simple scaling operations we can modify the given A without changing its mathematical meaning. These modifications often reduce x ( A ) to a value that gives a more accurate y. In a similar way, we can show (4.29) The net effect is that we always loose accuracy during the operation of an optical neural network, but if we reduce x ( A ) to the vicinity of 2 or less, this may not be a major concern. The second appproach is using some of the optics margin over electronics to tolerate error better. Takeda and Goodman (1986) hint at encoding the numbers spatially to gain accuracy. The point is not that the accuracy problem has been solved. Rather, the point is that it is explorable. We are limited by nerve and imagination, not by technology. 4.6

Applications

Analog optical processors are well developed and have found a wide range of applications. Here, we describe two of their most successful applications in synthetic-aperture radar and spectrum analysis. 4.6.1

Synthetic-Aperture Radar

The formation of high-resolution terrain maps using an airborne radar is of great interest. The “azimuthal” resolution of an airborne radar with an antenna of diameter D is roughly ).RID, where E, is the wavelength of the radar signal and R is the distance between the antenna and the ground. For a radar working in the microwave region, assuming i= 10 cm, D = 1 m, and R = 10 km, the resolution is about 1 km. To get a resolution of 10 cm using the same values of iand R as before, D should be 10 km, which is certainly impractical. The synthetic-aperture radar approach is a practical solution to

190

MIR MOJTABA MlRSALEHl

this problem. The idea is to synthesize the effect of a large antenna using the data obtained by a short antenna. In the synthetic-aperture radar, an aircraft carries the antenna to a sequence of positions, where the antenna radiates a pulse and then receives and stores the reflected signal. The reflected signal is recorded and processed to obtain a map of the terrain. The synthetic-aperture concept was first studied by Carl Wiley in the early 1950s (Sherwin et al., 1962). The major problem was that the amount of information needed to be stored and the number of computations that were required was so large that real-time analysis could not be achieved. In the early 1960s, the University of Michigan's Radar and Optics Laboratory group solved this problem by developing a coherent optical system for processing the synthetic-aperture radar data. Here, we briefly describe their approach. For more information, the reader is referred to the papers by Cutrona et al. (1960, 1961, 1966) and Brown and Porcello (1969). lmage Recording. Consider the side-looking radar system shown in Fig. 17. In order to obtain an accurate map of the terrain, a high resolution is required in both the direction of flight (azimuth), and the direction perpendicular to it (ground-range). The ground-range resolution depends on the duration of the pulse radiated by the antenna, and it can be improved by using short pulses. The azimuthal resolution, on the other hand, depends on the antenna pattern, and it can be improved by using the synthetic-aperture radar. To analyze the problem, let a target be located at ( x o , y o ) , a distance r from the airplane, and let the wave transmitted from the antenna be represented by UT(t)= A exp ( - j o t ) ,

(4.30)

where A is the amplitude and w is the angular frequency of the transmitted wave. The signal reflected from the target and received by the antenna is &(f)

= A' exp[ - j o ( t - 2f)],

(4.31)

where A' is a complex amplitude and c is the speed of light. In the radar system the received signal, UR,is first amplified to U k which has the same amplitude as UT,and then mixed with the transmitted signal. The mixed signal is squarelaw detected. The output of the detector l ( x ) is l(X)

1

=

2I UT(t) + Uk(t)12 (4.32)

OPTICAL AND OPTOELECTRONIC COMPUTING

191

Z

0

*Y

DIRECTION OF GROUND- RANGE

OF FLIGHT FIG.17. Geometry of the synthetic-aperture radar

where the asterisk indicates the complex conjugation. The above equation can be written as

+ cos(2kr)],

[(x) = ( A ( 2 [ l

where k

= w/c. The

r

(4.33)

distance r can be expressed as 2

=

J r i + ( x - xo)

2 rn

+ ( x - .Yo) 2ro

2

’

(4.34)

where r n is the distance from the antenna to a line that is parallel to the direction of flight and passes through the target. Substituting for r from Eq. (4.34) into (4.33).we obtain

(4.35) This expression is equivalent to the expression for a one-dimensional Fresnel zone plate with a focal length of r0/2 centered at x = xo (Jenkins and White,

192

MIR MOJTABA MlRSALEHl

1976). In very simple terms, a zone plate acts as a lens. That is, when it is illuminated with a parallel beam of light, it focuses the light to a point. In practice, there are other targets at distances other than r,, each reflecting the radar pulse at different times. At the receiver, the returned echo from the target is used to modulate the raster of a cathode ray tube (CRT) display. As soon as the radar pulse is emitted, the scanning raster on the CRT starts along a vertical line as shown in Fig. 18. The intensity of the raster is stepped up when the reflected signal is received by the antenna. The closer the target is to the airplane, the sooner the pulse returns, and the closer the bright point on the CRT will be to the point at which the raster has started. Signals from the targets at the same distance will appear on the same point on the CRT. The CRT screen is imaged on a running photographic film where the echo from equidistant targets is recorded as horizontal lines. The brightness of the display point on the CRT corresponds to I ( x )as given by Eq. (4.35).As a result, the amplitude and phase information of the reflected signal is recorded as Fresnel zone plates. Image Reconstruction. In reconstructing the image, a plane wave produced by a coherent light source is used to illuminate the developed film. Due to the Fresnel zone characteristic of the recorded pattern, the light transmitted through the film will produce an image of the terrain. The distance of each point on this image from the film is proportional to the distance of that point on the ground from the antenna. As a result, the image will be constructed on a slanted plane. To correct for this, a cylindrical lens should be placed behind each one-dimensional zone plate. These lenses can be combined to form a conical lens as shown in Fig. 19. The coherent light emerging from the filmconical lens combination is collimated in the x direction, but it has no focal property in they direction. The spherical lens in Fig. 19 focuses the beam in the

START OF SWEEP

0 LENS

INTENSITY MODULATION

I

END OF SWEEP BIAS

+

BIPOLAR VIDEO

Fici. 18. Recording process in synthetic-aperture radar (Brown and Porcello, 1969).

; @yx,

193

OPTICAL AND OPTOELECTRONIC COMPUTING

PI

SPHERICAL

p2

@

DATA F I L M

CYLINDRICAL LENS

CONICAL LENS FIG I9

x2

Optical procesor for synthetic-aperture radar analysis

x direction and the spherical-cylindrical lens combination images the beam in

the y direction. The reconstructed image of the terrain can be recorded on a photographic film or observed on a CRT screen. Very high resolution terrain maps have been obtained using this imaging radar system (Cutrona et al., 1966). 4.6.2 Spectrum Analysis

Thc heart of an optical spectrum analyzer is an acousto-optic Bragg cell which is made of a transparent material such as glass (Fig. 20). One end of the cell is attached to a transducer which receives an electric signal with a frequency usually in the M H z range and launches an acoustic wave inside the cell. The other end of the cell is tapered to prevent the reflection of the sound wave. The acoustic wave produces a mechanical strain which results in a periodic change in the index of refraction of the material. Such a periodic structure is known as grating, and it has special characteristics. An important property of the gratings is their capability of diffracting a beam of light at particular directions. It should be mentioned that since the speed of light is much higher than the speed of sound, the grating looks essentially stationary to the light wave. If a beam of light impinges on a grating at a particular angle (known as the Bragg angle), it will be diffracted. The Bragg angle (O,J is related to the wavelengths of the light wave (2) and the acoustic wave (A) by Be = sin- '(j42A). Another important parameter is the diffraction efficiency, which is the ratio of the power of the diffracted light to the power of the incident light. The diffraction efficiency depends on the parameters of the Bragg cell and the optical and acoustic waves. Acousto-optic cells have several applications (Berg and Lee, 1983).They are used for spectrum analysis (Anderson rt al., 1977),performing operations such

194

MIR MOJTABA MlRSALEHl

/ DIFFR, CTED

INPUT

LIGHT

\

UNDILZTED

t

w FIG.20. Acousto-optic Bragg cell, where (1) is the angular frequency of the electronic signal which is used to produce the acoustic wave.

as convolution and correlation (Rhodes, 198l), and algebraic processing (Casasent, 1984). Here, we describe their application in spectrum analysis. Figure 21 shows the schematic diagram of a spectrum analyzer. The device consists of a light source, a spatial filter, a Bragg cell, a Fourier transform lens, and a detector array. Because of its compactness, a diode laser is usually used as the light source. The light produced by the laser is collimated by the spatial filter and illuminates the Bragg cell. At the same time, an acoustic wave is launched in the cell by the transducer. As described above, part of the incident light will be diffractedat an angle that depends on the frequency of the acoustic wave. The diffracted light passes through a Fourier transform lens and is focused at the focal plane of the lens where a detector array is located. The position of the light on the detector array can be used to determine the frequency of the acoustic wave. The device described above can be fabricated in a compact form using integrated optics technology. Figure 22 shows the schematic diagram of an integrated-optical spectrum analyzer. Except for the diode laser, the other components of the device are fabricated on a common substrate. Among the materials that have been used for substrate are glass, lithium niobate, and gallium phosphate. To produce acoustic waves, surface acoustic wave (SAW)

195

OPTICAL AND OPTOELECTRONIC COMPUTING

LASER

SPATIAL FILTER

BRAGG CELL

FOURIER TRANSFORM LENS DETECTOR ARRAY

ZERO ORDER STOP

t

W

Fici. 21.

Schematic diagram of a spectrum analyzer.

PHOTO DE TECTOR

I

MIXER FILTER

Fi(t

12

Schematic diagram of an integrated-optical spectrum analyzer (Tsai, 1979)

196

MIR MOJTABA MlRSALEHl

transducers are used instead of the bulk Bragg cell. This type of transducer can be easily fabricated by depositing a thin film of a conducting material on the substrate. To analyze microwave signals, the heterodyne technique can be used. In this technique, an input signal with a frequency of v1 is mixed with a signal of a fixed frequency v2 produced by a local oscillator. The output of the mixer consists of two frequencies v1 + v2 and v1 - v 2 . The latter is separated by an intermediate frequency (IF) filter and it is used to drive the SAW device. Today, integrated-optical spectrum analyzers are manufactured by several companies (Goutzoulis and Abramovitz, 1988). These devices are very compact and powerful. They have bandwidths of several hundred MHz, frequency resolutions of few MHz, and can detect pulses with only 100 nsec length. 5.

Digital Processors

Although analog optical processors can perform computationally intensive operations, such as Fourier transformation and correlation, they suffer from two shortcomings: low accuracy and inflexibility. The accuracy of these processors is about 1% or less. To achieve higher accuracy, digital processors are needed. Also, analog optical processors are limited to specific operations. Digital processors, on the other hand, are more flexible and can implement any operation. These advantages motivates one to develop digital optical processors. Extensive interest in performing digital operations with optical signals started in the mid 1960s, when reliable lasers became available. The direction pursued at that time was to use nonlinear characteristics of materials to perform logic operations (Tippett et al., 1965). Although the possibility of performing logic operations with optics was demonstrated, the lack of materials with high nonlinear characteristics made the development of large scale systems impractical. Some critics claimed that digital optical computers will never succeed (Keyes and Armstrong, 1969). As a result, the research projects in this area were mostly abandoned. Starting in the mid 1970s, a second phase of research on digital optical processing was initiated and the interest in this area has increased in the 1980s. This new interest is partially due to the growing need for parallel processing and partially due to the development of new materials for optical bistability and, recently, new models for neural networks. Attention is made to develop architectures that take advantage of the parallelism of optics. Optical signals, unlike electronic signals in integrated circuits, are not restricted to planar structures. Numerous information channels can be distributed over a twodimensional array and propagate along the third dimension. Using 2-D arrays of spatial light modulators and detectors, the information in all channels can be processed and detected in parallel.

OPTICAL AND OPTOELECTRONIC COMPUTING

197

During the last two decades, a large number of digital optical processors have been proposed. However, because of technological problems, the proposed architectures have been demonstrated only as small prototype systems, and it is not clear yet which ones will be successful in implementing large and practical systems. As for any new field, there is not a standard classification of digital optical processors at this stage. Here, we categorize these systems according to three aspects: number representation, computing techniques, and architectures.

5.1

Number Representation

An important issue in digital processing is the number representation. Digital electronic systems generally use binary numbers. This is mainly because electronic bistability is easy to achieve. However, due to the carry propagation problem, binary numbers are not very suitable for parallel processing. Two number systems that do not suffer from this problem are residue and signed-digit. 5.1.1 Residue Number System (RNS) The foundation of residue arithmetic dates back to the first century A.D., when the Chinese mathematician Sun-Tsu published a verse in which he gave an algorithm for finding a number whose remainders on division by 3,5,and 7 are known. A general theory of remainders (now known as the Chinese remainder theorem) was established by the German mathematician K. F. Gauss in the nineteenth century. The application of residue arithmetic in computers, however, is relatively recent and was first introduced in the mid 1950s by Svoboda and Valach (1957) in Czechoslovakia. Unlike the commonly used binary and decimal number systems, the RNS is an unweighted system. The base of a residue system consists of n pairwise relatively prime (having no common factor) numbers, m , , m 2 , . . . , m,, called moduli. Any integer X can then be represented by an n-tuple (xl, x2,.. . , x,), where .xi = /XI,,,, (read X mod mi)is the positive remainder that is obtained from the division of X by mi.This representation is unique for a dynamic range of M = I mi. An important feature of the RNS is that the fixed-point arithmetic operations can be performed on each digit individually. That is, if X = (xI,.x2 ,..., x,) and Y = (y,, y2,..., y,) are two numbers of the same residue system, Z = X * Y = ( z l , z 2 ,..., z n ) , where zi = I(xi * pi)/,,,, for i = 1, 2 , . . ., t7, and * represents addition, subtraction, or multiplication. Division can be performed, but it is difficult except for the remainder-zero case (Szabo and Tanaka, 1967).

nl=

198

MIR MOJTABA MlRSALEHl

As an example, consider the set of four moduli (5, 7, 8,9}. These moduli cover a dynamic range of 2520. In this residue system, the decimal numbers X = 42 and Y = 3 1 are represented as X = (2,0, 2,6) and Y = (1, 3, 7,4). The results of performing addition, subtraction, and multiplication on these numbersareX+ Y = ( 3 , 3 , l , l ) , X - Y = ( 1 , 4 , 3 , 2 ) , a n d X . Y = ( 2 , 0 , 6 , 6 ) , which are the residue representations of the correct answers, i.e., 73, 11, and 1302, respectively. The first optical system that utilized residue arithmetic was the photoelectric number sieve invented by Lehmer (1933). The machine was constructed with 30 spur gears, one for each of the prime numbers from 2 to 113, and it was capable of sifting 20 million numbers per hour. The system was used for finding the factors of large numbers such as the Mersenne number 279 - 1. The application of residue arithmetic in modern optics was first investigated by Huang (1975) and Huang et al. (1979). They proposed the use of different devices, such as prisms, optical waveguides or fibers, and gratings, to implement optical residue processors. Other architectures were later proposed that utilized spatial light modulators (Collins, 1977), optical correlators (Psaltis and Casasent, 1979), photo-diodes (Horrigan and Stoner, 1979), optical programmable computation modules (Tai et al., 1979),and holograms (Guest and Gaylord, 1980).

5.7.2 Signed-Digit Number System The signed-digit system, introduced by Avizienis (196l), is a weighted system in which both positive and negative digits are allowed. Of special interest is a system with three permissible digits: -1 (or i),0, and 1, This system is known in the optics community as the modified signed-digit (MSD) system (Drake et a/., 1986). As an example, the decimal number 5 can be represented as (liOl)Ms,,, since (lTOl)MsD= 1 x 2O + 0 x 2' - 1 x 22 + 1 x Z 3 = ( 5 ) , 0 . The negative of an MSD positive number can be obtained by complementing each digit of that number. The complement of 1 is - 1, of - 1 is 1, and of 0 is 0. Thus 1 -+ T , i + I, and 0 -+ 0. For example, the decimal number -5 can be represented by (iloi)M,D. The MSD system is redundant, i.e., there is more than one representation for each number. For example, _ _ _ the decimal number 5 can also be represented as (loI)MsD, (loii)M,D, (1 101 I),,,, etc. This redundancy can be used to limit the carry propagation only to the next significant digit (Drake ec al., 1986).This, in turn, makes it possible to perform an operation on all digits in parallel. The first optical MSD processor was proposed by Drake et al. (1986). Their architecture was based on arrays of prisms, holograms, and bistable devices to realize the functions needed for addition and subtraction. Other architectures based on truth-table look-up technique (Mirsalehi and Gaylord, 1986b) and symbolic substitution (Li and Eichmann, 1987) have also been proposed.

OPTICAL AND OPTOELECTRONIC COMPUTING

199

5.1.3 Binary Number System

Although the residue and signed-digit number systems are very suitable for parallel processing, most of the proposed architectures for digital optical processors are based on the binary number system. This is due to the fact that the existing digital systems work in the binary system. Therefore, any new processor which is based on a different number system should convert its result into binary in order to communicate with other systems. Unfortunately, the conversion of these systems (especially the RNS) into binary is not easy. Also, the RNS suffers from shortcomings in performing operations such as division and magnitude comparison (Szabb and Tanaka, 1967). As a result, these number systems are suitable only for special-purpose machines that use their strengths and avoid their weaknesses. A general-purpose machine, on the other hand, should be capable of performing any operation and is generally designed to work in binary. 5.2

Computing Techniques

In order to develop practical optical computers, techniques should be used that utilize the strengths of optics. These are not necessarily the same methods that have been developed for electronic systems. Present digital electronic computers are based on the von Neumann machine designed in the mid 1940s. I n this machine, a memory cell is accessed by the central processing unit through an address unit (Fig. 23). The advantage of this technique is that the number of required interconnections for accessing the memory is significantly reduced. For example, a 64-Kbit memory can be accessed by only 16 lines. The price that is paid for this reduction is that only one cell can be accessed at a time. This puts a limit on the speed of the processor, and is known as the von

ADDRESS MECHANISM

DATA INPUT

PROCESSING UNIT

FIG.23. Schematic diagram of a von Neumann machine.

RESULT OUTPUT

200

MIR MOJTABA MlRSALEHl

Neumann bottleneck. Optics is capable of massive interconnection. Therefore, there is no reason to be restricted to the von Neumann machine. Several techniques for optical computing have been proposed that utilize the parallelism and interconnectivity of optics. Some of these techniques are described below. 5.2.1

Threshold Gates

A binary threshold gate has n inputs and one output. The output z is obtained from the inputs ( x l , x 2 , . . . , x,) using the equations z

=0

if w i x i < T,

z

=

1

if w i x i 2 T,

(5.1)

where w i is the weight factor corresponding to the input x i and T is a threshold value. The conventional logic gates, such as AND and OR, are special cases of threshold gates. It can be shown that using threshold logic, a function can be implemented with significantly fewer gates than with the conventional gates. A problem with electronic implementation of threshold gates is that, as the number of inputs increases, the construction of these gates becomes impractical. This is due to the fan-in problem in electronics. Optics does not suffer from this problem; numerous beams of light can be connected to the input of an optical system. The realization of optical threshold-logic gates has been investigated by Arrathoon and Hassoun (1984). In their proposed architecture, the multiplications are performed with Bragg cells and the summation is obtained by a lens (Fig. 24). The output light is converted to an electric signal by a detector and the resultant signal is compared with a threshold voltage. The proposed architecture can be implemented on one substrate using the integrated optics technology. 5.2.2

Table Look-Up

There are two methods for implementing a digital function. In the first method, the function is divided into a series of basic operations which are implemented by logic gates. For example, the addition of two n-bit numbers can be realized by cascading n one-bit adders. In the second method, the information about the outputs for all possible inputs are first stored in a memory. To perform a process, the output for a particular input is obtained from the stored information. This method, known as the table look-up technique, in general provides the output faster than the first method. However, for many functions of practical interest, the amount of information

OPTICAL AND OPTOELECTRONIC COMPUTING

r.'i(;.

20 1

14. Integrated optical threshold gate (Arrathoon and Hassoun, 1984).

that should be stored is too large and the table look-up method becomes impractical. One solution to this problem is to use the RNS. Using the RNS, a large table can be replaced by several small tables. The total number of entries in these small tables is much smaller than the number of entries in the large table while the information contents of the two are equal. Each small table corresponds to a particular modulus and is independent of the other tables. Therefore, the operation on all moduli can be processed in parallel. The table look-up technique was first proposed by Cheney (1961). He described the use of magnetic core elements to implement residue functions. Huang et al. (1979) proposed several optical implementations for the table look-up technique. They used position coding to represent residue numbers, and showed how residue functions can be realized by spatial maps. These maps can be implemented by mirrors, prisms, gratings, optical waveguides, or fibers. In general, two types of maps are needed: fixed maps and changeable maps. Fixed maps are needed for performing an operation on the input number with a specific value-for example, adding the input number by 3. Changeable maps are needed to perform an operation on the input with a variable datafor example, adding a second number ( Y ) to the input number ( X ) . Using these two types of maps, any operation on residue can be implemented. In particular, the table look-up technique is very suitable for performing matrixvector multiplication. Another optical implementation of the look-up technique was proposed by Horrigan and Stoner (1979). They used an opto-electronic system to realize tables similar to Cheney's. They also introduced an electro-optical parallel summer which can be used for matrix-vector multiplication. Guest and Gaylord (1980) used the table look-up technique to realize truth tables. In their truth-table look-up processor, all the input combinations that

202

MIR MOJTABA MIRSALEHI

produce a one in each output bit are stored. The process is then performed by comparing the input pattern with these prestored patterns, known as the reference patterns. If the input combination matches one of the reference patterns, a one is designated to that output bit, otherwise a zero is designated to it. This is a type of content-addressable memory (CAM). The advantage of the CAM over the location-addressable memory (LAM) is that logical minimization techniques can be used to reduce the number of reference patterns. As a result, the amount of information that should be stored is significantly reduced. More reduction can be achieved by multilevel coding (Mirsalehi and Gaylord, 1986a). For example, the 16-bit full-precision addition of two multilevel coded residue numbers requires the storage of only 300 reference patterns. 5.2.3

Symbolic Substitution

Symbolic substitution as a method for optical computing was first proposed by Huang (1983). In this technique, the input data are spatially coded. An operation is performed by detecting specific patterns and replacing them with patterns that correspond to the result. For example, the binary values zero and one can be coded as shown in Fig. 25a. To perform an operation on two numbers, their codes are put on top of each other and the substitution rules are applied. The rules for addition are shown in Fig. 25b. The addition of two n-bit numbers is performed simply by applying these rules n times. Using an optical system, the substitution can be performed on all bits in parallel. An optical symbolic substitution processor is proposed by Brenner et al. (1986). This processor has four parts: a pattern splitter, pattern recognizers, pattern substituters, and a pattern combiner. The pattern splitter is used to copy the coded input data as many times as needed. For example, in the binary addition case, four copies of the inputs are made, since binary addition has four different rules. Each of these copies is checked by a particular pattern recognizer that corresponds to one of the possible patterns, and the locations that match that pattern are detected. The patterns detected by each recognizer are changed to the corresponding output patterns using a pattern substituter. The outputs of all substituters are then combined to obtain the output. The symbolic substitution technique is not limited to binary arithmetic. Any Boolean logic operation can be performed by this method. In fact, any arbitrary operation that can be expressed by some rules can be implemented by symbolic substitution. The input data can be coded by different techniques. The method described above is called position coding. In this method, two pixels are needed for each bit. Another technique is to use two orthogonal polarizations to represent 0 and 1. This method of coding has the advantage that it reduces the device area by half.

203

OPTICAL AND OPTOELECTRONIC COMPUTING

0 0

I

8-5

E-0

oto

=

00

u-0 It0 =

01

OtI

=

01

8-% 1+1

=

10

FIG.2.5. (a) Coding procedure for logical values 0 and I . (b) Substitution rules for addition (Brenner and Huang. 1985).

5.3

Architectures and Technologies

Digital optical processors have been proposed and realized using different technologies. Some of the architectures are described below. 5.3.1

SLM-Based Processors

Spatial light modulators are widely used in the architectures of digital optical processors. In most cases, a SLM is used toenter the input data into the system. In other processors, it is used to perform digital or logic operations. In this section, only some of the architectures in which the SLMs are the major elements for their operation are described. The application of liquid crystal light valve (LCLV) for digital processing has been investigated by Collins ( 1977). He used the birefringent effect of

204

MIR MOJTABA MlRSALEHl

the LCLV to rotate the polarization of light. Using the cyclic nature of polarization, the device was capable of performing residue arithmetic. In a later work, Collins r t al. (1980) used a Hughes LCLV to realize logic gates. Athale and Lee (1979) have developed a type of SLM called an optical parallel logic (OPAL) device. The device consists of a layer of twisted nematic liquid crystal which is used as an electro-optic material. This layer is covered by a thin film of SiO, and CdS with a checkerboard pattern. Athale and Lee fabricated an array of 8 x 8 pixels of the OPAL and showed how different logic functions can be realized with this device. Of particular interest is the implementation of a half-adder which can be implemented by two OPAL devices. An architecture in which the SLM is the key element is shadow casting. Digital processing by shadow casting was first proposed by Ichioka and Tanida (1984). In their proposed system, they used spatial coding to represent binary numbers. Two 2-D arrays were used to code the input numbers. These arrays were put in close contact to each other and were illuminated by four light-emitting diodes (LEDs) that were placed at the corners of a square. A coding mask was used at the output plane to spatially filter the desired function. Using this technique, it is possible to create all 16 logic functions for two binary inputs. Tanida and Ichioka described the use of color and computer-generated holograms for coding the data in their system. The shadow-casting architecture was extended to multiple-input, multipleoutput functions by Arrathoon and Kozaitis (1986). They also described how multiple-valued logic (MVL) can be implemented by shadow casting. The advantage of MVL over binary logic is that more information can be handled by each channel. This is achieved at the price of more pixels and LEDs than used in the binary case. Li et al. (1986) proposed the use of two orthogonal polarizations for encoding the data. A method for the design of logic units with polarization shadow casting is described by Karim et al. (1987). One of the advantages of the shadow-casting architecture is that it utilizes the parallel-processing capability of optics. The inputs are entered as 2-D arrays of data, and the desired logic operation is performed on all elements of the arrays. Another advantage of this architecture is that it is relatively simple. The system does not require a laser; it works with the incoherent light produced by the LEDs. Also, no lenses are needed. In spite of these advantages, shadow casting is presently limited to small systems. To use this technique for large and practical systems, spatial light modulators with a large number of pixels and fast operating speeds are needed. 5.3.2

Holographic Processors

The application of holography for digital processing was first introduced by Preston (1972). He used different phases (0 and 180") to encode the digital

OPTICAL AND (3PTOELECTRONIC COMPUTING

205

inputs of 0 s and 1’s. To realize a logic operation, a hologram was first recorded. The hologram was then developed and used to perform that operation on the input data. In the output plane, the result was obtained as bright and dark spots representing 1’s and 0’s. Preston experimentally demonstrated the logic operations IDENTITY and EXCLUSIVE OR on two input variables. He also described how the logic operations AND and OR can be realized (Preston, 1972). Preston’s work was extended by Guest and Gaylord (1980).They proposed two types of holographical truth-table look-up processors that function based on logical EXECUTIVE OR and NAND operations. Figure 26 shows their

Flci. 26 NAND-based holographical digital optical processor: (a) recording the truth-table holograms arid (h) example of multiplication with the processor. LSR is the least significant bit and MSR is the most significant hit (Guest and Gaylord. 1980).

206

MIR MOJTABA MlRSALEHl

NAND-based processor. In both systems, the information in the truth tables of the desired operation was stored in a photorefractive crystal, such as LiNbO,, as thick holograms. The crystal was then used to obtain the result of that operation on the input data. During the recording process, two different phases (0 and 180") were used for encoding zeros and ones. In the reading process, however, the zeros and ones were encoded by opaque and transparent pixels. With some modification in the data encoding, the NAND-based processor can be used for multi-valued functions (Mirsalehi and Gaylord, 1986a). The application of phase-only holograms for optical symbolic substitution has been investigated by Mait and Brenner (1988).These holograms are made of non-absorbing materials, hence, they have the advantage of high power efficiencies.

5.3.3 Acousto-optic Processors Acousto-optic processors are based on the interaction of sound and light waves. The heart of these devices is an acousto-optic Bragg cell described in Section 4.6.2. Several architectures for performing matrix operations with acousto-optic cells have been introduced (Rhodes and Guilfoyle, 1984; Casasent, 1984). Most of these architectures function as systolic processors. That is, the elements of a vector or a matrix are entered in a specific order at specific positions on the acousto-optic cell, and the output is obtained by a time-integrating detector. An example of a systolic matrix-vector multiplier is shown in Fig. 27. The system realizes the simple equation

Two short electric signals are sequentially entered to the driver of the acoustooptic cell. The amplitudes of these signals are chosen such that they produce two propagating gratings with diffraction efficiencies proportional to x 1 and x 2 . Another set of electric signals is used to drive the light-emitting diodes (LEDs) and produce light intensities proportional to the elements of the matrix. For correct operation, the two signals should be synchronized. After the first clock pulse, the acoustic wave corresponding to x1 will be in front of the lower LED. At this moment, the lower LED creates a light wave with an intensity proportional to a,, . The created light will be diffracted by the grating and detected by the upper detector. The power of the diffracted light will be proportional to a l , x l . After the second clock pulse, the acoustic wave corresponding to .xz will be in front of the lower LED. At this moment, the lower LED creates a light wave with an intensity proportional to u12.The

207

OPTICAL AND OPTOELECTRONIC COMPUTING

ACOUSTOOPTIC MODULATOR

LEDS

a2LBi a12 QII

g

1,1

INTEGRATING DETECTORS

-

y2

[I----,

FIG.27. Acousto-optic systolic matrix-vector multiplier (Rhodes and Guilfoyle, 1984)

power of the diffracted light will be proportional to u I 2 x 2and detected by the upper detector. Since the detectors are of integrating type, the total power detected by the upper detector will be y 1 = a l , x l + u l 2 x 2 . Similarly, the output y , will be obtained. Although the computational power of the systolic acousto-optic processors is high ( 10'" operations/s), they suffer from low accuracy ( - 8 bits). As a result, they are suitable for applications where high accuracy is not essential. More accurate acousto-optic processors can be achieved by sacrificing the throughput of the system for accuracy. Guilfoyle (1984) has developed a system that is 20-bit accurate and has a computing power equivalent to 2.5 x 10" multiply-add/s. To get high accuracy, he has used an algorithm introduced by Whitehouse and Speiser ( 1976) to perform digital multiplication by analog convolution (DMAC). According to this algorithm, the result of multiplication of two binary numbers can be obtained by convolving the two strings of ones and zeros that correspond to those numbers. The result will be in the mixed binary system which has a base of 2 but digits larger than 1 are allowed. For example, consider the multiplication of 19 and 27. The binary representations of these numbers are 10011 and 1101 1. If these two strings are treated as signals f ( t ) and g ( t ) and then convolved, the result will be f ( t ) * g ( t ) = 110231121 which is equivalent to the decimal number 513. Figure 28 shows a schematic diagram of the Guilfoyle's matrix-vector multiplier. The system consists of two acousto-optic cells which are driven by signals proportional to the elements of the matrix and vector. The two lenses a t the left side are used for expanding and collimating the laser beam.

-

208

MIR MOJTABA MlRSALEHl

Fig. 28. Systolic acousto-optic binary computer (SAOBiC) configuration (Guilfoyle, 1984).

The other lenses are used for imaging and Fourier transformation. The final result is imaged on the detector arrays. It should be mentioned that subsequent analysis (Psaltis and Athale, 1986) has shown that DMAC is not an efficient algorithm, so alternative numerical methods are being explored. Acousto-optic cells can be used to implement optical programmable logic arrays (PLAs). The implementations of full-adder and 2- and 3-bit multipliers have been described by Guilfoyle and Wiley (1988). One advantage of these processors is that, unlike electronic PLAs, they do not suffer from the fan-in problem. The OR operation on a large number of optical beams can be performed by using a lens to focus those beams on a detector. The combinatorial architecture is very powerful since any digital function can be written as a sum-of-products expression and be realized by an optical PLA. 5.3.4

Integrated-Optical Processors

The field of integrated optics started in the mid 1960s when the thin-film technology was used to develop optical waveguide devices. An optical waveguide is made of a transparent and dense material which has a higher index of refraction than its surrounding media. As a result of this characteristic, a light wave can propagate inside the waveguide without leaking out.

209

OPTICAL AND OPTOELECTRONIC COMPUTING

Optical fibers, which are widely used for communication today, are one type of optical waveguides. In integrated optics, optical devices are manufactured on a flat substrate by techniques similar to the ones used in integrated electronics. Here, we are interested in the applications of these devices for optical computing. Integrated-optical devices have several advantages, including small size, small power consumption, and high speed. On the other hand, they are restricted to two dimensions and cannot utilize the 3-D capability of optics. Optical waveguides have several structures. The simplest structure is the slab waveguide which consists of a thin film on a substrate. The index of refraction of the material used as the thin film is higher than the index of refraction of the substrate. Different materials have been used for fabrication of optical waveguides. The most widely used materials are LiNbO,, GaAs, and Si. So far, no material has been found that has all the properties desired for integrated optics. Most of the developed devices are made of LiNbO,, since the fabrication with this material is easier. On the other hand, GaAs has the advantage that it can be used to fabricate both passive and active devices. Integrated-optical devices can be used for logical and digital operations. One of these devices which is based on the Mach-Zehnder interferometer is shown in Fig. 29. The device, in general, has two electrodes surrounding each arm. A beam of coherent light created by a laser enters from the left side and is split into two beams. If a voltage is applied between the electrodes of one arm, due to the change in the index of refraction, the optical wave in that channel will experience a phase shift. The applied voltage is normally chosen such that the amount of phase shift is 180".The two optical beams add coherently in the output and produce the result. Using the Mach-Zehnder interferometer, logic operations can be realized. For example, consider Fig. 30a where a voltage proportional to the logic variable ( I is applied to the electrodes. If a = I , the optical wave in the top channel will be shifted by 180' with respect to the wave in the lower channel. As a result, the two waves cancel each other and there will be no light in the output, indicating a logical 0. If a = 0, the two waves will remain in phase and there will be an output light indicating a logical 1. Therefore, the device works

INPUT LIGHT

-

FIG 29

-

OUTPUT LIGHT

The Integrated optlcal mplementatlon of the Mach-Zehnder interferometer

210

MIR MOJTABA MlRSALEHl

as an inverter. Using this technique, other logic functions can be realized. Some examples are shown in Fig. 30 (Taylor, 1977). Integrated-optical devices can also be used for numerical operations, especially matrix-vector and matrix-matrix multiplications (Hurst, 1986). As an example, Fig. 31 shows a matrix-vector multiplier proposed by Verber (1984). A beam of coherent light impinges on a metallic grating which is fabricated on a planar waveguide. The electrodes of this grating are connected to voltages that are proportional to the elements of the vector. As a result, the diffracted light beams will have intensities that are proportional to the elements of the vector. These gratings are used as beam splitters to produce three beams of light with equal power in each channel. Each of the produced beams then impinges on a metallic grating which is connected to a voltage proportional to one of the elements of the matrix. Three lenses are used to

a

a

b

a@b

a

b

1

1

FIG. 30. Integrated optical implementations of logical functions: (a) NOT, (b) EXCLUSIVE OR, (c)AND (Taylor, 1977).

OPTICAL AND OPTOELECTRONIC COMPUTING

211

LiNb03 Planar Waveguide

1PI 1p2

I

p3

F c . 31.

Integrated optical matrix-vector multiplier (Verber, 1984).

combine the beams that correspond to each output term. The outputs are obtained from the detectors placed at the focal points of the lenses. The advantage of this multiplier is that it is fully parallel. Using the systolic processing techniques, the integrated-optical processors can be used for matrix-matrix multiplication (Verber, 1984). Another integrated-optical processor is the device that performs the Givens rotation (Mirsalehi et cil., 1986). The elementary rotation matrix may be expressed as

The Givens rotation is obtained when $ is chosen such that d = 0. A schematic diagram of the integrated-optical Givens rotation device is shown in Fig. 32. The device consists of a metallic grating and two phase shifters. An array of this device can be used for matrix triangularization which is an important operation for solving systems of linear equations (Gaylord and Verriest, 1987).

212

MIR MOJTABA MlRSALEHl

FIc;. 32. Integrated optical Givens rotation device (Mirsalehi et

6.

a/., 1986).

Hybrid Processors

The inherent two-dimensional nature of optical systems makes it possible to work with large two-dimensional arrays of data. A second advantage of optical systems is their capability of operating at tremendous data rates. Since signals travel through the optical systems with the speed of light, the throughput of these systems is enormous. With state-of-the-art technology, pure optical systems have some drawbacks. First, the majority of optical systems are analog in nature. As with other analog systems, high accuracy is difficult to achieve. The second problem is that optical systems by themselves are not capable of making decisions. The simplest decision based o n comparison cannot be made without the help of electronics. A third problem is that optical systems cannot be programmed as a general-purpose computer. Purely optical systems can only perform specific tasks, and they can be considered analogous to hardwired electronic analog computers. These deficiencies of optical systems are the strong points of electronic systems. Accuracy, control, and programmability are some of the advantages of digital electronic computers. To combine the nice features of both optical and electronic systems, hybrid processors have been developed.

OPTICAL AND OPTOELECTRONIC COMPUTING

213

One of the first suggestions to combine optical and digital electronic processors was made by Huang and Kasnitz (1967). In the early 1970s, several hybrid systems were proposed to perform specific tasks. The articles by Thompson (l977), Casasent (l98l), and Leger and Lee (1982) have extensive reviews of the literature on hybrid processors. In the following sections, we introduce some of the hybrid processors. First, we describe the general design characteristics of a hybrid processor, and then we present some specific systems. 6.1

General Hybrid Processor

Depending on the tasks that they are designed for, hybrid processors have different architectures. Figure 33 shows a general design which includes most of the existing hybrid systems. Since a hybrid processor is a combination of an optical system and an electronic one, interface devices between the two systems are essential elements of the processor. The input signal can be in three different forms: ( 1 ) a two-dimensional optical signal, (2) a plane wave modulated by an electrical-to-optical (E/O) interface that converts an electronic signal to the optical transmittance of a transparency, and ( 3 ) an array of light sources, such as laser diodes (LDs) or light-emitting diodes (LEDs). Beyond the input interface, the signal propagates as an optical signal and is manipulated by the optical system. The optical system may include SLMs whose transmittance functions are controlled by the central control unit. The output of the optical system can be read as an optical signal or converted to an electronic signal by an optical-to-electrical (O / E )interface which can be a one- or two-dimensional array of photodiodes.

OPTICAL OR ELECTRICAL INPUT SIGNAL

INTERFACE

~

OPTICAL SYSTEM

CENTRAL CONTROLLER AND DIGITAL PROCESSOR

-

6, DEVICES

FIG 33. Schematlc diagram of a general hybrid processor.

214

MIR MOJTABA MlRSALEHl

6.2

Algebraic Processors

Hybrid algebraic processors represent an important class of optical processors. These processors perform the basic operations of linear algebra with flexible architectures. In the last two decades, several optical systems have been proposed that implement the basic linear algebra operations. Cutrona suggested a simple matrix-vector multiplier in 1965. Multiplication of two matrices by coherent optical techniques was proposed by Heinz et al. (1970), and was demonstrated experimentally by Jablonowski et al. (1972) for the simple case of 2 x 2 matrices. An alternative method for matrix-matrix multiplication was introduced by Tamura and Wyant (1977). Schneider and Fink (1975) proposed an incoherent optical system for matrix-matrix multiplication. Krivenkov et al. (1976) and later Bocker (1984) described methods for matrix-matrix multiplication using incoherent light. A very important method in multiplying a vector by a matrix was suggested by Bocker (1974) and an improved version of this method was described by Monahan et al. (1975). Recently, attention has been focused on optical architectures that perform matrix multiplication by systolic, engagement, and outer-product techniques (Caulfield et al., 1984; Athale, 1986). 6.2.1

Matrix-Vector Multiplication

One of the first operations in linear algebra that has been implemented optically is the matrix-vector multiplication (Goodman et al, 1978). As shown in Fig. 34, the vector x is represented by an array of light emitting diodes (LEDs)and the matrix A is represented by an optical mask. The light produced by each LED is expanded horizontally and illuminates the corresponding row of the optical mask. The light emerging from each column of the mask is collected vertically and illuminates one element of the photodiode array. As a result, the output of the photodiode array represents the vector b where A X = b.

(6.1

This system is capable of multiplying large-size vectors and matrices. The limitation on the size is influenced by the electro-optic devices used and the medium on which the matrix is written. Another advantage of this system is its high speed of operation. Since the system operates in parallel, the size of the matrix does not affect the speed of the operation. The disadvantage of this system, which is the result of its analog nature, is low accuracy (- 1%). 6.2.2 Matrix-Matrix Multiplication

Another important linear algebra operation is matrix-matrix multiplication. This operation can be implemented optically by several techniques

OPTICAL AND OPTOELECTRONIC

LED ARRAY

COMPUTING

MATRIX MASK

215

PHOTODIODE ARRAY

FIG. 34. Schematic diagram of an optical matrix-vector multiplier.

(Bocker, 1984; Barakat, 1987). One method is to use the matrix-vector multiplier shown in Fig. 34. Let A be an N x K matrix, 5 be a K x M matrix, and C be their product. The multiplication A 5 = C can be written as (6.2) where bi and ci represent the vectors obtained from the ith columns of the matrices B and C , respectively. To perform this operation using the matrixvector multiplier, we write b, on the LED array and read the vector c, from the photodiode array, and then we keep feeding one column of 5 and read one column of C at a time. In M cycles we obtain the result of the operation. The optical matrix-vector and matrix-matrix multipliers described above can be used to solve many linear algebra problems. In the following section, we present an optical hybrid system capable of solving systems of linear equations, matrix inversion, eigenvalue problems, and other linear and nonlinear problems with high accuracy. A[b,

lb2l+Ml

= CClIC2I.+MI.

6.2.3 The Bimodal Optical Computer Solving a system of linear equations and determining the eigenvalues and the eigenvectors of a system are important problems. These problems arise in solving many physical problems, such as signal extrapolation, phased-array radar, and image enhancement. In most practical cases, the matrices involved

216

MIR MOJTABA MlRSALEHl

are very large ( - 1000 x 1000) and the required computations are very intensive. Although very powerful digital electronic computers have been developed, solving many practical problems is still very time consuming. Because of its inherent parallelism and high speed, optics seems a natural choice for solving this class of problems. Analog optics is very attractive for optical information processing and computing. As shown in the matrix-vector multiplier of Fig. 34, all the elements of the vector can be processed in parallel. At almost the same time that we write the input x, we can read the output b. If the optical path length between the input and output planes is 6 cm, the whole operation of the matrix-vector multiplication can be done in 200 psec. For N = 1000, the number of operations needed to perform Ax = b is on the order of lo6. Therefore, the speed of the processor is about 5 x lOI5 operations/sec. This illustrative example shows the high speed of the analog optics in performing linear algebra operations. Unfortunately, this high speed of operation is combined with low accuracy which is the nature of all analog systems. Analog optics is very fast but inaccurate. On the other hand, digital electronics is very accurate but not as fast as analog optics. The advantages of both analog optics and digital electronics can be achieved in a compromised hybrid system that slows down the processor speed but in return increases the accuracy substantially. The bimodal optical computer (BOC) introduced by Caulfield et al. (1986) is based on this idea. In the following discussion we will show how to solve a system of linear equations using the BOC. Consider an N x N matrix A, and N x 1 vectors x and b. A and b are given, and we want to solve the equation Ax=b

to find the vector x. Equation (6.3) can be solved by analog optic techniques, such as the relaxation method (Cheng and Caulfield, 1982). Consider the hybrid system shown in Fig. 35. First, an initial value for x is assumed and is written on the LEDs. Then the vector x is multiplied by the matrix A. The resultant vector y is compared with b using a difference amplifier, and the difference is fed back to correct x. This process of multiplying the new value of x by A and comparing y with b continues until the difference between y and b becomes negligible and the value of x converges to the solution of Eq. (6.3). This method of solving a system of linear equations is very fast. Its speed is limited only by the speed of the electro-optic devices and the feedback electronic system used, and can be in the nanosecond range. Let us consider now the accuracy of the system. In writing the matrix A on the optical mask (which can be a photographic film or a spatial light modulator) and the vector x on the LED array, large errors are introduced because of the nature of these analog devices. Also, reading the vector b from

A PHOT0DI0DE ARRAY

LED

PROCESSOR *

1

' I

FIG.35. Schematic diagram of a bimodal optical computer.

218

MIR MOJTABA MlRSALEHl

the photodiode array cannot be done accurately. Therefore, the system in Fig. 35 does not solve Eq. (6.3) but instead it solves (6.4)

Aoxo = bo,

where the subscript zeros indicate inaccuracies in the optics and electronics. The solution xo of Eq. (6.4) can be refined to get the vector x using the fol I ow ing algorithm . (a) Solve the system in Eq. (6.4) for x, using the analog optical processor. (b) Store the solution xo with high accuracy in the ditigal processor. Use a dedicated digital processor to calculate the residue r = b - Ax, = A(x

-

x,) = A Ax.

(6.5)

(c) Use the optical analog processor to solve the new system of linear equations Ay = sr, (6.6) where y = s Ax and s is a “radix,” or scale factor, which is chosen to use the dynamic range of the system efficiently. (d) Use the digital processor to refine the solution xo for x 1: x 1 = x0

+ AX.

(6.7)

If the refined solution x , is accurate enough, terminate the iterations. Otherwise, return to (b) for a better solution. The convergence and speed of the solution for the system of linear equations has been studied by Abushagur and Gaulfield (1986, 1987), and Abushagur et ul. (1987).The convergence of the iterative solution depends on two main factors. The first factor is thecondition number of the matrix A , , i.e., z(A,). The smaller the condition number is, the faster the solution will converge. The second factor is the error involved in reading and writing A , x, and b using the electro-optic devices. The higher the accuracy of these parameters are, the faster the convergence will occur. A bimodal optical computer has been built and tested experimentally for solving a system of linear equations (Habli ef ul., 1988).Although the accuracy of the analog processor was about 2 bits, a solution with 16-bit accuracy was obtained by the hybrid system after few iterations. These experimental results show that a highly accurate hybrid system can be obtained from a lowaccuracy analog processor. 6.3

Diffraction-Pattern Sampling Processor

The diffraction-pattern sampling processor is one of the most developed hybrid systems. It is used for automatic data analysis, especially in applications that require the analysis of large amounts of data (Lendaris and

OPTICAL AND OPTOELECTRONIC COMPUTING

219

Stanley, 1970; George and Kasdan, 1975).The detector that is usually used in this type of processor is a unique photodiode array which consists of 32 wedge-shaped and 32 ring-shaped photodetectors. The ring-shaped detectors provide information about the spatial frequency content of the diffraction pattern. The wedge-shaped detectors provide information about the orientation of the object. The output from the 64 detectors is digitized and read by a digital computer. Recognition algorithms are developed for specific applications. The diffraction-based processor has several applications (Casasent, 1981). Here we mention some of them. One of the initial application of this device is the analysis of aerial imagery. Urban imagery can be distinguished from rural imagery based on their diffraction patterns. Urban imagery contains highresolution details and regular structures such as streets and buildings. These features appear as high-spatial-frequency components in the diffraction pattern. A classification algorithm can be developed based on the highfrequency and angular content of the diffraction pattern. Another application of the system is the detection of muscular dystrophy. This is possible because of the significant difference in the diffraction patterns of healthy and diseased cells. This hybrid system has industrial applications, especially in quality control of the products. For example, hypodermic needles can be automatically inspected by the system while they are on the conveyor belt. Bad needles generate vertical components in their diffraction pattern. The wedgeshaped detectors detect the presence of these components and the needles with defects are rejected. Versions of this system are now commercially available (Clark, 1987) and are being used in various industrial environments.

7.

Conclusion

Since the invention of the laser in 1960, there has been an increasing interest in the field of optical information processing. Elements of optical computers, such as memories, logic arrays, and input/output devices, have been developed with various degress of success. The development of holographic memories pursued in 1970s was not very successful because of the material problems. Optical disks, on the other hand, are promising devices and have found applications in data storage. Although the feasibility of optical logic arrays has been demonstrated, large arrays of practical interest have not been developed yet. The input/output devices for optical computers, especially the spatial light modulators, are among the key elements that should be improved in order to use the full advantage of optics. One of the important features of optics is its capability of global interconnections, which is far beyond what can be achieved with electronics. Also, optical interconnections can be made programmable. This feature is useful for performing digital operations, such as discrete Fourier transformation, or for implementing neural networks.

220

MIR MOJTABA MlRSALEHl

Optical processors can be classified in three categories: analog processors, digital processors, and hybrid processors. Although advances in all three areas have been achieved, analog processors have been more investigated and developed. Among the operations that can be easily performed with optics are Fourier transformation, correlation, convolution, and spatial filtering. Also, optical processors have found applications in image processing. Among other important applications of these processors are spectrum analysis and synthetic-aperture radar data analysis. There is an increasing interest in developing digital optical processors. At this stage, most of the efforts in this field are concentrated on developing computing techniques and architectures. Among the promising techniques are symbolic substitution and table look-up. Various architectures using different technologies are under investigation for optical computing. Among the most promising technologies are acousto-optics and integrated-optics. Also, processors based on holography and spatial light modulators have been developed. Hybrid processors combine the advantages of analog optics with those of digital electronics. These processors have found applications in both laboratory and industry and are promising architectures for computationally intensive problems such as solving large sets of linear or nonlinear equations. Today, research in all areas of optical computing is pursued, and this makes the prediction of the future of the field difficult. Knowing the pitfalls of predicting with greater clarity, we tackle the more modest task of doing a linear extrapolation of the present activities. We suspect that the future will hold more, not less. In terms of what will be achieved during the next 10 years, here are our thoughts. 1. Optics will find more applications in electronic computers. Optical disks which are now commercially available, will become more popular. In particular, erasable optical disks will be more developed, and probably will replace the present hard disk memories. Optics will be used for clock distribution in VLSI circuits of high speed systems. 2. Hybrid optoelectronic processors will offer high speed for linear and near-linear algebra problems. 3. Coherent and partially coherent optical pattern recognition will find industrial and military applications in the field. 4. The utility of optics for expert systems will be established. 5. Optically implemented multi-layer neural networks will allow real time control of complex systems.

Even if only these accomplishments take place, optical computing will have established a major role in the total computer arena.

OPTICAL AND OPTOELECTRONIC COMPUTING

221

ACKNOWLEl>GEMEN’I

We are grateful to Joseph Shamir for helpful comments on a draft of this article RFYEKENcES Abbe, E. (1873). Beitrage zur theorie des mikroskops und der Mikroskopischen Wahrnehmung. Archif.. Mikroskopische Anot. 9, 41 3 -468. Ah-Mostafa. Y . S., and Psaltis, D. (1987).Optical neural computers. Scientific American 256 ( 3 ) , 8n 9s. Abushagur. M. A. G., and Caulfield, H. J. (1986).Highly precise optical-hybrid matrix processor. Proc. Soc. Phoro-Opi. Instrum. Eng. 639, 63- 67. Abushagur. M. A. G.. and Caulfield, H. J. (1987). Speed and convergence of bimodal optical computers. Opr. En