Advances in Computers.. v. 2 (AP Inc.,U.S)(

Advances in COMPUTERS VOLUME 2 This Page Intentionally Left Blank Advances in COMPUTERS edited b y F R A N Z L A ...

Author: Author Unknown

12 downloads 448 Views 22MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Advances in COMPUTERS VOLUME 2

This Page Intentionally Left Blank

Advances

in

COMPUTERS edited b y F R A N Z L A L T National Bureau of Standards Washington, D. C.

associate editors A. D. BOOTH

R. E. MEAGHER

VOLUME 2

Aoademio Press New York London 1 9 6 1

COPYRIGHT @ 1961, BY ACADEMIC PRESS INC. ALL RIGHTS RESERVED NO PART OF THIS

BOOK

MAY

BE REPRODUCED IN

ANY

FORM, BY PHOTOSTAT, MICROFILM, OR ANY OTHER MEANS, WITHOUT WRIlTEN

PERMISSION FROM

THE PUBLISHERS]

EXCEPT A S STATED I N THE FIRST FOOTNOTE ON PAQB

m a mr

a.

PAGE 135

ACADEMIC PRESS INC. 111 FIFTHAVENUE NEWYORK3, NEWYORK

United Kingdom Ediliun Published by ACADEMIC PRESS INC. (LONDON) LTD. 17 OLDQUEENSTREET, LONDON, S.W. 1

Library of Congress Catalog Card Number 59-15761

PRINTED IN THE UNITED STATES OF AMERICA

Contributors to Volume 2

PHILIPJ. DAVIS,National Bureau of Standards, Washington, D . C. JIMDOUGLAS, JR., Department of Mathematics, Rice University, Houston, Texas SAULI . GAS, Federal Systems Division, International Business Machines, Washington, D . C. ROBERTMCNAUGHTON, Moore School of Electrical Engineering, University of Pennsylvnnia, Philadelphia, Pennsylvania PHILIPRABINOWITZ, Department of Applied Mathematics, Weizmann Institute of Science, Rehovoth, Israel KENNETH R. SHOULDERS, Applied Physics Laboratory, Stanford Research Institute, Menlo Park, California

V


Preface

The success of the first volume of Advances in Computers confirms the expectation that there was a real need for publications of this kind. Indeed, this could have been inferred from the emergence, during the past few years, of similar series in a large number of fields of knowledge. What is it that distinguishes these series from the more traditional media of publication? It seems that they possess three outstanding characteristics, all interrelated and reinforcing each other. First, they cover a wide range of topics, so wide that hardly anyone can claim to be expert in all of them. One may conjecture that this is felt by many readers as a welcome antidote to the ever growing specialization of technical fields. Second, and necessitated by the first, there is the level of competence required of the reader. These are not “popular” articles by any means; they represent solid technical writing, yet, as we said in the Preface to Volume I, they are intended to be intelligible and interesting to specialists in fields other than the author’s own. And finally, there is the length of the individual contribution: shorter than a monograph but far longer than an article in a conventional technical journal; long enough to introduce a newcomer to the field and give him the background he needs, yet short enough to be read for the mere pleasure of exploration. These are our aims. If we have fallen short, we ask the reader’s indulgence, and invite him to contemplate the new editorial difficulties created by this type of publication: the need to find a group of authors willing to engage in the time-consuming business of expository writing, rather than pursuing their own special interests; to find just enough of them for a volume of manageable size; and to persuade them to time the completion of their manuscripts so as not to keep each other waiting. Even if there were universal agreement on what constitutes a balanced mixture of topics, a proper level of technical writing, and adequate expository treatment, one could hardly hope to accomplish more than a crude approximation to the ideal. FRANZ L. ALT October, 1961

vii


Contents CONTRIBUTORS TO VOLUhfE 2 PREFACE . CONTENTS O F VOLUhfE 1

V

vii

...

Xlll

A Survey of Numerical Methods for Parabolic Differential Equations JIM DOUGLAS, Jr.

1. Introduction . . . . . . . . . . . 2. Preliminaries . . . . . . . . . . . 3. Explicit Difference Equations . . . . . . 4. The Backward Difference Equation . . . . 5. The Crank-Nicolson Difference Equation . . . 6. An Unconditionally Unstable Difference Equation 7. Higher Order Correct Difference Equations . . 8. Comparison of the Calculation Requirements . 9. Several Space Variables . . . . . . . . 10. Alternating Direction Methods . . . . . . 11. Abstract Stability Analysis . . . . . . 12. The Energy Method . . . . . . . . 13. Stefan Problem . . . . . . . . . 14. Parabolic Systems . . . . . . . 15. Integro-Differential Equations . . , . . 16. Extrapolation to the Limit . . . . . . . References . . . . . . . . . . . . . ,

,

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

,

.

.

.

.

.

. .

31

. . .

1 2 4 13 18 24

.

. . .

.

.

. .

.

37

.

.

.

.

.

.

.

.

.

.

41 44 46 48 49

.

.

.

50

.

.

. .

.

52

56 56 58 59

.

.

.

25 30

Advances in Orthonormaliring Computation PHILIP J. DAVIS and PHILIP RABINOWITZ

PART I: THEORETICAL

1. Introduction . . . . . . . . . . 2. The Geometry of Least Squares . . . 3. Inner Products Useful in Numerical Analysis 4. The Computation of Inner Products . . . ix

,

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

,

,

.

CONTENTS

X

. . . . . . .

60 63 64 68 69 70 73 75

.

79 81 83

5 Methods of Orthogonalization . . . . . . . . . . 6. Tables of Orthogonal Polynomials and Related Quantities . 7 Least Square Approximation of Functions . . . . . . 8 Overdetermined Systems of Linear Equations . . . . . 9 Least Square Methods for Ordinary Differential Equations . . 10 Linear Partial Differential Equations of Elliptic Type . . . 11 Complete Systems of Particular Solutions . . . . . . . 12 Error Bounds; Degree of Convergence . . . . . . . . 13. Collocation and Interpolatory Methods and Their Relation to Least Squares . . . . . . . . . . . . . . 14 Conformal Mapping . . . . . . . . . . . . . 15. Quadratic Functionals Related to Boundary Value Problems PART 11: NUMERICAL

16. Orthogonalisation Codes and Computations . . . . . . 17. Numerical Experiments in the Solution of Boundary Value Problems Using the Method of Orthonormalized Particular Solutions . . . . . . . . . . . . . . . 18 Comments on the Numerical Experiments . . . . . . 19. The Art of Orthonormalization . . . . . . . . . . 20. Conclusions . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . .

.

85

89 116 121 122 123

Microelectronics Using Electron-Beam-Activated Machining Techniques

.

KENNETH R SHOULDERS

. . . . .

1 Introduction . . . . . . . . . 2 Research Plan . . . . . . . . . 3 Microelectronic Component Considerations 4 Tunnel Effect Components . . . . . 5 Accessory Components . . . . . . 6 Component Interconnection . . . . . 7 Substrate Preparation . . . . . . 8 Material Deposition . . . . . . . 9 Material Etching . . . . . . . . 10 Resist Production . . . . . . . . 11. Electron Optical System . . . . . . 12. High-Vacuum Apparatus . . . . . 13. EIectron Microscope Installation . . . 14 Demonstration of Micromachining . . .

. . . . .

.

. . . . . . 137 . . . . . . 144 . .

. .

. .

. .

. .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. . .

. . .

. . .

. . .

. .

. .

. .

. .

. .

. .

.

. 150 . 158 180 . 190 . 197 . 204 . 224 . 230 . 236 . 260 . 275 . 276

CONTENTS

15. Summary References .

. .

. .

. .

. .

.

.

. .

. .

xi

.

.

.

.

.

.

.

.

.

. .

.

.

.

.

.

283 289

Recent Developments in linear Programming SAUL I. GASS

1. Decomposition Algorithm . . . , . . . . . 2. Integer Linear Programming . . . . . . . . . 3. The Multiplex Method . . . . . . . . . . 4. Gradient Method of Feasible Directions . . . . . 5. Linear Programming Applications . . . . . . 6. Summary of Progress in Related Fields . . . . . 7. Linear Programming Computing Codes and Procedures 8. SCEMP . . . . . . . . . . . . . . 9. Linear Programming in Other Countries . . . . . References . . . . . . . . . . . . . .

. . . .

296 302 309 314 317 322 325 361 363 366

The Theory of Automata, a Survey ROBERT McNAUGHTON

379 1. Introduction . . . . . . . . . . . . 2. Finite Automata . . . . . . . . . . . . . 385 3. Probabilistic Automata . . . . . . . . . . . . 391 4. Behavioral Descriptions . . . . . . . 393 397 5. Various Concepts of Growing Autoiiinta . . . . . . 6 . Operations by Finite and Growing Automata, Renl-Time and General . . . . . . . . . . . . . . . . 402 7. Automation Recognition . . . . . . . . . 407 8. Imitation of Life-Like Processes by Automata 411 416 References . . . . . . . . . . . . 423 AUTHORINDEX . . . . . . . , . . . . SUBJECTINDEX. . . . . . . . . . . . . . 430 ,

,

,

,

,

,

,

,

,


Contents of Volume 1 Gcncrttl-Purpose Programming for Business Applications CALVIN C. GOTLIEB Numerical Weather Prediction NORMAN A. PHILLIPS The Present Status of Automatic Translation of Languages YEHOSHUA BAR-HILLEL Prograniniing Computers t o Play Games ARTHURL. SAMUEL Machine Recognition of Spoken Words RICHARDFATEHCHAND Binary Arithmetic GEORGE11'. REITWIESNER

xiii


A Survey of Numerical Methods for Parabolic Differential Equations JIM DOUGLAS, JR. Rice University

Housfon, Texas

1. Introduction

. .

2. Preliminaries .

.

. .

. .

. .

.

. 3. Explicit Difference Equations .

.

. . . . .

. .

. .

. .

.

.

.

4. The Backward Difference Equation . . . . 5. The Crank-Nicolson Differcnce Equation . . . 6. An Unconditionally Unstable Difference Equation 7. Higher Order Correct Difference Equations . . 8. Comparison of the Calculation Requirements. . 9. Several Space Variables . . . . . . . . 10. Alternating Direction Methodp . . . . . . 11. Abstract Stability Analysis . . . . . . . 12. The Energy Method . . . . . . . . . 13. Stefan Problem . . . . . . . . . . 14. Parabolic Systems . . . . . . . . . 15. 1nt)egro-Differential Equations . . . . . . IG. Extrapolation to the Limit, . . . . . . . Ileferenres . . . . . . . . . . .

.

.

.

.

.

.

.

.

. .

. .

. .

. .

. .

. .

. .

. .

1 2 4

. .

.

.

.

.

.

.

13

.

.

.

. . . . .

.

. .

. . . . . . . .

,

,

,

.

. . . .

. . ,

. .

,

.

,

,

.

.

,

,

.

.

.

. . . .

. . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . .

.

,

. .

. .

.

.

.

. . .

.

. .

. .

.

18 24 25

30 31

37 41 44 46 48 49 50 52

1. Introduction

The purpose of this survey is to introduce a theoretically minded, but not highly mathematically trained, scientist to finite difference methods for approximating the solutions of partial differential equations of parabolic type, Differential equations of this type describe diffusion processes of many kinds. In recent years much progress has been made in developing more efficient finite difference procedures and in methods for determining whether or not the numerical solutions are indeed good approximations to the solutions of the differential equations. Both of these advances will be discussed. Clearly, the better methods should be presented for the use of the applied scientist. Since the variety of physical problems leading to 1

JIM DOUGLAS, JR.

2

parabolic equations that cannot be solved by classical methods is much larger than could be anticipated by the mathematician, and since many difference analogues of the differential system are either inefficient or divergent, it is necessary that the practitioner understand at least the fundamentals of techniques for demonstrating the convergence of a numerical solution to the solution of the corresponding differential system. In order to make clear the salient features of each method of proof to be presented, the proofs will usually be given for the heat equation,

although the difference equations will be generalized to linear parabolic equations with variable coefficients and to nonlinear equations. At first the number of space variables will be limited to one. To illustrate the process of obtaining better difference equations for a, given problem, an orderly refinement of difference analogues will be derived. Also, to avoid technical complications, the detailed derivations will be presented only for the heat equation. The sequence of difference equations will progress from one for which the error is first order in the time increment and second order in the space increment and that is subject to restrictions between these increments to ones that are second order correct in the time increment, fourth order in the space increment, and independent of restrictions between the increments. The sequence will not be the same for one space variable and more than one space variable. Before beginning the discussion of difference equations, a few preliniinary definitions and facts about difference analogues of derivatives will be collected together. The new book of Forsythe and Wasow* contains a good introductory chapter on parabolic equations, although it appeared too late to be referenced in this survey.

2. Preliminaries

The solution of a difference analogue of a differential equation is defined only on a lattice of points in the region of interest. Let Ax, Ay, . . . , At be increments of the independent variables, and let xi = iAx, yj = jay, . . . , t, = nAt. Then, the lattice consists of the points (zi, yj, . . , t n ) . Let f(xi, yj, . . , t,) = fi,j.. . .,n. The subscripts are not necessarily integral; * Forsythe, G. E., and W. R. Wasow, Finite-LXfference Methods for Partial LXffer-

.

ential Equations. Wiley, New York, 1960.

.

NUMERICAL METHODS FOR PARABOLIC EQUATIONS

+

3

i.e., t,+ 4 = (n $) At. The values of the space increments could vary with the indices (x,+~= x, Ax,, etc.), but for the sake of simplicity let us assume that they do not; however, At is not assumed equal to Ax, although we shall frequently assume Ax = Ay = . . . . Throughout the paper the symbols u and w will bc used to denote the solution of a differential equation and the solution of a difference equation, respectively. Frequently, the symbol z will be used to denote u - w. The Landau order notation will be used. Thus,

+

means that as x tends to some stated or obvious limit. Similarly,

f(x)

= o(g(4)

means that

as x tends to its limit. I n particular, f(x) = o(1) implies that lirn f(x) = 0. Let the function f ( ~x2, , . . . ,x,) be defined on a closed region R. Then,

j CPLP2.. .. (2.5) i f f has pz contiiiuous derivatives with respect to 2,.Iff is 5t function of several variables, f E C@ (2.6) implies that all partial derivatives off of order not greater than /3 are continuous. If R is unbounded or not closed, we also assume these derivatives to be bounded. I n most of our applications, the highest order derivatives need only be bounded; however, this relaxation of the hypotheses will not be discussed. The following results are proved in any text on numerical analysis and most calculus texts. First,

JIM DOUGLAS, JR.

4

The s i arise from the use of the extended mean value theorem. Denote the centered first difference by Axfi

=

(fi+i

Also, d"(Xi) dx2

-

fi+l

- 2fi

+ + fi-1

As the second difference will appear very frequently, let' (2.10) (2.11)

(2.12) m d dx4

=

Az4ji

+ O((Ax)')),

where

+

- 4f~t-i 6fi

f E C6,

+

4fi-1 fi-2)/(AXI4* It follows from elementary trigonometric identities that AzYi

=

(fi+2

Ax2 sin rpx

=

-- 4

(Ax)

-

sin2

rpAx sin apx. 2

(2.13) (2.14)

(2.15)

3. Explicit Difference Equations

Let us begin with the numerical treatment of parabolic differential equations by considering the boundary value problem for the heat equation in one space variable, since this problem is the easiest example t o discuss for difference equations. Let 0 < x < 1, 0 < t I T, ut = U J Z , 0 < x < 1, u(x, 0) = f(x), (3.1) 0 C t 5 T, 4 0 , t) = go(t), 0 < t 5 T. ~ ( 1t,) = gi(t), Let Ax = M-I and At = TN-', where M and N are positive int,egers. Assume that a solution u(x, t ) exists for (3.1) and, moreover, that 1L

E 6412.

(3.2)


5

Assume that an approximate solution w is known on the lattice points up through time 1,; then a melhod must, tw spccified to advance the solution to time Clearly, the valucs o f w,,,+1 for 2 = 0 niid J: = I should be those assigned to 1 1 : (3.3)

At a point (xi,tn), 0 < i < M , the differential equation will be replaced by a difference equation. The simplest replacement is to approximate the time derivative by a forward difference and the space derivative by the centered second difference a t (xi,t,,). The resulting difference equation is

- W.n

Wi,n+1

At

=

Az2win,

i

=

1,

. . . , M - 1.

(3.4)

Equation (3.4) call be solved for W;.~+I :

+

i = 1, . . . ,M - 1. w,, AtAz2w,,, (3.5) As only five arithmetic operations are required to evaluate (3.5) for each choice of i, the approximate solution can easily be advanced a time step by (3.3) and (3.5). As the solution was prescribed for t = 0, the approximate solution can be obtained a t all (z,, tn), i = 0, . . . , M , n = 0, . . . ,N . Equation (3.5) is frequently called the forward difference equation. The question of the accuracy of the approximation arises immediately. There are several ways to study the relation between w and u. The most obvious way is to take examples for which closed form solutions of both the differential and the difference problems can be obtained and to compare these solutions directly. This procedure has been used by several authors [l-41; however, the method lacks the generality required to treat more complex linear and lionlinear problems. A particularly simple analysis [5-81 based on the concept of a maximum principle can be applied to analyzing the difference between u and w , and this analysis will be presented. For most of the more refined difference methods to be discussed below, more general analytical methods are required, and these techniques will be discussed later; nevertheless, the maximum principle analysis has broad application. Since w is defined a t a finite number of points, no differential equation can be found for w; thus, it is helpful to find a difference equation satisfied by Ztn = Ucn - Win. (3.6) As u E W2, (2.7) aiid (2.9) imply that i = 1 , . . . , M - 1. Ui,n+l = ?!in AtAZ2U,, O((At)' (Az)'At), w,,,+1

+

=

+

+

(3.7)

JIM DOUGLAS, JR.

6

Subtract,ing (3.5) from (3.7), we obtain Zi,rA+1

+

= z , ~ AfA,%,,

+ O((At)%+ (Az)%At),

i

=

1,.

. . , M - 1. (3.8)

As w agrees with

?A. initially Z;o

and

011

= 0,

thc boundary, i = 0 , . . . ,M , n = 0, . . . , N .

zO, = za1, = 0,

(3.9)

Let r

=

At (AX)'

Then, (3.8) can be written in the form

+

(3.10)

*-

+

= rzi+I,,, (1 - 2r)zi, 4-rzi-i,, o((At12 -I- (Az)'At). (3.11) Kote that the three coefficients on the right-hand side of (3.11) sum to one for any choice of r and all are nonnegative if (3.12) O < r l + . Assume (3.12) to hold; then Zi,n+l

THEOREM.Let u E C4s2, and let w be dejined b y (3.3) and (3.5). If 0 < r 5 $, then maxlu;, - win] 5 B ( ( A z ) * At), 0 I z i I 1, 0 I t, 5 T .

+

The value of B depends on upper bounds for u t land u,,,,, as well as on T. Thus, the forward difference equation satisfies the most important property required of a difference analogue of a differential equation; i.e., at least under some conditions its solution converges to that of the differential equation as A x and At tend to zero. It is clear that the condit,ion (3.2) can be relaxed to u E C2J. (3.17)

7


Then, by (2.7), (2.9), and the argument above,

llznll

= 41)

(3.18)

as Ax aiid At tend to zero. The condition on r , however, cannot be removed; it can be relaxed trivially to

(3.19) r 5 t -I- ()(At), as will be seen presently. The necessity of (3.19) will be discussed later. One of the most interesting features of the numerical treatment of psrtiul differential equations is that many of the numerical methods and many of the proofs for linear equations with constant coefficients carry over directly to nonlinear equations. Consider the differential equation Ut

=

#(x, t,

U, u,,

uZ2),

0

< z < 1,

0

< t _< T ,

(3.20)

again subject to initial and boundary values. To assure that this problem is well posed (i.e., physically stable) assume that (3.21)

Then a forward difference equation can be written as wiSn+I

=

Win

+ At4(Xi, tq

win, AzWinl A z 2 ~ i n ) ,

i

= 1,

- . . ,M

1. (3.22)

Now, assume also that (3.23)

in the region. Then, by the mean value theorem O((At)' -I- (Az)*At)i ~ i , , + 1 = in At+(Xi, t n , U i n , AzUinf Az'uin)

+

(3.24)

provided 'u E C"2. Then, subtracting (3.22) from (3.24) and applying t'he mean value theorem again, we obtain

+ 0((W2+ ( A x ) ~ A ~ ) , where the partial derivatives appearing on the right-hand side are evaluated ;It a point between (xi, t,, I L ; ~ A2u;,, , AZ2u;,) and (xi, t,, Win,&Win, &'Win) as required by the mean value theorem. Equation (3.25) can be rearranged as

a

JIM DOUGLAS, JR.

The analysis for the heat equation is applicable here provided that we can choose At and A x so that the coefficients are nonnegative. Now, (3.27) if

Az

5 2a/0.

(3.28)

Thus, (3.28) implies that the first and third coefficients are nonnegative. Also, (3.29) if

O
Ui,n+l

= in

that the forward difference equation was treated. Again, it is necessary to restrict T to be not greater than one-half, and it is easy to see that llZnlI

= O((A2)'

+ (At)').

(3.37)

Note that (AQ2 is dominated by ( A Z ) ~ thus, ; the error is of the same order as before. Consequently, no significant improvement was obtained. The trouble lies in that, while the local error in the time direction was improved, the local error in the space direction was not. As

the local space error can be improved by using the difference equation Wi,n+ 1 = Win

+ AtAZ'Win + [&(At)z - &(A~)'At]Az~t~in.

In this case, it can be shown that llZnll

=

+

O ( ( A Z ) ~ (A2)'At

+ (At)2)

(3.39) (3.40)

if T 5 3. The analysis above implies a lower limitation T 2 Q as well; however, a more refined argument shows that the restriction is unnecessary. Note that the error has now been reduced significantly and, fortuitously, the restriction on r has been eased slightly. Unfortunately, (3.39) cannot be applied to the bounded region problem unless the solution can be extended across the boundaries by symmetry considerations, since otherwise

JIM DOUGLAS, JR.

10

the formula could not be evaluat,ed a t the grid points next t o the boundary points. The discussion of the initial value problem has 80 far been oriented toward the derivation of difference analogues; let us turn our attention more toward the convergence issue. Fritz John [9] has given a very thorough treatment of convergence for explicit difference analogues of the linear parabolic equation (3.41)

where a&, 2) is positive and bounded away from zero. Equation (3.41) was generalized to the slightly nonlinear equation where a2(z,t)u d ( x , t ) is replaced by d(x, t, u).Professor John’s paper is without doubt the most outstanding contribution to the mathematical analysis of numerical methods for parabolic equations. His results are too extensive and many of his arguments too complex to present here; unfortunately, we shall have to be contented with a summary. The general explicit difference equation for (3.41) is of the form

+

Wi,n+1

- C ciniWi+jvn 3

+ Aldsn,

--m

0,

mini =

17

(3.54)

Cjffini= 0. 3

j

As these conditions hold for difference equation (3.22)for the linear differential equation (3.41)for sufficiently small Axl a convergence theorem similar to the one proved earlier follows from the general result, provided

are bounded. A direct appeal to the definition shows that the conditions

2 0, C cini 5 1 + O(At) cini

(3.56)

j

also imply stability. So far the existence and the required smoothness of the solution of the differentialequation have been assumed. John bases his proofs of the results stated above in part on certain a priori bounds for the solution of the difference equation. The norm of the solution is shown to be less than a multiple of the norms of the data f(x) and d(x, t ) , the magnitude of the multiplier depending on the smoothness of the coefficients of the differential equation. These a priori bounds are natural starting points for proving existence and uniqueness of the solution of the differential problem, as well as the convergence of the difference solution to this solution. The arguments are clear but long. A typical result is the following.

THEOREM. If a0 E al E: C1.0,a2 € C1lO, d E C1*O1f f C2and a0 is bounded away f r o m zero, then there exists a unique solution u E C2J of the initial value pToblem for (3.41). C2101

Generalized solutions exist under much less restrictive conditions. Fundamental solutions were also discussed. John [lo] has also relaxed the


13

requirement that the derivatives appearing in the discretization error tcrms bc bouiidcd as J tcnds to infinity. Rittermnii, a student of Professor John, htis trcatcd tlic boundary problem in a similar fashion, but, liis Lhesis 11I] is as yct unpublishcd. The author lias not seen it. The convergence proofs given above all depend 011 considerable snioothness of the solution of the differential equation; with this smoothness it has been possible to establish the rate (ie., exponents on Ax and At) at which the solution of the difference equation converges to that of the differential equation. John, in discussing generalized solutions, obtained convergence with much weaker smoothness conditions and with those on the data rather than on the solution, but he sacrificed establishing the rate of convergence. Juncosa and Young [2, 3,4] and Wasow [12] have studied the dependence of the rate on the smoothness of the initial condition f(x) for the simplified version of (3.1) when go(t) = g l ( t ) = 0, for several difference methods. As might be expected, the rate goes down as the restrictions on f(x) are removed. In particular, the effect of isolated discontinuities has been studied. See the papers cited for details. 4. The Backward Difference Equation

The necessity of imposing a restriction on r for the forward difference equation can be partially explained by considering the domains of dependence of the solutions of the difference and differential equations. For the bounded region the solution of the differential equation at the point ([, q ) depends on all the data for 1 = 0 and that portion on the boundary for which t < q. If an explicit difference equation is of the form

the solution at (xi, in) depends only on the data not above the lines t = 1,

+ min [5 mAx At (x - xi)

1 .

(4.2)

Thus, for the domain of dependence for the difference equation to converge to that of the differential equation it is necessary that At = AX) as Ax tends to zero. Obviously, this does not imply bounded r nor any particular choice of the bound on r ; however, it does indicate a way around the inconvenience of having to limit the time step excessively. If we wish to choose Ax and At independently, the difference equation should be chosen so that its domain of dependence is independent of Ax and At and coincides with that of the differential equation. An easy way to accomplish this coincidence is to approximate part or all of uzza t time In+1. The backward

JIM DOUGLAS, JR.

14

difference equation results from replacing all of uZza t the advanced time; i.e., Ar%Wi,n+l =

W,,,,+l

- qfl,n

(4.3)

At

corresponds to the heat equation. Now,

if u E: C4v2. The crucial step in the convergence argument for the forward difference equation was the obtaining of (3.15). Since the second difference is nonpositive at a relative maximum, it is easy to see for the boundary value problem that (3.15) holds for the backward difference equation [7, 81 without any restriction on r. Consequently, 11~~ll = O((Az>' 4- AL) (4.5) as Ax and At tend to zero independently. Thus, the desire to be able t o choose the increments independently has been realized; however, as the error is second order in Ax and only first order in At, it can be shown [8] that the optimum choice between Ax and A,? is st,illconstant T . The number of time steps required to reach the time T may be reduced somewhat in comparison to the forward equation by using a value of r larger than one-half. Notice that three unknown values of w appear in the difference equation ~ + ~ and a system of written z = 2,; hence, (4.3) defines ~ i , implicitly, linear algebraic equations must be solved a t each time step, Fortunately, this system is a special case of the following tridiagonal system blfl C l l 2 = rllt i = 2, . . , M - 1, Uifi--l+ 6ifi ctrts1 = q2, (4.6)

+

WfM-1

+

+

.

= VM.

6MSM

A normalized form of Gaussian elimination can be applied very efficiently to obtrainthe solution of (4.6). The algorithm is the following: P1 = bl, (4.7) ccl Pi = bi - a9 i = 2 , ..., M , Pi-1

-,

y1 = rll PI

i

= 2,.

., (4.9)

fM = Y M , fi =

yi

- &, Pi

i=M-1,

. . . , 1.


15

If the coefficient matrix does not change from time step to time step, needs to be computed only once. I n the case of the heat equation only four arithmetic operations per grid point are necessary to evaluate Z O ~ , ~ + ~ , after pi is obtained. Note that this is one less than that required to evaluate the forward difference formula. It has been shown [13] that this method of solving the linear equations does not introduce large errors in the solution due to round-off errors in the calculation. Cuthill and Varga [14] have proposed a different algorithm which is slightly more efficient for rclf-adjoint equations with time independent coefficients; however, if the coefficients vary with time, the calculation required is greater than that required by the above algorithm. The round-off error results have been extended to the algorithm of Cuthill and Varga for the self-adjoint case by Blair [15]. Let us consider other boundary conditions than assigning the value of the solution. The conditions (4.10)

or all

4)

+ D(0u = y(t)

(4.1 1)

occur very frequently in physical problems. I n (4.11) the sign of a(l)b(t> must be such that the physical problem is stable. Rose [l6] has discussed (4.11) in connection with a Crank-Nicolson type difference equation (to be discussed in Section 5) for a nonlinear parabolic equation of the form (3.20) under the assumption that b(t) is bounded away from zero. If his argument is restricted to the backward difference equation, he shows by means of a sequence of comparisons that, if u z is replaced by a one-sided difference, the truncation error is O(Ax At) for sufficiently smooth u. Kote that the rate of convergence has been reduced. The reduction results from low order correctness in the replacement of the boundary condition. Lotkin [lT] has obtained related results for a more general problem. Similar sets of conclusions can be drawn for boundary condition (4.10) and for certain nonlinear generalizations of (4.10) and (4.11). A second order replacement for u,(O, t ) is

+

(4.12) 13attcw [I81 has showii very rccciitly that the use of (4.12) in eithicr (4.10) or (4.11) leads to an over-all error of O((Ax)2 at) for the backward difference equation. It is not necessary that b(t) # 0. Consider next the backward difference equation for the nonliiiear equa-

+

JIM DOUGLAS, JR.

16

tion (3.20). If (3.21) holds, then the implicit relation can be solved for u.~. Assume the differential equation to be in the form (4.13) ~ z = z $4, t, u, uz, U t ) , where (4.14) The backward difference equation becomes A z z w i , n + l = $(xi, L+l, W i , n + l , A z W i , n + l , ( W i , n + l - ~ i r J / A t ) . (4.15) The convergence of the solution of (4.15) to that of (4.13) for the boundary value problem is proved by a similar argument to the one for the heat equation [S, 191. The error is AX)^ A t ) , and no restrictions on r arise. The algebraic problem may have become quite complicated, since the algebraic equations are no longer linear. However, an iterative method is easily devised. Let W t( 0, n) + l = Win, (4.16) and determine successive approximations to wi++1 by the linear, tridiagonal equations AZ2w$",'+'{ - Aw@fl) t.n+l = -Aw:s+l (4.17) $ ( x i , tn+1, ~ I : + l r AzW8!+1, (wt(%+~ - win)/At), where A is a positive constant. The optimum choice of A is

+

+

(4.18)

If (4.19) and At is sufficiently small, then W ~ Z converges +~ to the solution of (4.15). Note that (4.19) is needed only in order to demonstrate the usefulness of the iteration and is not needed for convergence of the solution of the difference equation to that of the differential equation. If (4.13) can be written in the quasi-linear form uzz a(x, t, u ) ? ~ , b(x, t, u) = c(x, t , u h t , (4.20) then a modification of (4.15) may be obtained for which the algebraic problem is linear a t each time step. Let At = AX)

+

Az2wi,n+i

+

4- a(xi, tn+i, Zuin)AsWi,n+l -I-b(xi, t n t i r Win) = c(-ci,

tn+l,

in)

Wi,n+I

- Win.

(4.21)

Al As wi,n+ 1 appears only linearly, the desired modification has been produced. The convergence proof remains valid [S]. A similar situation arises where the differential equation is in the self-adjoint form


17

(4.22)

It can be shown that

is a second order replacement of (au,), for sufficiently smooth a and u. An obvious alteration of (4.21) using (4.23) gives a backward difference equation for (4.22) for which the algebraic equations are linear. The same convergence proof holds. Of course, (4.22) is a special case of (4.20), but it is frequently more convenient to leave it unaltered in form. The backward difference equation can be applied ‘readily to heat conduction problems in one space variable with adjoining regions of different properties. As an example, consider the problem (4.24) Assume that both the temperature and the heat flow are continuous a t the transition. Then, lim u ( x , t ) = lim u ( x , t ) , z

X t z *

4 x*

(4.25) au au lim PI - ( x , t ) = lim pz - ( 2 , t ) . z.tz* ax z&z* ax In addition, assume that the temperature is specified initially and along the ends of the material. For convenience let x* = X E = kAx. Then, the problem reduces to finding an analogue of (4.25), and a simple one is Wk--l,n+l 81 Wk,n+l Ax

=

82 W k + l , n + l - W k d - I . Ax

(4.26)

Notice that (4.26) is a tridiagonal linear equation; thus, the algebraic problem is changed only trivially. It can be shown that the difference system incorporating (4.26) is convergent with an error that is O(Ax -I- At). Lotkin [171 has discussed generalizations and improvements of this procedure. The time step has been assumed to be independent of the time in all the arguments given above. Frequently, the solution of a parabolic differential equation tends to smooth out as time progresses, and it would be desirable to take advantage of this knowledge to reduce the computational effort for such problems. Now, it is usually inconvenient to change Ax in the midst of a calculation, but At may be altered easily. Gallie and the author [20] have shown that At may be increased following the relation

18

JIM DOUGLAS, JR.

for the backward difference equation for the heat equation without a reduction in the rate of convergence, provided the derivatives appearing in the truncation error terms decay exponentially with time. The result also holds for certain analogues of the heat equation in several space variables [all, and the ideas of the proof extend to any of the other difference equations discussed in this paper for linear parabolic equations. The relation (4.27) has also been applied successfully many times to nonlinear equations by the author and his colleagues.

5. The Crank-Nicolron Difference Equation

The global truncation error for each of the difference equations treated so far is of the same order as the local error, provided the stability restrictions are satisfied. It would seem reasonable to hope that increasing the local accuracy would lead to the same increase in the global accuracy, and such is often the case. Let us begin by decreasing the local error in the time direction by deriving the Crank-Nicolson difference equation [22, 231. The concepts involved are credited to von Neumann in both references. It is frequently convenient to limit the number of time levels appearing in a difference analogue to two, although multilevel equations are both possible and useful. If this limitation is admitted, then it is natural to + I Win)/At. Now, this difference is first order correct replace ut by ( w ~ , ~ at any point (xi, t) for tn 2 t 2 tn+l. At the particular choice t = tn+h it is bounded. In order to becomes centered and is second order correct if utCt take advantage of this increase in accuracy it is necessary to replace u,,at (xi, tn+i). Assume that u € Ca.This implies that u,,,, = u f rand uzZCC = utlf are bounded. Then,


19

Rearranging, Ui,n+l

- Uin

At

- 1

- 2 Az2(Ui,n+l

+

Uin)

+ O((&)' + (At)').

(5.2)

Consequently, let us define the Crank-Nicolson equation to be (5.3)

Flatt [24] has shown that stability in the uniform sense does not hold for r > R, where R depends on the length of the rod. For a rod of length one, R = 4 - W2.Another method of analysis must be introduced to treat the convergence of the solution of the Crank-Nicolson equation. This procedure will be based on a combination of Duhamel's principle and harmonic analysis. Let us consider the boundary value problem. The difference equation for the error is i = l ,. . .

(5.5)

where (a)

Zink

= 0,

(b)

&+I

=

WL'z:t+i 4-Ateik,

n

5 k,

n

=

k

+ 1,

(5.6)

It is easy to see by direct substitution that (5.5) and (5.6) provide the solution of (5.4) ; thus, the analysis of the error is reduced to estimating t,he solution of the homogeneous initial value problem, starting from the initial data z 2 k + 1 . First, note that (5.6b) can be rewritten in the form

which is the backward difference equation with z i k k replaced by eikAt and At by At/2, By the argument for the backward equation it follows that

max[z:t+l[ 5 At maxleikl = O((Az)'AClt i

i

+ (At)".

(5.8)

20

JIM DOUGLAS, JR.

Now, consider the auxiliary problem

vio

(5.9)

arbitrary,

v ~ , n + l = oM.n+l

= 0.

Equation (5.9) is a linear homogeneous difference equation with constant coefficients subject to homogeneous boundary conditions. The analogous differential problem is the standard textbook example for the method of separation of variables; consequently, it seems natural for us to try to separate variables here by assuming a solution of the form (5.10) (5.11) Thus, the following eigenvalue problem arises:

,M 40

=

4ild

=

- 1,

(5.12)

0.

It follows from (2.15) that the eigenfunctions are p = 1, . . . , M &P = sin ?rpxi,

- 1,

(5.13)

and that the corresponding eigenvalues are 1 pp

= 1

- 2r sin2rpAx/2.

+ 2r sin*?rp~x/2

(5.14)

The natural topology for discussing eigenfunction expansions for differential equations is the Lz topology, and such is also the case here. Define the inner product of two vectors defined on the points xi, i = 0, . . , M , as

.

M

(u,V) =

C i-0

u~v~*Ax,

where vi* indicates the complex conjugate of jlvllz =

vi.

(v, w2

(5.15)

Let (5.16)

denote the corresponding norm. The coefficient Ax is introduced to maintain uniformity in estimates ae M tends to infinity. The standard facts of finite dimensional vector spaces will be assumed [25]. I n particular, the set of eigenvectors of a symmetric operator form a basis for the space, and eigenvectors corresponding t o distinct eigenvalues are orthogonal. Hence,


21

sin n-px and sin r q x are orthogonal for p # q, provided that p , q - 1. Actually, it is easy to see [26] that p , q = 1, . . . , M - 1. (sin q x ,sin m p ) = $Sp,,

=

1,

. ..,

M

Let us apply these results to (5.9). Denote the vector Then,

Vik

by

(5.17)

vk.

c cpcp',

(5.18)

2(VO, P).

(5.19)

M-1 2'0

=

p=l

where cp

=

As the solution of the recursion &L=

PO =

PPl

(5.20)

CP,

Pn

is (5.21)

it follows that. (5.22)

11vn112

As

IpPl

< 1 for any choice of

=

3

c

(5.23)

M-1

ICPI'PP2n.

p=l

T, IIunJIz

5

(5.24)

II~'o112.

A diflerence equation for which (5.24) holds for the homogeneous error equation is said to be stable in the Lz norm. Note that this is merely a change of norm from stability in the uniform sense. The general relationship between stability and convergence will be discussed later. Let us show here that the above stability implies tJheconvergence of the solution of the Crank-Nicolson equation. For 2'0 = z:+I, (5.24) implies that IIZnkllz

I II~:+lilz,

+ 1.

rL 2

(5.25)

Relation (5.8) and the definition of the L, norm imply that llZ:+i][2

=

O((Az)'AL

+ (At)').

(5.26)

Thus,

5

I I Z ~ I I ~ 'E111Znkl12 =

k-0

~o((Aw~

+

= ~((AX)'

+ (at)')

(5.27)

22

JIM DOUGLAS, JR.

as nAt I T. Thus, as Ax and At tend to zero, the error tends to zero in the grid LZnorm. Of course, this is not the usual integral L2topology; however, it can be shown [27] using interpolation to define the solution over the rectangle (0 5 x 5 1,0 5 t 5 T) that the integral Lz norm of the error is also O ( ( A X ) ~ ( A t ) z ) . Note that we have preserved the local accuracy in the global error. The above argument is a special case of this method of the author [27]. It will be shown later that the optimal relation between A,? and Ax for the Crank-Nicolson equation is that the ratio At/Ax be constant. Consider the algebraic problem associated with the evaluation of the solution of the Crank-Nicolson equation. No essential difference between the linear equations in this case and in the backward difference equation case arises. Only the right-hand side is affected; consequently, the same elimination method may be applied. Indeed, a considerable increase in accuracy has been obtained for a very small increase in computing. The comparative aspect of the computing requirements will be discussed later. Strang [28] has also treated the above convergence problem by different methods. His proof is based on matrix manipulations requiring explicit knowledge of the eigenfunctions of the difference operator. Although these eigenfunctions were used above, it will be seen in Section 11 that exhibiting thcm is not necessary for the genera1 method used. As mentioned earlier, Juncosa and Young [2,3,4] and Wasow [12] have studied this convergence problem using the explicit solutions of both the difference and the differential equations. The Crank-Nicolson method may be easily formulated for problems of a more general nature. For the differential equation (4.13) the difference equation becomes

+

a A z 2 ( W , n + l +W,n)

= $(xi, t n + : , i ( W i . n + l

+

Win), 4 A z ( W i , n + l

+

Wsn),

(Wi,n+l - w i n ) / A t ) . (5.28) The convergence of the solution of (5.28) was first proved by Rose [IS] under a restriction on T . More recently Lees has constructed a proof based on energy estimates [19,29], and the restriction on T has been removed. The algebraic problem associated with (5.28) is essentially the same as that for (4.15), and an iteration method analogous to (4.17) can be written down. For the somewhat simpler differential equation (4.20) the difference equation simplifies to %Az2(w;,n+~ Ww) ia(ri, f n + t , i ( w i , n + l Win)) ALr(lDi.n+l ~ , n ) (5.29) [)(xi, t n + i , i ( w i , n + l win))

+

+

+

+ +

+


23

'I'hc soliitioii of the slgchraic proldcm may hr ol,tninctl hy itcmtiiig on w,, I 1, using thr old itrratc in tlic r1v:iliint ion of t hc corfficienth. If an arciirule cstimatc of thc solutioii at timc l n + : could ho provided, it, \vould 1)c unnecessary to iteratc a t all. The author [30] proposcd to usc the forward difference equation to predict wn++ from w,; however, the proof offered was incornpletc. Perhaps a bctter method would be to use the backward difference equation (4.21) for a time step of At/3 to predict wn+;, and then modify (4.29) to

+

%Az'(~,n+l

ujm)

+

%,(,it

k+:,

Wt,n+J)Az(wz,n+l

+ b(.~7,

tn+;t

=

c(.rt,

+

WZ")

w,,n+;)

W,n tn+;, ~ 7 , n + i )

(5.30) 1

- Wtn.

A convergence proof has been roiistructed in case the differential equation is of the almost linear form ( u ( q t)u,),

+ b(r,t , u )

= c(s, t)W.

(5.31)

Another method for predicting wn+i would be to extrapolate from w, and w,,-~. Whether this can be done without iteration is not known. So far, only the advantages of the Crank-Nicolson equation have been pointed out. A disadvantage of the Crank-Nicolson method with respect to the preceding methods is that greater smoothness is required of the solution of the differential equation to insiirc convergence. For some particular examples the solution of the backward difference equation may actually be better than that of thc Crank-Kicolson equation; however, if the differential problem has a sufficiently smooth solution, it is to he expected that the Crank-Kicolson equation will produce superior accuracy. Crank-Nieolson equations can, of coursr, be applied to problems for which slope conditions are specificd a t a boundary rathcr than the values of the solution; however, the manner of replacing the normal derivative is not analogous to that for u,,in the differential equation. The values of thc solution a t only the advanced time should appear; no avcraging to obtain a timcwise centcred replacement should he atternptcd. To see this, considcr the following example. Let 14(5,0)

= 0,

o> n, that as n 3 a, this "discrete" process is also a convergent one. I n problems where the contours are composed of line segments or some such simple curves, and f is simple, our process (13.4) may actually coincide, up to roundoff, with (13.3). In most problems we select m = 2n or 3n. The case m = n (which also can be run via orthonormalizing codes) is the caEe of straight interpolation (13.2). It is felt that the value of 2 or 3 for m/n provides enough "slack" in the system to forestall any divergencies of interpolatory type. I n Section 18, case XV, we report some numerical experience with collocation which shows such divergencies. Three boundary value problems were solved by collocation over an H domain. We took boundary

81

ADVANCES IN ORTHONORMALIZING COMPUTATION

values which came from (1) an entire harmonic function, (2) a, harmonic function possessing a singularity outside but near the boundary of the domain, (3) boundary values arising from the torsion problem (+(z2 y') ). Results in (1) were comparable to the least square solution. Those in (2) were less accurate, while those in (3) were bad, with errors exceeding 100 times those of the least square solutions. These results can be explained by arguing by analogy to the case of equidistant interpolation to functions of one variable. If the interpolated function is regular in a sufficiently large portion of the complex z plane, the interpolation process is demonstrably convergent. If it is not sufficiently regular or is not analytic a t all, the interpolation process may be divergent.

+

14. Conformal Mapping

Conformal maps are obtainable from complex orthogonal functions (see Bergman [14.2], Bochner [14.3], Carleman [14.5], Szego [14.1]). Szego's work employs a line integral as inner product and the other authors stress the area integral. Let B be a simply connected domain of area S possessing a piecewise smooth boundary b of length 1. Let zo be an interior point of B and let w = m(z, zo) designate the function which maps B conformally onto the circle IwJ< R and such that w(zg, zo) = 0 and w'(zg,20) = 1. Let p,(z) = k , p . . . , k, > 0 , be a set of complex polynomials such that

+

/

-

p,(z) p,(z) ds

= 6,,,

m, n = 0, 1, 2, . . .

(14.1)

Then, (14.2) where

(14.3) of the circle onto which z is mapped is given by 1 1 r=---* (14.4) 2?rK(z0,zo) The function K(zolz ) is sometimes known as Szego's kernel junction for the domain B. If, on the other hand, the polynomials p,(z) are orthogonal in the following sense : The radius

T

a2

PHILIP J. DAVIS AND PHILIP RABINOWITZ

then, (14.6)

and (14.7)

where

KB(ZO, 2)

=

2 P,(z)

n=O

pnO.

(14.8)

The function KB(z0,z) is known as the Bergman kernel function for the domain B. For monographs and text books which expand and develop these formulas see Szego [5.2], Bergman [3.1], Nehari [11.7], pp. 239-263, Rehnke and Sommer [14.8], Chapter 111, $12, Kantorovich and Krylov [10.8], pp. 381-389, and Walsh [14.10], pp. 111-151. For multiplyconnected domains, see Bergman [3.1], Chapter VI, and Nehari [11.7], Chapter VII. For an application of complex orthogonal polynomials to a 2-dimensional Poisson equation, see Kliot-Dashinskiy [14.71. Nehari [14.6] has given an estimate of the truncation error in the computation of the Szego kernel function of a plane domain with an analytic boundary 6. Let u,(z) be analytic functions in the domain which are complete and orthonormal over C. Then K(z, f ) = C:-ou,(z)u,(r>. Nehari shows that IK(Z,~)

- j2 ui(z> =l

5 sn(z)sn(l)

(14.9)

where

a,(t) =

1%@

2T c z - t

ds.

The functions s,(z) +0 as n t 00, and the estimate may be carried out after the orthogonal functions have been determined. A similar estimate for the area integral orthogonalieation (corresponding to (14.5)) does not seem to be available. Exterior maps are also related to orthogonal polynomials. Let z = m(w) = cw co (cl/w) (cz/w2) . . , ,c > 0, map the exterior of the unit circle IwI = 1 conformally onto the exterior of B. Let w = +(z) be the inverse function (mapping the exterior of B onto the exterior of [wl= 1). Then, with the normalization (14.1) we have

+ +

+

+

(14.11)


83

uniformly in the cstcrior of l?. The lending rocfficicnts k , posscss the following asympt otic hhnvior : k,tl liin __ = 1,'~. (14.12) n 4 m li,, The coiistaiit c coincides with the transfinite diametcr of B, a concept which was introduced by Yeketc [14.4] and which has been given very simple and clegaiit gconictricnl definitions for very geiierd point sets. This concept is closely rclatcd to that of the electrostatic capacity of a region, referred to rrpeatedly in the bibliography of Section 15. The followiilg theorem of Fekete aiid Wnlsh [14.9] relates the leading coefficicnt of complex orthogonal polyrioniials with the transfinite diameter: Let I!: consist of a finite number of rcctifiable Jordan arcs and let p,(z) = k , ~ . . . , k,, > 0 he complex orthonormsl in the sense that ,

+

(14.13) lini 71-+*

(i)"" =

trniisfiiiitc di:tinetcr of 11;.

(14.14)

15. Quadratic Functionals Related to Boundary Value Problems This is an area in which orthonormalizing techniques are potentially of great nunicrical value, but as the authors have had little computational experience, the topic will be passed over with a few general remarks plus an extensive bibliography. A typical problem here is the following. Tat u(r, y) designate the solution of the Xeumann problem A I L= 0 i n 13 (15.1) = j on as, dn

Compute the value of the Dirichlet integral (15.2)

+

In the particular case wheiif = (d/ds)$(xz y'), D is intimately related t o the torsional rigidity or the stiffness of the plane domain B. iLIuch work has bcen done to provide upper aiid lower bounds for I). Two inequalities valid in inner product spaces are pertinent to this work. The first is the familiar Bessel inequality (15.3)


a4

where ( r , ]is orthonormal and y is arbitrary. The second is a modification of this. Let y he arbitrary and z bc selected so that ( z , y) = (y, y). Let n linearly independent vectors w;be selected so that (y, w,)= 0, i = 1, 2, . . . , n. Then we have,

c a&, 7I

(Y,Y)

I (z,z> -

a=1

(15.4)

Wl)

where thc ronstaiits a; solve the linear systrm of equations

2 a,(w,, w,)=

(2, W I ) ,

j = 1,2, . .

. , n.

(15.5)

2=1

Lower hounds to (y, y) arc provided by (15.3) while upper bounds are given by (15.5). Int.roduce the Dirichlet inner product (15.6)

Tlicii

u = (11, ti,)

(15.7)

whrre u solves the problcin (1 5.1). Now Green’s identity leads to (1 5.8)

If in (15.3) we select y = u and xi = ui, where ui is any set of functions which are ort,honormal with respect t,o (15.6), then, we have (15.9)

Under appropriate completeness conditions (15.9) bccoincs an equality when n = 00. If z(r, g) = f c, on the p separate contours of dB (c, constant, p = 1, 2 , . . . , q), then (2, u) = (u, u).If u1 = 0 on dB, then (u, u,) = 0. With z and u, thus selected we may insert, y = u,z = z, wz = u,in (15.4) and obtain upper estimates for B. These matters are explained fully by Diaz [15.30, 15.491. Numerical examples have been worked out by Synge [15.45]. Though much effort has been spent on the idea of separate upper and lower bounds, and though it is of great theoretical interest, some of its devotees lose sight of one numerical fact. To obtain separate upper and lower hounds which are very good requires an increasing amount of computation, the accuracy of which cannot be given. The computed upper and lower bounds are therefore only approximate, and one might do well to confine one’s numerical work to a single convergent scheme such as (15.9) with n = 00.

+


85

PART II: NUMERICAL

16. Orthogonalization Codes and Computations

There are several papers in the literature describing routines for genelating orthogonal polynomials by means of a 3-term recurrence relation. Barker [6.12] gives a fairly complete treatnieiit of the subject. He describes a routine using equations (5.6) to (5.9) which was written for the NORC, a floating point computer, and which has the following properties. The input consists of N pairs of data-point coordinates ( L , y,), N associated weights wt > 0, and a parametcr m, the degree of the least-square approximating polynomial fm(z) desired. The output consists of as much of the following as is desired : (a) P*,,(zJ,i = 1, . , . , A;; 1 1 = 0, 1, . . . , m (b) (P*,, P*,) = 6,,, i, j = 0, 1, . . . , m. Orthonormality check. (c) The coefficients u , , ~of the polynomial expansion of P*,(J) = C;=oa,,,~.],j = 0, . . . , z ’ ; v = 0, . . . , m (d) fm(z,), i = 1, . . . , N The Fourier coefficients O,, L’ = 0, 1 , . . . , rn;fm(.c) = Z‘=O b,l-’*,(c) The residuals e(z,) = yyz- f m ( x , ) ,i = 1 , . . . , N bv2,v = 0, . . . , m Sums of squares of residuals eVTwe,= yTwy e,(.cJ = yz - Z = O b,P*,(z,). e, = { e u ( x j ) ] = (yLb,Z’*j(c,): y = (yt>, w = diagonal matrix [w,]. Student’s t, = d N - v - l/e,Twe,, = 0 , 1 , . . . ,m Coefficients d, of the polynomial expansion of fm(z) CLOdux”, ti = 0, 1, . . . , m Check 011 the above coefficients by evaluating f,n(n(zl) from the polynomial expansion and comparing with results of (d) P:’(zJ, I’;’’(zJ, i = 1, . . . , N ; v = 0, 1, . . . , m bJ’;’(z,), i = 1, . . . , N CLo b*Pf‘(z,),i = 1, . . . , N I n addition, Barker gives 2 sets of tables for the case of equally spaced points. The first set gives the coefficients in the polynomial expansion of Z’g,N(z), the polynomial of degree k orthoiiormal over the set of N equally spaced points in the interval [ - 1, I], for selected values of k and N as well as the values of these polynomials at these points. The second set gives coefficients which cnable oiie to calculate the variance. a fixed-point computer, Ascher and Forsythe [5.7], working with SWAC, describe two routines which use floating vectors. The first routine generates LJ

xr=o

-


86

a set of polynomials Pk(x) orthogoiial (not orthoiiormal) over a given set of N points 2 , using the recurrence equations

Pk+l(x) ffh+l

+

=

hX(z -

=

(xpk, Pk)/(Pk]I>/,), p/, ==

ffk+l)Pk(X)

flLI’h-1(2),

PO

=

1,

PO

hh(Pk1 PL)/hk-l(pk-lf

=

0 hL = 1, PL-1).

It then computes the Fourier coefficients b k of a given set of values yi, b k = (y, P k ) / ( P k , PA). The f f k , f l k , and b k are printed out together with a k 2 = ( N - k - l)-l(ehl ek). Knowledge of these quantities is sufficient to determinc the polynomial approximation to y and hence Ascher and Forsythe do not concern themselves with finding the coefficients de of the least square polynomial fnL(s). Instead, their second routine computes f m ( x I ) for a given set of values x,, which may include the original x,, given the quantities ahl o h , and b h . Clerishaw [16.1] is concerned with those computers which have a small amount of high speed storage. He has written a routine for the DEUCE which generates by recurrence the coefficients of j ’ h ( X ) in a Chebyshev series. Here the P L (x )are normalized differently by taking Po = 3 and h k = 2. If P k ( Z ) = ; f ) o , k f 711,kT1(2) . . . -k l J k - - l , k T k - i ( L ) Tk(Z), then P j , k + l = 7 ) j t l . L f 1)1j-i1 .h - 2ah+lpj,L - P k r ) j , l - l . Thus OlllY the P , - l , k and P l , k need be stored instead of the values of P L ~ ( Xand ) I ’ k ( s ) which are ge:icrated when riccded to compute f f h arid PA. The output consists of the cocfficicnts of the least square approximation polynomials given as a Chebyshev series and the largest positive and numerically largest negative residuals. Efroymson [1G.2] has written up a program for solving least square problems on a computer using normal equations. Given a set of vectors x, and a vector y, this program expands y as a linear combination of only those vectors x k on which y is really dependent. There are many other write-ups of least square routines on electronic computers of which we mention only that of Sciama [lG.3]. The first paper in the literature on orthonormalizing a set of vectors 011 a computer was that of Davis and Rabinowitz [1.1]. Davis [1.3] brought the subject up to date. The latest publication of the “National Bureau of Standards School” in this subject is a write-up by P. J. Walsh and 13. V. Haynsworth dated November 10, 1959 of a routine Bs ZRTH for use on the IBM 704 submitted to SHARE. The features of this routine are as follows: (a) The generalizcd definition of inner product is used.

+

+

is a givcii vector arid W = w,is a real, positive definitc synimetric matrix of weights, generally diagonal arid often the identity matrix.

87


(11) ‘J’11e Gram-Schmidt process i u rccwrsivc form is iibetl. Givcii :L set, .fL,f?, . . . , j,L t h c . objcvt, of thc (:rani-

of linearly iiidcpwitlcnt vectors

Schmidt process is to produce a set of orthonormal vrctors which are linear combinations of the vectors j , , 41 = U l l S I

42

= anlfl

6,= a,,f,

+$

=

. . . z,,,)

( ~ ~ 1 ,

+ @?.h +

a,,&

+ . . . + a,,,,f,z.

Then in terms of thc orthonormal functions &, . . . , & a given vector

where d, follows :

=

41 = f

Cr=l(y, 4k)ukj.The orthonormal vectors D1 =

1 / h

+i

are computed as

(fl,fl)”2

i = 2 , ...,12. (c) Matrix operations are used. The central and recurrent feature of thc orthonormalization and expansion scheme is the construction of a vector of the form

- . . . - (g, +A)&, k = 1, . . . , n where g is one of the vcctors . . . ,f,,,yl, . . . , yf2. g* = 9 - (g,41)41

?: )

We have

Hence, if we designate the N X k mntris d2,. . . , &) tiy aPk, we have q* = g(I - N’% @kT). In addition wc need (g*, g*) = ( g * T ) ( W ) ( g * ) . (d) A “straightening out” of the orthonormal vectors is used. Let us suppose we have a system of k vectors (PI, . . . , +M, C& such that (4,) &) = 6,,


00

. .

( z , ~= I ,

. . . , 1;

-

I ) ; (&, &)

=

1 h i t ($/, 4,)

=

~ , , j= 1 , 2, .

. ,k

- 1.

'I'hr L l 1-1

4/L = 4)--

c

J=1

6-1 (61j

4J)41 =

B / b

-

c '.f4,

3=1

is much closer to orthogonality. (e) General augmented inputs arc possible. With an input of the vectorsf, and y3, the following quantities can be computed : 1. The orthonormal vectors 4t 2. The residual vectors e, 3. The sums of the squares of the residuals E,r 4. The standard deviation c1k = ((el.+,e,k)/N - I c ) ' ' ~ 5 . The normalized Gram dctcrminant G* 6. The Fourier coefficients (y,, &).

=

DID2 . . . D,,)'

Ilfll?~

*

*

llfnll*

To get the matrix A = (G,)and the coefficients dJ and related quantities such as the covariance matrix C = ATA and the variance-covariancc matrix V = u2C, we augment each vector fa by the n-component vector whose j t h component is 6,, and each vector y3 by the n-component zero vector. If we then apply the Gram-Schmidt algorithm to these augmented vectors with the provision that inner products are taken as before only over the N original components, then the vectors a,, are replaced by the columns of A and the zero vectors by the coefficients -d,. This process can be generalized as follows: The input vectors f z form an n X N matrix # which wc augment by an n X p matrix T = ( t l l ) . Then Il.A = ip where @WaT= I , and a t the same time T A = E. If T = LIl. for some p X N matrix I,, then E = La. If p = N = n, E = T$-*@,$-* = T-'EWaT. I n particular if T = I , W = I , then E = A and #-I = A@'". This gcneralized augmentation is useful in many cases where one approximates data by, my, a polynomial. What one is really interested in is some further quantity obtained from the polynomial by a linear process, perhaps an integral or derivative at specified points. If L designates a linear operator and f a function, this reflects the working rulc : Approximation to L(f) = L(approximation to f). Now L(approximatioi1 to f ) may be computed by augmenting each vector f z by the vector L(f,) and yl by a zero vector. Then this zero vector will be replaced by

Further work a t the National Bureau of Standards has consisted of the writing of routines to gciicrate particular solutions of the harmonic and


a9

biharnionic eyuat ions, orthonormalizing them by Gram-Schmidt using a double-precision floating routine, and also routines to orthonormalize a :et of complex vectors for the purpose of computing the exterior mapping function and transfinite diameter of planar domains. Finally, Rutishauser [16.4] has writtcn an orthonormalization routine using ALGOL58. I n this routine he writrs CP = #RN arid computes @ and RN,given J.. He then computes the Fourier coefficients b , for a given vector y. The coefficients d, in the expansion of y terms of the original vectors f t are then computed by solving the system RYD = B by back substitution.

17. Numerical Experiments in the Solution of Boundary Value Problems Using the Method of Orthonormalized Particular Solutions

The preseiit experiments were carried out on an IBM 704. 1:or couvenicnce, we shall elaborate the meaning of several terms which occur in the tabulation of the data. Equation: The partial differential equation solved. That is, either A u = 0 in 2 dimensions, A u = 0 in 3 dimensions, or AAu = 0 in 2 dimensions. Domain: The portion of space over which n solution is sought. Points: (xl,y,) or (xl,yl, z,), i = 1, . . . , N . Selected points on the boundary of the domain. Somet inies the points and corresponding weights were chosen accordiiig to a high order quadrature rule 011 the houndary. At other times points were chosen to be equidistant in one or more coordinates along the boundary. I n some instances the points were chosen so as to exhibit the characteristic features of the domain. IVeights: ur, : Weights were chosen either corresponding to a quadrature rule, were taken t o be rqual, or, i n thr 2 dimensional case, wrrc computed by the forniu1:i 20,

=

a[((&

- X+l)?

+ (UL-

&L)pl

+

- Z,)Y

+ (g,+1 - g,)9’2],

that is, they are equal to the average of adjacent chords. Special Functions: ul, . . . , u,,are the particular solutions of the differential equation, selected from among a complete set. Boundary Values: A function f defined on the boundary points P,.We designate by max f and min f the maximum and minimum values of f o n I>,. This information is included to indicate the range of the boundary \.nlucs.

90


Root Mean Square Boundary Value: V n : This is defined by

arid is the weighted average of the given boundary values. This is not the usual definition of the root mean square. It was used for ease of computation. A similar definition occurs below with En and the relative error ER/Vn is the same as it would be with the conventional defintion. The Boundary Value Problem: Laplace Equation: This can be written in the following way. Let a(P,) and b(P,) be two functions defined on Pi. We wish to determine that linear combination u* = aiui for which

fits f a t Pi in the sense of least squares. The Dirichlet problem is the case a = 1, 2, = 0. The Neutnann problem is thc case a = 0, b = 1. For general a, b we have the mixcd or Robin’s problem. Biharmonic Equation: No mixcd boundary conditions were considered. We wish to determine that linear combinatioii u* = x?=a1m for which u*(PJmatches an fi and au*(Pi)/anmatches an fi a t the points Pi,simultaneously, in the sense of least squares. What i s Minimized: Laplace Equation:

Biharmonic Equation:

Theoretical Value at Isolated Interior Points: u ( P ) , available only when exact solution is known in simplc closed form. In the present work, this occurs only when f has been computcd from a simple, closed form solutioii given in advance. Computed Value at Isolated Interior Point: u*(P), where P is the giveu poiiit. Maximum Error an Boundary: E,,,,, Laplace Equation:


91

Hiharmonic ICquation:

Relaiive Root Mean Square Error: If8/Vli The average error relative to the average boundary value. Exact Error at Isolated Interior Point: E ( P ) The absolute value of the difference between the theoretical value u(P) and the computed value u*(P) a t the given point P. Case I

Equatio I I :

Au = 0.

Domaiii :

Cube. -1 I z, v, z 5 1.

Points :

N = 386, spaced uniformly a t an interval of 0.25 in each coordinate.

Weights:

EquaI, 1.

Special funct,ions: n = 25.

a

+b +c

=

0(1)4. See Miles and Williams [11.11]

W n is the j t h iterate of the Laplacian.

Boundary values : (a) f = [(1.5 - z)z These are unweighted.

+ (1 - y)? + (1.25 -

z)?]-I’~

92


( l ) ) f = I forz (0)

f

=

I,!=

> 0, j

= 1 for z

=

Oforz

< 1.

0 for z

5 0.

Boundary value problrm : Dirichlct problrm.

Case (11)

Case (a)

V R

Em ER

LX

ER/Vn

Number of spwi:il functions taken

Citae

1" 4"

Cnm (I))

0.43 0.24 0.136 0.08 0.05

!J"

16" 25" a

( i ~ )

0.88!) 0.636 0.439 0.327 0.397

Thwe linear conibinatioiis cwrreq)ond to Oth, lst, Case I(a), Error

Point Error

0.0 0.178

0.I; 0.019

tit

((5)

1.o 0.0 0.6772 0.39 0.1804 0.266

1.o 0.0 0.4581 0.376 0.136 0.297

1.789 0.2556 0.502 0.22 0.025 0.05

Max f Min f

Case

Case

(1:)

0.736 0.332 0.324 0.272 0.266

. . . , 4th degree polynornials

Interior Points, 10 Special Functionu 0.0 0.000

0.3 0.024

-0.3 0.001

-0.6 0.012

-0.9 0.057

Point 0.9 refcrs to (0.9, 0.9, O.!)), c x t c * , Case II

Equation : Domain :

AU

=

0

+ + z'

_< 1 Points : N = 266, 24 equidistant points on each of 11 parallels of latitude plus 2 poles. Sphere is divided into 266 equal areas. Points are centers of these areas. Weights: Equal, 1. Special functions: n = 25. Same as Case I. Sphere: z?

g'l


93

Uouiidury values :

+ ( I - y)? + (1.25 -

(a) f

=

[(1.5 - J - ) ~

(h) f

=

0 for z 2 0, f

( c ) .f

=

0 for z 2 0, f

(d) f

=

0 for z 2 0, j

(e) f

=

0 for x 2 0, f

(f) f

=

0 for z 2 0, f

~)~]-l/?

0 ; f = 0 , y 5 0. (b) f = I, z > 0 ; f = 0 , 2 5 0. (c) f = 5, 2 > 0;f= 0, 2 5 0 . (d) f = 2,2 > 0;f = 0, 2 5 0. (e) f = x3, 2 > 0;f = 0, z 5 0 . (f) f = 2 4 , 2 > 0;f = 0,z 5 0 . (g) f = 56,2 > 0 ; f= 0,z i 0. (h) f = exp (z2 - zy + 2y2). (a) f = 1, g

+ 4 + (y - 1)'). f = log ((z + 1.5)2 + v').

(i) f (j)

=

log (z

Boundary value problem: Dirichlet problem.

Case VI, Rapidity of Convergence, ERIVH Number of Case functions" (b) 1 2

3

4 5

fi 7 8 16

0.707 0.337 0.337 0.249 0.249 0.207 0.207 0.182 0.133

Case

Case (4

Case

Case

Case

Case

(c)

(4

(f)

(9)

(j )

0.7833 0.3370 0.0793 0.0793 0.0390 0.0390 0.0245 0.0245 0.0086

0.8360 0.4735 0.1591 0.0251 0.0251 0.0092 0.0092 0.0014 0.00078

0.8064 0.5609 0.2639 0.0759 0.0091 0.0091 0.0027 0.0027 0.00018

0.8860 0.6199 0.3440 0.1429 0.0368 0.0036 0.0036 0.0009 0.000032

0.8997 0.6626 0.4063 0.2026 0.0766 0.0180 0.0015 0.0015 0.000015

0.8177 0.2016 0.0713 0.0295 0.0133 0.0063 0.0031 0.0015 0.0000038

* Because of symmetry, only real parts of z" contributed to solution. Function i is thus R(zi-l).

Case VII, Results

*o Q

Maxf Minf

0.522 0.0

V R

1.0 0.0 0.136

Em,,

0.48

Ell ER/VR

0.025 0.18

0.048 0.0020 0.042

0.048

0.272 0.0 0.022 0.0097 0.00058 0.026

0.142 0.0 0.0106 0.0034 0.00021 0.020

O.Oi4 0.0 0.00518 0.0013 0.00009 0.0168

Exact error at origin: Case (j), 2 X 10-8; Case (k), 2 X 10-7.

0.039 0.0 0.0026 0.0006 O.oooO1 0.0151

0.575 0.012 0.0565 0.024 0.0018 0.032

2.71 1.02 0.309 0.040

0.0030 0.0095

1.82 1.31 0.329 0.001 8.4 X 2.6 X

lo-'

1.41 -0.238 0.181 3 x 10-7 9.6 X 1O-O 5.3 X lo-'

1.99 0.02 0.24 2 x 106 1.6 X 10-7 6.8 X l o '

100


Case VII

Equation

AU = 0.

Domain :

"Bean" shaped design. See Davis a i d ltabiriowitz [1.2].

N = 84. Distributed on boundary with more points where curvature is greater. Weights: Equal to average of adjacent chords. Special functioiis: n = 31: 1, Re zn,Im zn, n = 1, 2, . . . , 15. Boundary values: (a) f = 1, z > 0 ; s = 0, z 5 0 (b) f = 2, z > 0 ; f = 0, z 5 0 (c) f = 22, z > 0 ; f = 0,z 2 0 (d) f = 2 " , > ~ O ; f = 0 , 5~0 (e) f = z4,z > 0 ; f = 0, z 2 0 (f) f = z5, 2 > 0 ; j = 0, z 2 0 (g) f = z? y2 (Torsion problem) (h) f = esp (2' - zy 2y2)

Points :

+

6) f

=

f

=

(j)

(k) f

+

+ y + (9 - I)') log ((x + 1.5)2+ V ) log (z

+

+ (y - l)?)

cos y log (z2 13oundary valuc problem : Dirichlet problem. = e"

Case Vlll

Equation: Domain : Points:

Au = 0. Lower Unit Semicirclc. N = 78. On z-axis: - 1(0.05)1. On circle: every 5'. Corner points are counted twice. Weights : Equal to average of adjacent chords. Special functions: n = 21 : 1, Re zn, Im zn, n = 1, 2, . . , , 10. Boundary valuo: (a) fi = 0,f2 = 1 (Flow t,hrough semicircular channel)

ADVANCES

(I))

IN Jl

( c ) $,

101

ORTHONORMALIZING COMPUTATION

=

+ (I - 2)' +

log (.rz

=

log

((J

$),f2

- _i_

=

!I)?),

1

+ .c'

=0

Boundary value problem: Mixed. a ( P ) = 1, b ( P ) = 0 on semicircle a(Z>) = 0, b(P) = 1 on s-axis Case VIII, Results

Case

Case (b)

(a)

Max fi on semicircle Max faon x-axis Min fi on semicircle Min ji on z-axis VR Emaxon semicircle Em,, on z-axis ER ER/ V R

0.0 1 .0 0.0 1.o

0.1601 0.0435 0.0234 0.00205 0.0128

Exact error at origin

Case -

0.693 -0.5 0.317 -1.0 0.1748 0.000% 0.00048 5.93 x 10-6 3.39 x 10-4 4.0 x 10-5

((!I 022 0.0 0.0015 0.0 0.0315 3.6 X 8.8 X 4.79 x 10-7 1.52 x 10-6 2.0 x 10-0

Case VIII, Rapidity of Convergence, E R / V R Number of special functions

Case (a)

Case (b)

1 3

1 .o 0.361 0.087 0.04:! O.WB 0.018 0.013 0.013

0.743 0.21 1 0.150 0.027 0.005 0.001 0.0003 0.000:3

4

8 12 10 20 21

Case (r)

0.467 0.113 0.041 0.006 0.00 1 0.0002 0.00004 0.000015

Case IX

Equation:

AU

Domain :

Compressor Blade. See Poritzky and Danforth [18.5].

=

0.

PHILIP J. DAVIS AND PHILIP RABlNOWlTZ

102

N = 75. Ji:ssmt)i,zlly rqu:tlly spncrd with rrq'cct to x. Equal to nvcragc of adjsceiit chords.

Poiiiis: Weights :

A : n = 15: 1 , Re z", Im z", n = 1, 2, . . . , 7. B : n = 10: log (z2 (y - 0.5)*), 1, Re z", Imz", n = 1 , 2 , . . . , 7 . (a) $(z? y2)(TorsionProblem) Boundary values : (b) f = 1, z > O ; f = 0, z 5 0 (c) f = 2, 2 > 0 ; f = 0 , x 5 0 Boundary value problem : Dirichlet problem.

Special functions:

+

+

Case IX, Results

Max j hlin f

vx

Em,,, n = 15 E,.,, n = 16 ER, n = 15 ER, n = 16 ERIVR,n = 15 ERIVR,n = IG

Case (a)

Case (b)

Case (4

1.446 0.02 0.1939 0.00516 0.00522 0.0004 0.0004 0.0021 0.0021

1 .o 0.0 0.2097 0.44 0.42 0.0401 0.0396 0.191 0.189

1.65 0.0 0.2059 0.OG7 0.081 0.0053 0.0045 0.026 0.022

Case IX(a), Rapidity of Convergence

ER

Emas

Special functions used 1% log, 1 log, 1, Re z, Im z, Re 9, Im z2 log, 1, . , Re 24, Im z4 All

. .

n = 15

n = 16

0.960 0.030 0.010 0.005

1.076 0.453 0.021 0.010 0.005

Case X

Equation:

AAu = 0.

Domain : Points :

Ellipse, x2

+ 4y2 5 1.

N

=

=

80, x

- 1(0.05)1.

n = 15

0.1367 0.0031 0.0012

0.0004

n

= 16

0.1717 0.0623 0.0023 0.0012 0.0004

103


wj’)= wj”. Equal to average of adjacent

Weights :

chords.

+

Special functions:

n = 38: 1, 5 , y, ‘x y’, lie z”, Im zn,Re 2 zn, Im z P,n = 2, . . . , 9, Re zlO, Im zIo.

Boundary values:

(a) fl = Itc (b) fl

=

(2 c z ) , f. -

’

0,

f2

a

Re an

(2 e z )

= 1

f.2 = exp (.I? - xy (c) fl = 0, (d) fl = exp ( x y - zy 2yz),f2 = 0 f? = 0 (e) fl = 2, f2 = x (f) fl = 0,

+

+ 2y’)

dU

Bouudary value problem: u = fl, - = f?. an Case X, Reeults Case (1%)

Case (d)

Cnse

Case (b)

(C)

Case (e)

Cnse (f)

-

2.718

- 0.388 5.137 0.0 3.57 x 1.42 x 1.33 x 7.29 x 2.0

x

10” 10-7

10-8 10-0

0 .o 0.0 1.0 1 .o 1.7 x 10-4 7 3 x 10-6 8. x 10-5 3. x 10-6

0.0 0.0

3.01 1.49 0.7 x 3.03 x 3. x 1. x

10-4 10-4 10-4 10-4

3.01 1.49 0.0 0.0 3.18 x io--a

2.35 x 10-4 1. x lo-: 1. x 10-4

1 .o

0.0 0.0

- 1.0 0.0 0.0 1.2 x 9.0 x 5.7 x 4.2 X

1 .O

- 1.0 10-8 10-5 10-4 10-i

4.5 1.7 2.1 1.0

x x

x x

10-4 10-4

10-4 10-4

10-s

Case XI

Equat,ion : Domaill : Casc! (A) Points : M’eigh ts : Special funct,ions: Case (H)

AAu = 0.

Square

- 4 5 x, y 5 4.

N = 100. Gaussian abscissas of order 25 on each side. wf’)= wj2’. Gaussian weights. n = 38, as in Case X.

Poiiit s :

N

IVcights:

wj’)= wj”. I;:qual l o average of adjacent chords.

= 104. Equally spaced on sides at iiitervals of .02. Corner poiiits taken twice.

Case X, Rapidity of Convergence

d

P 0

Number of special functions

Case (a)

1 2 4 7 11 15 19 23 27 31 35

2.11 0.85 0.38 0.79(-1) 0.12(-1) 0.17(-2) 0.17(-3) 0.16(-4) 0.13( -5) 0.10(-6) 0.42(-7)

Emax, /I

Case (f)

0.0 0.48 0.48 0.16 0.61(-1) 0.57(-1) 0.106(-1) 0.108(-1) 022( -2) 0.23(-2) 0.45(-3)

Case (a> 5.44 3.61 1.34 0.23 0.36(-1) 0.49(-2) 0.61(-3) 0.65(-4) 0.74( -5) 0.76(-6) 0.15(-6)

Case (f)

1.0 0.56 0.56 0.30 0.22(-1) 0.28(-1) 0.34(-2) 0.23(-2) 0.75( -3) 0.40(-3) 0.17(-3)

Case

Case

(4

(f)

0.84 0.40 0.28 0.52(-1) 0.72(-2) 0.93(-3) 0.96(-4) 0.90(-5) 0.77( -6) 0.75(-7) 0.16(-7)

I

ER.f.

ER. I1

Emax, j z

0.0 0.27 0.2i 0.65(-1) 0.29(-1) 0.25(-1) 0.44(-2) 0.46(-2) 0.90(-3) 0.94(-3) 0.21(-3)

Case (a) 1.69 1.37 0.33 0.58(-1) 0.12(-1) 0.20(-2) 0.31(-3) 0.40(-4) 0.44( -5) 0.43(-6) 0.73(-7)

Case (f

1

0.58 0.38 0.38 0.17 0.15(-1) 0.20(-1) 0.22(-2) 0.14(-2) 0.46( -3) O.l9(-3) 0.10(-3)

Case (a) 4.85

3.56 1.12 0.19 0.32(-1) 0.52(--2) 0.76(-3) 0.95(-4) O.lO( -4) 0.99(-6) 0.16(-6)

Case (f)

1.42 1.13 1.13 0.42 0.77(-1) 0.75(-1) 0.118(-1) 0.115(-1) 0.24( -2) 0.23(-2) 0.58(-3)


+

18: 1, z, y, z? y?, Re z", Im zn, Rc 2 5'1, In1 2 x " , n = 2, 3, -1, Ite x5, Im z5.

Spccisl functions:

TL =

Case (C) Points :

N = 21. Equally spaced on sides a t intervals of 0.1. Corner points taken twicc. As in Case (R). n = 18, as in Case (13).

Weights: Special functions: Boundary values:

a

ji = Re (W), j 2 = -lie an (b) j i = 0, J-2 = 1

(a)

(c> fl ( 4 fl

13oundary d u e problem:

=

0,

j 2

=

z

=

z,

ji

=

0

Case R, a

A, a 0.30 -0.194 1.63 -0.614 4.5( -6) 2.0( -5) 3.4( -6) 8.6( -6)

(W)

a11 an = J2.

ZL = j 1 ,

Cnsv

a

105

Case c, a

0.3!) -0.195 1.63 -0.614 4.5( - 6 ) 2.0( -5) 3 . q -ti) 7.4( -6)

0.39 -0.103 1.63 -0.614 5.3(-6) 1.0(-5) 4.0( -6) 8.1(-6)

4.4(--6)

4.0( -6)

A, b

Case

Case B, b

0.0 0.0 1 .o 1.0 8.1(-3) 1.0( -3) 4.7(-3) 5.8( -4)

0.0 0.0 1.o 1.0 8.2( -3) 1.0( -3) 4.0( -3) 4.6( -4)

A refers to Cusc A after only 18 special functions were used. Case XI, Results Case c, b

0.0 0.0 1.0 1.o 8.0(-3) 1.2(-3) 5.1(-3) 6.1(-4)

Case

A, c 0.0 0.0 0.25 -0.25 1.0(- 2 ) 2.0(-2) 5.3(-3) 7.4(-3)

Case B, c 0.0 0.0 0.25 -0.25

1.0(-2) 2.0(-2) 4.5(-3) 6.1(-3)

Cnse

c, c 0.0 0.0 0.25 -0.25 8.4( -3) 1.2(-2) 5.0(-3) 8.3(-3)

Case A, d 0.25 -0.25 0.0 0.0 3.3( -2) 1.8(-2)

1.8(-?) 7.7(-3)

Case B, d 0.25 -0.25 0.0 0.0 3.3( -?) 1.8(-2) 1.9(-2) 6.7(-3)

Case

c, d 0.25 -0.25 0 .0 0.0 3.6( -2) 8.0(-3) 1.9(-2) 5.2(-3)


106

0.39 -0.194 1.63 -0.014 1.2 x 10-8 5.8 X 3.4 x 10-9 1.7 X 5.5

x

0.0 0.0 0.1 0.1 2.7 x 2.5 x 1.2 x 1.2 x

0.0 0.0 0.25 -0.25 6.8 X 1.2 x 3.0 x 3.5 x

10-2

10-3 10-2 10-3

-

10-10

0.25

--0.25 lOP 10-2 10-3 10-3

0.0 0.0 1.17 X lo-* 1.24 X lo-* 6.5 X 10-3 4.2 x 10-3

-

Case XI1

IGquation:

AAu = 0.

Domain: Poirit,s:

Lower Unit Semicircle.

Weights: Special fuiictions: Boundary values :

A : w;’)

N

=

78, as in Case VIII.

w;’); L3: n = 38, as in Case X. =

=

0.l~j”.

(a) f1 = 0, f2 = 1 (b) f i = 0, f2 = 2 (c) f i = 2,f2 = 0 (d) Uniform loading. f1 = (1 - x2)2 on x-axis fi = 0 on semicircle, f2 = 0 (e) Concentric loading. f l = (t x2) log 3 $(I 2% 1x1 I fi = (t 22) log3 Z(1 - 22), 121 2 fi = 0 on semicircle, fi = 0 (f) Circular loading.

+

+

I4 I 3,

-

+ + +

+ +

= 8(1 z’) (3 4x2) log 2, 121 2 fi = 0 on semicircle, f2 = 0.

fl

3 3

B,

Case XI, Rapidity of Convergence

-

Number of special functions

0

Case (b)

Case

Case

(C)

(4

1.4(-3) 5.8(-3) 1.7( -2) 6.7( -3) 1.2( -3)

1.2(-1) 3.3(-2) 7.4(-3) 4.4( -3) 3.5( -3)

v

4 13 19 29 35

8.2(-2) 8.1(-2) 7.2(-2) 2.8(-2) 2.T( -2)

5.8(-2) 1.8(-2) 1.0(-2) l.l(-2) 6.8(-3)

2.3(-1) 3.7(-2) 3.3(-2) 2.7(-2) 1.2( -2)

1.4(-3) l.O(-2) 3.3(-2) 1.6(-2) 2.5( - 3 )

2.5(-1) 7.7(-2) 2.0(-2) 1.2(-2) 1.2( - 2 )

7 . 7 ( - 2 ) 4.8(-2) 2.5(-3) 4.7(-2) 1.8(-2) 4.0( -2) 7.4(-3) 1.2(-2) 1.2( -2) 1.2( -2)

5.0( -2) 9.7(-3) 5.3( -3) 6.0(-3) 3.0( -3)

2.0(-1) 1.9(-2) 1.8( -2) 1.8(-2) 6.5( -3)

5.4(-2) 1.4(-2) 7.7( -3) 3.7( -3) 4.2( -3)


108

(g) Linear loading. fl

= -(z6

fi

=

- 2s3

+ z), on z-axis

0 on semicircle, fi

=.

0

aU

Boundary value problem: u = fi, - = fi. an Cusc

hiax f i

0.0 0.0 1.0 1.o 5.4(-2) 1.6(-2) 1.5(-2) 4.9(-3)

Min f l Max fj Min j 2 E m a x , /I

Emax, / r

ER,

/i

ER, I?

XII, RosiiltH

0.0 0.0 0.0 0.0 1 .0 1.o 1.0 -1.0 4.6(-2) 5.3(-2) 3.7(-2) 3.4(-2) 1.2(-2) 1.4(-2) 1.7(-2) 1.1(-2)

0.0 0.0 1.0 - 1.0 4.5(-2) 4.5(-2) 1.2(-2) 1.5(-2)

I.0

1 .0 -1.0 0.0 0.0 0.0 0.0 :J.7(-2) 2.4(-2) t.l(-2) 2.4(-2) '3.2(-3) 8.6(-3) 1.4(-3) 8.1(-3)

- 1.0

Case XII, Results Case A, d

Cme d

1 .0 0.0 0.0 0.0 7.3(-4) I .8( -3) 3.7( -4) 4.7(-4)

I .0 0.0 0.0 0.0 1.8(-4) 3.1(-3)

Caw A, c

CSK!

Case A, g

13, 0

Case B, g

~~

9.q-5)

0.2 0.0 0.0 0.0 2.q-3) 2 . q -3) 1.2( -3)

7.6(-1)

5.5(-4)

0.2 0.0 0.0 0.0 1.7( -3) 6.1 (-3) 8.0( -4) I .8(-3)

1.28 0.0 0.0 0.0 1.8( -3) 1.7(-3) 8.6( -4) 6 . q -4)

0.29 1.28 02!) -0.20 0.0 -0.29 0.0 0.0 0.0 0.0 0.0 0.0 1.5( -3) 1.7(-3) 1.3(-3) 2.7(-3) 1.4( -3) 3 4 -3) 6.9( -4) 8.2( -4) 5.7( -4) 1. I (-3) 6.1(-4) 1 . q-3)

Case Xlll

Equation :

Au = 0.

Domain :

Isosceles right triangle, vertices at (0.25,0.25), (0.25, -0.25), (-0.25, -025).

Points :

N

Weights :

Gaussian.

Special functions:

n

=

=

75, 25 point Gaussian rule on each side. 31 as in Case VI.

Boundary d u e problcm: Mised. n(1')u

au + b( P )--an = ,f, o ( P ) = 1 ,

b(") = 2. Case XIII, Rcwlts

Maxf Minj

0.56 0.0 V R 0.063 E,,, 4.3 x 10-7 ER 8.7 x 1 0 - 9 ERIVR 1.4 X lo-'

8.15 - 18.77 1.051 3.6 X l o - ' !).7 x 10-6 9.2 x 10-6

4.71; 7.67 - 18.77 - 18.77 0.665 1.515 5.1 x 1 ~ 5 5.1 X lOV3 1.5 x 2.4 x 10-4 2.2 x 10-8 1.G X 10-4

5.6'2

- 14.01

0.916 1.8 X 10-1 6.0 X lou6 6.5 X

Case XIV

l?quntion : Domniii :

AAtL

=

0.

Points: Weights:

Isoscrlrs right triangle, vertices at (0.25,0.2.5), (0.25, -0.25), (-0.25, -0.25). N = 75, 25 point Gaussian rule on each sidr. Gaussian ; = w ~ ~ ' .

Sprcial f u n d ions:

n

Round:try vnliies:

().

~1;')

=

38, as in C'asc X.

a - Hr a?%

.fi

=

Re 2f+,

,f?

=

(h)

fi

=

= 1

.fl

=

0, 0,

$2

(c)

j-2

= T

r,

.f? = 0

(d) .fl

=

(re.)

110


Houndary v a l prohlent: ~

11, =

a16

.fi, - = fi.

an

Case XIV, Results

Case

Case

(4

(b)

Case (c)

0.0

0.39

0.0 1 .0 1.o 3.37 X lo-* 2.58 X lod2 1.08 X lowa 7.0 X

-0.14 1.63 -0.77 1.39 X 6.10 X 10-8 3.68 X lo-# 2.13 X 10-6

0.0 0.0 0.25 -0.25 8.77 x 10-3 7.75 X 2.96 X 1.26 X lo-'

Case

(4 0.25 -0.25 0.0 0.0 1.3 X 5.55 x 5.93 x 1.20 x

10-2

1010-3 10-3

Case XV

Equation :

AU = 0.

Domain :

H-shape. Vertices a t (0.7, 0.8), (0.7, -0.8), (0.3, -0.8), (0.3, 0), (-0.3, 0), (-0.3, -0.8), (-0.7, -0.8), (-0.7, 0.8), (-0.3, 0.8), (-0.3, 0.4),(0.3, 0.4)) and (0.3, 0.8).

Points:

N

Weights: Special functions : Boundary values :

Average of adjacent chords.

= 96, equidistributed in steps of 0.1 wit,h ovcrlap at every corner.

31, as in Case VI. (a) f = +(x2 y?) (Torsion problem) (b) f = log ( x 2 (y O.LL)*) (c) f = log (x' (y 0.8)Y) (d) f = log (x2 (y - 0.8)') (e) f = 1, 2 > 0 ; f = 0, x 5 0 (f) f = 2: 2 > 0 ; f = 0, x 5 0

n

=

+

+ + + + +

Case XV, Results

--

4

Masj Minj v-8 Emax

ER ERI5-R

0.565 0.0 0.095 0.138 0.019 0.2

0.66 -2.41 0.324 0.786 0.092 0.284

1.12 -2.41 0.296 0.113 0.010 0.035

1.12 -2.41 0.309 0.117 0.011 0.036

1.o 0.0 0.207 0.47 0.05 0.242

0.7 0.0 0.110 0.16 0.0125 0.114

0.49 0.0 0.071 0.11 0.012 0.169

0.343 0.0 0.0485 0.10 0.0103 0.212

0.240 0.0 0.0336 0.079 0.0057 0.229

0.168 0.0 0.0227 0.057 0.0053 0.233

2.01 0.346 0.326 0.97(--9) 0.71(-10) 0.22(-9)


112

(g) f = (h) f = (i) f = (j) f =

(k) f

=

> 0;f

0,z 5 0 0, 2 5 0 z4,z > 0 ; f 0, z 5 0 z6, 2 > 0 ; j = 0, z L 0 Ite ez = ez cos y z2,z

x’, 1: > 0 ; f

= = =

Eoundary value problem : Dirichlet problem, Case XVI

Equation :

AAu = 0.

Domain :

“H” Shape (as in Case XV). N = 96, equidistributed in step of 0.1 with

Points :

overlap a t cvery corner. Weights :

wjl)= w;’)= average of adjacent chords. n = 38, as in Case X. Special functions: Boundary values : (a), (b), (c), (d), as in Case XIlr. au Boundary value problem: IL = fi, - = f2. an Case XVI, Results

Max fi

Min f l Max fz Min j 2 Enlax, /I

Em.,, ER,

/$

Ji

Ex, In Exact error at (020)

2.14 -0.35 3.54 -2.00 7.!)6 x 10-8 1.63 x 10-7 2.83 X 6.43 x 10-8

0.0 0.0 1.0 1 .o 4.28 X 9.18 X 1.81 X 2.58 X

1.40 x 10-9

-

10-I

10-1 lo-‘ lo-’

0.0 0.0 0.7 -0.7 9.95 X 10-1 2.54 X lo-’ 5.68 X 7.38 X

0.7 -0.7 0.0 0.0 5.06 X 6.20 x 2.88 X 2.48 X

-

10-1 10-1

10-I lo-’

-

Case XVll

Problem :

+ . . . over

To compute orthonormal polynomials alLzn

a point set and to compute its transfinite diameter.

Point, set:

Two eolliricar linc segments S: [- 1,

-$I,

[+, I].


Ittiicr product :

(J, q )

113

= Jsj(r)g(x) (1.r.

N = 50, a 25 point Gaussian rule in each segment. Gaussian. Weights : Special functions: n = 21; 1, r , z?, . . . , .$I' . Points:

Case XVII, Convergelice to Transfinite Diaiiieter an

71

0 1 2 3 4 5

-

(i i

8 9 I0 I1 12 13 14 15 1ti li 18 I!) 20 m

(o,*/an+*)1'2

1.oooooo 1.309307 4.601790 6.193236 23.82221 31.86801 125.353% 167.388ti 633.4655 885.3688 3521.162 4697.453 18715.48 24963.57 99565.1-4 132791.5 529991.0 706811 .O 2822310.0 3763744.0 I 5033710.0 (theoretiral valuta)

0.46616 0.45979 0.43951 0.44084 0.43594 0.43633 0.43467 0.43481 0.43408 0.43414 0.43375 0.43379 0.43356 0.43358 0.43343 0.43344 0.43334 0.43335 0.43328 -

0.4330127

a,,-"q 0.76376 0.46616 0.54454 0.45264 0.5004 1 0.44700 0.48120 0.4438!) 0.47048 0.44191 0.46366 0.44054 0.45893 0.439L3 0.45546 0.43877 0.45282 0.43816 0.45073 0.43767 -

Case XVlll

Problem : Point sct :

Same as in XVII. Three collinear line segments. R : [ - 1 ,

Inner product:

Same as in XVII.

Points:

N = 75, a 25 point Gaussian rule in each segment.

Weights :

Gaussian.

I%, 11.

Special functions: n

=

21: 1, r , . . . ,x?ll .

-41, [-4,

a],


114

Case XVIII, Convergenco to Transfinite Diameter a

0.8164967 1.297772 2.557682 5.76441 1 9.536194 22.947 78 44.04354 78.68162 201.1679 334.6985 722.6411 1619.348 2648.467 6798.31 8 12413.51 23011.70 58823.60 95330.98 218701.1 465947.6 773404.5

0 1 2 3

4 5 6 7 8 9 10 11

12 13 14 15 16 17 18 19 20 a

0.433

< theoretical

value

0.5650 0.4745 0.5179 0.5012 0.4653 0.5400 0.4679 0.4848 0.5276 0.4546 0.5224 0.4880 0.4618 0.5435 0.4594 0.4913 0.5186 0.4423 0.5317

-

0.5213 0.5144 0.4812 0.5077 0.4949 0.4850 0.5086 0.4775 0.4990 0.5018 0.4737 0.5072 0.4864 0.4871 0.5069 0.4721 0.5017 0.4977

-

-

-

0.6292 0.5650 0.5213 0.5409 0.5132 0.5145 0.5207 0.5024 0.5125 0.5073 0.5015 0.5098 0.4994 0.5027 0.5050 0.4971 0.5034 0.4994 0.4978 0.5025

< 0.5. Case XIX

Problem :

Domain :

To compute the complex orthonormal polynomials corresponding to a given domain, pn(z) = anZn . . . , to compute an approximation to the exterior mapping function of the domain from the ratio pn+l(z)/pn(z), and to compute the transfinite diameter of the domain. Same as in Case VII, the “bean.”

+

Points: N = 84, as in Case VII. Weights: As in Case VII. Special functions : R = 31 : 1, z, . . . ,z30. Error in exterior mapping function: Max 6 2 9 , i = 0.0774 i

($ ,xa;), N

a=t

li2

= 0.0281

115


0 1 2 3 4

5 6

7 8

9 10 11 12 13 14 15

0.48794905 0.51270368 0.50368673 0.50441978 0.50784869 0.50711472 0.50705196 0.50776177 0.50822067 0.50764419 0.50733756 0.50798193 0.50821394 0.50749173 0.50772655 0.50823471

16 17 18 I9 20 21 22 23 24 25 26 27 28 29

0.50776548 0.50732470 0.50808456 0.50815499 0.50731565 0.50757485 0.50832095 0.50760986 0.50733903 0.50784640 0.50783172 0.50708396 0.50761722 0.50827258

W

Transfinite diameter

Case XX

Problem :

Same as XIX.

Domain :

Square: Same as XI.

Inner product,:

(.f, 9)

=

Points :

N

100, as in XI, Case (A).

Weights:

As in XI, Case (A). n=35:1,z 9 4.

N

Special functions:

=

.E W i f k i ) s ( z i > . a=1

) . ' . )

Error in exterior mapping function: Max 633,i = 0.008 i

(z;x s%,)''2 I N

=I

=

0.001

-


116

Cum SX,Coiivc:rycwcr 6o Trnnsfuiitc Diamotcrn

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

0.28867513 0.29580399 0.30304576 0.28544961 0.29562620 0.29506199 0.29451547 0.29440303 0.29490287 0.29487829 0.29489984 0.2M85711 0.29496020 0.29497102 0.29498217 0.29497104 0.29500655

17 18 19 20 21 22 23 24 25 26 27 28 29 80 31 32 33

0.29501340 0.29501968 0.29501633 0.29503205 0.29503620 0.29503996 0.29503893 0.29504709 0.29504972 0.295052 12 0.29505188 0.29506660 0.29505836 0.29505997 0.29506002 0.29506298 0.29506420

Theoretical value: 0.295085 = o.5[r(1/4)1~/4T3~~

18. Comments o n the Numerical Experiments

The tables giving rapidity of convergence may be used to estimate the number of special functions needed for prescribed accuracy. Case I

.

Case (a) asks for a harmonic function which is regular in a larger region than the cube. But the singularity is close to the boundary. This yields considerably better results than the discontinuous problems (b) and (c). In discontinuous problems, E,,,, is of no great significance since we must inevitably pick up a discrepancy of the order of half the jump. Note that the error in the interior is considerably less than the error on the boundary. In case (a), the maximum error occurs at a vertex and is the point nearest the singularity. Part of the error is due to proximity to the singularity and part to the discontinuities at edges and vertices. Case II

For regular boundary values (a), the result on the sphere is better than on the cube by an order of magnitude. This comparison is somewhat unfair because the singularity is closer to the cube than to the sphere. For dis-


117

coiit,inuous boundary values (b) there is little difference in Ee/VR between the sphere and the cube. The family of solutions (b)-(f) exhibit the degree of approximation possible for prescribed boundary values of increasing smoothness. Theoretical discussions of this point do not seem to be available in three dimensions. If fd? 011 the sphere, the approximating functions of degree n 2 cause the greatest decrease in E,t/J’R whereas functions of degree n 3 contribute practically nothing. Case (f) is worse than case (e) where one might expect better results. But in (f), the best approximants would bc of dcgree five, and these were not included.

+ +

Case 111

Odd powers and imagiiiary parts of even powers were discarded in view on the outer boundary is an order of magnitude better of symmetry. Emax than on the inner boundary. The Dirichlet integral of this solution gives the capacity of the annulus (see, e.g., Diaz [15.30]). Case IV

Only the functions I , lie 9,log Iz f $1, lie ( x f + ) - I , Re (2 f +)+ played a significant role in (a). In case (b) eveii Re (z f +)+ contributed little. Convergelice to zero of 13x/Vli is very slow. As might be expected, no significant difference in EIR/VRbetween (a) and (b) turned up. The first treatment of this problem via least squares and punch card machines is in Reynolds [18.2]. Case V

This case was suggested to us by S. Bergniari [18.1]. Note that the roundiiig of the corners reduces E,,,,,,by and EIlt/V1t by +. The method apparently prefers smooth boundaries. Yet the error is relatively large in both cases (probably due to the noiiconvexity of the domains) and we do not feel that the results are sufficiently sharp to reflect the differences in the true solutions of (a) and (b). I n judging the table of rapidity of convergence bear in mind that we go up four degrees with each new function.

+

Case VI

This case is well worked over in the literature of numerical analysis. For polynomial boundary values prescribed on an ellipse the solution of the Dirichlet problem is a polynoniial (see Vodicka [18.3]). Note the family of houiidary values of iricreasiiig snioothness (L)-(g) and tlie resulting deand E’,t/Va. As in case 11, for feC” the approsiniating funccrease in E,,,,, tioii of degree n 2 causes the greatest decrease in ER/VR.

+

118

PHILIP J. DAVIS AND PHILIP RABINOWITZ Case

V11

Case VII(k) was first done in 1955 using SEACand eleven special fuiictions, see Davis and Rabinowitz [l.2]. The present computations lead to maximum errors which are lop3 times the old errors. This reflects the increased speed and capacity of the preseiit generation of computing machines and leads us to hope that similar progress will he made on more refractory problems over the riext five years. We have used almost the same set of boundary functions in V I aiid VII. The results show that the errors arc significantly smaller over the ellipse than over the “beau.” The differeiices become greater as the boundary function becomes smoother. Howevrr when the solution is harmonically contiriuable across the boundary, this ceases to be true and VII(j) is better than VI(j). The singularity is closer to the ellipse than to the “bean.” The nonconvex bouiidary of the “bean” does not seem to play a role in (j), but it is apparently the cause of the difference in behavior of VI(b)-(i) vs. VII(a)-(f), (h), and (i). As in VI, approximation by harmonic polynomials is strongly sensitive to the contiiiuity class of the boundary data and we would normally expect (g) to yield better results than (e)-(f). The actual results are something of a mystery. The errors are of same order of magnitude as in V. Maximum errors for (a)-(i) occur iii the vicinity of the innermost coilcavity. In (j) and (k) they occur at the boundary points nearest the singularity. Case Vlll For (a) sce Synge [2.2]. Note that the derivative discrepancies are of same order as the functional values. Note the superiority of (c) to (b) and the relative locations of their sirigularities which are a t the same distance from the boundary. The maximum error in (a), (b), and (c) occurred at the corners. Case IX

The addition of the logarithm doesn’t make much difference in the long run. Porit,zky and Danforth [18.5] tried collocation on t8hetorsion problem for this contour. Thirteen points and functions were employed and unsatisfactory results reported. Case X

Iio low order coiitinuity data was prescribed. Apart from (a) which was a “set-up,” the other cases yielded results roughly of the same order of magnitude. In (a), exact error a t (0.0) was an order of magnitude less than


119

the avcragc crror. I t is intcrcsting that in (h-(f) the derivative is approxiinatcd more accurately than the function. The table of rapidity of coilvergerice shows that while I, naturally, decreases monotonically with the number of special functions, the separate crrors El,,,, j,; B,,,, I?; En, I,; and ER, j 2 rieed not. Case XI

For the entire biharmonic boundary value (R,a) the errors are of the same order as the cllipse X. For the other boundary value problems common to X and X I , the results on the ellipse are better by two orders of magnitude. We were interested in comparing the discrete inner product with the integral inner product and to obtain the latter with a high degree of accuracy, Gaussian weights and abscissas were employed in XI, (A). No observable improvement resulted. I n (C), the number of equidistributed points taken was reduced by 80%. The number of special functions was 18,and the number of points was, essentially, 40. At these points, the errors are comparable to (A) and (B). When the explicit approximate solution was evaluated a t the boundary points, the errors were comparable to the errors achieved in the interior, In other words, no divergencies of collocation type were observed (compare with XV). Case XI1

Cases (d)-(g) were suggested to us by J. Nowinski. 111 (a) the semicircle yields accuracy comparable to the square. In (b) and (c) the results are one order better than the square and an ordcr worse than the ellipse. Note that (.4)weights the values and those of derivatives equally (this was done elsewhere with the biharmoitic eyuat,ion) but (B) gives the values 10 times the weight of the derivative values. Accordingly we would expect Ell, in (B) to be better than in (A). Similarly, Em:,,,f2 and E R ,j 2 thatE should be worse. However, it is interesting to note that the values computed did not change significantly, and improvemelit in fi was less than the = 0.5w(I),practically worsening 0f.L In a case not reported here, where w(?) no change was observed. In the cases suggested by Xowinski, momeiits M,, M,, M , , Me were computed and thc match twtwecii M , and M , at the corners had a 5%) error. Case

Xlll

Case (a) has similarities to the torsioii problcm. The error is much lcss than for the torsion problem for the nonconvex regions studied in V, VII, and IX. Cases (b)-(e) have harmonically continuable solutions whose singularity in each case is at the same distance from the domain, h u t differ-

120


ences in accur:Lry show up nnd may hc nt,tributed to rr1:ttivc. local ions of Ihc singularity. Case XIV

This case is comparable in accuracy to the square, XI. Case XV

Although this domain presents difficulties, very high accuracy was achieved for the entire harmonic solution (k). The case of torsion (a) was very bad, and none of the other cases (e)-(j) was much better. The continuable cases (c) and (d) are an order of magnitude better than (a). The poor showing of the continuable case (b) may be attributed to the position of its singularity in the interior of the convex hull of the H . Cases (a), (b), (c), and (k)were also solved by collocation with 31 points. For case (k)resulting coefficients agreed fairly well with the corresponding theoretical coefficients. For cases (a)-(c), the resulting function, when evaluated at intermediate points on the boundary, exhibited errors of several orders of magnitude greater than the already large errors achieved by least squares. Case XVI

Comparing this case with XI (a) is same order of magnitude whereas (b) , (c), (d) are worse by about two orders of magnitude. This bears out our previous experience that for entire harmonic or biharmonic functions, the shape of the domain is of little consequence. Case XVll

This is an extension of previous computation (see Davis [18.4]). This computation was carried out in two ways: (a) Gram-Schmidt, double precision floating orthonormalization of the powers, (b) single precision floating recurrence. In view of the symmetry of the point set, the coefficient of in the orthonormal polynomial should be zero. With method (a) and for n = 17, this theoretically zero coefficient was of the same ordcr of magnitude as the leading coefficient. With method (b) a-id for n = 20, this coefficient was times the leading coefficient, i.e., the full precision of the computation. Here is strong evidence for the superiority of rccurrencc vs. Gram-Schmidt, wheiicvcr npplicablr. Case XVlll

We are not aware of any closed form expression for the answer to this problem. The computation was done by recurrence. The poor convergence


121

(when compared with XVII) indicates the numerical difficulty associated wit,h this type of problem. The last column should theoretically converge to the proper answer. This is true of the two previous columns, if convergent. Case XIX

Cases XIX and XX are the only examples of complex orthogonalization carried out. For earlier computations, see Davis and Rabinowits [1.2]. Case XX

The results are far superior to those of XIX. The convergence of a,/a,+, is almost monotonic and is within 0.00002 of the theoretical value. The boundary errors in the mapping function are of an order of magnitude better than in XIX.

19. The Art of Orthonormalization

As in many problems run on computers, great care must be taken to see that there are no mistakes in the input data. For this reason, in running a series of computations, it is advisable if possible to insert one problem whose answer is known. The orthonormalizing routine is very sensitive to errors in the data of the function to be expanded. For this reason it is important to examine the deviations at each individual data point. By inspecting these discrepancies, one can detect a t a glance any errors in the input data. Usually the discrepancies a t all points are of the samc order of magnitude, so that when the discrepancy a t a particular point is greater than the average by one or two orders of magnitude. it is advisable to recheck the input data for that point. Experience has shown that that piece of information has been incorrectly prepared. Another important thing to remember when expanding a function by least squares is that the sum of the squares of the absolute deviations is minimized. This may result in large relative errors occurring a t several data point,s. If this is undesirable, a possible way to remedy the situation is to assign as weights the inverse squares of the values of the function, i.e., W ;=

1/Fi2.

The computations reported herein were almost all done in doubleprecision floating. For some of the scaling problems involved when working in fixed point see Davis and Rabinowits [1.11. For small problems involving up to 10-12 functions, single precision arithmetic (36 bits) should be adequate. However, because of roundoff , double precision is advisable for work wit,h more than that number. With single precision words of greater length,

122


the iiumber of functions for which single precision is adequate naturally increases. As has been mentioned several times, whenever one want,s to fit data by II polynomial, the %term recurrence relation should be used rather than Gram-Schmidt. This is so for reasons of accuracy, but even if comparable accuracy could be achieved, 3-term recurrence is much quicker than GramSchmidt and requires much less storage. The time for the former is proportional to nN,for the latter, to n2N. For solving the harmonic, biharmonic, and other linear elliptic equations, the relative advantages and disadvantages of orthogonal functions versus finite differences should be considered. The use of orthogonal functions has the advantage that it gives the answer as a closed expression, gives a good idea of the error committed, and can be used with any shape domain. On the other hand, the computations may take longer, and every new differential equation requires a new program to generate particular solutions.

20. Conclusions

The set of experiments reported in Section 17 indicates the accuracy that can be achieved with the least square method in a wide variety of situations. This method, without undue expenditure of machine time, yields answers of considerable accuracy in some problems and of quite acceptable accuracy in a great many problems. The interested reader, confronted with his own particular problem, can “extrapolate” from the cases given and obtain an idea of what the method is likely to do for him. Theory indicates and computation bears out the sensitivity of the method to the boundary and to the boundary data prescribed. The most favorable condition is that of boundary values which come from entire harmonic functions or from harmonic functions which are regular in large portions of the plane. Here we may expect geometric convergence, and it seems t,o matter little whcther the boundary itself is analytic or piecewise analytic. Boundary data leading to solutions which do not cont,inue harmonically across the boundary, or boundary data which is of low continuity class, is a less favorable situation. Convergence is oiily arithmetic and observed errors are correspondingly greater. Here it seems to matter what the boundary is, with analytic boundaries responding more favorably than piecewise analytic ones and convex boundaries than nonconvex ones. There is a progressive falling off in accomplishment as we move from simply-connected domains to multiply-connected domains and from two dimensions to three dimensions. The more coniplicatcd the geometrylmundary data combination, the more we must work to obtain a solutinn


123

of given accuracy. Special devices, “tailor-made” for the configuration , might have to be called into play. This undoubtedly is the case with all numerical methods, and leads to the conclusion that for unusual geometrical configurations, the potential problem or the biharmonic problem cannot,, even now, he regarded as an open and shut matter. We regret that the return of one of the authors to his permanent post in September 1900 has prcvrnted us from continuing t hese iiivestigatioiis. We would likc to have looked into the equations AIL= 0 and AAu = 0 tackling more complicated boundary problems. Thus, we would like to have solved mixed problems for two-dimensional multiply-connected and t hree-dimensional simply- and mult iply-connected domains. The biharmonic equation with its wide variety of boundary value problems still needs considerable investigation, and we would like to h a w solved problems over t hree-dimensional domains. In these harder problems, we feel that, comparable accuracy can he achieved by taking 50-80 special functions. We would like to know something about the optimd ratio of n to N. When n = N we have, as has been pointed out, the rnsr of straight collocation with t he possibilities of serious divergencies. We would have liked-and this is of considerable importance-to h a w carried out comparisons of the present method with other methods currently employed. Though these things have not been possible, we feel we have laid a solid ground for anyone who wishes to carry this work further.

References SECTION

1. Introduvtion

1.1 Davis P., and Rabinowitz, P., A multiple purpose orthonormalizing code and its uses. J. Assoc. Cornpuling Machinery I,183-191 (1954). 1.2 Davis, P., and Rabinowitz, P., Numerical experiments in potential theory twin3 orthonormal functions. J. Wash. Acad. Sci. 46, 12-17 (195G). 1.3 Davis, P., Orthonormalizing Codes in Numerical Analysis, Lectures presented a t NBS-NSF Training Program in Nunirrirul Analysis, 1957 (J. Todd, ed.), to he published. SECTION

2. The Geometry of Least Squares

2.1 Schreier, O., and Sperner, E., Modern rllgebra and Matrix Theory, c;hapt. 11. Chelsea Publishing Co., New York, 1951. 2.2 Synge, J. I,., The Hypercircle in Mathematical Physics, pp. 7-124. Cambridge IJniversity Press, London and New York. 1957. 2.3 Taylor, A. E., Introduction to Functional Analysi,~,p. 106. Wiley & Sons, New York, 1958. 2.4 Davis, 1’. J., Haynsworth, E., and MarcuE, M., Bounds for the P-condition number of m:ttrires with poeitive roots, J. Research,Natl. Bur. Standards 658, 13-14 (1961).

124 SECTION


3. Inncr Products Useful in Numoricd Analysis

3.1 Bergman, S., The Kcrncl Function and Conformal Mapping. Am. Math. SOC., New York, 1950. SECTION

4. The Computation of Inner Products

4.1 Birkhoff, G., and Young, D. M., Jr., Numerical quadrature of analytic and harmonic functions. J. Math. and Phys. 99,217-221 (1950). 4.2 Tyler, G. W.. Numerical integration of functions of several variables, Con. J. Math. 5, 393412 (1953). 4.3 Hammer, P. C., and Stroud, A. H., Numerical integration over simplexes. Math. Tables Aids Comput. 10, 137-139 (1956). 4.4 Hammer, P. C.,and Wymore, A. W., Numerical evaluation of multiple integrals I. Math. Tables Aids Comput. 11, 59-67 (1957). 4.5 Hammer, P. C., and Stroud, A. H., Numerical integration of multiple integrals 11, Math. Tables Aids Comput. 19, 272-280 (1958). 4.6 Albrecht and Collatz, L., Auswertung mehrdimensionaler Integrde. 2. angew. Math. Meeh. 38,l-15 (1958). 4.7 Stroud, A. H., A bibliography on approximate evaluation of integrals. Math. Tables Aids Comput. 15, 52-80 (1961). 4.8 Stroud, A. €I., Quadrature methods for functions of more than one variable. Ann. N . Y . Acad. Sn'. 86,776-791 (1960). SECTION

5. Methods of Orthogonalkation

5.1 Courant, R., and Hilbert, D., Methoden der Mathematischen Physik, Vol. I , pp. 40-47. Berlin, 1931. 5.2 Seego, G., Orthogonal Polynomials. Am. Math. Soc., New York, 1939. 5.3 Shohat, J. A., Hille, E., and Walsh, J. L., A bibliography on orthogonal polynomials. Bull. Natl. Research Council (U.S.) 103, (1940). 5.4 Peach, M. O., Simplified technique for constructing orthonormal functiona. Bull. Am. Math. SOC.50,556-564 (1944). 5.5 A. Erdblyi el al., Higher Transcendental Functions, Vol. 2, Chapters 10 and 12. McGraw-Hill Book Co., New York, 1953. 5.6 Forsythe, G. E., Generation and use of orthogonal polynomials for data fitting with a digital computer. J. SOC.Znd. Appl. Math. 5 , 74-88 (1957). 5.7 Ascher, M., and Forsythf, G. E., SWACexperiments on the use of orthogonal polynomials for data fitting, J . Assoc. Compding Machinery 5, 9-21 (1958). 5.8 Weisfeld, M., Orthogonal polynomials in several variables, Numerische Math. I, 38-40 (1959). SECTION

6. Tables of Orthogonal Polynomials and Related Quantities

-

6.1 Russel, J. B., A table of Hermite functions, J . Math. and Phys. 19,291-297 (1933). [ee-Z'/2Hn(z): x = 0(.04)1(.1)4(.2)7(.5)8, n 0(1)11, 5D.l 6.2 British Assoc. for the Advancement of Sci., Legendre Polynomials, Mathematical Tables, Vol. A. Cambridge University Press, London and New York, 1946. [ P ~ ( z )z: = 0(.01)6, TZ = 1(1)12, 7-8D.l 6.3 Wiener, N., Extrapolation, Interpolation, and Smoothing of Stationary T i m Series. Wiley & Sons, New York, 1949. [Laguerre Polynomials Ln(x) = e@(e-V)(n). n = 0(1)5, z 0(.01).1(.1)18(.2)20(.6)21(1)26(2)30, 3-5D.l 6.4 National Bureau of Standards, Tables of Chebyshev Polynomials S,(x) and Cn(Z),


125

Appl. Math. Ser. No. 9. U.S. Govt. Printing Office, Washington, D. C., 1952. [ x = 0(.001)2, n = 2(1)12, 12D. CoefTicienta for ri = 0(1)12.] 6 5 Saleer, H., Zucker, R., and Capuano, R., Tables of the zeros and weight factors of the fist twenty Herniite polyrioniids. J. Research N a f l . Bur. Slriritlurds 48, 111116 (1952). 0.0 Karmazina, L. N., Tablrtay Polirroinov Yalohg. Izdatclstvo Airad. K;uiil< S.S.S.11.

1 1

Moscow, 1954. [Jarobi Polynomials: C,,(p, q, z),

0.9 0.10 6 11

6.12

0.13

6.14

(12

= 0,

# 71; G,,(p,q, 2) = zR . . . ; z = O( 01)1, q = . l ( . l ) l , p = 1.1(0.1)3, n = 1(1)5, 7D.l Iiopal, Z.,Nuwierical Aiialyhis. 'CYiley & SOIIS, K e n Yolk, 1955. Davis, P., and Rabinonitz, P., Abscissas and weights for Gaussian quadratures of high order. J . Research Nall. Bur. Slandards 56, 35-37 (1956). Davis, P., and Rabinowitz, P., Additional abscissas and weights for Gaussian qiiadratures of high order. J. Research Natl. Bur. Sfandards 60,613-614 (1956). Head, J., mid 'CYilson, W., Laguerre futictions: tables and properties, PTOC. Inst. Elec. EUQPS. P f . C 103, 128-441 (195G). Fishman. H., Numerical integration constants. Math. Tables Aids Coinpulers 11, 1-9 (1957). Barker, J. E., Use of orthonormnl pol) nornials in fitting curves and estimating their first and second derivatives, Naval Proving Ground Ilept. KO.1553, Suppl. 1, Tables 1 and 2. Dahlgren, Virginia, 1958. Gawlik, H. J., Zeros of Legendre polynomials of orders 2-61 and weight coefficients of Gauss quadrature formulae, A.R.D.E. Memo. (U) 77/58. Fort Halstead, Kent, 1958. HxbinoNitz, P,,and Weiss, G., Taldrs of abscissas and weights for numerical

?it

0.7 G.8

+

(1 - z)P-'W-~G,G,

cvaluation of integrals of the form

lJm

e-zrry(z)dx. Malh. TabZes Aids Compul.

13, 285-294 (1959). 6.15 Rabinowitz, P., Abscissas and weights for Lobatto quadrature of high order. Math. o j C'oiripulalzo?i 14, 47-52 (1960). (I. 16 Hochstrasbcr, U. W., Orthogonal polynorniitls, in EIarulbook of Mathematical Punclions, Chapt. 22. U.S. Gov. Printing Office, Rashington, D. C. in press. SECTION

i.1

7.2 7.3

7.4 7.5.

7.6 7.7 7.8

7. Lrast Square Approxiination of Functions

Snctlecor, G . W., Statislical Methods. Iowa State College Press, hiies, Iowa, 1937. Kendall, M. G., The Advanced Theory of Statistics. Griffin, London, 1946. Spcncer, It. C., and Parke, N. G., 111, A Matrix Treatment of the Approxiination of Power Series Using Orthogonal Polynomials, Including Application, Electronics Itcwarch Division Antenna Laboratory. Air Force Cambridge Research Center, Cambridge, Massachusetts, 1952. Cvetkov, li., A new method of computation in the theory of least squares. Auatraliati J . A p ~ lSci. . 6, 274-280 (1955). Gale, L. A., A niodiF.ed-equiLtioris method for the least squares solution of condition equations. T r a m . Am. f;eophp. Union 36, 779-791 (1955). Shrnitzer, A,, C'hebjshev approximation of a continuous function Ly a class of funvtioiis. J . Aaaoc. ('ompuling Machrtiery 4, 30 35 (1957). hluehly, l i . J., First Interim Progrestj Report 011 llational Approxirn:btiori, lechnicd Report. Princeton Uiiiversity, Princeton, New Jersey, 1958. ;\lurnaghan, F. D., and-Wrench, J. \V., The Approsirnation of DiBereutiable


126

Functions by Polynomials, David Taylor Model Basin Report No. 1175. 1958. 7.9 Stiefel, E.L., Numerical Methods of Tchebycheff approximation in On Numerical Approzirnalion, (It. E. Langer, ed.), University of Wisconsin Press, Madison, Wisconsin, 1959. 7.10 Deming, L. S.,Selected bibliography of statistical literature, 1930 to 1957: I. Correlation and regression theory. J. Research Natl. Bur. Slandards 648, 55-68 (1 960). 7.11 Maehly, H ., and Witegnll, C., Tsrhebyscheff -Approximationen in kleinen Intervalleii I. Approximation durch Polynome. Numer. Math. P, 142-150 (1960). 7.12 Veidinger, L., On the numerical determination of the best approximations in the Chebyshev sense. N u m w . Math. 9, 99-105 (1960). 7.13 Box, G. E. P., Fitting Empirical Data, Mathematical Research Center Tech. Summary Report No. 151. University of Wisconsin, Madison, Wisconsin, 1960. 7.14 Hartley, 1%.O.,The ModiEed Gauss-Newton Method for the Fitting of NonLinear Regression Fiinctions by Least Squares, Statistical Lab. Tech. Report No. 2.3.Iowa State University, Anies, Iowa, 1959. SECTION

8. Overdetermined Systenis of Linear Equations

8.1 de la VallBe-Poiissin, C. J., Sur la mdthode de I’approximation mininium. SOC. sci. Bruzelles, Ann. SBr. 11, 35, 1-16 (1911).(English translation by H. E. Salzer available). 8.2 Goldstein, A. A,, On the Method of Descent in Convex Domains and Its Applications to the Minimal Approximation of Overdetermined Systems of Linear Equations, Math. Preprint No. 1. Convair Astronautics, 1956. San Diego, Cal. 8.3 Goldstein, A. A,, Levine, N., and Hershoff, J. B., On the “best” and “least Q “ ” approximation of an overdetermined systttni of linear equations. J. Assoc. Cornputing Machir~ery4,341-347 (1‘357). 8.4 Goldstein, A. A., and Chenry, W., A finite algorithm for the solution of consistent linear equations and inequalities and for the Tschebycheff approximation of i~iconsistentlinear equations. Pacijc J . Math. 3 , 415-427 (1958). 8.5 Zukhovitskiy, S. I., An algorithm for the solution of the Chebyshev approximation problem in the case of a finite system of incompatible h e a r equatiGXl8 (Russian). Uokladg Akad. Nauk S.S.S.R. 79, 561-564 (1951). 8.6 Hemez, E., Ge:efmalComputational Methodsjor Chebyaheu Approximation (Itussian), Part 11. Iedatelstov Akad. Nauk Ukrajrisk S.S.Il., Kiev, 1957. 8.7 Stiefel, E., Note on Jordan elimination, linear programming and Chebyshcv approximation. Numer. Math. 9, 1-17 (1960). SECTION

9. Least Square Methods for Ordinary Differential Equations

9.1 I’irone, M., Sul nietodo delle minime potenze ponderatc e sul metodo di ltitz per il calcolo approssimato nei problmii della fisira-matematica. Rend. Circ. mat. Palermo 59, 225-253 (1928). 9.2 Faedo, S.,8ul nietodo di Ritz e su quelli fondati sul principo dei niininii quadrati per la risoluzione approssimata dei problemi della fisica matematica. Rend. Mat. e A p p l . Uniu. Roma I d . nazl. alla mat. 6, 73-94 (1947). !) 3 Collatz, I,., Nirmerischr BehaiitllrtrLg uoii I)i~ercritinlgleichririyeri.Spririgcr Ikrlin, 1951.

9.4 Faedo, S., Sulla maggioraziorie dell’errore nei metodi di Ilitz e dei mininiiquadrati. Atla. Accad. nazl. Ltncei. Rend. Classc sci. Js. niat. 1 2 rial. 14,466-470 (1953).


127

F o x , I,., 1Vunu~rimlSoliitiori 0.f 7’iiio-l’oinL Boii?ulnr~/Problrms i i i Ordiri(iri/ I I t f Jwcnlid Eqitnlioris. Oxford I’nivwhit y Prws, 1,nndon and Nrw 1-ork, 19.57. . Llftch. 9.6 Iiadner, If., Untrrsurhungcn ziir Kolloc:ttiorihnicthod(,. 2. A t ~ g r c ~Math. 40,99-11:( (1960). (3.7 Hildebrand, F. B. ltnd Crout, P. D., A Least Square Procedure for Solving Integral Equations by Polynomial Approximation, J . M n l h . mid Phys. PO, 3 10335 (1941), 9.8 I,onseth, h. D., Approximate Solutions of Fredholm-Type Integral ltlqiiationci, 1 3 i ~ I l AWL. . M d h . SOC.60,415-430 (1954).

!I5

SECTION

10. Linear Partial Differential Equations of Elliptic Type

10.1 Zaremba, S., L’equat,ion 1,iharmonique ct. une classe remarquahle do fonrtions fondamentales harmoniques. RuLE. intern. acad. sci. Cracouie, 147-196 (1907). 10.2 Zaremba, S., Sur le calcul nunikrique des fonctions demandhes dans le p r o b l h e de Dirichlet, et le prohkme hydrodynamiqiie. Bull. intern. acad. sci. Cracovie 1 , 125-195 (1909). 10.3 Bergman, S., uber die Entmicklung der harmonischen Funkt.ionen der Ebene und des Raumes nach Orthogonalfunktionen. Math. Ann. 86, 237-271 (1922). 10.4 Merriman, G. M., On the expansion of harmonic functions in terms of normalorthogonal harmonic polynomials. , 4 ~ J. . Math. 53, 589-596 (1931). 10.5 Bergman, S., The Kernel Function atrd Conformal Mapping. Am. Math. SOC., New Pork, 1950. 10.6 Fichera, Q., Risultati concernenti la resoluzione delle equazioni funzionali lineari dovuti all’Istitnto Nazionale per le Applicazioni del CaIcolo. Atti. ri ccad. rind. Liiirei Mem. Classe Sci. fis. mat. e nat. Sez. [ 8 ] , 3, 1-81 (1950). 10.7 Firhera C., On some general integration methods employed in connection with linear differential equations. J . Math. and Ph.ys. 29, 59-68 (1950). ’ 10.8 Kantorovich, L.V., and Krylov, V. I., Approzinmte Methods of Higher .4ml Intersciencc Publishers, Xew York, 1958 (1950 ed. translat.ed by (’. D. Bens 10.9 Bcrgman, S., and Schiffer, RI., K ~ r w Fi~7irtion.s l and Elliptic Differci~tialEqiiationa Academic Press, New York, 1953. i n Mathematical Phy. 10.10 Pieone, M., Exposition d’une methode d’int.6gration num6riqiie deu systbrncs d’6quations IinEaires aux derides part,ielles, mise en oeuvre a I’Instit,rit Kational pour lcs Applications du Calrul. REsultats obtcnus et rEsult,at,squo l’on pourrait att.eindre. Les machines ii ralcirler el la pensbe humnine. Colloq. intern. centre nnt. rerherche s c i . (Paris) 37, 23-261 (1953). 10.11 Picone, M., On tho mathematical work of the Italian Institute for the Application of Calculus during the first quarter contury of its existence. Piibbl. I d . nicz. .4ppl. del (”alrolo, Roma, No. 362 (1953). 10.12 Sokolnikoff, I. S.,Maltie,natiral Ttirory of Elastin‘ty, 2nd rd., hirGraw-Hill Hook (!o., New York, 1956. 10.13 Zolin, A. F., An approximate solution of thr polyharmonic prohlem. Dolrlarly A k d . NO,/& S.sT.S.R. 122, 971-973 (1958). 10.14 Lieherstein, H . &f,, A Continuons Method i n Kumtrrical Analysis Applied t,o $:samples From a New Class of Boundary Value Problems, Matjhemat,ic:tl llesearch Center Tech. Summ:~ry Report No. 175. [Jnivcrsity of Wisronsiii, Madison, Wisconsin, 1960. 10.15 Fichera, G., On a unified theory of boundary value problems for elliptic-parabolic equiitions of sccond order, in Bourdary Problems i n Differenlid Equations,

128


h'ls1.h. Rrsrarch Crntcr Symposium (R. E. Langrr, cd.), pp. 97-120. Univmily* of Wisconmin Press, Madison, Wisconsin, 1960. JU.10 Lowan, A. N., On the Picone 'l'reatment of Boundury Vahio Problcmv for Partial Differential Equations, Report 5310, Vniversity of California Radiation Laboratory, Livermoro, Califmiia (1958). SECTION

11. Coniplcte Systems of Particular Solutions

11.1 Bergman, S., Zur Theorie der ein-und mehrwertigen harmonischen Funktioneri des dreidimensionalen Ilaumcs. Math. 2. 94, 641-669 (1925). 11.2 Kellogg, 0. D., Foundation of Potential Theory. Frederick Ungar, New York, 1953. (Reprint, Dover Publications, New York). 11.3 Fichera, G., Teoremi di completezza connessi all'integrasione dell'equaeione A4u = f. Giorn. mat. Battaglini 77, 184-199 (1947). 11.4 Fichera, G., Teoremi di conipletczza sulla frontiera di un dominio per taluni sistemi di funzioni. Ann. mat. pura e appl. 97, 1-28 (1948). 11.5 Vekua, I. N., Novye Melody rcsheniya ellipticheskikh uravneni. OGIZ, Moscow and Leningrad, 1948. 11.6 Bers, L., Theory of Pseudoanalytic Functions, Department of Mathematics, Notes. New York University, New York, 1951. 11.7 Nehari, Z., Conformal Mapping. McGraw-Hill, New York, 1952. 11.8 Zwieling, K., Grundlagen einer Theorie der biharmonischen Polgnome. Verlag Technik, Berlin, 1952 11.9 Vekua, I. N., On completeness of a system of harmonic polynomials in space. Doklady Akad. Nauk S.S.S.R. [N.S.] 90,495-498 (1953). 11.10 Vekua, I. N., On completeness of a system of metaharmonic functions, Doklady Akad. Nauk S.S.S.R. [N.S.] 90,715-718 (1953). 11.11 Miles, E. P., Jr., and Williams, E., A basic set of homogeneous harmonic polynomials in k variables. Proc. Am. Math. SOC.6, 191-194 (1955). 11.12 Vekua, I. N., Sgsteme von I)iffcrentialgleichungen erster Ordnung vom elliptischcn

Typus und Randuertaujgahen (German translation). Math. Forschungsberichte, Berlin, 1956. 11.13 Henrici, P., A survey of I. N. Vekua's theory of elliptic partial differential equations with analytic coefficients. 2. Angew. Math. u. Phys. 8, 169-202 (1957). 11.14 Horvath, J., Basic sets of polynomial solutions for partial differential equations. Proc. Am. Math. SOC.9, 569-575 (1958). 11.15 Miles, E. P., Jr., and Williams, E., Basic sets of polynomials for the iterated Laplace and wave equations. Duke Math. J. 96/35-40 (1959). 11.16 Henrici, P., Complete systems of solutions for a class of singular elliptic partial differential equations, in Math. Research Center Symposium (R. R. Langer, ed.) University of Wisconsin Press, Madison, Wisconsin, 1959. 11.17 Bergman, S., Integral Operators in the Theorg of Linear Partial Differential Equations. Springer, Berlin, 1960. 11.18 Krzywoblocki, M. 2. v., Heryman's Linear Integral Operator Method in the Theorg of Compressible Fluid Flow (see Appendix by P. Davia and P. Rabinowitz). Springer-Verlag Vienna, 1960. 11.19 Gagua, M. B., On completeness of systems of harmonic fnnctions. Soobshcheniya Akad. Nauk Gruzin. S.S.R. 19, 3-10 (1957).

ADVANCES IN ORTHONORMALIZING COMPUTATION SECTION

129

12. Error Bounds; Degree of Convergence

12.1 Marcolongo, R., Sulla funzione di Green di grado n per la sfera. Rend circ. mat. Palermo 16, 230-235 (1902). 12.2 Walsh, J. L., On the degree of approximation to a harmonic function. Bull. Am. Math SOC.33, 591-598 (1927). 12.3 Humbert, P., Potentials et Prepotentials. Paris, 1936. 12.4 Nicolescu, M., Les Fonctions Polyharmoniques. Paris, 1936. 12.5 Walsh, J. L., Maximal convergence of sequences of harmonic polynomials. Annals of Math. 38, 321-354 (1937). 12.6 Miranda, C., Formule di maggioritzione e teoreme di esistenza per le funzioni biarmoniche di due variabili. Gwrn. mat. Batlaghi 78, 97-118 (1948). 12.7 Walsh, J. L., Sewell, W. E., and Elliott, H. M., On the degree of polynomial approximation to harmonic and analytic functions. Trans. Am. Malh. Soc. 67, 381420 (1949). 12.8 Fichera, G., Sulla maggiorazione dell'errore di approssimazione nei procedimenti di integrazione numerica delle equazioni della fisica matematica. Rend. accad. rci. $5. e mat. Soco reale Napoli [4] 17, Pubbl. Ist. nazl. appl. del Calcolo, Roma No. 289 (1950). 12.9 Collatz, L., Fehlerabschatzung bei der ersten Randwertaufgabe bei elliptischen Differentialgleichungen. 2. angew. Math: Mech. 39,202-211 (1952). 12.10 Griinsch, H. J., Eine Fehlerabschiitzung bci der dritten Randwertaufgabe der Potentialtheorie. 2. angew. MaM. Mech. 32, 279-281 (1952). 12.11 Shaginyan, A. L., On approximation in the mean by harmonic polynomials (Russian). Doklady Akad. Nauk Armyan. S.S.R. 19, 97-103 (1954). 12.12 Collatz, L., Numerische und graphische Methoden, in Handbuch der Physik (S. Flugge, ed.), Vol. 2. Springer-Verlag, Berlin, 1955. 12.13 Nehari, Z., On the numerical solution of the Dirichlet Problem, in Proc. Con!. Differential Equations, pp. 157-178. Univeraity of Maryland, College Park, Maryland, 1956. 12.14 Hochstrasser, U., Sumerical experiments in potential theory using the Nehari estimates. Malh. Tables Aid8 C'omput. 1 9 , 2G-33 (1958). 12.15 Duffin, It. J., and Nehari, Z., Note om Polyharinonic Functions, Tech. Report No. 32. Carnegie Inst. Technol., Pittsburgh, Pennsylvania, 1960. 12.16 Agnion, S., Maximum theorems for solutions of higher order elliptic equations. BUZZ. Am. Math. SOC.66,77-80 (1960). 12.17 Bramble, J. H., and Hubbard, B. E., A class of higher order integral identities with application to bounding techniques, in press. 12.18 Bramble, J. H., Hubbard, B. E., and Payne, 1,. E., Bounds for solutions of mixed boundary value problems for second order elliptic partial differential equations, in press. 12.19 Bramble, J. II., arid Paync, I,. E., New hounds for the deflection of elastic plates, in press. 12.20 Bramble, J. H., and Paync, L. E., Bounds for solutions of mixed boundary value problems for elastic plates, in press. , SECTION

13. Collocation and Interpolatory Methods

13.1 Poritzsky, IT., and Danforth, C. E., On the torsion problem, in Proc. 3rd U.S. Natl. Congr. of Applied Mechanics, pp. 431-441, New York, (1958).

130


13.2 Walsh, J. L., Solution of the Dirichlct problem for thc ellipse by interpolating harmonic polynomials. J . Math. Mech. 9, 193-196 (1960). 13.3 Curtiss, J. H., Interpolation with harmonic and complex polynomials to boundary valucs. J . Math. Mech. 9, 167-192 (1960). SECTION

14. Conformal Mapping

14.1 Szrgo, G., Uber orthogonale Polynomc, die zu einer gegebenen Kurve der koniplexen Ebene gehoren. Math. Z. 9,218-270 (1921). 14.2 Bergman, S., ffber die Entwitbklung der harmonischen Funktionen der Ebene untl des Raumes nach Orthogonalfunktionen. Math. Ann. 86,237-271 (1922). 14.3 Bochner, S., Uber orthogonale Systeme analytischer Funktionen. Math. Z. 14, 180-207 (1922). 14.4 Fekete, M., Uber die Verteilung der Wurzeln bei gewissen algebraischen Gleichungen mit ganzzahligen Koeffizienten. Math. Z. 17,228-249 (1923). 14.5 Carleman, T., uber die Approximation analytischer Funktionen durch lineare Aggregate von vorgegebenen Potenzen. Arkiv Mat. Astron. Fys. 17 (1923). 14.6 Nehari, Z., On the numerical computation of mapping functions by orthogonalization, Proc. Natl. Acad. Sn‘. U.S.37, 369-372 (1951). 14.7 Kliot-Dashinskiy, M. I., On a method of solution of a plane problem of potential theory (Russian). Inthen-Stroit. Inst. Sb. Nauc. Trudou (Leningrad) 17, 11-27 (1954) * 14.8 Behnke, H., and Sommer, F., Theorie der analylischen Funktionen einer lcomplexcn Veranderlichen. Springer, Berlin, 1955. 14.9 Fckete, M., and Walsh, J. L., On the asymptotic behavior of polynomials with extremal properties and of their uses. J . Anal. Math. 4, 49-87 (1955). 14.10 Walsh, J. L., Interpolation and Approximation by Rational Functions i n the Coinplex Domain, rev. ed. Am. Math. Soc., Providence, Rhode Island, 1956. 14.11 Davis, P., Numerical computation of the transfinite diameter of two col1ine:ir line segments. J . Research Natl. Bur. Standards 58,155-156 (1957). SECTION

15. Quadratic Functional8 Related to Boundary Value Problem

15.1 Trefftz, E., Ein Gegenstuck zum Ritzschen Verfahren, in Proe. 2nd Intern. Congr. for Applied Mechanics, 131-137. Zurich, 1926. 15.2 Friedrichs, K. O., Die Randwert und Eigenwert Probleme aus der Theorie der elastischen Platten (Anwendung der Direkten Methoden der Variationsrechnung). Math. Ann. 98, 205-247 (1928). 15.3 Trefftz, E., Konvergenz und Fehlerschiitzung beim Ritzchen Verfahren. Math. Ann. 100,503-521 (1928). 15.4 Friedriehs, K. O., Ein Verfahren der Variationsrechnung das Minimum eines Integrals alu dlts Maximum eines anderen Ansdruckes darzustellen. Nachr. Gca. Wiss. Gbttingen Math.-Physik. K l . I 13-20 (1929). 15.5 Weber, C., Eigengrenzung von Versehiebungen mit Hilfe der Minimalsatze. Z. angew. Math. Mech. 99,126-130 (1942). 15.6 Courant, R., Variational methods for the solution of problems of equilibrium and vibrations. Bull. A m . Math. SOC.49, 1-23 (1943). 15.7 Diaz, J. B., and Weinstein, A., Schwarz’ inequality and the methods of RayleighRitz and Trefftz. J . Math. and Phys. 96, 133-136 (1947). 15.8 P61ya, G., lhtinixting electrostatic capacity. A m . Malh. Monthly 54, 201-206 (1947).

ADVANCES

IN ORTHONORMALIZING COMPUTATION

131

15.9 Prayer, I%'., and Syngc, .I. L., Approximat,ions in elasticity based on the concept of frinct.ion space. Q w r t . Ap1~l.Math. 5, 241-269 (1947). 15.10 Topolyanskiy, D. B., On bounds for Dirichlet's integral. Prilclad. Mal. i Mekh. 11 (1947). 15.11 Dim, J. B., and Greenberg, 11. J., Upper and lower bounds for the solution of the first biharmonic boundary veluc problem. J. Math. and Phys. 97, 193-201 (1948). 15.12 Diaz, J. B., and Greenberg, H. J., Upper and lower bounds for the solution of the first boundary value problem o f elasticity, Quart. Appl. Math. 6, 326-331 (1948). 15.13 Diaz, J. B., and Weinstein, A., The torsional rigidity and variational methods. A,tn. J . Malh. 70, 107-116 (1948). 15.14 Greenberg, H. J., The determination of upper and lower bounds for the solutions of the Dirichlet problem. J . Math. and Phys. 27, 161-182 (1948). 15.15 Greenberg, H. J., and Praget, W., Direct determination of bending and twisting moments in thin elastic plates. A m . J . Math. 70, 749-763 (1948). 15.16 Weinstein, A., New Method8 for the Estimation of the Torsional Rigidity, Proc. 3rd Symposium in Appl. Mathematics (Am. Math. SOC.),1949, 141-161. McGrawHill Book Co., New York, 1950. 15.17 Fichera, G., Sulla maggioraeione dell'errore di approssiniaeione nei procedimenti di integrazione numerica delle equaeioni della Fisica Matematica. Rend. accad. sci. JS. e mal. Soc. nazl. sci. Napoli [4], 17, 1-8 (1950). 15.18 Fichera, G., Risultati concernenti la risolueione delle equaeioni funzionsli dovuti all'Istituto Nazionale per le applicaeioni del Calcolo. Atti. accad. nazl. Lincei, M e m ~CZasse . sci.fis. n d . e nat. I81 3,3-81 (1950). 15.19 Fichera, G., On some general integration methods employed in connection with lincar differential equations. J . Math. and Phys. 29, 59-68 (1950). 15.20 Funk, P., and Berger, E., Eingrenzung fur die grosste Durchbiegung einer gleichmiissig belasteten eingespannten quadratischen Platte, in FederhoferGirkmann Festschrijt, pp. 19+204. F. Deuticke, Vienna, 1950. 15.21 Mykhlin, S. G., Direct Methods in Matheinalieal Physics. Moscow, 1950. 15.22 Picone, hI.,arid Fichera, G., Neue funktionalanalytische Gruridlagen fur die Existenzproblenie und Losungsmethoden von Systemen linearer partieller Differentialgleichungen. Monalsh. Math. 54, 188-209 (1950). 15.23 P6lya, G., and Weinstein, A., On the t,orsional rigidity of multiply connected cross-sections. Ann. Math. [2] 52, 154-163 (1950). 15.24 Diae, J. B., Upper and lower bounds for quadratic functionals, in Proc. Symposium on Spectral Theory and Differential Problems (Oklahoma Agri. and Mech. College), pp. 27S289. Stillwater, Oklahoma, 1951. 15.25 Diaz., J. B., Upper and lower bounds for quadratic functionals. Colleclanea Malh. (Semirrnrio mat. de Barcelona) 4,l-50 (1951). 15.26 P61ya, G., and Szego, O., Isoperimetric 1nequ:tlities in hlathenistical Physius, Ann. Math. St,udies No. 27. Princeton Universit,y Press, Princctori, New Jersey, 1951. 15.27 Iteitan, D. K., and I-Iiggiiis, T. J., Calculation of the electrical capacitance of :L cube. J . Appl . Phys. 92,223-226 (1951). 15.28 Bertolini, F., Sulla capacith di un condensatore sferico. Nuouo ci,naento 9, 852-851 (1952). 15.29 Cooperman, P., An extension of the method of Trefftz for finding local bounds on the solutions of boundary value problems, and on their derivatives. Quart. Appl. Math. 10, 359-373 (1952). 15.30 Diax, J. I3., On the estimation of t,orsional rigidity and other physical quantities,

132

PHILIP J . DAVIS AND PHILIP RABINOWITZ

259-263 in Proc. 1st U.S. Natl. Congr. of Applied Mechanics (Am. SOC. Mech. Engs.), 1952. 15.31 Gross, W., Sul calcolo della capacith. elettrostatica di un conduttore. Atti. accad. nazl. Lincei Rend. Classe sci. fis. mal. e nat. [8] 19,496-506 (1952). 15.32 M y k h h , S.G., The Problem of the Minimum of a Quadratic Functional. Moscow, 1952. 15.33 Slobodyanskiy, M. G., Estimate of the error of the quantity sought for in the solution of linear problems by a variational method. Doklady Akad. Nauk S.S.S.R. [N.S.] 86,243-246 (1952). 15.34 Daboni, L., Applicazione a1 caso del cub0 di un metodo per eccesso e per difetto della capacith, elettrostatica di un conduttore. Atti. accad. natl. Lincei Rend. Classe sn'. $8. mat. e nat. [8], 14,461466 (1953). 15.35 Kato, T., On some approximate methods concerning the operators T * T . Math. Ann. 196,253-262 (1953). 15.36 McMahon, J., Lower bounds for the electrostatic capacity of a cube. Proc. Roy. Irish Acad. 55A, 133-167 (1953). 15.37 Washizu, I . Ihvironment . . . . . . . . . . . . 14. Heating Effects . . . . . . . . . . . F. Cathode Formation and Properties . . . . . . C. . Vacuum Encapsulation . . . . . . . . . 11. (!oinponcrit Lifetime . . . . . . . . . . I . Solid Stat(&'l'un~irl1CtTcc.t 1)evicw . . . . . .

. . . . .

. . . . 137 . . . . 137 . . . . 139 . . . . 142 . . . . 141 144

. .

. .

. .

. .

. .

.

.

.

.

. 145

145

. . . . . 146 . . . . . 146 . . . . . 147 . . . . . 148 . . . . . 148 . . . . . 14!1 . . . . . 150 . . . . . 150 . . . . . 150 . . . . . 156 . . . . . 158 . . . . . 15!) . . . . . 1G3 . . . . . 163 . . . . . 165 . . . . . 1G6 . . . . . 167 . . . . . 172 . . . . . 175 . . . . . 178

* This article was prepared with the sponsorship of the Information Systems Branch, Ofice of Naval Research, under contract Nonr-2887(00) . Reprotkuclion i n whole OT ifl part as permilled f O T any purpose of the United Stales G'overnmenl 135

.

KENNETH R . SHOULDERS

136

V . Accessory Components

. . . . . . . . . . . . . . .

. . . . . . . A. Secondary Emission Devices B. Light Detectors . . . . . . . . . . . C. Light Generators . . . . . . . . . . . D. Micro Document Storage . . . . . . . . . E. Electrostatic Relays . . . . . . . . . . F. Electromechanical Filters . . . . . . . . . VI . Component Interconnection . . . . . . . . . A. Vacuum Tunnel Effect Amplifiers . . . . . . . B . Memory Devices . . . . . . . . . . . C. Electromechanical Components . . . . . . . . . . . . . . . D. Steerable Electron Guide E. Plasma System . . . . . . . . . . . . VII . Sitbstrate Preparation . . . . . . . . . . A. Mechanical Formirig . . . . . . . . . . B. Substrate Cleaning . . . . . . . . . . C. Substrate Smoothing . . . . . . . . . . D. Terminals . . . . . . . . . . . . . E. Substrate Testing Methotls . . . . . . . . VIII . Material Deposition . . . . . . . . . . . A . Thermal Evaporation . . . . . . . . . €3 . Substrate Heater . . . . . . . . . . . C . Reactive Deposition . . . . . . . . . . 1). Single Crystal Growth . . . . . . . . E. Instrumentation Methods . . . . . . IX. Material Etching . . . . . . . . . . . A. Molecular Beam Etching . . . . . . . . . B. Atomic Beam Etching . . . . . . . . . C. Ion Beam Sputtering . . . . . . . . . . D. Depth Control . . . . . . . . . . . . X. Reairct Production . . . . . . . . . . . . A. Evaporated Resists . . . . . . . . . . B. Chemical Decomposition Resists . . . . . . C . Multilayer Resist Production Methods . . . . . D. Compatibility with Electron Optical System . . . XI . Electron Optical System . . . . . . . . . . A. Micromachining Mode . . . . . . . . . B . Scanning Electron Microscope . . . . . . . C. Scanning X-Ray Fluorescence Probe . . . . . D . Mirror Microscope . . . . . . . . . . . E . Multiple-Cathode Field-Emission Microscope . . . F. Pattern Generator . . . . . . . . . . G . Construction Details . . . . . . . . . . XI1. High-Vacuum Apparatus . . . . . . . . . . . A . Requirements . . . . . . . . . . . . B . High-Vacuum Apparatus . . . . . . . . . C. Ultrahigh-Vacuum System . . . . . . . . D Integration of Apparatus . . . . . . . . XI11. Electron Microscopo Installutior! . . . . , . .

.

180

. . . . . 180 . . . . . 183 . . . . . 184

.

.

.

.

. 185

.

.

.

.

.

188

. . . . . 189 . . . . . 190 . . . . . . 190

. . . . . 193 . . . . . . . . . . . .

. . 195 . . 195 . . 197 . . 197

.

.

.

.

.

.

.

.

. . . .

. . . . . . . . . . . .

. . . . . . . . . . . . . .

. .

. .

.

.

.

.

.

.

.

.

.

. . . . . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. .

. . . . . . . . . . . . . . . .

. . . .

.

.

. . . . . . . . . . .

.

.

.

. . . . .

. . . . .

.

.

. . . .

. . . .

. 198 . 199 . 201 . 202 . 203 . 204 . 205 . 213 . 215 . 221 . 222 . 224 . 224 . 2'28 . 229 . 229 . 230 . 231 . 232 . 234 . 235 . 236 . 237 . 246 . 249 . 251 . 253 . 25'4 . 256 . 260 . 260 . 261 . 265 . 273 . 275

MICROELECTRONICS

S I V . Domonst.rat,ionof hlirroma(.tiining A. Substrate Prcpn.ratiori . . . . . . 13. Film Drposition . . . . . . . . C. Resist Exposing . . . . . . . . D. Etching. . . . . . . . . . . E. Disciission of IlrsiiltR . . . . . . . S V . Summary . . . . . . . . . . . A. hIicrorlect,ronic Component C'onsidcr,ztions B. Tunnel Effect Components . . . . . C. Accessory Components . . . . . . D. Component Interconnection . . . . . E. Substrate Preparation. . . . . . . F. Material Deposit,ion . . . . . . . G. Material Etching . . . . . . . . H. Resist Production . . . . . . . . I. Electron Optical System . . . . . . J. Vacuum Apparatus . . . . . . . References . . .

137

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . . .

. . . .

. . . .

. . . . . . . .

. 27G

. 277 277 278 280 281 283 283 . . . . 285 . . . . 2% . . . . 286 . . . . 287 . . . . 287 . . . . 287 . . . . 288 . . . . 288 . . . . 289 . . . . 289

. . . . . .

1. Introduction

A. General Concepts

It has become increasingly apparent that electronic data processing machines have been applied as of now to only a few relatively simple problems, compared to the vast array of complex problems of the future. It is conceivable that in only a few machine generations, TIC will be aided by complex machines designed to perform economically many of the functions of a n intelligent technician. Machines of the kind that we postulate might well require over lo1' electronically active components-a number comparable in size to that of highly developed biological systems. The greatest single obstacle to fabricating machines complex enough to cope with future problems is our iiiabilit,y to organize matter effectively. Many pertinent basic electronic effects are now known and usable; more will undoubtcdly become acceFsible. To use these effects for the construction of complex systems, processes permitting the manipulation of matter must proceed a t video frequency rates with due regard to the specialized properties of electronic materials. If we are to build machines to permit performance approaching-and perhaps exceeding-that of biological systems, why not attempt to use biological building methods and biological information-processing methods? After all, the processes do work, and they can do so in a garbage can without supervision. Nevertheless, it is suggested that there are other means within the scope of our developing technology that are more suitable to our present,

138

KENNETH

R. SHOULDERS

and future needs. Let us examine a few of the major differences betwrrn biological organisms and our proposed electronic systems. We would like to remove the severe environmental limitatioiis of biological systems. Preferably, our proposed machines should be able to function in a varuum, in corrosive atmospheres, i n a furnace, in Conditions of strong nuclear radiation, and under the forces induced by high acceleration or any other disorganizing conditions. Obviously, such machines should give superlative performance under what is presently considered “normal” environmental conditions-for example, in such an important matter as long lived stability. Furthermore, such environmental immunity should permit efficient coupling to the new and more energetic energy sources that are becoming available. The rugged environment suggested above would eliminate most, if not all, biological procesees. There are, however, electronic effects and materials that can survive and function in such an environment. We would like to pass on-from one generation of ninchinc to the nexta i l organization that has been acquired or “learned” by the previous generation. In the higher biological species, offspring must learn by personal experience much of the information accumulated by their predecessors. I n man, the learning process may take a third of man’s useful life-span. It would appear advantageous for our projected electronic machine to have built-in, initially, a considerable amount of previously acquired organization; it, should also have the additional capability of reorganization and expansion. This impoees on the building process a necessary capability for passing detailed data on to the machine that is being built. Data in the form of instructions as to components, interconnections, and states appears to be needed; a previously organized machine would serve as a model. From the foregoing considerations, important constraints have been imposed on the selection of building processes, ueable electronic effects, and materials. The process must be such that all the fabrication data of a successful predecesEor can be communicated to the successive generations of machines, in an economical length of time and with a minimum of human mpervision. The fabrication processes themselves should be rapid enough to tic able to take advantage of the very high rates of electronic communication now available. It appears mandatory to use parallel processingprocessing that would involve the fabrication and interconnection of many components simultaneously. The ability to function in extremely severe environments demands the selection of refractory metals and dielectrics possessing the highest binding energies. The properties of these materials make normal mechanical or thermal methods of fabrication for complete machines too difficult to cope

MICROELECTRONICS

139

with ;subtle chemical reactions, which can replacc “brute force” mechanical working or thermal burning, are required. Cleanliness, in extreme form, is imposed as a necessary condition for reliable long-term operation. Such cleanlincss of process can be achieved a t present only in ultrahigh vacuum in which material is transported in elemental or vapor form during fabrication. What size ehould our proposed machine be? Let us assume that the desired 10” electronic devices are built layer by layer on two-dimensional surfaces; then the average size per device is determined principally by the resolution of the process that can selectively localize necessary materials. The process to be presented here has achieved a resolution of several hundred angstrom units. From this consideration, and others described later, the size of the ultimate machine appears to lie in the range between one and several cubic inches. One may speculate on the implications of the portability of such a complex machine. It would appear to be possible for it to accompany a human-acting as a highly organized clerk-in order to aid him in solving with unmatched speed and precision complex problems a t the site of the problem. Direct communication between man and machine, in a language understood by man, through optical, acoustical, or other means compatible with the human senses, would be essential to surmount the artificial barriers usually imposed. Looking beyond the boundaries of this program, it becomes apparent that ultimately we would wish to divorce ourselves from the limitations imposed by the low binding energies of solid materials. We require organization of matter to effect order, permitting growth in our intelligence machine; a t the same time, increased operating rates imply increased temperatures, with an ensuing tendency to disorganize. To surmount the low binding energies of solids, it may be possible to construct plasma-like structures in which elementary particles are organized spatially and temporally in dynamic equilibrium, under the influence of applied fields. Several aspects of the present program can be extended to form a natural transition into this postulated class of machine. In summary, we would like to build machines operating a t electronic speeds, almost completely immune to the effects of a harsh environment, with components and versatility sufficient to attain a critical degree of organized complexity such that the machine itself may participate in the further development of ever more powerful machines. 8. Background of the Present Program

This report is a qualitative and semiquantitative discussion of the various problems that are being dealt with in an effort to explore the possibilities and limitations of a class of integrated microelectronic devices, systems, arid

140

K E N N E T H R. SHOULDERS

fabrication methods. It is hoped that this review will help others select a similar field of endeavor and that some of the devices and techniques discussed here will have direct application. It is recognized that this project is a very ambitious one and that the end goal will not be reached for some time. On the other hand, the author does not feel compelled to deviate from the course he has taken of pursuing the end goal, rather than directing effort toward interim devices and methods; the latter approach would surely lengthen the time required to attain the more desirable goal. I n the beginning of the writer’s work in this field, vacuum material manipulation methods were used to fabricate resistors, capacitors, inductors, vacuum tubes having secondary emission and field emission cathodes, photoconductors, and electrostatic relays. Masking methods were used with conventional thermal evaporation sources and the resolution of these masks was carried to an extreme degree in order to investigate the properties of the microelectronic devices. No important deviations from the expected characteristics of the devices were found due to scaling down the size, but many material deposition limitations of masking and thermal evaporation methods resulted in poor over-all results for the components. Various unsuccessful methods were tried in an effort to overcome the masking limitations. Among these were the application of ion beam techniques, xerographic methods, optically produced thermal effects, and optically produced chemical resists in vacuum. The use of electron beams bccame the most promising method of producing the desired localized images on a surface during the micromachining mode and also for the analysis of various effects produced on the surface. Among the first methods tried, a n electron beam was used to bun1 through a low tempemturc masking material such as condensed carbon dioxide, but the energy density requiremeiit was very high and breakdown of adjacent arcas resulted from inadvertent electrical discharge. Later, oil films were polymerized on various surfaces in a fashion described by Carr [ 11 and the resulting composite was developed in vapors of substances such as mercury and zinc. Although interesting patterns could be produced, they had very limited electronic application. It was then discovered that silicone pump oils could be decomposed to silica by the action of an electron beam, and that this resist layer of silica could serve to protect an underlying film of material during a subsequent etching process. Eventually, this etching process was carried out in a vacuum chamber a t reduced pressure and became compatible with the deposition and resist production operation. At the present time the basic micromachining process is this: to deposit a material to be selectively removed later; to deposit a thin layer of resistproducing material and to expose this with elcctrons in some desired

MICROELECTRONICS

141

pattern; then filially l o etch thc first lnycr of material with :t mol(wd,zr heam of an element that inakcs a volatile compound of thc mixterid bring machined. Ilcfinemcnts in apparatus and tcchniqucs hnvc produccd rcsolutions in excess of 100 A for the process with very short csposure times a t low current density for the electron beam. Vacuum apparatus was developed for the processing of the chemicals that had been introduced into the system. This consisted essentially of providing w r y high bakeout temperatures for the apparatus and using materials that were not affected by the chemicals. A great deal of mechanical manipulation is used in the various phases of the process. To introduce motion it has been found advantageous to use differential pumping methods in which the ultrahigh-vacuum chamber is surrounded by a vacuum of mm Hg in which “0”rings for the many entry ports can he used. The inner ultrahigh-vacuum chamber, on the other hand, which can he raised to temperatures of 900°C, does not require the use of such seals. I n this way the manipulation can be introduced into the ultrahighvacuum chamber through small holes without undue leaks or contamination from organic materials. The design of an electron optical system has been undertaken with the aim of providing the many functions of analysis and micromachiniirg that are needed without impairing the ultrahigh-vacuum condition desirable throughout the processing of a system of electronic components. Deposition equipment has heen improved; it is nearing the point where knowing what to do with it is of more concern than further improvements. The chief effort in the deposition equipment design is in iiitegrating it) properly with the vacuum apparatus and other equipments so that it is not limiting their performance. The design of a mass spectrometer is being undertaken, and upon completion it will be incorporatcd in the process control of the deposition apparatus. Aftcr working with and considering many components of thc film typesuch as cryotrons, magnetic devices, and semiconductors-for application to microelectronic systems, it appears that devices based upon the quaiitmi mechanical tunneling of electrons into vacuum (field emission) possess many advantages. They seem relatively insensitive to temperature variations and ionizing radiation and they permit good communication with other similar components in the system as well as with optical input and output devices. The switching speeds seem reasonably high, and the devices lend themselves to fabrication mcthods that could economically produce large uniform arrays of interconnected componeiits. These componeiits are bascd on the phenomena of field emission into vacuum, which has been under investigation for many years by competent people and has a firm scientific

142

KENNETH R. SHOULDERS

basis. In our future discussion of thc hasic emission process the term “ficltl emission" will usually bc used, and thr complete roniponrni will he referred to as a “tunnel effect component.” At prcsent the phase of integrating many isolated hits of thought and apparatus has been reached. The isolated bits seem to fit together in a n interesting pattern; however, only by actually going through the motions of physical tests can the complete plan be verified or modified to accomplish thc desired end. To accommodate unforeseen changes in any of the various aspects of the program, the maximum flexibility in equipment design has been sought. Flexibility tends to increase the initial effort and cost, hut] this should be the most economical course in the long run. The author would like to acknowledge the invaluable technical and moral support of his many associates through the past years, as well as the financial support of Stanford Research Institute; the Information Systems Branch of the Office of Naval Research; the Electron Tubes Division, Techniques Branch, of the U. S. Army Signal Research and Development Laboratory; and the Solid State Branch of the Air Research and Development Command, Air Force Cambridge Research Center. C. Objectives

The general objectives in this area of research are to develop the proper apparatus and techniques for electron-beam-activated micromachining and to apply these tools to the fabrication of integrated groups of electronic components. Ultimately, such a program, aided by logic and system studies, will lead to the fabrication of complete machines that could be used for real-world general pattern recognition, real-time language translation, and should have problem-solving and decision-making capabilities which are not available in present machines. The specific objectives for this program would be to first obtain the proper tools for electron-beam-activated micromachining, such as the electron optical system, ultrahigh-vacuum apparatus, and mass spectrometer controlled deposition and etching apparatus. Following the development, of these tools, the techniques for micromachining would be developed to an advanced stage while being applied to the fabrication of small groups of selected electronic components. The components to be developed would he selected to integrate well with themselves, their construction process, and the microscopic size range in which they must function. These components, based largely upon the quantum mechanical tunneling of electrons into vacuum, would seem to suffice for high-speed computer switching and memory, provide low noise amplification and communication filtering, and

MICROELECTRONICS

143

give optical coupling means in and out, of the system. The specific objectives are listed as follows: ( I ) Provide an electron optical system capable of analyzing the physical, chemical, and electrical properties of microscopic components so that they can he optimized without undue difficulty in spite of their reduced size, This electron optical system u ould consist of a mirror microscope, xannirig electron microscope, and X-ray fluorescence probe. The mirror microscope would be capable of voltage measurements down to 0.2 volts and have a maximum rePolution of approximately 500 A. The scanning microscope would h a w a resolution limit of 100 A. The X-ray fluorescence probe would have the ability to analyze 10-13 grams of material to 1% accuracy. (2) Integrate the electron optical system with mabs spectrometer coiltrolled deposition and etching apparatus in an ultrahigh-vacuum system capable of reaching 10-lGmm Hg in one-half hour or less. (3) Deviee vacuum tunnel effect devices of micron size with switching times in the 10-lo Fec region which (a) operate a t about 50 volts (b) have high input impedance (c) are insensitive to temperature effects u p to 1000°C (d) are insensitive to ionizing radiation effects up to the limits of the best knovn dielectric materials, and (e) have a useful lifetime of many hundreds of years. (4) Devise light sensing devices of fractional niicroii size based upon photoelectron emission from metal-dielectric matrices under high electric field, which have a frequency response in the range of 100 Mc/sec, and which have environmental immunity similar to the tuniiel effect devices. (5) Devise cathodoluminescent light generating sources of fractional niicroii size based upon the stimulation of a refractory phosphor by field-emitted electrons. (6) Deviee electromechanical devices ubing microscopic diaphragms operated by electric fields in the lo7volt/cm region for low-voltage relays and communication filters. (7) Deviw modular groups of interconnected vacuum tunnel devices which take advantage of the large dynamic range of lolo in operating current to permit very low power dissipation in the quiescent operating states. These circuits would t e specifically designed to eliminate lowvalue plate resistors in active memories in order that the power dissipation levels of both states of two-state devices are approximately 1O'O below the dissipation level during switching. (8) Investigate the limitations of periodically focused, electrically steerable, electron beam guides to provide interconnections between active components which will be less lossy than submicron sized wires, and

144


to reduce power requirements and heating effects by conducting a n electron bunch or pulse through relatively long paths giving up energy only upon termination; making use of the interconnection flexibility such that a considerable amount of logic can be performed by interconnection control rather than by dcpendcnce on the logical properties of the active devices only. (9) Develop eclf-formation nicthods for active components in which a chemical process, during formation, is controlled by a significant electrical property of the component in such a way that the component characteristics are automatically modified to conform to some previously specified range of operation. (10) Develop a document storage system by using electron-beamactivated micromachining processes to store information in the form of 1O'O defined areas on a one-inch-square glass plate, each area representing ten optical intensity levels-to be read by fractional micron-sized lightsensing devices.

D. Time

Schedule for Various Objectives

The work outlined above will not contribute to the nest generation of machines made from micromodule components. I n the following generation of devices, employing solid state integrated techniques, there may be some slight interaction with electron-beam-activated machining methods such as the selective masking of semiconductor surfaces for control of diffusion; the various electron beam analysis tools may also contribute information about the microscopic behavior of diffusion processes and give data on surface effects. The third generation of machiiies would seem to be the one to which these techniques and devices would begin to contribute strongly. A great deal of effort is needed to carry this program to the point of showing completed systems being fabricated in a few hundred hours, but one thing that should become evident with such an electronic construction process is that there would be a very short delay between engineering and manufacture of electronic systems made by these methods. No tooling u p is needed for this electronic material handling process othcr than the construction of additional identical processing chambers and auxiliary control equipment. II. Research Plan

The problems listed iii the outliiic that follows represent most of the areas of investigation that were indicated at the beginning of this work. Initially, most of the scientific problems were worked on in a superficial

MICROELECTRONICS

145

or qualitative fashion iii ordcr to cxplorc economically thc wide latitudc of problems and efficiently converge on desired results. This exploration has removed the need for work in a great many areas and has shown the confines of the experimental work, which in turn allows the design and construction of integrated equipment without the fear of obsolescence. Future phases of the work will begin to fill in scientific gaps and generate new problems which cannot be seen in this first phase. The following paragraphs detail in summary form the gcneral work plan:

,4.General Component Considerations I . Application of scaling laws to determine optimum size range for various electronic components for complex data processing systems. 2. Investigation of manufacturing feasibility for various one-micronsized electronic components.

13. Tunnel Effect Component Design and Interconnection 1. Component Design

(a) Determination of area of application for solid state and highvacuum devices. (b) Determination of optimum impedance for vacuum tunnel effect devices and device of a geometry that optimizes gain and bandwidth while minimizing current intercepted by control and screen grids.

2. Interconnections and Circuits (a) Design of low-loss circuits having negligible quiescent power dissipation. (b) Investigation of direct coupling, secondary emission coupling, and optical coupling methods to provide minimum constraints for logic operations. (c) Investigation of periodically focused electron beam guides to be used as electrically steered communication paths in microelectronic systems. (d) Investigation of electron beam parametric amplifiers for lownoise amplification in microelectronic systems.

3. Fabrication (a) Determination of stability of metal-dielectric combinations under operating conditions involving high thermal shocks and intense electron bombardment.


1 46

(1)) Investigation of encapsulation effcctivcncss using thin filnis

made by reactive deposition methods. (c) Investigation of self-formation mcthods for vacuum tuniicl effect devices having multiplc-tip cathodcs.

4. Test Methods (a) Use of field emission microscopy and field ion microscopy to determine physical, chemical, and electrical properties of materials used in tunnel effect, devices. (b) Development of electron mirror microscopy to show operating voltages and switching waveforms of microelectronic components. C. Accessory Component Design 1. Investigation of high current density, low voltage secondary cmission surfaces for tunnel effect component interconnection. 2. Investigation of photoelectron-emitting light detectors coupled with current amplifying secondary electron multipliers. 3. Investigation of light emission from field emitted electron stimulated phosphors.

4. Investigation of microdocument storage system using fractional wavelength optical reading methods. 5. Investigation of electrostatic electromechanical relays.

6. Investigation of electrostatically driven electromechanical filters.

D. Substrate Considerations I . Materials Determination-Determination of materials to bc used and the optimum shape and size of the substrate.

2, Mechanical Preparation Methods

(a) Investigation of economical limits to grinding and polishing operations. (t)) Study of the effects of vacuum firing for normalizing polishrd surfaces. (c) Investigation of porosity of raw samples. 3. Terminals

(a) Study of the termination of thin films to gross terminals and their encapsulation to prevent corrosion a t high temperature.

147

MICROELECTRONICS

(1)) Investigation of sturdy strain-frcc lend wire connections to subst rat P .

4. Cleaning and Smootliirig

(a) Optimization of ultrahigh-vacuum electron beam cleaning and smoothing methods for substrate preparation and intralaycr smoothing.

(b) Application of test methods to indicate faulty surface.

E. Material Deposition 1. Thermal Evaporation-Determination the maximum cleanliness of deposits.

of methods for obtaining

2. Reactive Deposition (a) Determination of smoothness, crystalline properties, and electrical properties of various metals, metal carbides, borides, osides, nitrides, silicides, and sulfides. (b) Investigation of diffusion characteristics between adjacent layers of material.

3. Crystal Growth (a) Investigation of single crystal growth in presence of sweeping thermal gradient on substrate with and without reversible chemical reactions. (b) Investigation of reduction of crystalline size by alloying and admixing materials so as to obtain uniform and stable polycrystalline materials.

4. Instrumentation (a) Design and construction of X-ray fluorescence probe film thickness monitor. (b) Design and construction of ion gauge evaporation rate controllers. (c) Design and construction of rf mass spectrometer for evaporation rate monitors, etching control, and destructive analysis of film materials. (d) Utilization of field emission and field ion microscopy to determine cleanliness of deposits, activation energy, and stabilities of materials.

148

KENNETH

R. SHOULDERS

F. Material Etching 1. Molccular Beam Etching (a) Fundamental study of low pressure surface chemical reactions so as to allow prediction of proper etchants and temperatures. (b) Study of surface migration of etchant with intention of controlling undercutting of resist and material redistribution. (c) Investigation of multiple component etchants to increase vapor pressure of material being etched. (d) Investigation of side reactions produced with multiple component etchants. (e) Study of etching rate of various crystal faces and methods of causing uniform etching by selective adsorption of different materials.

(f) Investigation of depth control methods with and without chemical barriers. 2. Atomic Beam and Sputtering Etch Methods (a) Determination of effect of molecular beam etch a t a surface bombarded by a high current density electron beam which produces atoms and ions. (b) Study of electrical property damage resulting from sputtering and atomic beams.

3, Instrumentation (a) Design and construction of high temperature substrate heater that is chemically nonreactive. (b) Design and construction of accurate temperature regulating devices to control substrate heat. G. Resist Production 1. Classification-Classification of electron sensitive reactions and application of the ones best suited to micromachining. 2. Efficiency Limitations (a) Study of polymerization mechanism in simple reactions and electron multiplication process in multilayer complex reactions. (b) Analysis of maximum attainable efficiency vs. resolution in optimum process.

MICROELECTRONICS

149

( c ) Study of electron absorption process in resist producing materials.

(d) Study of efficiency as a function of temperature and electric field. 3. Resolution Limitations (a) Investigation of electron scatter processes and X-ray production and adsorption as fog-producing mechanisms. (b) Study of material migration due to thermal and field enhanced mechanisms. (c) Investigation of etch-back methods to reduce background fog effects. 4. Integration Investigations-Development of optimum resistproduction methods which integrate well with other system functions, such as noncontamination of lens and film materials and the adaptability of resist materials to vacuum handling methods. 13. Electron Optical System-Determination of the system most suited to micromachining and analysis, which includes making a comparison between magnetic and electrostatic instruments, cylindrical vs. spherical lens elements, and a select ion of optimum electron velocities for various modes of use. 1. Resolution and Sensitivity Limits

(a) Determination of rrsolution and coiitrast limit for scanning microscope.

(I)) Determination of accuracy, sensitivity, arid resolution limit for X-ray fluoresccncc probe. (c) Study of voltage sensitivity, bandwidth, and resolution limit of mirror microscope. (d) Determination of resolution limit, for multiple field emission cathode imaging microscope. 2. Distortion Liniitations (a) Investigation of distortioii effects xhich would limit uniforniity of field. (b) Determin:Ltioii of maximtiin 1iuml)cr of rcsolva1)lc bits per field. (c) Study of mechaiiical and electrical instability which would limit rcproducibility and registration.

150

KENNETH

R. SHOULDERS

(d) Study and suppression of contamination effects and stray charging. 3. Accessory Apparatus

(a) Design and construction of combiiicd X-ray detector and electron multiplier. (b) Design and construction of electrically operated substrate micromanipula tor. (c) Design and construction of pattern generator for micromachining. (d) Assembly of monitor console to integrate various display functions.

I. Equipment Integration 1. Ultrahigh-Vacuum Apparatus (a) Design and construction of rapid-access equipnieiit capable of being baked a t high temperature to remove chemical residue betwceii deposition and encapsulation operations. (b) Design and construction of contamination-free pumps for ultrahigh vacuum. 2. Valves-Design and construction of large diameter, ultrahighvacuum valves for separation of various operations having volatile reaction products. 3. Manipulation-Design and construction of multichannel manipulator having great, operational flexibility.

111. Microelectronic Component Considerations

A. General Considerations

1.

CONSTHUCTION PROCESS CONSIDERATIONS

Our over-all component size is constrained by the construction techniques to be within the limits of one-tenth of a micron and two microns. The lower limit is set by the resolution of' the machining process. The upper limit is indirectly determined by the maximum thickness to which films can be deposited with desirable properties. Films thicker than a few microns can be obtained, but i n this size range mechanical iorces come into play which tend to cause severe strain a t the film boundaries unless the thermal expansion coeficients of the various materials are matched. These strains

MICROELECTRONICS

151

niay not rcsult iii pecliiig of the filiris, hiit, tlicy ciihaiicc tliv diffuhion of forcign matcrial along boundaricx Thc dimensional aspect ratio of a d ivc devices is prcdoiniiitlully coiitrolled by consideration of the impedancc level, both of the device itself and of the coupling means between similar components. For semi-conductor and vacuum devices, this aspect ratio usually approaches unity. Thus since the component thickness will approximate one micron, the width and length will also approach one micron, and the resolution of the machining process must considerably exceed this value so as to produce the detailed structure needed. When the Component size approaches the resolution of the construction process there is a severe problem in obtaining uniformity. At times it may be possible to couple a self-forming process with the principal construction process and effect an improvement in over-all resolution without complicating the primary process. An example of this that can be found in normal machine processes might be the use of electropolishing to remove burrs caused by an inexact machine tool cutting process; the electropolishing does not necessarily affect the basic dimensions. Some components seem inherently simple in geometry but require properties of materials which are either difficult to obtain or are unstable. Other components require simple material properties, but have complicated geometry. A transistor is an example of the first class and a transformer or saturable reactor is an example of the second class. An ideal component taxes neither the geometry nor the materials. One reason for the selecqion of vacuum tunnel effect devices is that they seem to fit both of these requirements better than any class of component examined thus far. A single metal and a single dielectric, chosen particularly for their stability, seem to be the only materials needed to give a large range of electronic effects. The geometry is extremely simple because the device consists principally of the termination of the various wircs coming into the active area, as shown in Fig. 1 (p. 160). A choice is always possible between fabricating two-state or multilevel devices. The multilevel dcvices usually require a higher resolution construction process for any particular over-all size. The stability requirements for the materials may also be higher, but carcful analysis of each case secms in order, because occasionally the two-state device design has latent possibilities of being a multilevel device with the same stability, and therefore is not being used in an optimum fashion. For the present, we will be concerned only with two-state devices.

2.

SCALING O F ELECTRICAL PROPERTIES

This program is investigating the reduction of size of electronic components by about three orders of magnitude, namely, from one millimeter

152


to onc micron. This reduction i n sizo will bring almit c.limgc~sin tdia hhavior of materials, componcrits, mid thc m:Lnncr in which thc compoiictits are used. (a) Trunsrnission Line Loss. Oric of thc more unforturiate effects in reducing the size of electronic systems is thc increased loss experienced by transmission lines and tuned circuits. Assuming no change in resistivity, a transmission line made from a suitably stable material like molybdenum-3000 A diameter, one inch long-has 100,000 ohms dc resistance. Materials like copper or silver, having lower resistivity, are considered too unstable for this purpose. The 3000 A diameter is the largest convenient wire size that could be used with a 10,000 A, or one-micron-sized component. Any calculation of line loss should consider the possibility of high temperature operation of line sections rcsulting from cnergy dissipation during component switching. If conventional transmission lines arc to bc used for iiitcrroiinecting microelectronic componcrits then the components should have a high input impedance to avoid excessive current in the line and the attendant loss. There is no rigorous relation between the input impedance of a component and its output impedance, but the two are not usually independent. Thus most semiconductor and magnetic devices operate, input and output, a t relatively low impedance levels, while a few semiconductor and most vacuum devices operate at high impedance levels. With the high resistance transmission lines described above, the choice of high impedance level devices becomes mandatory if line loss is not to be excessive. For example, if one component is to couplc with another located 10,000 component diameters away, approximately one inch, then the input impedance of the most distant component should be a t least 100,000 ohms, considering only dc losses in the transmission line. Highcr input impedances would be required if ac losses were included. (b) Q of Circuits and Filtering. The Q of tuned circuits is drastically reduced when the components are scaled down uniformly by a factor N . The resistance R varies as 1/N, the capacitance C and inductance L vary as N , the product LC varies as N 2 , and the product RC is constant. The Q of a circuit can be defined as wLlR. If the frequency w remains the same, then the Q scales down as N 2 . Thus if all dimensions for a n inductor are reduced from 1 centimeter to 1 micron-a scale change of 10-4-the Q of the inductor would be reduced by a factor of lo", for the same frequency. It will be difficult indeed to perform filtering operations in the conventional manner unless other energy storage mechanisms are used. RC filters are not affected by scaling their size, but they require stable gain elements to achieve stable over-all performance. This stability may be achieved by degeneration of active elements up to the limit of stability

MICROELECTRONICS

153

imposed by the temperature sensitivity of the resistance elements. In scaling down a resistor all dimensions are assumed to be reduced, but in fact the present day resistors are already films so that it would be difficult to scale the thickness. The net result of this is that microresistors will either have to have very low resistance values, and thereby couple poorly to the active elements, or they will have to he made from materials of high resistivity. A n unfortunate circumstance of high-resistivity materials is that they are usually temperature sensitive to a very high degree, thus causing large changes in filter characteristics when used in a n RC network. Similarly, reactatnces-capacitive and inductive-may be produced by active semiconductor dcviccs, hut these devices also exhibit considerable teniperat ure instability . The very fast thermal time constants of micro devices could tcmperaturemodulate filters in the megacyclc region and if the filter is temperature sensitive a spurious signal results; i n addition, a drift is caused by slow heating. A useful filter for the micron size range can be devised, using bistable switching elements in a data processing unit which handles the incoming signals as digital data after appropriate time sharing and quantization circuits have sliced up the signal. These methods are not very good for handling small signals in the presence of large ones and may have to be coupled with drift-prone RC filters to improve the over-all results. Mechanical and acoustical filters scale nicely into the micron size range for radio frequencies. The most important rcquirement for these devices is that they be mechanically uncouplcd from the solid lattice that surrounds them. This problem is somewhat equivalent to the vacuum encapsulatioii problem where a membrane must be suspended free of the lattice over most, of its area. Effective driving and pickup transducers for these mechanical filter clrments can be clcctric field operated, being quite effective when subjected to bias fields of about lo7 volt/cm. Ultimately some of the molecular resoiiance arid adsorption properties of materials may come into UFC as filters, but none have been investigated in any detail. (c) Optinziiin Time Constant. On first examiliation it would appear that scaling down all dimensions of active componcnts would dccrease the RC time coilstant and therefore increase the system opcratiiig speed almost indefinitely. There is, however, a lower resistance set by efficient energy transfer requirements, i t . , the resistance should not be less than the maximum transmission line resistance between two interconnected devices. Furthermore, in most active devices the current rises faster than linearly with applied field, and the resistancc scales down more rapidly than linearly. Present, semiconductor tunnel diodes represent a component that exhibits

154


too low an impedance for a system composed of one-micron-sized devices. Although the vacuum tunnel tetrode, which is discussed in a later section, tends to be a high impedance device, it may be possible to reduce its impedance level to around the optimum of 100,000 ohms. Such a micronsized device will have an interelectrode capacitance of approximately 10-l6 farads, yielding a time constant of about 10-" sec. This value appears optimum when based upon efficient energy transfer between components, one inch apart. It is further to be noted that a one-inch transmission line operating with a velocity of propagation approaching that of light, has a 10-lo sec delay, a time compatible with the switching time of the above device. (d) Heating Eflects. The integration of a large number of components into a closely packed structure introduces thermal coupling between elements to a degree not experienced in large-scale systems. Thus, if a number of devices are to operate simultaneously, they must either operate a t considerably reduced power density, be spaced sufficiently far apart, or the circuit design must be such that large numbers of adjacent devices will never be energized simultaneously. Generally, high power density is required for high operating speed. Furthermore, low operating power density two-state devices usually are quite temperature-sensitive. For example, the cryotron is very temperature sensitive and is also a very low power density switching device; ferroelectric and ferromagnetic devices, which operate at medium power density, are fairly temperature sensitive; the power density and temperature sensitivity of semiconductor devices depend somewhat 011 the width of the energy gap of the material used. The vacuum tunnel effect device appears to be the only device which can satisfy most of the heating requirements. It operates at a very high powcr density and up to temperatures barely under those which destroy the refractory materials from which it is made. Individual micron-sized devices, having a large surface to volume ratio, and made of refractory materials, such as molybdenum and aluminum oxide, can be projected to operate continuously a t a power density of lo* watt/cm2 C2.31. It appears possible to operate LL vacuum tunnel device with a dynamic dissipation range of loiobetween active switching and quiescent states. If the dissipation during switching is watts for a 10-13 watt quiescent condition, then the quiescent power for 10" such devices would be watts, an entirely acceptable level. In comparison with the above, a conventional solid state tunnel diode, with a dynamic dissipation range of only 20 to 1, would involve a quiescent power dissipation of watts per device, and a total of lo6 watts quiescent dissipation for 10" devices. Solid state tunnel devices made from only metals and dielectrics, as will

MICROELECTRONICS

155

be discussed later, seem to leiid themselves to low standby powcr but would not exhibit a negativr resistance characteristic.

3.

SCALING OF MATERIAL PROPERTIES

For metallic films 3000 A thick the bulk rrsistivit,y values may be used. Traiismissioii lines haviiig 3000 A diiznicters may also exhibit bulk characteristics even though the surface arca and electron scatter is very high. Dielectric breakdown strength increases to loBvolt/cm for 3000 A thick films of material that arc properly prepared [4].This is primarily because the thickness is near the value for the electron mean free path in the dielectric. An electron avalanche is not likely to occur under such conditions and breakdown is forestalled. High field electrostatic devices can use this increased breakdown strength to advantage. As the size of our coniponents is scaled dowii the surface-to-volume ratio rises and surface-tension forces play a profound role in determining the shapes of the objects we can make. Materials tend to minimize their surface energies and assume a spherical shape instead of remaining sharply pointed. One of the limits to the smallness of the tip size that can be used for field emission cathodes is the blunting caused by material migration and surface tension forces at high temperatures [5, 61. Another limit that can frequently be seen is the way that a surface layer of material, that is to be diffused into a semiconductor device, tends to agglomerate and form into sniall droplets [7]. This is partly caused by impurities accidentally present a t the interface, and partly by surface tension forces. Special problems in crystallinity are encountered when dimensions are reduced to the few thousand angstrom unit region. Single crystal materials require an unusually high degree of perfection over a large area if pinholes, dislocations, or other anomalies, which could cause accidental diffusion of materials from one laycr to another, are to be avoided. Polycrystalline films are usually composed of arrays of single crystals with few grain boundaries parallel to the film surface thickness but with many grain boundaries perpendicular to the surface, permitting easy diffusion paths in the thickness direction. Polycrystalline films having many crystals in series in the thickness direction can be formed by evaporation methods, but these are unstable and tend to recrystallize a t elevated temperature to crystals having all sides approximately equal to the film thickness. By proper admixing of materials such crystal growth can be forestalled, but even with 300 A crystal size, which is considered small, there would be only 10 crystals in a 3000 A-thick film, and the possibility of having a direct diffusion path is large. Certain electronic devices, like semiconductors, have detrimental sur-

156

KENNETH

R. SHOULDERS

face recombination effccts which can adversely affect the electrical properties of the device with the reduction in thc sizc of these devices by thc ensuing increased surface-to-volume ratio. The surface under consideration includes not only the free surfacc but internal boundaries which represent discontinuities in properties.

B.

The Over-All System Specification

Many detailed considerations are needed to dictate the requirements of a complex data processing machine [7a]. At this early stage of the program the opinions formed about the details of the eventual machine are not likely to be firm conclusions; however, opinions are needed to guide the day-by-day activity. The tentative specifications for a complete machine are presented here in order to show why various endeavors have been undertaken. 1.

NUMBER OF COMPONENTS

Based on a number of coiisidcrations a maximum system sizc of 10" components, contained wit,hin a one-cubic-inch volume, has been selected. The most important single factor in thc selection of this size is the resolution of the machining process. Other factors that are strongly related to size are : electronic effects, interconnection problems, uniformity of manufacture, and stability of components. Furthermore, the economics of fabricating such a large system, with the methods to be described, have been considered and found to be reasonable. 2.

THE SHAPE AKD NUMBER O F LAYERS

A cube, one inch on a side, has been selected to house the data processing system because of the adaptability of this shape to the manufacturing process. Vacuum deposition mcthods lend themselves to the deposition of material on a singlc surfacc. Past cxperiments have shown the feasibility of depositing fifty layers of material 3000 A thick. More layers could conceivably be dcpositcd but the resulting unevenness of thc top layers is difficult to cope with. When thesc layers of materials arc used to make components, wiring, shields, and crossovers, approximately ten layers of components could bc made on each substrate from the 50 deposited layers. The thinnest substrate that would secm reasonable to handle in the polishing and lead wire connection operations would be 0.01 inch thick. The major dimensions of the substrate are belicvcd to bc optimum a t about one inch square. Larger sizes arc difficult to heat uiiiformly to the high temperatures needed for reactive deposition and arc too flexible during exposure by the electron image. Smaller sizcs are more desirable to work with but entail a rrduction of thc numbcr of lead wircs that can bc attached

MICROELECTRONICS

157

to the substrate as well as a reduction of the area available for optical communication to the machine. One hundred substrates would thus stack to a one-inch cube. Approximately lo8 one-micron-sized components can be accommodated on one layer of a one square inch substrate when a 25% packing factor for components is used. A cubic inch machine mould thus be composed of 100 substrates having tell layers or lo9 components per substrate, for a total of 10" components. 3.

EXTEItNAL COMMUEiICATION

Electrical, optical, and meclianical methods would bc available for coupling the microelectronic system to the surrounding world. Lead wires can be attached to each of the substrate edges and up to 80 wires per plate could be secured; however, the difficulty of attaching and accounting for these wires would tend to h i t the number to a smaller value. A few hundred megacycles bandwidth per wire appears attainable. These wires could serve as intersubstrate coupling and coupling to external devices. Optical coupling between the adjacm t substrates could serve to connect up to lo4parallel data channels with over 10 Mc of bandwidth per channel. It is estimated that approxiinatcly 10% of the components would be used as electrooptical generating and receiving t raiisducers. The light detecting device would be a simple photoemissive surface coniiectcd to a vacuum tunnel effect tetrode employcd as an amplifier. If more sensitivity is needed, a secondary electron niultiplicr could be interposed between the two units without undue complexity. l'he light generator would be a refractory phosphor stimulated by elect roils drawn from a field emission source and coiltrolled by a grid stiucturc in a triode geometry. It is necessary to have a t,hin transparent substrate in order to allow many channels of communicatioii between substrates without the use of a lens. The light emitted from a small fluorescent source diverges until it strikes the photoreceptor. If the substrate is 0.01 inch thick, then the light that falls on the adjaceiit substrate could be confined to within approximately 0.01 inch by moderate collimation a t the source. A 0.01 inch resolution yields lo4 separate areas on the suhstratc. The optical channel or surface that is to communicate with the external surroundings of the machine could use a lens; this \voulCl allow over 10: optical channels in the input field. The output of the machiiic to human observers or to rcmotely located machines would also contain 106 individual channels. It would also be possible t o ohtaiii ~nrchanicalor acoustical interconunuriication with machiiic cireuith by providing thin hard diaphragms as the encapsulation that vary the spacing of field emission diodes, thus causing a current change. Thcse transducers could be czs m a l l as one micron.

158

4. ENERGY

KENNETH

R. SHOULDERS

SOURCE AND DISTRIBUTION

Under conditions of the maximum rate of processing data, up to 100 watts of power could be dissipated by this one-cubic-inch machine. The data processing rate a t this power input would be approximately 10l6bits per second. It is not expected that this rate would be used often, but, in the event it was reached, the prime energy source would be a 100 volt direct current supply drawing 1 amp. Auxiliary low current supplies for electron multipliers and the like could be provided from external sources or from internal converters. The power distribution system would have to be an elaborate arrangemcnt to prevent faulty sections from disturbing the entire machine. Fuse links built into the machine construction in the form of marginal wire diameters would serve as an irreversible protection device. Electrostatic relays could be employed as fault sensing devices and act as reusable circuit breakers.

IV. Tunnel Effect Components

The quantum mechanical tunneling of electrons through an energy barrier under the action of a high electric field is the basis of many known effects. The source of electrons for an electrical avalanche leading to destructive breakdown niay frequently be traced to field emitted or tunneling electrons. The Malter effect [8] is reported to be a process for obtaining very high secondary emission yield from a surface in vacuum; the effect is caused by a positive charge being deposited on the surface of a thin dielectric which covers a metal base and results in field-emitted electrons being drawn from the metal through the dielectric and into the vacuum system. Cold cathode tubes based on self-sustained secondary emission from MgO have been reported to use a Malter effect or field emission source for their emitted electrons [9]. Some dc electroluminescence has been reported to stem from field emission [lo] and the Mott and Gurney explanation of corrosion processes in thin films invokes quantum mechanical tunneling as the source of the electrons needed to carry on a thin film corrosion process [ll]. The utilizatioq of quantum mechanical tunneling in tunnel diodes has recently become an active area of investigation for electronic components, and before this, investigators of high-vacuum field emission have applied the phenomena to various microwave, tetrode, and cathode ray tube designs [12, 13, 13al. It is our intention to use the principal and best understood form of field emission, namely, the emission from a metal into vacuum, as the basis for

MICROELECTRONICS

159

active rlrrtroiiic~romponriit in n microrlrci roiric. data processing systrm. 111 addif ioii, infrared rlrtrc.tors a i d olrcf roluminrsc.enrc! genrrators will hc iiivcstigatcd that niakr U ~ C of ' licltl emissioii from aiid into semiconductors. A type of solid state tunnel effect c.oinpoiicnt has brcn investigated in which only a metal aiid a dielectric arc used, the mctal supplying the electxons and the dielectric giving the ricc~ssaryenergy gap. Experience thus far with this class of dcvice show ail unfavorably low impedance, as well as other problems such as traps caused by poor crystal structure in the dielectric. The greatest problem in applying field emission to devices is the instability of the final device due to impurities and material migration. Field emission can be a very high energy density effect, and when it is coupled with low activation energy materials, as is done in most applications attempted thus far, material decomposition and instability is the inevitable result. By using stable materials and very specific geometries to avoid electron collision with dielectric supports, a much higher stability could be expected. a11

A. Geometry

The most essential design criterion in a vacuum tunnel effect device is to provide a method of changing the field a t the cathode in such a way that the electrons that are emitted under the action of the field are not intercepted by the control electrode (grid) that caused it. Thc use of the emitted electrons depends upon normal elcctron ballistics, and conventional tube design considerations are adequate. Many geometries are possible that will achieve the desired results; two geometries that lend themselves to film construction techniques are shown in Figs. 1 and 2. These configurations could be constructed by successive deposition and machining operations. The only part of the process that is not obvious upon inspection is the method of supporting the conductor layer over the cavity in the lower dielectric, and the closing of the entire cavity with encapsulating material. These details will be discussed in Section IV, G. The multilayer geometry shown in Fig. 2 is more difficult because of the higher registration needed between layers. 1.

THE CATHODE

The cathode preferably is made in the form of an array of small tips superimposed upon a larger cathode lead wire. Tips having a radius of approximately 100 A would be desirable from the standpoint of lowering the operating voltage but wouId be difficult to fabricate and have a tend-


160

CATHODE

ANODE

CONTROL G R I D

SCREEN G R I D

TOP VIEW

CATHODE

-SCREEN

~

CONTROL G R I D

-

GRID

-ANODE

SIDE VIEW

FIG.1. Tunnel cffcct vacuum tetrode-single-layer

type.

ency toward instability due to surface tension forces. It would be desirable to fabricate an array of at lcast ten of these tips on the cathode area. Some of the pertinent fabrication processes will be considered in Section IV, F. The work function of the cathode material should be approximately

TOP VIEW

SIDE VIEW

FIG.2. Tunnel effect vacuum triode-multilayer

type.

MICROELECTRONICS

161

FIG.3. Field emission current density J as a function of applied surface electric field F lor three values of work function in clcctron volts.

4.5 ev, the same as tungsten. If highcr work functions are used the voltage requirements of the device are increased, as are heating effects. Low work functions reduce the slope of the emission 10 I current-grid voltage characteristic and thus lower the gain of the device. If the work function is carried to very low values, loosely bound monolayers of materials such as cesium are usually used and instability results. At very low work functions the devices also becotne slight,ly temperature sensitive. Graphs taken from “Advances in Electronics,” [G, pp. 94, 961 and plotted in Figs. 3 and 4 show the dependence of cathode cur- 5 rent density upon field, work function, and -2 temperature. The range of current densities of interest for 1O-Io cm2 cathodes is between , -4 5 X lo6 amp,’cm? during tthe conducting 1 2 3 4 5 IO’/F (F IN VOLTS/CM) state to 5 X amp/cm2 during the quiescent state. This is a dynamic current FIG. 4. Field emission current swing of loLo, which corresponds to 5x1 actual density J as a function of apcurrent swirlg of from 10-4 to 10-14 plied surface electric field F a t and can be achieved by a change in field a t four temperatures for a 4.5 electron volt work function. the cathode of approximately five to one. Young [14] has observed a velocity distribution of 0.15 ev for field emitted electrons, indicating that this is not a noisy source and that

-

I

2

I

I


162

the energy spread is low enough to allow easy control of the emitted electrons. 2.

GRIDS

A geometry must be selected for the control and screen grids that will allow their fields to effect the cathode without the electrons emitted from the cathode being intercepted by the grids. The control grid would normally operate positive with respect to the cathode potential and, by keeping the grid near the plane of the cathode, electron current to it can be reduced to very low values. The control grid may be run negative if it has a higher work function than the cathode, or is smoother than the cathode. Normally the grid would be formed smoother than the cathode so that negative operation is possible, but a negative grid is more difficult to incorporate in circuits because it cannot be connected directly to a preceding anode as a positive grid can. At a field intensity of lo7 volt/cm the mechanical force on the grids amounts to about (540psi, and this force increases with the square of the applied field. The grid electrodes must be firmly supported only a short distance from their active regions to prevent distortion in the field and unwanted mechanical resonances due to electromechanical effects. These effects will be discussed later in the section on electromechanical filters. The screen grid has the dual role of shielding the control grid from the anode field while acting as a bias field electrode. In an effort to lower the switching voltage of tunnel effect components it may be desirable to maintain the screen grid at a potential of 100 volts while the control grid and anode swing between 3 and 10 volts. The high field from the screen would serve as a bias field at the cathode while the control grid caused the necessary current change. The secondary electrons and reflected primary electrons at the plate must be effectively suppressed and prevented from returning to the screen grid if this method is to reduce over-all heating effects, including dissipation a t the screen grid. Secondary electron coupling effectsto be discussed later would be ineffective a t low voltages and therefore low voltage operation would not be desirable if secondary emission effects were required. 3. THE

ANODE

Efficient collection of the emitted electrons is the principal function of the anode, and heating effects caused by electron impact must be dissipated to the surrounding lattice. The anticipated current density at the anode is lo4amp/cm2 and the average electron velocity may be anywhere from 3 volts to 100 volts, depending upon the final outcome of some of the techniques for producing and stabilizing cathodes.

MICROELECTRONICS

163

The power density in a one-square-micron area could reach lo6watt/cm2 in operation. As will be discussed later in Section IV, E on Heating Effects, the power density is lower by two orders of magnitude than the maximum allowed. The high field strength used in tunnel effect components effectively suppresses space charge effects. It has been shown that current densities of lo' amp/cm2 can be obtained in the region of the field emission cathodes with only a 25% space charge effect [15]. This low space charge would allow efficient collection of electrons a t the anode for a current density of lo4amp/cm2. By shaping the anode in the form of a shallow cup, a space charge cloud may be purposely established as a means for suppressing secondary electrons. B. Time Constants

By considering the major energy storage mechanism, a n order of magnitude calculation of time constant may be made for the vacuum tunnel tetrodc. In this case, the interclectrode capacitance between screen grid and anode is of major importance. Roughly this capacitance is 10-l6 farads, assuming no increased capacitance due to interconnection wiring or charge carriers in the grid-plate region. The discharge resistance, based on a current of amp and a voltage swing of 50 volts between screen and anode, is 5 X lo6 ohms. The RC time constant is thus 5 X lo-" sec. The transit time of an electron between cathode and anode in this particular micron sized device, operating a t 50 volts, would be about 10-13 see, and no grid loading effects would be observed even a t 100,000 Mc, thus reducing the need for complex geometries such as traveling wave tubes to achieve high frequency gain. C. Tube Characteristics

Tube characteristics for vacuum tuniiel effect devices have been obtaiiied by both analytical and experimental methods; however, the analytical methods have been applied only to geometries that involve high voltage operation, in an effort to compare operation with the large-sized cathodes that have been thoroughly explored by previous investigators. I n early experimental work by the author, diodes and triodes were fabricated by film techniques using high resolution masking and vacuum evaporation methods. The masks were guided into position by a micromanipulator using a point projection field emission microscope to view the results. The geometries employed had largely undetermined dimensions, but over-all operating characteristics could still be determined. Operating voltages in the range between 20 volts and 100 volts were readily obtained with currents as high as 100 p a for short times. The devices were formed

164


in a poor vacuum and not encapsulated, so that effects of contamination caused the destruction of the device a t high current levels. This contamination causes a build up of a carbonaceous material on the cathode and other structures by electron bombardment of the pump oils in the vacuum system. The build up process produces a runaway emission and eventual destruction. Tube characteristics showing an amplification factor up to 100, a plate resistance of several hundred thousand ohms, and a transconductance of over 1000 pmhos per ma have been obtained in experiments operated in the higher current range for a short time before destruction occurred. Both negative grid and positive grid characteristics were obtained; the most favorable case of positive grid operation gave a grid current sufficiently low to measure a dc power gain of lo4. Recent work a t Stanford Research Institute [l5s] on applied research in microminiature field emission tubes has partly confirmed the earlier findings by showing diode operation a t voltages as low as 25 volts for 10 pa of current, yielding a power density of lo4watt,/cm2 on an anode of stainless steel which had not been outgassed. In other tests using molybdenum anodes a power density in excess of lo7 watt/cm2 has been obtained. A spacing of 2000 A was used in this test, or essentially the same dimension anticipated for the final grid-to-cathode spacing. By using a 1000 mesh screen of copper as a grid and micromanipulating a 1000 A radius tungsten field emitter into one of the mesh openings, triode characteristics have been obtained even though the anode was located more than one centimeter from the grid. The voltage amplification for the triode geometry used was over 5000 and the grid current was less than lo4below the anode current when the grid was positive. A grid voltage of around 400 volts was necessary for this experiment because of the large hole size in the 1000 mesh screen. Tentative results using 5000 A hole diameters have shown good cont,rol grid action with 25 volts applied. A geometry similar to the triode shown in Fig. 2, but having a single emitter tip, has been chosen for analytical treatment. The dimensions chosen were: cathode radius 1000 A, cathode-to-anode spacing 7750 A, control grid hole diameter 5000 A, spaced 7750 A from the anode. The analysis considered grid emission and intercept currents. An amplification factor of 20, a transconductance of 50 pmho/ma, a plate resistance of lo7 ohms, and a power gain of 20 were found for the geometry chosen. The grid voltage was 250 volts for an anode voltage of 1500. These operating characteristics are far from ideal, but are in line with the large tip radius used in the calculations. Complete data on this project can be found in [15a]. It is the aim of this program to investigate analytically and experimentally the properties of field emission tubes in an effort to optimize their geometry arid determine the compatibilit,y of certain metal and dielectric

MICROELECTRONICS

165

c.on~binat,ioiisfor thcir coiistruction. Thc expcrinicntal appsrat 11s uscd oil this program is csscntially a point projection electron microscope using tho field emission tip as thc source. Rlicromaiiipulators arc provided that can position the emitter in 5000 A diamctcr grid holes while under observation by the microscope a t magnifications up to onc million diameters. The micromanipulators have been calibrated to give about 40 A of motion per dial division in any onc of three mutually perpendicular axes so that variations in gcometry can be investigated. It has been the aim of this program to use methods that are independent of electron beam micromachining so that early results can be obtained to help guide the later application of micromachining work. The features that are readily observed from this work to date arc that the operating current range is exceedingly large-which will be a great asset in obtaining low quiescent power dissipation-and that the power gain and voltage gain are adcquatc. D. Environment

It can be seen from Fig. 4 that the effect of tcmpcraturc on a cathode having 4.5 electron volts work function is negligible for the conducting state until a temperature of 3OOO"Ii is reached. The most important effect of increased temperature for the entire array of 10" components is the increase in quiescent current for the various deviccs. If 10" devices each having 10-'0 cm2 of cathode area are raised to a temperature of lOOO"K, under conditions which correspond to a current density of 5 x amp/cm2 per device and a field intensity of 2 x 107 volts pcr centimeter, then approximately 50 watts would bc dissipated by the entire group of components c volts). Since thc maximum (total current 0.5 amp, opcratiiig v o l t ~ ~ g100 continuous powcr dissipation for the system was postulated to be 100 watts, this leavcs only 50 watts available for dissipation of all the components that are not in thc quiescent state, thus reducing the data processing rate t o less than one half the original rate. A temperature limit may also be imposed by electrical conduction within the aluminum oxide lattice used for the dielectric of the system. Tests have been made by the author, which are described in Sections IV, I, and VIII, C, on aluminum oxide a t elevated temperatures by depositing the alumina onto the tips of field emission microscopes, observing the emission current that can be conducted through the lattice a t various temperatures, and studying thc ability of the surfacc to retain a charge. The tests indicate that properly deposited alumina is an adequate diclcctric for our purposes to temperatures in the ordcr of 12OO0Iedtungsten filamcnts which contain thoria, t o increase thc stability of thc surfacc. In actual operation these small tips would always be under the action 011 an electric field to prevent dulling whenever heating occurs. The possihility of the small tips hecoming sharper during operation seems less likely than the sharpening of larger tips because crystallographic binding forcrs become stronger as the size is reduced. It has been noted that it is not possible t o smooth a surface of anything except glassy materials beyond a roughness of a few hundred angstrom units by heating processes alone, because of the tendency of materials to crystallize. Mueller [24] has shown the application of field evaporation using exceedingly strong fields to smooth surfaces, but these fields exert more force than we use by a factor of approximately 100.

:ionapparatus used in the micromachining program. XIV. Demonstration of Micromachining

A demonstration of micromachining has been conducted using simple vacuum deposition equipmciit and a commercial electron microscope. The optimum properties of the process cannot be secured by using such makeshift methods, but the goal of denionstrating the machinability of materials by electron beam techniques is attainable to some small degree. Some applications for electron beam machining would conceivably need no more elaboration than is used in this demonstration. Briefly, this demonstration consisted of depositing niolybdeiium metal on a thin film of aluminum oxide which is supported on an electron microscope specimcn screen, coating the molybdenum with the triphcnylsilanol, cxposiiig in the clcctron microscopc with a rcducctl pattern of n scrcci~ wire mesh, and etching thc molybdeiium in chlorine. Thc fiii:il rcsulf, is shown by a photograph taken in the elcctroii microscope.

MICROELECTRONICS

277

A. Substrate Preparation

A c o p p c ~wweii of 200 mrsh is covrixl with a collodion film made by spreading a 2% solution of ccllulosc nitrate in ainyl acetate on water, :ih described i i r :uiy st:indard text OII dcctron microscopr specimen screen pi*rpnration. The collodioii is typicdly 200 A thick. This s(’re(’n is thrn put i n a \~acuunisystem, swh a< thc oiir shown in Fig. 31, and coated \L ith :L 800 Li-tliickfilm of nlruninurii oxide I)y cuaporatiiig from a tungsten 1)uskct hratrd by rlcctroii t)oml)ardrncnt iii the apparatus shown in Fig. 13 and described in Section I’IIT, A on Matrrial Drposition. The copprr screen and films are heat c d t o urouiid 800°C by the srniill ceramic substratc heater shown in Fig. 24 to drivr off the collodion a d stabilize the evaporated :Llumiiia. Higher tempcrat u r ~ sivould he desirable but, the coppcr srreeii is prone to evaporate : u d mrlt hcfore thc rrcrystallizatiori of the aluiniiia a t 900°C; this is caused I)y thc fact that the copper is hottcr than the film of alumilia, since they :irr i i o t in I~lackbodycqiiilibrium with thc sritistr:it(>heater. B. Film Deposition

?‘he inoly1)deiium as evaporated on1o the aluiniiia surface with the apparatus shown in Fig. 13. The molybtlrnum sourcc is simply a selfFupporting rod of vacuum melted niolybdrnum that is heated by electron of the molybdenum film is optimum for our tiombardment . The thick1 purposes when betwcen A :~nd200 A. Heavy drposits cannot be penetrated by the electron beam during h t r r aiialysis for surface defects, such as spotty resists or piles of foreign material, that may have been produced. Thinner films fail to give adequate coiitrast iii the micrograph duc t o electron scattering from thr substratr. The tempcraturc of the substrate during the deposition is not important but is usually in the region of 300°C t o prcvrnt oil films from \)ring dcpositcd siinultarirously with the molyhdciiuin. The vacuum was rarcly better than 5 x 10-5mm IIg and the oil pumps have hrcii k i i o ~ ~to t i hckstrcani :md coiitaniiiratc~v:trious samplrs. Thc rntc of deposit ion was fairly slow for t molybdenum source that l i : ~ lieen used I)ecnusc of the small diameter. A typical drposition required three minutrs for a 150 A film of molyhtlriiiim. Aftcr deposit ion, the film iis heated to tiround 700°C to stabilize and rccrystallizc the molybdeiiuni. A micrograph of thc film after pro is shon-ii in Fig. 41, and a diffmctioii pntterii iii Fig. 4%. Thc f i l i n is c h o o l ( 4 to room trniprmtui~cand a cwatiiig of tripht’irylsilaiit,I \\:is :ipplicd l)y \ w u u i i i ( ~ v : t p ( ~ ri oaiti froni :1 m i i l l glass crricith with t h ~ : L I I I Capparat us usrd previoiihly. ‘I’hc. cruci\)l~ is hcld in :I iiiolyldriiuin wirr holdcr \vhicdi rcceivrs the elect roil 1)oriihardmciit for heating. A filrri thickh ( b


FIG.41. Electron micrograph of 200 A-thick molybdenum film deposited on 200 A-thick aluminum oxide by thermal evaporation.

iiess of about 50 A is optimum for the resist layer. Thicker films tend to agglomerate and grow into feathery shaped patches. Thinner films can be used but the possibility of developing a hole in the resist is higher. Upon completion, the compositcb of films was removed from the vacuum system arid inserted i n the rlrct,mn microscope for exposing. C. Resist Exposing

The 1Iitach@IU-l0 electron microscope pictured iii Fig. 40 and described in Section XI11 has been modified slightly by installing a removable 500

MICROELECTRONICS

279

mesh screeii above the objective leiis in such a way that) the screeii can be dernagnified by 200 times. The focal length of the objective lens is 0.5 mm during normal operation aiid thc w r w n riiust be spaced about 100 mm above the principal plane to obtaiii the 200 t o one demagnification. The specimen holder for the microscope was modified by extending it approximately one-half millimeter below thr principal plane of the lens. This specimen holder was used for csposing only, aiid iiot for microscopy. .4pproximately 50 specimen holders conic with the iiistrurneiit so that modifying oiic is of little coiisequciice. The illumination system was stopped down by usiiig a small aperture iii the second condenser h i s . This leiis has three movable apertures that can be changed during the oprration of the instrumcnt without breaking the vacuuni. The aperture size used for cspoxiiig resists was approximately

FIG.42. Electron diffraction pattern of ruolybcleiiriin-LIlumirla sample shown in Fig. 41.

280


0.002 inch in diameter. The illuminating intensity of tlhe instrument was reduced t o the lowest possible value by increasing the grid bias on the electron gun with the switch provided for that purpose. The intensity was further reduced by reducing the filament temperature. The only method of rontrolling the aperture angle of the illuminating electron beam into the objective lens is by adjustiiig the second rondenser lens current. The condenser lens thus determines the number of grid wires seen arid the ohjective lens controls the size of the image, although strictly spraking there is only one magnification that is properly focused, namely 200. The current density under typical exposure conditions is about 10 ma/cmz and the beam velocity is 50 kv. During exposure, the intermediate and projector lens were used as a microscope to observe the position of the specimen screen. These lenses do not, have enough resolution to observe the fine detail of the image used for exposing nor is the brightness high enough to see any detail. At best the microscope was used to tell whether or not the exposure was being made on a n open area of the specimen screen. The method that was most effective for determining corrrct focus was to cause t,he beam to converge to a point on the specimen screen, as indicated by the point projection niicroscopy produced, and then to increase the objective current a small but known amourit]. Another indication of focus can be observed in the diffraction pattern of the sample whereby the minimum number of spots appear when the beam is at, crossover. After proper alignment the exposures were made by moving the manipulator for the specimen screen in small increments, observing on the fluorescent screen that, the exposures were being made on the open spaces of the specimen screen. The exposures had to be long enough to prevent blurring of the image during the time in moviiig froni one sample to the next because beam blanking was not used. A oiie to three sccond exposure was normal since the manipulator could be moved rather rapidly froni one spot to the next. D. Etching

After exposing, the samplc was retumed to the vacuum system and inserted in the small substrate heater. The chamber was evacuated to the mm Hg and the sample was heated to 600°C. A stream region of 5 X of chlorine was admi d to the system through a valve and a length of aluminum tubing. The tubing was tcrmiiiated within 5 inches of the surface being et,ched; duc to the pumping action of the vacuum system the pressure was highrr :It thc surfncc than is indicated 011 a remote gauge. Thc chloriiw prcssurc was rchgulatecl l)y hand to a value uf 5 x 10 ' rnrn Hg, as iiidicatd by operating oiic of the evaporator positions as an ion gauge. 'l'he etching

MICROELECTRONICS

281

was complctcd in a fcw seconds; the chlorine prcssurc \vas rccluwd; aiitl the suhstratc tcmperaturc lowered. 0t)scrvatioii of the surface during etchiiig revealcd that the appearailre chaiiged flwn L: rcflcctiiig metallic

surface to a traiisparent dielectric surface. Thc sample was removed from the vacuum system and inserted in the electron microscope for viewing. Figurcs 1.3 and 44 show typical samples. E. Discussion of Results

ils indicated on Fig. 44, the image size is such that the scrccii wire spacing is 2500 A or 100,000 mesh per inch. 'l'he image has heen distorted

FIG.43. Low inngnificatioii electron niirrogr:ipti of ~nic~oma.ctiiiied molyhde~lum 2OU A-ttiick on slumiriurii oxide substrate.

282

KENNETH

R. SHOULDERS

FIG.44. High magnification electron micrograph of micromachined molybdenum 200 A-thick on aluminiiin oxide substrate.

into a pincushion shape by the electron optical system so that the current density falls off approximately as the square of the distance from the center of the axis. This effect is beneficial for determining the condition of exposure and the effect of the etching on underexposed areas. As can be seen, the transition region between properly exposed areas and unexposed areas produces patchy etching caused by having the resist too thin. I n the center region the transition is rapid enough to produce relatively sharp edges. The resolution shown iii the micrograph is in the region of 300 A, with the

MICROELECTRONICS

283

priiicipal error resultiiig frc~mthe raggctlness causecl I iy tlic l):~(:kgrouiid fog effect. The background fog, or patclics ( J f niolybdeiium abut 300 A in diameter, have been mused hy migration of unstable, recrystalliziiig aluminum oxide during the various hcating cycles. This aluminum oxide forms a chemically resistant film around the small granules of molybdenum and prevents further etching by chlorine. This tiackground fog can be removed by using a very short etch of phosgeiic gas to remove the thin film of alumina. Close inspection of Fig. 44 reveals that most of the molybdenum particles lie adjacent to a light area in thc supporting substrate, which could be interpreted to nican that the alumina came from these arms. By using high temperature screens aiid stabilizing the alumina to alpha form, most of the background van be removed, although there is still a very small reaction bctwecii thc niolybdeiium aiid the alumina. The dark areas i n the molybdenum film are caused by selcctiw rlcct roil scattering within the molybdenum crystallite. When these films are being observed in the electron inicroscope the arcas alternate between light and dark, and under darkfield illuminatioil they are even more pronounced. The thickness of the film is apparently uniform. Figure 45 shows a specimen scrceii of silicon on silicon dioxide made h y techniques comparable to the oiics used for the micromachining of molybdenum. The specimen support film had been torn from the screen and rolled back to produce the dark ragged line adjacent to the light area. As can be seen, several images were exposed on the same screen opening. These samples were oxidized slightly in transferring between the electron microscope and the vacuum deposition chamber, which resulted in another type of background fog, namely, silicoii dioxide on silicon. The only completely effective remedy for this is to keep the silicon under vacuum a t all times by incorporating the electron optical system with the deposition apparatus. This denionstrat ion has purposely been doiie with relatively crude equipment aiid techniques, with the exception of the electron microscope, so that they could he carried out without modification in any establishment outfitted with iiornial deposition apparatus, and having access to an electron microscope. With only minor additions, almost ally electron microscope with rcsolution greater than 100 A can be used for exposing. XV. Summary A. Microelectronic Component Considerations

The fabrication of one-micron-sized components, incorporated into a n electronic system having over-all dimensions of one inch on a side and

284


coiitaiiiiiig 1W coinpoiicrits per cubic inch, is prcsciitly limited by coiistriw 1 ion tcchiiiqucs. Intcrconiicctioii betwccii widely separated components is coiistraiiicd by t hr high rlwt ricd rcsistiLii(ac of traiismissioii lilies, which can h a w a dc rcsistaiice of l o 5 ohms per liiiear inch for 3000 A diametcr conductors. To keep potentially high transmission line loss to acceptablc

FIG.45. Electron Inicmgraph ol niic.roiii:ic.liiiic,tl silicon 200 A-thick on silica 100 A-thick.

lcvels, active compoiiciits of high iinpcdaiice are required. Uiidcr these conditions, :ui optimum switchiiig time coiist,aiit for the active component lies in the 10-11 sec rcgioii; shorter switching times than lo-" scc result in exccssive interconncctioii loss at room tcmpcrature and above. Powcr densities of 10* watt/cm2 appear pcrmissiblc for onc-micron-sizcd components; these high power densities permit high data processing rates. For large arrays of cornpoileiits, a low quiescent, power for each component

MICROELECTRONICS

285

is required; peak power densit>yduring operation should he as high as 10'0 above the quicsccnt state. LC filtcrs of onc-micron-size would 1)c cupectd to have a Q as much as lox below the Q for convciitional sized filtcrs, thus making thrm relatively useless. RC filters would bc prone to tcmperature drift when scaled down in size because of the necessity for scaliiig up resistivity and the attendant highcr tcmperature coefficient of resistivity. Electrostatically operated elcctromcchaiiical filters that apprar ncll suited to the micron size rarigc and coiistruction processes arc discusscd. The rcsistivity of conductors is not affected in the size range under iiivcstigation, hut in the submicron range thc dielectric breakdown strength of insulators can bc incwascd to lox volt 'cm. Scaling dowri in size gives a high sur.facc-to-volume ratio that cauws materials to migratr under the action of surface t crision forces, and iii addition causes iiicrc:iscd carrier recomhiiiation at thc surfnres of srmiconductors. The most difficult problems in inaterial handling involvc tlicl protluction of uniform crystalliiiity in films. 8. Tunnel Effect Components

Electmiiic components 1)asrcl upon the quantum 1nechiic:d t uaneliiig of electrons from a metal into a high \ w i i u r t i are proposed; with this tcchiiique, only stable metals and dielectrivs arc employcd iii various grometrics to produce diodrs, triodcs, and tctrodcs. With suital)lc drsigii that makes iisc of the l~allisticaproperties of thc clcctroris in vacuum, control grids and srrerii grids of such deviccs caii he operated at a positive potrritial without drawing apprcciable currents. Operatiiig voltagrs as low as 10 volts, with currcmts in the region of 100 p unip for lo4 :mp/rmz currciit densities, appear possMc without aclvcrsc space chargc eff the high field iiitciisities used. Normal grid voltage variations used in various circuits would produce a chaiige in cathode current of about lolo, which is highly desirable for low poner quicscciit operation. The amplification factor for triodes can be in thr vicinity of 100, with a traiisronductaiicc of 1000 p i t h o / ma and a platc rcsistancc of about lo5ohms. With a power driisity of lofi wat,t,/cm2, a switrhing time constant of 10-lo sec appears possit)lc for a oiie-microii-sizr.d devicc, with an indicated transit time lag of src. High-frequeiiry grid-loading would bc iicgligiblc a t 100,000 hlc because of the short transit time lag. An array of 10" vacuum tuiiriel cf'fcct devices containcd iii oiie cubic inch of spacc could opcratc from arhitrarily low tempcratures up to arouiid 800°C. A t a inasimuni data proccssiiig rate of lot5bits per second, thc machine would self-hcat, to around 800°C. The iriseiisitivit,y of thc dcvice to local crystallographic

286


properties would indicate an immunity to radiation damage about los timrs greater than present single-crystal semiconductor devices. Methods of potentially forming stable, mutiple-tip field emission cathodes having radii of 100 A are discussed ; here, self-formation methods would be ueed to generate arrays of over lo8 active components per layer. The methods would have to form components with uniform characteristics by degenerative processes. Vacuum encapsulation methods are discussed that are applicable to large arrays of film components, and corrosion methods of testing the encapsulation are described which indicate a component lifetime of several hundred years. Solid-state tunnel-effect amplifiers are considered ;however, high grid current, low impedance, temperature sensitivity, and device nonuniformity make them less desirable than vacuum devices. C. Accessory Components

Secondary electron emission effects having high stability, high current density, negligible time delay, and temperature insensitivity are discussed for appliration t o transmission-type electron multipliers and for coupling between tunnel effect components. Multiplier phototubes having 100 Mc bandwidth, negligible transit time ft-c appear possible spread, and a sensitivity adequate for detecting using film techniques. Such tubes would make photodetectors available with diameters ranging between 0.01 inch aiid 0.2 microii for application to the intercoiinection of large arrays of componrnts, and for microdocument reading. A method for using electronic micromarhiniiig techniques to record data 011 glass plates, with subsequent electrooptical read-out, yields a document storage scheme with a data density of 10'' hits per square inch and a readout rate of 1O'O bits per second. Electrostatically operated mechanical relays, operating in vacuum a t 30 volts, mid with frequencies up to 10 Mc, seem applicable to switching low level signals and for power distribution. Electromcchaniral filters composed of simple metal and dielectric diaphragms could serve as communication filters between 4 Mc and 600 Mc. Field strengths as high as lo7 volts make possible electromechanical coupling coefficieiits up to O.G, which would permit temperature compensL' t'ion by electrical interaction with internal thermal bimetal capacitors. D. Component Interconnection

I n the absence of thermally stable fixed rcsistors, tunnel effect diodes will be employed for some resistor functions in an effort to match thc ternperaturf~._cocfficic.iitsof active elements. Low dissipation circuits are dis-

MICROELECTRONICS

287

cussed in which the wide current swing of tunnel effect components is utilized. Thew low dissipation circuits would be employed in an active niemory in which the switching time may be 10-'O src, with quiescent power of watt for either a negative re ance or a flip-flop type of circuit element,. Low noise amplifiers arc discussrd in nhich a field emission cathode produces a virtual, space-charge-limited cathode for a iicgative grid tube. A method of using elcctrirally stccrable rlcctron guides for the purpose of eliminating transmission liiics from future systems and for increasing the logical freedom of a system is also discussrd. E. Substrate Preparation

Sapphire platrs, 1 inch square and 0.01 inch thick, are considered to be an optimuni substrate size. The platrs are mrchanically ground and polishrd ; clcanrd by high-vacuum methods using the explosive removal of a filmof niatcrial from the surface; then smoothcd in vacuum by drpositing a film 011 a temporary fill material, volatilizing the fill material, and sintcririg the film into the substrate. Terminals are prepared by firing molybdenum metal electrodes onto the plate iti a high-vacuum furnace.

F.

Material Deposition

Electron boriibardnient and rt4stively-heatcd t h r r n d evaporation sources have been built pcrmitting automatic regulation of evaporating material to within 2 7 , of a predetcrmined value 1)y using ion gauge monitors and electronic regulators. A bubstrate heatrr is described that is heated by a 3 kv, 0.1 amp elwtron lwani. The heater is completely enclosed by ceramic and is capable of achieving 1700°C operating temprratures for use with reactive deposition. Reactive deposition methods are discusscd in which dual thermal cvaporators are used to deposit stable, dcnsr, and pinhole-free niaterials like molybdenum and aluniiiium oxide films. These films wcre tested in the form of an encapsulatrd capacitor having a 200-Athick dielectric a t temperatures of 800°C and field strengths of 10' volt/cm to reveal impurities or imperfections. Solid state tunnel emission was obsrrved to orcur bct\seeii elcctrodcs. Tcsts for stability of aluminum oxide and molybdenum were carried out on the tips of a ficld emission microscope; the tests revealed no impurities caused by the dcposition and etching process. Mrthodx of potentially growitrg single crystd films of mattvials iiivolvirig reactive deposition and a sweeping t hcrmal gradient are discussed relative to deposition processes. G. Material Etching

Using molecular beam techiiiqucs in a high-vacuum chamber, both solid and gaseous sources are employed to etch the surface of materials siich

288


as molybdenum and aluminum oxide, converting them to volatile compounds. Tests for the clcariliiiess of this process have been carried out on field emission microscope tips and 110 cvidence of etchant residue was observed. The parameters necessary to etch holes wit,h depth-to-diameter ratios of 10 : 1 in a molybdeiiuni film are discussed. Atomic beam etching methods arc discussed for application to low temperature etching of materials, such as the room temperature etching of silicon by atomic hydrogen. Alet,hods of using ion beams to sputter material from a substrate have shown very straight sided etching with no undercutting. Depth control methods effcrtivr to 1% are considered for various etching methods. H. Resist Production

Evaporated niat,er.ials such as silica and alumina can make effective etching resist,s by deposit,ing t,hem through masks; however, the resolution is limited by t,he mask. Resists can be produced by electron bombardment of materials such as triphenylsilnnol, which decomposes to form silica with a quant,um yield of about. one and a resolution of over 100 A for films 20 A t,hick. Multilayer met,hods of producing resists have shown a quantum yield of lo4molecules per electron with a resolution of 300 A. Consideration is given to finding resist-producing processes that are compatible with vacuum processing and t'he electron optical system. One of the chief requirement's is to maintain the surfacc uriipot)eritialduring resist formation, thus preveiitiiig electron bcsm distortion. I. Electron Optical System

The desigii of an clectrori opt,ical system is described that is irit,ended to become a micromachining electron source, a scanning electron microscope, a mirror microscope, arid an X-ray fluorescence probe. An elcctrostatic lens syst,em made from met,alized ceramic parts is currently being built; t,he lcns syst,em is capable of being baked in ultrahigh vacuum to a temperature of 900°C wit,hout t,he need for mechanical realignment t o obt.ain 200 A reso1ut)ion. Problems in obtaining registration between adjacent fields are discussed, and a regist.rat,ion of 500 A is predicted. The scanning microscope may approach a resolution of 200 A, which is equivalent to the micromachining mode. Only lo6bits per field would be expected for these two modes of operation because of limitat,ions in the deflection system. A n electron multiplier t.hat is integrated with the lens should provide most of the neccssary gain for the scanning microscope. The X-ray fluorcscence mode of operation should ultimately give a 1yo quantitative analysis on grams of material, with a resolution of one micron. Operation as a mirror microscope would give the ability t,o measure volt'ages down tjo 0.2 volt with a resolution of' 500 A and make dynamic voltage

MICROELECTRONICS

289

~nct~snrcinrnl s to 1000 M c . By usiiig the leiis as an emission microscope, multiple cathodc ficld cmihsion arrays could be imaged with 300 A resolution and thus greatly assist in forming uniform arrays. The requirements for various pattern generators are discussed; the ultimate geiicrator would be able to take full advantage of the lens hy producing a 10*-hit pattern every tenth of a setwid. The construction techniques for producing accurate lens elements are reviewed and some of the tests performed on the elements are described. J. Vacuum Apparatus

A 24-inch diameter, metal, bell jar-type vacuum system is described which is pumped by a 30 rfm roughing pump and a 6-inch oil diffusion pump. An adapter spool having 18 entry ports for accessories is shown arid ories such as cooling-water manifolds, electrical lead-ins, an optical port, and mult,iplc chaiiiicl manipulators are described. A demountable ultrahigh-vacuum attachmrwt is descrihcd which when completed, should be capable of attaining a vacuum of 1 X lo-'" mm Hg in 25 minutes. A 900°C bakeout] temperature is employed in this differentially pumped unit with a molten metal seal separating the high-vacuum and the ultrahigh-vacuum regions, the ultrahigh vacuum being produced by ion pumping. A range of accessories for the ultrahigh-vacuum unit are discussed, and considerations for iiitegration of the electron optical system and the deposition and etching apparatus are presented.

References 1. Chrr, P. H., K\'cw method of recording elertrons. Rev. S c i . Instr. 1, 711 (1930). 2. Niaon, W. C'., The point projcction X-ray rnicrosc+opcas a point source for microhcain X-ray diffr:wtion, i n S - R a y Mirroscopy aiicl Microrcrdiogruph?! (V. E. Cosslet, A. Ehgstrom, and H. H. Pattee, cds.), p. 336. Academic Press, New York, 1957. 3. Itovirisky, 13. hf., Lutsau, V. G., and Avdeyenko, A. I., X-ray microprojector, in X - R a y Microscopy and Microradiography (V. E. Cosslet, A. Engstrom, and H. €1. Pattee, eds.), p. 269. Academic Press, New York, 1957. 4. Powers, D. A,, and Von Hippel, A., Progress reports show consistent breakdown strengths between 50 and 200 megavolts per centimeter for film thicknesses between 700 and 4000 :tiigstronrs using aluinino d i r a t c glass. Afassachusetts Inst. of Technol. Lab. for Insulation Research, Cambridge, Massachusetts. 5. Good, It. €I., Jr., and Mueller, E;. W., Field emission, in Randbuch der Physik ( S . Fldgge, ed.), Vol. 21, Part I, p. 202 Springer-Verlag, Berlin, 1956. 6. Dyke, R. P., and Dolan, W. W., Field emission. Advaiares in Electronics and Electron Phys. 8, 153 (1956). 7. Giraedigiier, 11. J , Some aspects of vac~i\iindeposition of metals in trnnsistor falxication, in Y'ratls. 6th Natl. S!jrrzpociutn otL Vacuum Technology, 1958, 11. 235. 7a. Shoulders, K. R., 011microelectronic coniponent~,interconnections, and system

290


fabrication, in Proc. WesternJoint Computer Covg. pp. 251-258. San Francisco, 1960. 8. Malter, L., Thin film field emission. Phys. Rev. 50, 48 (1936). 9. Firth, B. G., Investigation in Utilization of Self-sustained Field Emission, Contract DA-36-039 SC-73051, DA Project No. 3-99-13-022, Signal Corps Project No. 112R. U.S. Army, 1957. 10. Zalm, P., Electroluminescence in zinc sulphide type phosphors. Philips Research Rep. 11,353, 417 (1956). 11. Mott, N. F., and Gurney, R. W., Electronic Processes in Ionic Crystals, 2nd Ed., p. 267. Oxford University Presa (Clarendon) London and New York. 12. Charbonnier, F. M., Brock, E. G., Sleeth, J. D., and Dyke, W. P., The application of field emission to millimeter-wave oscillators and amplifiers, Final Report, Contract DA-36-039 SC-72377, DA Project No. 3-99-13-022, Signal Corps Project No. 112B. U.S. Army, 1958. 13. Martin, E. E., Pitman, H. W., and Charbonnier, F. M., Research on Field Emission Cathodes, WADC Tech. Report No. 5IT97 (ASTIA Document No. AD-210565). Wright Air Development Center, Cincinnati, Ohio, 1959. 13s. Griffith, J. W., and Dolan, W. W., Field emission cathode ray tube development, WADC Tech. Report 5&8 (ASTIA Document No. AD-155723). Wright Air Development Center, Cincinnati, Ohio, 1958. 14. Young, R. D., and Mueller, E. W., Experimental determination of the total energy distribution of field emitted electrons, paper presented a t Field Emission Symposium. University of Chicago, Chicago, Illinois, 1958. 15. Dyke, W. P., and Dolan, W. W., Field emission. Advances in Electronics and Electron Phys. 8, 109 (1956). 15a. Hansen, W. W., Applied Research in Microminiature Field Emission Tubes, Contract DA-36-039 SC-84526, Task No. 3A99-13-001-01, Signal Corps. Research and Development Lab. Quart. Progress Report Nos. 1 4 . U.S. Army, 1960. 16. Mueller, E. W., Field ion microscopy of damage in tungsten by bombardment with alpha particles and cathode sputtering. Paper presented a t 20th Ann. Conf. on Physical Elcctronics. Mass. Inst. Technol., Cambridge, Massachusetts, 1960. 17. Dyke, W. P., and D o h , W. W., Field emission. Advances i n Electronics and Electroii Phys. 8, 158 (1956) 18. Holm, R., Electric Contacts Ziandbook, 3rd ed., p. 27. Springcr-Verlag, Berlin, 1958. 19. Boyle, W. S., Kisliuk, P., and Germer, L. H., Elcctricsl breakdown in high vacuuni. J . A p p l . Phys. 26, No. 6, 720 (1955). 20. Good, 11. H., and Mueller, E. W., Field emission, in Handbuch der Physik (S. Flugge, ed.), Vol. 21, Part I, p. 213. Springcr-Verlag, Berlin, 1956. 21. Good, R. H., and Mueller, E. W., Field emission, in Handbuch der Physik (S. Flugge, ed.), Vol. 21, Part I, pp. 212,214. Springer-Verlag, Berlin, 1956. 22. Dyke, W. P., and Dolan, W. W., Field emission. Advances in Electronics and Electron Phys. 8, 158 (1956). 23. Herring, C., The use of classical macroscopic concepts in surface energy problems, in Structure and Properties of Solid Surfaces. (It. Comer and C. S. Smith, eds.), p. 80. University of Chicago Press, Chicago, Illinois, 1953. 24. Good, R. H., and Mueller, E. W., Field emission, in Handbuch der Physik (S. Fliigge, ed.), Vol. 21, Part I, p. 218. Springer-Verlag, Berlin, 1956. 25. Good, R. IT., and Mueller, E. W., Field emission, in Handbuch der Physik (S. Flugge, ed.), Vol. 21, Part I, p. 202. Springer-Verlag, Berlin, 1956. 26. Powell, C. F., Campbell, I. E., and Gonser, B. W., Vapor Plaling, p. 37. Wiley & Sons, New York, 1955.

MICROELECTRONICS

291

27. Sugnta, E., Yishitana, Y., Iianeda, S., Tatcishi, M., arid Yokoya, II., Furidainentnl Rcsearches for observing specinims in gas layers, in Z’roc. -3rd ~ n t c r n .(‘ortf. otL Elref)on Microscopy, 1954, p. 452. The Royal Microscopical Socicty, London, 1956. 28. Martin, E. E., Trolan, J. I_ 0.

The latter condition is rclated to the Lagrange multiplier procedure for obtaining the optimum. If the nonnegativity condition on the variables is included, then on multiplication by -1 these conditions acquire the form in which the general constraints are given, and are therefore included with these constraints. One is always faced with the problem of whether an optimum is global or local, I n studying solution procedures, to simplify matters, f(x) is assumed to be convex or concave depending on the type of optimum required and the region is assumed convex in order to insure a global optimum and avoid the problems of a local optimum. The equation J(x) = c defines a one-parameter family of surfaces. Each value of c fixes one of the surfaces of the family. When x = ( ~ 1 x 2 )is a two-variable vector, one has a set of contours in thc plane. Thus, each feasible point must fall on a contour of j(x) and the problem is to move to the contour on which the optimum is located. Since the gradient points in the direction of greatest increase, it is natural to follow the direction of the gradient. Most gradient procedures involve suggesting good, i.e., quick and converging directions, which when successively followed from

RECENT DEVELOPMENTS IN LINEAR PROGRAMMING

315

uric feasible point on one mcmbrr of thc family to another fcasiblc poilit, on the same member or 011 another member, lead to an optimum whether local or global. Note that if P is a global maximum, then f(P) 2 f(x) for any feasible x. We next study with Zouteridijk [227, 2281 his method of feasible directions. We first consider the case of linear constraints:

5 a,,x, 5 b , (i

=

1 , . . . , m).

3=l

In the matrix notation Ax vectors ai where

=

b let the matrix A consist of m column

Then each inequality may be written as

(i = 1, . . . , sn.), - UiTZ 2 0, where uiT is the transpose of ui and x is regarded here as a column vector 2/i(Z) = bi

rather than a row vector as first considered above. Note that for a feasible point 55, yi(P) 2 0 (i = 1, . . . , m ) . Those i for which Z lies on the boundary of the corresponding hyperplane, i.e., yi(Z) = 0, are denoted by (il,. . . , ik). We indicate a new feasible4 point by (x As) where s is a unit vector with initial point P and 0 5 X 5 X for some X > 0 (since the point is feasible, X > 0 exist.s). We have point.ed out that t.he gradient locally points in the direction of maximum increaee; it is t.herefore reasonable to determine t.hat s 5 (s,, . . . , s,) which makes the smallest angle with t.he gradient a t Z (the best feasible direction). Since we are working locally, we use the condition of footnote 4, with the fact that s is a unit vector, to determine that, s which yields the best feasible direction. Thus, we must solve (we UPC the t,ranspose of the gradient since all vectors are involved as column vectors) vjT(P)s = maximum subjcct to . . ujTs5 0 , z = zl, . . . , i,,, STS = 1.

+

A ncwssary condition for the feasibility of j ; aiTS

5 0,

+ As)

= bi,

- ailT(? + As)

holds at a feasible point.

. . . ,i a ) . > 0. Then since yi,(X)

(i = i l ,

Thus, suppose that for some i, say i ~ ,arils yi(X

+ As is that

= -uiITXs

= bi, - allT = 0, < 0. This is a contradiction since 1 ~ ; > 0

SAUL 1. GASS

316

Note that if the first constraint holds for no i, s must, be aloiig the gradient. It is assumed that if the gradient vanishes a t P, then a new feasible starting point in the neighborhood of P is chosen at, which the gradient does not vanish, otherwise P is a local maximum. Since it is desired to increase the objective function moving out of P along S, it is sufficient to find s such that the scalar product to be maximized is positive. This is insured by replacing the last condition by S T S 5 1. But this is a nonlinear constraint which may obviously be replaced by the equivalent condition - 1 2 s 5 1. Once the maximizing s, i.e., Z is determined, its value is substituted in the constraints and in the objective function. Now X is determined from yt(E AS) 2 0. Thus, X = minz ( ~ , ( E ) / u , ~ E )for those i for which U , ~ S> 0. Then one maximizesf(% AS) with respect t o X subject to 0 5 X I X. If we denote our successive feasible points by Z k (Ic = 0, 1, 2, . . .) we have &+I = Pb XkSk. The reader is referred to ref. [227] for a convergence proof. If one has a mixture of linear and nonlinear constraints

+

+

+

i = 1, . . . , m1 i = ml 1 , . . . , m

5 b, aLTx 5 11,

g,(x)

+

where gl(x)are convex for all i, then one uses the vector q z ( x ) E (ag,/axl, . . . , ay,/ax,), which is the outward pointing normal of the hypersurface g l ( x ) = b,. Assume also that arTx= b, for i = . . , i,. A vector s with initial point E is now feasible if q,'(x)s < 0 for i = ill . . . , ik (the set of subscripts of those nonlinear constraints for which g l ( x ) = b , ) and a t T s 2 0 for i = . . . , ip. To find s one introduces a variable u and then finds s satisfying Ql*(X)S

-(Vf)S

. .

+ L0 a,Ts 2 0 + 50

. .

. . . , ak % = %k+1, . . . 2 p z =

u

21,

u

sTs21 u =

or

-1Lsll

maximum.

If u > 0, then s is feasible (as can be seen from the first, second, and third of the constraints) and the objective function will increase in the direction of s. If the solution yields u = 0, then it follows that there exist nonnegative quantities XO and X, such that

ni I

cxi+xo=

i=l

1,

x,20,

x i 2 o ( i = l,...,m).


317

If A0 > O,Vf(Z) is a nonnegative linear combination of the outward-pointing normals in 5t and hence j z is a maximum. The treatment for A0 = 0 is excluded due to its infrequent occurrence in practice. As before one maximizesf(Z As) as a function of A, etc.

+

5. linear Programming Applications

One is continually amazed by the diverse origins of linear programming applications. This is certainly indicated by the large section an applications in the bibliography by Riley and Gass [179]. In this section we will not attempt to review all of the new applications which have appeared since thc publication date of that bibliography, but it is felt that we should highlight a few unusual and possibly not so wcll known applications. Thc rcadcr is referred to the references of this article for additional material on applications. 5.1. Critical-Path Planning and Scheduling

The application of linear programming to planning and scheduling by the critical-path method is described in Kelley and Walker [123,124] and Kelley [120]. Fundamental to the critical-path method is the basic representation of a project. It is characteristic of many projects that all work must be performed in some well-defined order. For example, in construction work, forms must be built before concrete can be poured; in research and development work and product planning, specifications must be determined before drawings can be made; in advertising, artwork must be done before layouts can be made, etc. These relations of order can be shown graphically. Each job in the project is represented by an arrow which depicts (1) the existence of the job, and (2) the direction of time flow from th8 tail to the head of the arrow. The arrows then are interconnected to show graphically the sequence in which the jobs in the project must be performed. The result is a topological representation of a project. Several things should be noted. I t is tacitly assumed that each job in a project is defined so that it is fully completed before any of its successors can begin. It is always possible to do this. The junctions where arrows meet are called events. These are points in time when certain jobs are completed and others begin. In particular there are two distinguished events, origin and terminus, respectively, with the property that origin precedes and terminus follows every event in t.he project. Associated with each event, as a label, is a nonnegative integer. It is

SAUL I . GASS

318

possible to label events in such a way that the event a t the head of a11 arrow always has a larger label than the event a t the tail. We assume that 1 events, events are always labeled in this fashion. For a project P of n t8heorigin is given the label 0 and the terminus is given the label n. The event labels are used to designate jobs as follows: if an arrow coilnccts event i to event j , then the associated job is called job (i,j ) . When the cost (labor, equipnlciit, arid materials) of a typical job varies with elapsed-time duration it usually approximates the form of the curve of Fig. 4. This represents what is usually called “direct” costs. Costs arising from administration, overheads, and. distributives are not included.

+

iv) -’

/rash

limit

V

:: b

I

I

Dij Duration of job (i,.j)

dij

Fia. 4.Typical job cost curve.

Kote that when the duration of job (i, j ) equals U,,, the cost is a minimum. On the surface, this ii a desirable point a t which to operate. Certainly management would seldom elect to require the job to take longer than the optimal method tirnc. We call D,,the normal duration for job (i,j ) . However, exogenous conditions may require that a job be expedited. This may be done in a variety of ways. But in any case there is a limit to how fast a job may be performed. This lower bound is denoted by d,, in Fig. 4 and is called the crash duration for job (i,j ) . It is thus reasonable to assume that the duration y t J of job (i,j ) satisfies

0 Idii

I yij 5 D,j.

(5.1) The cost of job (i,j ) is now approximated in a special way over the range defined by inequalities (5.1). The type of approximation used is dictated


319

1)y the mathematical technique involved in what follows. Thus, wc must :~ssumcthat the approximate cost function is a pieccwise linear, noniiicreasing and convex fund ion of y7,. Usually, in practice, iiisufficient data are available to makc more thaii a lincar approximatioil. There are cxccptions, of course T n the linear case me may write

+

Cost of job (i, .I) = U , ~ . V ~ , b,,, (5.2) where a,, 5 0 and b,, >_ 0. This is iridicated by the dotted line in Fig. 4. On the basis of job cost functions just developed we can determine the (direct) cost of any particular schedule satisfying inequalities (5.1) by simply summing the individual job costs. That is, Project (direct) cost

=

C (aijy2j+ b i j ) .

(5.3) It is clear that there arc gcnerally inany ways in which job durations may be selected so that the earliest completion times of the resulting schedules are all equal. However, each schedule will yield a different value of (5.3), the project cost. Assuming that all conditions of the project are satisfied by these schedules, the oiie which costs the least would be selected for implementation. It is therefore desirable to have a meuiis of selecting the least costly schedule for any given feasible earliest project completion time. Within the framework 11e have already constructed, such “optimal” schedules are obtained by solving the following linear program : Minimize (5.3) subject to (5.1) and (Tj.4) ? / > I 5 t, - t , , and f” = 0, f,, = x. (5.5) Inequalities (5.4) express the fact that thc duration of a job cannot exceed the time available for performing it. Equations (5.5) require the project to start at relative time 0 and be completed by relative time X. Because of the form of the individual job cost functions, within the limits of most interest, X is also the earliest, project completion time. A convenient tool for generating schcdules for various valucs of X is the method of parametric linear programming with X as the parameter. Intuitively, this technique works as follows. Initially, we let yI, = D,,for every job in the project. This is called the all-normal solution. We then assume that each job is started as early as possible. As a result we can compute t L ( 0 ) for all events. I n particular, thc earliest project completion time for this schedule is X = f n ( O ) . We now force a reduction in the project completion time by expediting certaiii of the critical jobs-those jobs that control project completion time. Kot all critical jobs are expedited, but

320

SAUL I . GASS

otrly thosn i,h:~I,tlrivo i,hc projoc t, cost, up a t n minimum rntc :LSt,hc project, completion t imc tlccrrnscs. As t lic project, completion is retluced, mow and more jobs become critical and thus different jobs may be expedited. This process is rcpeatcd uiitil no furthcr rcduction in project completion time is possible. Mathematically speaking, the process utilizes a primal-dual algorithm. The restricted dual problem is a network flow problem involving both positive upper and lower bound capacity restrictions. A form of the FordFulkerson network flow algorithm is used to solve it. The critical jobs that are expedited at each stage of the process correspond to a cut set in the graph of all critical jobs. This process produces a spectrum of schedules (characteristic solutions in the linear programming sense) each at minimum total (direct) cost for its particular duration. When the costs of these schedules are plotted versus their respective durations, we obtain a nonincreasing, piecewise linear, convex function.

5.1.1

COMPUTATIONAL EXPERIENCE

The critical-path method has been programmed for the UNIVACI, 1103A, and 1105. The limitations on the size of problems that available computer programs can handle are as follows: UNIVAC1-739 jobs, 239 events; 1103A-1023 jobs, 512 events; 1105-3000 jobs, 1000 events. An IBM 650 Computer program for critical-path planning and scheduling is available on a fee basis from Mauchly Associates, Inc., Ambler, Penna. Generally, computer usage represents only a small portion of the time it takes to carry through an application. Experience thus far shows that, depending on t,he nature of the project and the information available, it may take from a day to six weeks to carry a project analysis through from start to finish. At this point it is difficult to generalize. Computer time has run from 1 to 12 hours, depending on the application and the number of runs required.

5.2 Structural Design I n Heyman and Prager [97], it is shown that the automatic plastic design of structural frames can be treated by the method of linear programming. The authors point out, however, that since the number of variables increases very rapidly with the complexity of the frame, this basic formulation is computationally feasible only for simple frames. In their paper the authors propose an alternate method which greatly reduces the size of the problem. This special procedure has been coded for the IBM 650, Stone [195], and the IBM 704, Kalker [105]. These programs handle


32 1

the following basic problem: Given the centerline dimensions of a plane structure of n bays and m storeys, find the cross sections of the various members such that the material consumption is a minimum. I n Lemke et al. [133], the authors pose a nonlinear problem in plastic limit analysis as an integral linear program. Arabi [5] considers a design problem which is associated with airplane control surfaces. The tail control surfaces of aircraft are equipped with several internal balance panels connected to the movable surfaces b y linkages. For added safety, extra weight can be added, when feasible, to those panels so that if one should become ineffective during flight, enough weight remains to counterbalance the control surface. The problem is to determine the minimum weight which must be added to a n airplane to prevent flutter of the control surfaces in the event of such a failure. 5.3 Other Applications

Maline [146] studied the planning problem faced by a machine shop required to produce many different items so as to meet a rigid delivery schedule, remain within capacity limitat.ions, and a t the same time minimize the use of premium-cost overtime labor. It differs from alternative approaches to this well-known problem by allowing for setup cost indivisibilities. As an approximation, the following linear programming model is suggested: Let an activity be defined as a sequence of the inputs required to satisfy the delivery requirements for a single item over time. The input coefficients for each such activity niny then be ronstructed so as to allow for all setup costs incurred when the activity is operated at the level of iuiity or at zero. It is then shown that in m y solution to this problem, all activity levels will turn out to be either uility or zero, except for t,hose related to a group of items which, in number, must be equal to or less than the original number of capacity constraints. This result means that the linear programming solution should provide a good approximation whenever the number of items being manufactured is large in comparison with the number of capacity constraints. Fort [72] describes a linear programming model of the gaseous diffusion process for srparating uranium isotopes. The model is intended primarily as a component of larger models involving interactions between the gaseous diffusion prowss, nuclear reactors, :~nd other facilities of the nuclear niaterials industry. Such models inay be useful in attacking such problems ns t hc svlcct ion of tlie h s t coinposit ion of wastt? niuterial from the gaseous diflusioii p r o w ~ b ,t tit, choicc anlong altcriiat ivc designs arid inodcs of operation of nuclear rmciors, thc choice nn1oiig altcrnativc patterns of material flow ilrnong the various elenient~of t lie nuclear materials industry,

SAUL 1. GASS

322

and the assessment of the cost or value of materials or electric power supplied to or yielded by the industry. The model depicted concerns an idealized version of the gaseous diffusion process in steady-state operation. Three types of relations are taken into account: (a) material balance within the plant; (b) the scale of plant required to generate given material flows; and (c) the irreversible nature of the gaseous diffusion process. The principal contribution of this paper is in suggesting the importance of the last consideration in certain applications, and in describing a way t o handle it by linear programming. I n Saaty and Webb [186] we have not only a discussion of the formultition and solution of a rather large problem in scheduling aircraft overhaul, we also have a report on the application of the analysis of sensitivity to critical parameters. The authors note that this sensitivity analysis yielded information regarding the required accuracy of the parameters, and the desirability of their inclusion in the final version of the solution. Longrange planniiig is interpreted by means of these parameters, hence tests of new policy or planning can be based on the sensitivity analysis. As an example, the system has been found more sensitive to man-hour distributions than to the error distributions of the number of items allocated, contrary to existing expectation. Another problem of interest is the prediction method for determining releases to overhaul of aircraft engines. h mathemat>icalmodel using renewal theory has bcen tested for use as a basic input to the distribution modcl. 6. Summary of Progress in Related Fields

6.1 Curve Fitting

Basic work in this area, which transforms problems of “best fit” into linear programming problems, is contained in Kelley [121], Charnes and Lemke [26], Wagner [214], and Wolfe [226]. Wagner treats the problem of “best fit” as applied to least absolute deviations and least maximum deviations. Ile shows that if the linear regression relation contains p parameters, minimizing the sum of the vertical deviations from the regression line is equivalent to a p-equation linear programming model with bounded variables; and fitting by the Chebyshev criterion leads to a standard ( p 1)-equation linear programming model.

+

6.2 The Theory of Games

Tlic rclatiowliip between the problem of linear programming arid tlio problem of the zero-sum two-person game is well known. Applications, computational tcchniques, and theoretical advances from either field


323

hliould bc continually evaluatcd as to how they affrct oiic anothcr. Daiitzig [37] considers a two-movc gamc with perfect information. This leads to the problem of finding a global niiriimum of a concave function over a convex domain. It is shown that the global minimum can be obtained by solving a linear programming systcni with the side conditions that at least one of certain pairs of variables vanish. This is equivalrnt to a linear programming problem with some iiitegcr valued variables. In Dartmouth [49], Kcmeny and Thompson rcview thc essential elcments of the fictit,ious play method for solving matrix games. The authors have incorporated certain computational features which tend to decrease the total computational time. The procedure produces very accurate answers and does not have any round-off error. The method has been coded for an IBM 704 and can solve problems with dimensions of m 5 200, n 5 200 and mn 5 4000. A 10 X 10 game was solved in 10,694 steps in 1 niin with an accuracy of 0.00004 and a 17 x 18 game in 35 sec. A 20 X 20 problem deliberately made up to be as difficult as possihle for the method took 15 min and had an accuracy of 0.0005. 6.3 Stochastic Linear Progromming

An excellent presentation and survey of trhe work done in this area is given in Madansky [140]. The author reviews two types of stochastic problrms, the “wait-and-sce” and the “here-and-now.” I t is the latter type problem which is of interest and RIadansky rcviews the work done in this area. 111 addition, he discusses a wide varicty of rcferences. Other rrccnt publications in this ficld are Madansky [I411 and Wagner 12091. 6.4 Nonlinear Programming

As it is planned to includc n separate survey article on nonlinear programming in one of the future volumes in this series, we will only make a few notes and references to this broad and important area. A general survey paper which describes the various problem areas a i d methods for solution is one by Wolfe [226a]. There the author classifies the problrms based on the combination of the linearity or nonlinearity of thc objcctive function and constraint srt. He also classifies available computing procedures as the walking (simples nicthod for linear programs), hopping (the simplex method adaptcd t o minimizing a eoiivrx function hubject to liiicar constraints), and crecping (gradient mcthods). In addition to the work of Frisch and Zoiitendijk which are described in this survey, we find variations of the gradient method in the work by Lenike [132], Rosen [181], and Fiacco rt al. [66n]. I n addition we have thc differential gradient method of Arrow et al. [7]. The study by Witzgall [224] contrasts and discusses the procedures of Frisch, Lemke, and Roseti

SAUL I. GASS

324

as they apply to the linear casc. Witzgall describes the close computational relationship between Frisch and Rosen and the relationship between the Lemke Computational Tableau as a generalization of the Simplex Tableau. Computer codes for the Frisch, Zoutendijk, Rosen, and F.S.B. procedures have been developed. For example, Rosen notcs that his method has been coded for a Burroughs 705 and IBM 704. For the special case of minimizing a quadratic objective function subject to linear constraints, we cite the work of Wolfe [226], Beale [ll], and Markowitz [152]. Wolfe’s procedure is an adaptation of the simplex method and has been coded for the IBM 704. The use of this procedure calls for a transformation into a much larger problem of the order of m n equations and m 3n variables. Wolfe notes that the largest problem solved with this program has 90 constraints and 192 variables and took 230 min and 359 iterations. I n addition to the above procedures, the cutting plane method of Kelley [121] and Cheney and Goldstein [29] has been applied to general convex programming problems. From Kelley’s work we cite the following brief description of the procedure. The method uses the notion of cutting planes that has been introduced by Gomory and others in connection with techniques for solving integer linear programs. It is of interest to note that while Gomory’s method reduces to the Euclidean Algorithm when applied to a special casc, Kelley’s method is formally analogous to Newton’s Method for finding roots. We address ourselves to the problem of minimizing a linear form on a closed convex set R. It should be noted that this problem is equivalent to minimizing a convex function F ( x ) on a closed convex set R’. The former reduces to the latter since the linear form is also convex. Conversely, let R = {(qy)ly 2 F@), z&’). Then the latter problem has the form of minimizing the linear form y on R. Assuming that the problem has a bounded minimum we imbed that part of R containing a minimum point in a compact polyhedral convex set S and minimize the form on S. This is a straightforward linear programming problem. We now cut off a portion of X containing the just determined minimum point b y passing a hyperplane (cutting plane) between the minimum point and R, obtaining a new convex set S. The linear form is now minimized on S. This process of making cuts and minimizing the linear form on the resulting convex set is continued. In the limit we obtain a point in R for which the linear form is minimized. As just described, the convex programming problem reduces to solving a sequence (generally infinite) of linear programs. I n practice, of course, this sequence is truncated after a finite number of steps a t a point when the desired degree of approximation is obtained.

+

+


325

7. linear Programming Computing Codes and Procedures

If the importance of a computing algorithm can be measured by thc number of different machines for which corresponding codes have brcn written, then the simplex algorithm and its variations rank high indeed. In fact, whenever a new computer is announced, one can usually find a linear programming code on the list of codcs already developed or to be developed. It is also interesting to note that in their recent book, Mathematical Methods for Digital Compulers, by Ralston and Wilf [177], the authors deemed it worthwhile to include a chapter on “The Solution of Linear Programming problem^."^ I n gathering information for this section we attempted to be as inclusive and exhaustive as possible. It must be recognized that data of this sort does not remain up-to-date for any great length of time. Most of the information was supplied by the manufacturers or other appropriate sources such as computer user groups. No attempt was made to measure the efficiency or adequacy of the cited codes. The time estimates or comparisons given are taken from the program write-ups. Unless otherwise specified, the write-ups of the codes and card decks are available from the manufacturer or the associated computer users group. Also, unless otherwise specified, for the general linear programming problem, m is the number of equations and n the number of constraints; for the transportation problem m is the number of origins and n the number of destinations. 7.1 Digital Computers

7.1.1

THE IBM

701

SHARE PROGRAMS

Table I1 gives a list of available program for the 704 which have been submitted t o the SHAREDistribution Agency. Most of the codes are described in this section. 7.1.1.1 The Rand Code (RS Lrsl) This linear programming system uses the modified simplex procedure with the product form of inverse. It will handle as many as 255 equations and there is no restriction to the number of variables that may be used. The use of double precision floating point arithmetic in the internal calculations is adequate to solve problems involving as many as 255 equations because there are stringent, but not unreasonable, limitations on the input data. With the algorithms used, the computing time per iteration increases very little for a n increase in the number of variables. A precise time esti(Chapter 25, which was written by I ) w n If. Arden.)

SAUL I . GASS

326

T.\HI,E

Itlrntity C E FLP C E SCRL IR M L ~ IB TFL IB TFM

MI C N F ~ N$

NY

LPS2 TRI

RS LPSl RS MI 13C MAP SC MUSH

sc

XPCI) SM LPMP

I’K IP01

1’K

IP02

PK I P O 3 RS

BPI

RS

QP2

II

IM(’

Tit Ir

7-21-59 12-2S59 10-12-59 624-58 (i -17-59 9-14-58 9-11-58 12-27-57 1-08-57 341-60 10-17-58 10-1 7-58 7-09-59 7-03-59 7-01-60 7-0 1-60 7-01-60 10-0 1-60 1-01-61

FORTRAN L. 1’. SCROLsystem Machine loading-grneralized transportation l‘ransportation-Himgarian mrthod I)rundess version of IB TFL Capacitatcd network flow Ilrumless version of RS LPSI Transportation-Hungarian nirthod “The Rand code” FORTRAN mathematical programming system Tnput-output for sc MAP Small-problem linear programming Transportation 1.incar programming matrix structure print Integer programming Tnteger programming Integer programming RS L P S ~with upper bounds The simplex method for quadratic programming

mate is impossible, but one typical problem with 50 equations and 100 variables was solved in 20 min. The basic machine needed for this system is the IBM 704 with 4 logical drums, 4 tapes, and 4096 words of magnetic core storage. One additional tape may be used optionally for storing output results. One other tape may be used for input to the data assembly program which precedes the main code. Hence the input is from cards or tape and the output is the on-line printer or off-line printer. The card punch is used by the data assembly and also for punching restart information. A modification of this code, N$ L P S ~ ,has been written which uses a 32K machine and no drums. The code allows for the use of a curtain which restricts the solution to a predetermined set of vectors. The curtain can, of course, be lifted during the later stages of the solution. Parametric linear programming of the constant terms is also available. A descript,ion of this code in French is given by Pigot [168]. 7.1.1.2 Pigot French Code (PF PLI) This code is a modification of RS m p l done by D. Pigot of SociBtB des PBtroles Shell Berre. It includes such changes as being able to solve problems with up to 640 equations (with parametrization), and important


327

reductions in computation time. A write-up of the code in French is available [1681.

7.1.1.3 SCROL Systeni

(CE SCHL)

SCROL is a system using the origiiial 704 L.P. codes developed a t Rand, as a basis but incorporating a whole new dimension of control for operational procedures. The system is designed to perform not oiily the composite simplex inethod including maiiy variations but alho to carry out automatically a number of the data handling, data editing, and peripheral fuiictions formerly provided for with auxiliary routines or not at all. These are activated by card-programmed, interpretive n~arro-iiistructioi~s. The whole repertoire of available operations is conibined into one system which exists physically on a system tape. The basic version is for an 8K or 32K 704 with drums. It will not operate on a 4K machine. There exists another version to enable one to run on the 32K machine without drums. For small to mediumsized problems, transformations will never get to tape with 32K. The basic version can be easily added to for spccial applications. Conceptually the system consists of three parts: (1) the system control which includes some of the minor macro-operations, (2) thr major macro-routines, and (3) the L.P. codes or subroutines. Other features include p:mnietric linear programming of the constant terms and cost coefficients; the finding of alteriiilte optimal solutions; the determination of the range of each cost which still maintains optimality ; the handling of multiple objective functions and right-hand sides; “curtains,” which restrict the selection of a new basis to the vectors in front of the curtain, if possible; and partitions, which are absolute barriers in that vectors after the partii ion cannot enter the basis. 11s LPSI,

7.1.1.4 SCR.IIiSII

MUSHis a subroutine fur solviiig a niediuni-size linear program. The size restriction permits miiiimum transfer of data into and out of core and the me of single-precision arithmetic. This results in a routine whose speed compensates for its limited range. A modified simplex algorithm usiiig siiiglc-precision floating-poiiit arithmetic is uscd t o determine the optimum solution. A starting identity basis is made up from available unit positive vectors in the input data with artificial vectors supplied as needed. Theee artificials are driven out of the basis by assigning a relatively high penalty to them in the cost functional. Each change in the basis matrix is recorded as a change in thc inverse, which is niaiiitained in core. Iiouiitl-off errors, which accumulate in the

328

SAUL I . GASS

continuously updated inverse, are periodically reduced by a purification method due to Hotelling. Restrictions of the code are: (a) Two physical drums are required. (b) Input data must be in core a t the time MUSH is called in. (e) A problem of a t most 55 equations may be solved. Virtually any iiumber of variables can be treated if the machine used has more than 4K core memory. I n the 4K machine, the practical limit is about 128. If NMAX is the maximum number of variables for which the program is assembled, then the coefficients of the constraint matrix are limited to 2048 - R’MAX nonzero elements stored in compressed form.

7.1.1.5FORTRAN Linear Programming

(CE FLP)

The purpose of this program is to provide a flexible and easily modified code for solution of linear programming problems by the simplex method facilities. The program includes subroutines for installations with FORTRAK for: Input, Phase I setup, arbitrary transformations, pricing for composite algorithm, ratio test for composite algorithm, and Gaussian elimination. Output is interspersed as needed. A checking subroutine is also included. Restrictions are: (a) Components required-minimal FORTRAN installation but 8K core assumed. (b) Other programs required-FORTRAN master tape to compile. (c) Data-maximum number of rows in tableau including all optimizing forms, max m = 51.This caii be changed by recompiling, within memory limits. Total columns in rest mint matrix, excluding right-hand side and any artificial slack vectors for Phase I, niax 71 = 91.(Same comments as for m.) The entire tableau m X (a 1) is kept in core a t all times. All data are input and output as fixed point; input not to exceed 10 digits of which 5 are decimal fraction.

+

7.1.1.G FORTHAN Mathematical Programming Systems (RSM I ) RS ~1 (Rand Corporation, Mathematical Programming System One) is a system of 704 programs for the solution of linear programming problems. In its design, emphasis has bccri placed on producing a system of routines which could be readily modificd in any of its parts to serve special needs. The basic system coiisists of twenty-two routines, of which three are concerned wilh input, four with output, six predominantly with control functions, arid iiiiie with computing. The system i i i its prcsmt form has bccw dcsigiied for 3 321i drurnless 704, usiiig all cquipmciit but the punch, and from one to four tape units. The program is self-contained, but requires the deletion of some alternate


329

sii1Jroutincs before using. The data format is tlic mine as that, for thc Rand liiiear programming cotlc I<S L P s l cwcpt iiig thni, tlccinial point Y iiiust l)c used and certain roiii rol c ~ t l of s LPSI must bc replnccd by thosc appropriate l o this system. The program consists of the FORTKAN I1 Bss loader followed by relocatable hinarics. Alost of the system parts were 11. compiled with FORTRAN The system employs the revised simples method, in which the simplex method basis-inverse is maintained in its esplicit form. It can handle linear programming problems having up to 97 const mints, 299 (nonartificial) variables, and 2499 nonzero niatris entries. The parts of the computation sensitive to round-off error may be done in either single or double prccision. Accuracy on small problems done in single precision has been close to the theoretical maximum. Two linear programming problems mere used cstensively in testing the system. The first has 33 constraints, the second 64. The computing times given below wcrc found with the Rand Linear Programming System 1 (RS Lpsl-all double precision, product form of the inverse, considerable use of tapes in computation), the Standard Oil Company of California linear programming code (sc MusH-revised simplex method with explicit inverse, single precision, uses drum), and the single and double precision versions of the present Rand hlathematical Programming System 1 (RS M I ) . The difference in numbers of iterations done between L p s l and the other routines arises from the fact that the latter search the problem for singletons (columns having just one positive entry in the constraint matrix) and form a starting basis from them. L p s l does not automatically do this. If the heavy use L p s l makes of tapes in its computation is kept in mind, and the fact that its pricing-out operation is done in double precision, which TABLEI11 Kiimber of equations

33 64 33 64 33 G4

RS LPSl

SC MUSH

RS M 1 (SP)

Times (sec)-Computation Only 26 16 220 1300 Number of iterations (after initial basis) 69 33 35 136 110 Seconds per iteration-computation only 2.76 0.79 0.46 9.56 2.00 190

RS M 1 (DP)

73 1230 35 110 2.08 11.18

330

SAUL I. GASS

is not the CRSC in M I , the relstive tines per iterntioil of L r d ant1 ~1 seem to be a convincing argument fnr thc rffiricnry of thr product form of th r iiiversr, as used in L m 1 . A new program lis ~3 is under constructioit which will handle UP l o 4000 equations and will use single precision for all computations except for a double-precision “tidying up phase.”

7 . 1 . 1 . 7 70.4 Linear Programming System of Bonner and Moore This code is available from Bonner & Moore Engineering Associates, Houston, Texas, and is a general purpose code with special consideration given to the needs of the petroleum industry. The Boniier & Moore 704 Linear Progranlming System includes, as well as the linear programming code itself, a complete set of subroutines and subprograms for data loading, assembly, prc-edi ting, output, and output . ~with ~ a few subanalysis. Most of the system is written in F O R T R11, routines coded in SAP.Single-precision floating-point arithmetic is used throughout, and the main solution algorithm is a modification of the simplex algorithm. Direct, dual, and composite problems may be handled. Minimum computer requirements are an 8K 704, with four tapes and four logical drums. The system is written to take advantage of larger memories, and additional tape units may be used to advantage. As mentioned above, the solution algorithm uses single-precision floatingpoint arithmetic. Several measures have been taken to minimize the development of round-off error, however, and to detect and correct it when it appears. Some of the more important of these include significance analysis during the computation of new matrix and inverse elements, provision for automatic reinversion and matrix “clean-up” from time to time, and a n improved path selection algorithm which reduces the number of iterations required for solution by 20% to 50% over that required by earlier algorithms. The program is limited to a maximum of 2047 variables, including slacks and artificials. There are no independent limits on the number of restrictions or variables, although practical considerations limit problem size to approximately 150 restrictions on an 8K machine. Experience to date indicates that problem solution time of the order of 20 to 40 iniri may be expected for 100 restriction problems on an 8K machine. This time mould apply where no use was made of such techniques as curtaining, priority control, or pre-editing. Judicious use of these techniques would materially shorten the above estimate, as would use of a 32K core memory. The system also includes provirioii for automatic parametrization of the objective function or the requirements vector, or both simultaneously. The user has a great deal of control over the output of the intermediate cases during a parametric operation. For example, he may tell the program to


331

suppress output automatically if the parametrization results in a long sequence of very small steps, waiting until the solution has moved a significant amount before taking more than token output. Provision has also been made for revisions to the right-hand side, object8ivefunction, or both simultaneously after solution has been obtained, followed by postoptimal re-entry into the solution mode and resolution of the altered problem. Complete provision is also included for dump and restart procedures, in which all essential information from drums, tapes, and core may be preserved on a single tape. This information may then be recalled a t a later time during the same run (after a user-written program has performed some auxiliary function, for example) or a t some later date, The pre-edit option makes it possible to take advantage of prior knowledge of the probable structure of the solution, to decrease the number of iterations required, and cut down the actual size of the matrix being handled. The pre-edit routine will prepare a condensed matrix tape, with restrictions and column vectors pertaining to variables whose filial status is definitely known in advance deleted from the matrix. Thc smaller condensed matrix will be solved, and then the solution automatically restated in terms of the original expanded matrix. Other techniques for taking advantage of advance knowledge of the probable solution structure are also available, such as curtaining, matrix tape ordering, and the designation of “priority” and “prohibited” variables. In addition to the conventional listings of basis activities and shadow prices (with associated ranges of optimality and feasibility), elaborate output analysis and display options have been included in the system. These provide for several varieties of postsolution arithmetic on the final activities, including such things as the accumulation of weighted sums of final activities, the multiplication of such summations by priccs or costs, accumulations of totals a t several levels, the back computation of TEL levels in gasoline blending problems, and the display of results in a readily readable matrix format, with Hollerith descriptive information inserted. 7.1.1.8 The Transportation Problem

(XY T R ~ )

This program is designed to solve the basic transportation problem. The machine requirements include a 4K core storage, card reader, printer, 1 drum, and 3 tapes. The data restrictions are (m is the number of origins and n is the number of destinations) :

+ n I 850 m + 5 1850 m + 5 7500 m

for 4K,

)&

for 8K,

II

for 32K.

SAUL I . GASS

332

+ + + + + +

For all cases n 5 800 and m(n 1) 25m(n 1)/256 5 900,000. Only high speed storage is used if mn 4(m n - 1) 600 I high speed storage available. The coefficients cannot exceed 10’O - 1 in magnitude. The “stepping stone method” for solving the transportation problem is used. The major characteristics of the method are these: The stepping stone method is an iterative procedure. A distribution must be furnished at the start. A distribution is any allocation of shipments which satisfies all the requirements without regard to the total cost. At each iteration a new distribution is constructed with a smaller total cost than the previous. When an improvement is no longer possible the distribution is optimal with respect to the total cost and furnishes the required solution to the ~ the initial distribution automattransportation problem. Code NY T R finds ically. The demand for each destination is assigned to the origin whose corresponding unit transportation cost is smallest, provided the origin has sufficient supply available. Otherwise, the next cheapest origin is tried, and so on until the demand is met. The epsilon method is used to guarantee that a basic feasible solution is constructed. As an option, an initial distribution may be furnished as part of the input. 7.1.1.9 The Transportation Problem-Flow

Method (IB

TFL)

The transportation-flow program is designed to solve the transportation problem on the IBM704 in accordance with the flow, or Hungarian, method as proposed by L. R. Ford, Jr. and D. R. Fulkerson. Particularly for large matrices this program affords substantial gains in time over NY T R ~ which , is based on the stepping stone met,hod. The minimum machine requirements are : 4096 words of high speed storage,

card rcader, 1 drum (logical unit #1), 1 tape (logical unit #6),

on-line printer or tape #10 for output. The problem restrictions are

5 600, no restriction on WL, n) 5 high speed storage available. rk

618

+ m(n + 1) + 2(m +

The numerical data supplied must not contain numbers exceeding 10’0 IB TFL and NY T R ~(stepping stone method) have given the results shown in Table I V for the main calculation (exclusive in- and output) : 1 in magnitude. Comparison tests between


333

TABLEIV hfatrix (ni X

71)

130 X 30 I60 x 30 190 X 3 0 220 x 30

NY T R ~

2'13" 4'56"

I B TPL

1'30" 3'34" 4'00" 4'58"

7'05" 11'58"

7.1.1.10 Transportation Code (sc XPCD) This code solves the standard transportation problem using the algorithm presented by Munkres [161]. It is well-suited for tall distributions, i.e., when one dimension is much larger than the other. It requires only one tape, assumes a n 8K core storage and four drums. Infinite costs arc indicated in a condensed notation and previous shadow values may be used to start problems. Restrictions on input data are: number of columns number of rows

5 31

I 1500

largest cost coefficient largest availability largest demand

5 131,071

5 67,108,836

5 33,554,431.

7.1.1.11 Transportation Algorithm and Code I I This code [67] uses a variation of the Hungarian method and was writ11. ItI is available from the author, Merrill M. Flood. ten in FORTRAN Computational experience yielded the following computing times for sample problems. These problems were developed by a random number generating scheme which is st part of the program: Size

No. of problems

Average time (min)

29 X 116 29 X 29 20 X 29

2 3 13

2.47 0.53 0.25

One of the 29 X 116 took 3.03 min with Code I1 and 3.17 min for IB TFL. The code can handle problems up to 60 X 150. 7.1.1.12 Capacitated Network Flow Program

(MI

CNF1)

The program determines a flow pattern over a general network so that a linear cost function of the branch flows assumes its minimum value.

334

SAUL I. GASS

The branch flows arc restricted to being nonnegative and less than or equal to the capacities of the branches, and flow into and out of the nodes is conscrved. The nctwork flow problem is a linear programming problem arid can be solved by general lincar programming techniques, but doing so complicates the solution procedure since the network structure can be utilized to give a more efficient method. Many of the problems that have been solved by linear programming techniques can he formulated in terms of network flow, thcrcby utilizing this procedure. The program is complcte with input and output routines so that information can be transferred into and out of the computer conveniently. The input is assumed to be in integer form and integer arithmetic is used throughout. The results therefore are exact and are given as integers in the output. The only restrictions on the problem due to computation limitations is that the cost value associated with each branch and the branch capacity values must be less than 262,142, and the number of branches and nodes in the network must satisfy the relation: Five times the number of branches plus the number of nodes must be less than the magnetic core memory size minus 530. I n terms of the network, the problem is defined as: minimize

N+ 1

C

cIPz,, (cost function)

%,3=0

subject to the constraints N

C (xlj - x j , )

=

s, (conservation of flow a t the i = 1, 2,

. . . ,N

nodes)

j=1

0 5 x,, 5 M,, (flow restrictions) where, for an arbitrary branch (i,j), x t J = flow through the branch directed from node M,j

=

capacity of branch

cl,

=

cost per unit flow (assumed nonnegative)

i to node j

s1 = outside flow supply at node i (may be positive or negative)

N

=

number of nodes in network excluding the source and sink nodes.

The computer program solves the above problem for any specified total flow into the network. The total flow is the sum of the flows in the branches corresponding to the positive 8, terms. If the total flow is the sum of these s,, then the maximum flow through the capacity restricted network may be specified. Unless some other part of the network restricts the flow to a smaller value, the total flow allocated will then be the sum of the positive s,.


7.1.1.13 The Machine Loading Problem

335

(IB M L ~ )

The purpose is to solve any problem which can be cast into the following inat hematical form : Subject to the linear restrictions

5 a,,x,j I M , (i

=

I, 2,

J=1

. . . , m)

find values x,, w hch niiuimize the lincar function C C c,,xt,. The program requires n 704 (without special devices) with 8K memory, 2 tapes, and 4 logical drums. If data input is to be from tape rather than cards, a third (input) tape unit is required. The admissible size of problems has the upper limit : number of rows plus number of culumiis not to exceed 318(m n 5 318). All of core storage is used. The program also Korks on a 32K machine for the samc size problems. The method used for solution is a gcneralizntion of the ‘'stepping stoile method” as outlined in [61].

+

7.1.1.14 Multidimensional Distribution Problem This program is designed to solve problems with the following formulation: Find a set of integers x,,~,such that j=lh=l

55

Zijh

=

bj

i=l h=l

and

f 2 2 c,,JLx,,his a minimum.

t=1 j=lh=1

Three methods have been programmed: approximation method, method of optimal regions, and method of reduced matrices. The dimensions of thc problcms for each method are as follows:

.I pproximats solufiori I,. 5 a0

c p, I900 1.

1=

1

SAUL I. GASS

336

p i is the number of “shipping points” in the i t h dimension. PlpZ * ‘ * pk 5 60,000 Computing example: k = 2, pl = 186, p2 = 15; time, 19.5 min.

Ic

=

4, pl

=

p2 = p ,

=

p 4 = 3; time, 12 sec.

Optimal regio I I s k = 2

p1

+ p2 I 1200

p2 I 10 cij < 104 This is for tall matrices. Uses 8K memory, 5 tapes, 2 drums. Computing example: 7 X 1152; time, 45 min. Reduced matrices k

5 20

k

c p i 2 900

i=l

PIP2 ‘ * ‘

pk _< ~00,000

Cijh

< lo4

Requires 8K memory, 6 tapes, and 1 drum. Time given approximately by s2/100 min, where s = p i . These codes are available from B. Caller, University of Michigan Computing Center.

c

7.1.1.15 Integer Programming

(PK

1 ~ 0 11, ~ 0 21, ~ 0 3 )

Codes I P O ,~ 1 ~ 0 2and , 1 ~ 0 are 3 all designed to solve integer programming problems; that is, to solve linear programming problems with the additional 2 restriction that the variables involved are integers. Codes I P O ~and 1 ~ 0use only the method of Comory [90a] ; consequently, all coefficients remain integer throughout and the round-off error problem is avoided. Code 1 ~ 0 3 uses the methods of Comory [89, 90a] and introduces fractions into the course of the calculation, controlling round-off by various methods. Code I P O ~uses a simple form of [goal and each individual pivot step is faster than , usually requires fewer a step in the more complicated 1 ~ 0 21;~ 0 2however, steps than I P O ~arid can solve some problems that are intractable for 1 ~ 0 1 . IPO3 is probably the most generally effecfive of the three, though 1 ~ 0 is 2 often better on degenerate problems. The performance of a11 the codes is quite unpredictable. Failures (i.e., endlessly long runs) have been recorded with as f c w as 10 variables, successes with as many as 90. Generally, the difficulty goes up with the number of variables, the size of the matrix coefficients, and the density (lack of sparseness) of the matrix. However,


337

cvcn tlicse characterist ics are not always rcliablc guitlcs as n1:it ricw 1 hat, arc almost, idcntic:d havc producc~lw r y diffcrriit ruiis. For any problem thc niaxinium number of variablrs, i i , for 1ro1 and 1 ~ 0 2is 35 for ail 8 K memory, 100 for 32K. The maximum number of constraints is 75 - n for the 8Ii mwhiiie, 200 - n for the 32K. All coef3 a 3°C core storage; three magnetic ficients must be integers; 1 ~ 0 requires tape units, and tape-to-printer equipment are required in addition to thc “mininium 704.” For any problem, thc iiuinbrr of variables, n, must not exceed 100, and for most problems thc total iiurnbcr of objective funct‘ ’ lolls - 72. and constraints must not cxcecd 200 All coniputatiotis iii the program are floating point. There is no round-off error unless some matrix element reaches a magnitude of the order of lo8. This usually occurs only when many elements of the matrix have begun to oscillate violently between very large positive and negative integers. This condition indicates that the problem is probably intractablc for the code and that it is unlikely that any solution will be reached ill a reasonable amount of machine time. For 1 ~ 0 1 the , time elapsed during onc iterative step varies widely with the size of the problem, the numbers currently in the matrix, and other factors. A rough estimate is 75 pivots per minute for a problem of 20 variables and 20 constraints, 45 pivots per minute for a 30 by 30 problem. The number of pivots required for any particular problem is almost wholly unpredictable, although it tends to increase with the size of the matrix and decrease with the sparsity. For 1 ~ 0 2a, rough estimate is 65 pivots per minute for a problem of 20 variables and 20 constraints, 40 pivots per minute for a 30 by 30 problem. For 1 ~ 0 3a, rough estimate is 50 pivots per minute for a 30 by 30 matrix, 80 per minute for a 20 by 20. 7.1.2

THE IBM

7090

7.1.2.1 m/90

This code was developed by CEIR, Inc., Arlington, Virginia, for a number of petroleum and chemical organizations. At, this writing, it was not generally available. The code is approximately 10 times faster than the comparable RS Lrsl code for the 704. I t can handle 512 equations including the objective function and a n indefinite number of variables. It has double precision floatingIt point arithmetic and has the parametric and other features of SCROL. automatically determines if the problem can be solved using internal core storage. I t allows for the naming of rows as well as columns which enables the selection of a specified subset of rows and columns to be solved as a subproblcm.

SAUL I . GASS

338

The system also includes a facility for updating the matrix and allows fnr the addition or deletion of a row or column and the change of an individual coefficient. It is a system with a built-in compiler which greatly facilitates the addition of a ncw routine or a change in an old routine. 7.1.3

IBM

650

7.1.3.1 Linear Programming-10.1 .001 This program is in fixed point arithmetic. All numbers are 10 digits with the decimal point assumed between the 6th and 7th digit. It is designed to accommodate systems of m rows and n columns where m 5 30, n 5 59, and m(n 1) < 1400. The values of m and n pertain to the system after the slack vectors and art,ificial vectors have bccn adjoined.

+

7.1.3.2 Linear Programming-10.1.002

This program has been written for processing linear programming problems having a maximum of 97 equations not including the objective functions and an unlimited number of variables. This program uses the method of “recursive generation of vectors for the modified simplex method” as described by Eisemann [6Oa]. 18-digit double precision floating point computation is used. The programmer has chosen for E in check sum computation for a problem consisting of 33 equations and having a magnitude of the largest element of 4 X loa. For problems of larger order and having the same magnitude may be more suitable. of the largest element it is felt that E = 7.1.3.3 Linear Programming-10.1 .OOg This procedure will solve linear programming problems containing up to 40 equations and any number of variables. Fixed decimal arithmetic is used, with five digits on each side of the decimal point. Using the modified simplex method, column vectors representing properties of the different variables are brought into the calculation one a t a time, multiplied by the B matrix to bring them up to date, and checked to see whether they should be brought into the basis or not. They are brought into the basis if the profit function is more negative than an arbitrary tolerance. Successive passes of the input cards for all the variables are made until, on an entire pass, no variable is brought into the basis. Then the tolerance is lowered and more passes are made in the same way until, with the smallest practical tolerance, a pass is made during which no variable is brought into the basis.

339


7.1.3.4 The TransportationProblem-10.1.003 This program has been designed to solve the standard transportation problem. It also is possible to obtain alternate optional solutions. A problem is “small” if m, n satisfy the following inequalities:

2(m

+ + [y]+ [m + n - 1 ] + [ m

+ ;

-

‘1 < n

1200,

< 100.

Similarly, a problem is characterized as being “big” if m,n satisfy one or both of the inequalities:

2(m

+ + [ y ]+ [”+; ‘1 + [m + n - 1 ] > 1200, 72)

-

n

> 100.

The brackets indicate that the fraction is rounded up. The present program will solve a transportation problem for which m, n satisfies either the “small” or “big” inequalities. I n the case of a “small” problem, the entire cost matrix (cij) is stored on the drum. Only a single row of the cost matrix is stored on the drum in a “big” problem and the cost matrix must he continually circulated through the read feed. It is therefore advantageous to process a given problem a s a small problem whenever possible as this eliminates the C i j card reading time inherent in the big problem. All input data are restricted to a maximum size of five digits. The iterative method employed is essentially the same a s the “stepping stone” method. All operations are performed using fixed point arithmetic. 7.1.3.5 Linear Programming Code for the Augmented 660-lO.l.006 The composite algorithm is used in somewhat the same manner as used in the RAND Corporation RS Lpsl 704 program. All computations are performed in single precision floating point. The size of the problem which can be handled is restricted by the relationship ( m 2)(n 2) 5 1900, where m is the number of restrictions and n is the number of independent variables excluding the original basis variablcs. The precise time required per iteration depends on whether the existing solution is feasible, and on the number of zeros in the matrix. As an approximate figure, a problem with m = 17 and n = 40 requires about 20 secs per iteration. The basic 650 with floating point,, index registers, and immediate access storage are used.

+

+

SAUL 1. GASS

340

7.1.3.6 Network Problems This code is designed to handle iietworklike problems of a type more general than the transportation problem. It can solve such problems having a maximum of 96 nodes. It was written by C. E. Lemke and T. J. Powers of the Department of Mathematics, Rensselaer Polytechnic Institute. 7.1.3.7 A Linear Programming System This code for the 650 has been especially designed to meet the requirements of the petroleum industry. It is probably the most widely used 650 linear programming code and is available on a fee basis from Bonner and Moore Engineering Associates, Houston, Texas. It is a group of inter-related programs, with the output of one program serving as the input for the next. All of the programs were developed for use with a minimum 650.The output procedures were designed for a 407, but could easily be adapted to a 402 or 405. The code uses a computational modification of the simplex method which permits omission of the identity matrix from computer memory. Running time is approximately 15 msec per element per iteration (based upon the number of elements in the full matrix, including the identity matrix). Limiting size is determined by the inequality (m 2)(n - m 3) < 1731,

+

+

where m and n refer to the number of restrictions and variables, respectively, including the identity matrix. For example, for a system of 40 restrictions, up t o 78 variables, including the 40 variables in the original basis, may be handled. Operating speed for a matrix of this size would be approximately 50 to 60 sec per iteration. Experience has shown that optimization is usually obtained in a number of iterations ranging from 1 to 2 times the number of restrictions, when using this calculation procedure. Provision is made for assigning objective function coefficients t o the variables in the original basis, if they represent alternate dispositions or slack variables with real debits or credits attached. The output of this section of the program consists of a single card punched a t the conclusion of each iteration, containing an iteration count, the current value of the objective function, and the identification of the variables entering and leaving the basis during the current iteration. There is no card reading or card handling of any sort during the process of optimization. When the optimum is reached, the program will load the answer punchout routine described below. Provision has been included for detecting and identifying an unbounded solution, and for resolving degencracy.


341

Although the entire calculation is done in fixed decimal arithmetic, special provisions have been made to minimize rounding errors. Experience with this program has shown that these errors do not accumulate to significant levels within the number of iterations usually required to achieve optimization. Although some preliminary scaling of data is necessary, this rarely needs to go beyond a careful choice of units. The program is completely trapped against loss of leading digits through overflow conditions. The answer program is read into the computer after optimization has been achieved, and controls the punchout of the final solution. This punchout consists of three parts. The first group of cards includes one card for each variable in the final basis. This card contains a four digit number identifying the variable (arbitrarily assigned by the user when preparing the original data), the final activity of that variable, (b%),the row number in the final matrix correvponding to the variable, and the range of values (C‘,) which its objective function coefficient could take without destroying ‘ ins one the optimality of the current basis. The second group of cards conta‘ card for each variable not in the final basis. The information on these cards includes the variable identification, the shadow price, the column number, and the range of nonzero activities which the variable may take without destroying the feasibility of the current basis. The third group of cards is optional. It is a reloadablr punchout of all of the nonzero elements of the final solution matrix. After completion of the punchout, the solution matrix remains uiidisturbed in memory. At this point, new objective function coefficients may bc introduced through the use of an auxiliary “cost change” program. The computer will search out the row or column in which the variable in question is located, revise the objective function coefficient, determine whether optiniality has been destroyed by the change (or changes), and reoptiniize if necessary. In similar fashion, it is possible to specify a new requirements vector. The computer will multiply the new requirements vector by the inverse of the original basis, deriving a new solution vector in ternis of the present basis variables. It will then test the new solution vector for irifeasibilities (negative elements). If any infeasibilities exist, they will be removed by manipulation in the dual simplex algorithm. Absociated with this code is a matrix generator which nutonlatically develops the input matrix for a large class of blending problcms. 7.1.3.8 Multidimensional Distribution Problem

The rcadcr is referred to the similar section in the 704 code descriptions for a definition of the notation and terms which follow:

SAUL 1. GASS

342

Approximation procedure

k 5 7 1 5 p1, pz, . . , pk 5 15 Cbjh 5 lo5. For k = 4, pl = pz = p3 = p4 = 3, the code took 2 min arid 35 sec to solve the problem. Reduced matrices k 5 7 1 5 P l , pz, . . * , p , L 15 Cijh

s (s

-k

5

lo4

+ 1) I 235,

k

where s =

C pi.

i= 1

7.1.3.9 Dual Simplex Algorithm This program uses the dual simplex algorithm and is available from Mrs. Ann L. Hillier a t the Applied Mathematics and Statistics Laboratory, Stanford University. Table V gives values for m and n based on the number of negative coefficients in the objective function (maximizing) and the number of equalities in the system. TABLEV Segative coefficients cf) Equalities ( e ) c = o O<eis the element of the matrix in the zth row and j t h column and b, is the right hand side of the i t h equation. In conjunction with the amendments routine described below, these ranges facilitate parametrization of one variable a t a time; a pre-inversion routine is about to he developed and will be used for solving problems for which a good solution is already known, with a consequent reduction of calculation time. It will also be used to eliminate rounding errors in problems in which an optimum has been reached where the results are possibly suspect; amendment routines have been added, enabling alterations to the costs or to the right hand side of the equations to be made, or

SAUL 1. GASS

352

the nmw of any variable to be changed; prohlems which havo nrgai ivv ri&, h:ind fiitlrs which arc not opposiitc artificial s1:ickR may Iw ~olvrtl.

TABLEI. L.P. PROBLEMS RUN USINGI’ROGRA\IWC (a) ( I h w w ONLY) Time (min)

Sizc of problem

No. of cycles

Input

Calculation

Outpiit

168 168 168 168

229 238 25 1 222

3 3 3 3

35 62 52 43

7 7 7 7

103 X 181 103 X 181 103 X 181

85 107 105

2 2 2

7 11 8

5 8

Typc ‘C’

84 X 127 84 X 127

167 146

2 2

29 24

9 9

Type ‘D’

139 X 210

299

3

93

7

Type ‘E’“

534 X 371

426

6

62

23

(4 Type ‘A’

Type ‘l3’

200 200 200 200

(v4 X X X X

5

a An exceptional problem containing many zero or unit elements (i.e., practically a transportation problem).

RUNUSINGPROGRAMME (b) TABLE11. L.P. PROBLEMS (MAGNETICTAPESAND DRUMS) Size of problem

No. of cycles

Time (including input and output)

Typc ‘F’

276 X 255 276 X 255 276 X 255

527 557 480

4 hr 14 min 4 hr 54 min 5 hr 5 min

Type ‘G’”

260 X 220 2G5 X 270

187 249

35 min 32 min

Type ‘H’*

337 X 308

355

1 hr 11 min

a This type of problem was special in that about 100 to 120 of the equations were merely upper bounds restricting a variable.

A direct comparison of the programmes (a), (b), and an equivalent programme writtjen by Pigot for the IBM 704 computer gave for a test problem of size 219 X 264:-


Pigot coding (with IBhI 704) Programme (a) (Mercury) Programme (1)) (Mercury)

353

50 inin 48$ min 25 min

The Pigot programme is a refinement of the RS Lrsl programme, which was written for the IBM 704 in Paris, and gave a considerable reduction in calculation time with many problems compared with the earlier method. (c) A programme based on the revised product form algorithm is being developed and is hoped to be able to solve much larger problems, although no estimate of the time requircd for calculation is yet available. The programme will accept up to 500 equations and will include full parametrization facilities.

B. Uiscrttf. P ~ o g r a ~ ~ ~ i t i g Work has been started on a programme to solve discrete variable optimization problcnis, following the method described by Benders el al. [20]. It will be possiblc to attempt problems with up to 120 constraints and 120 variables but if the number of constraints approaches this figure the calculation time will probably be very long. This work is being undertaken to give a time comparison with the Goinory method which is also being coded.

C. Transportation Progra~nming The stc.pping-stone nicthod has been coded lor the hfercury computer without the use of the magnetic tapes. An itrration in the stepping-stonc mcthotl conaists of 3 main phases. ( I ) Finding the shadow costs, u Land ul. (2) Finding t h r most negative L U ~=~c I , - u, - itI, where c,,’s are the costs. (3) Transformation. In phase (2) the search for the negative u’zl involves examination of the t n X n elements of the cost matrix but phases (1) and (3) refer only to the m 11 - 1 elements of the solution. In the programme the elements are sorted, each cycle into :in order from which the shadow costs can then be found in two passes through the solution. The time for both phases and thc sorting can then be made proportional to n only. A further modification has been made to decrease the time spent in phase (2); a list of :t number of the most negative UI,,is formed and is used in succession in minor cycles cwnsisting of transformation and sorting only and phases (1) and (2) are callrd in only when necessary. The time for a minor cycle becomes proportional to 111 TZ and for a major cycle to ) J I x n ; the total time is probably proportional to (rn ~ h ) ~ although there may be a. small elemcnt proportional to n m ( m rh). Theoretical limitations on size are:

+

+

+

+

(I)

ni

(ii) (iii)

11

/)/

< 128 < 381 X [rt/32]

I 772

In prac.tire the total culculation timr works out to be approximately seconds.

n.

I)WlJtIl

(111

+ r1)*/60

O h 1 I / 0 !I

L)rrompositioii nicthotls huvc bccn titutlicd and a progranmw 10 c o i i j i i r i c . t ioii M itli thci simplex algorithm ia uridcr dcvelopmerit. (it)

+

IM

thoin i n

,

354

SAUL I. GAS5

(b) A programme has already been developed to solve a linear programming problem consisting of a number of transportation problems linked to one central simplex problem. The linking variables may consist of any or all of the availabilities and requirements of the transportation-problems. The method of solution consists of the following stages:-

I. A feasible solution of the simplex problem is found. 11. The transportation problems are solved for the values of the itvailabilities and requirements given in the simplex solution.

111. The resulting shadow costs from the transportation problems are fed as costs on the variables of the simplex problem.

IV. With these costs, R number of iterations are performed on the simplex problcm. Stages 11, I11 and IV are repeated until no further progress seems likely V. The complete linear programming matrix is assembled. VI. Iterations are repeated to reach the optimum. Preliminary and very approximate indications give a calculation time equal to the time required to solve the simplex problem, plus 50%, plus 3 seconds for each transportation variable. If T is the number of transportation problems of 7n, X n, (t = 1, 2, . , . , T) attached to a simplex problem of size 700 X na, then the following conditions are the theoretical limitations on the size of probleins possible to solve with this coding. (i)

YNO

+ 2 ( m ; + n; - 1) I1023

(ii) no

5 1023.

(iii)

710

+ (total no.

(iv)

?)It

I 64

(v)

/It

I128

of allowed movements in the trunsportation problems) about 4000.

(vi) 2’ I 40.

7.1.16 FERRANTI MAHK I The following report prepared by the programming group of The Koninklijke/Shell- Laboratorium in Amsterdam describes their work and a number of programs for the Mark I. At the KoninklijkelShell-Laboratoriumin Amsterdam (KSLA) a small group is working on the mathematiral and computational aspects of mathematical programming problems. The group forms part of the industrial mathcrnatics department] which among other things operates a Ferranti Mark I* computer (fast memory about 800 words (20 bits), drum nicmory 32000 words; fixed point binary arithmetic; speed about 1 msw per instruction, ont’ :Lddress rod(&). 1 . L i t ~ r u rl’rogrumminy

(a) The straightforward simplex algoritlini was coded for the Ferraiiti macllirie in 1956. A great number of linear programming problems has successfully been solved with this code including problems with about 150 constraints and 200 non-basic variables


355

(in about, 4-8 hours). Sitice 1950 it ntinrhrr of aincndmrnt routiiws and post-optiin:il programs havc hcrn m:idr. (b) In 1959 a romparativr stucly was m:dr of srveral roniputational algorithms for the simplex method, viz., the straightforward algorithm, the explicit inverse algorithm, the product-form algorithm and thc reviscd product-form algorithm. The last algorithm was developed at the same time: it will br rnrntionrd in point l(r). In this study the number of multiplication oprrations, the numhrr of additional bookkeeping instructions, the quantity of numbers to tie proressed, thr arcurary, the flexibility and the complexity of the codes werr ronsidrred. Sevrral assumptions were made about the fullness of the LP matrices, about the way variables in t h r original basis behavc etc. Thesc assumptions were tested with the hvlp of some practiral large linear programming problems and by using a special program which counted the number of multiplications in the different operations of the product-form algorithm as coded for the IBM 704 by the Rand Corp. There appeared to be good agreement between the theoretical formulae and the experimental results. The conclusion of the study was that the revised product-form algorithm i R the fastest for large problems and a large romputrr, the explirit inverse algorithm being the sccond best. ( c ) The revised produrt-form algorithm already nicntioned in point l(b) haB been developed. It is a major revision of the prodwt-form algorithm and diflrrs from it in the following respects: (i). Instead of calculating the shadow prices anew in each iteration, only those are recalculated which actually change, i.e., the new shadow prices are found by calculating the pivot row elements and by transforming the old shadow prices with the help of these pivot row elements. (ii). The main row calculation is performed in the same way as the calculation of the shadow prices in the product-form algorithm, hence consists of two parts. However, in the first part (backward transformation) those prices are already found which belong to variables that have once been in the basis. This reduces the second part considerably and makes it possible to remove columns from the original matrix which are or have been in the basis. (iii). If a variable re-enters the basis the forward transformation (updating the main column) is started there where the variable last left the basis. (iv). During a re-inversion pivot elements are chosen in such a way that a minimum number of non-zero elements is created in the new “eta vectors.” If a singular matrix is re-inverted there will be an automatic rejection of one or more columns. More information on the comparative study of point l(b) and the revised productrform algorithm can be found in Zoutendijk [228].

2. Transportation Programmang The stepping-stone method has been coded for the Ferranti Mark I* computer. As a starting procedure a revision of Vogel’s Approximation Method is used in which that column is always selected which is not yet full and for which the product of the requirement and the difference between the smallest and second smallest cost figures is largest. Scveral other starting procedures havc also been coded, e.g. the NW corner rule and the procedure of taking the columns in consecutive order and filling them in the best possible way. Comparison of computing times showed that the modified VAM method is about 30% faster than the best alternative. 3. Xetwork Flow Problems

The flow method (primal-dual method) for solving network flow problems taking into account both cost figures and capacity limitations for the flow through the branches of

356

SAUL 1. GASS

the network has been coded for the Ferranti Mmk I* computcr. I t can handle a maximum of 150 nodes and 1200 branches in the network, and is about 5 times faster thari the linear programming code written for the same computcr. Incorporated are: (a) A method for finding a pair of initial solutions (one for the primal problem and one for the dual problem) which satisfy the complementary slackness relations. The method is in particular applicable if some cost figures are negative. (b) A method for making post-optimality studies. If a network-flow problem is solved by the flow method, a new problem obtained by changing the total amount of flow, some capacities, of some cost figures, can be solved by starting from the pair of optimum solutions obtained for the original problem and its dual problem. A publication of the details of these methods is being prepared. 4 . Integer Programming Problems

The combinatorial method for solving integer linear programming problems in which all variables may assume the values 0 or 1, Benders el al. [20], has been coded for the Ferranti Mark I* computer. The program is written for problems involving about 40 inequalities and 40 variables. The calculating time is proportional to the number of constraints and depends exponentially on the number of variables involved. Experiments have shown that problems involving up to about 25 variables can be solved within a reasonable time (a few minutes to half an hour). 5. Nan-Linear Programming A complete set of procedures has been developed for the convex programming problem, i.e. the problem of minimizing a convex function in a convex region. These procedures are called methods of feasible directions and work along the following lines: (a) The starting point must be feasible (if such a point is not available, a phase I procedure must be applied to find one). (b) Each iteration consists of the determination of a new feasible point with a lower value for the objective function. It can be divided into two parts: (i). The determination of a usable feasible direction (i.e. direction along which it is possible to make a step without immediately leaving the feasible region, and along which the value of the objective function decreases). (ii). The determination of the length of the step to be taken in that direction. (c) The sequence of points obtained will be such that the values for the objective function converge to the minimum of the function restricted by the constraints. The different methods of feasible directions differ in the way the directions are determined. These direction-finding problems are always linear programming problems which can be solved by means of the simplex method. The methods are finite in the linear and quadratic case and can also be used if the convexity assumptions are not fulfilled (although a local minimum may be found in that case). The methods have been published in Zoutendijk [227, 2281. I n [228] equivalence is shown between many existing methods for the linear, quadratic or convex progrrtmming problem and the methods of feasible directions.


7. I . I7

357

FEI~RANTI PEGASUS

7.1.17.1 Cotlc $or S o b i n g tkc ( k t u t w l

Litiuir

l ’ r o ~ r ~ ~ ~ ~ il’roblctri ~tiitig

SIMPFIX Code uses contracted simplex method, fiscd poiiit arithmetic, and the problem sizes for the models of the computer are as follows: 4096 Work Drum Simpfix 16 Sinipfis 32 Simpfix 48

rta+2

n + l

16 32 48

192 96 64

7168 Word Drum Simpfix 7168: nz mri

5 166

n?

+

IL

_< 205

+ 9n + m _< 5973

Exnmplcs of times for solution: m

11

Total time (min)

11 35

180 62

20 20

LINPLEX In this code the matrix is read from tape and the basis is inserted for each iteration. The problem size is m 5 13 and n unlimited. A sample problem for m = 13 and n = 3070 took 15 min per iteration. 7.1.17.2 Codes for the Transportation Problem TRANSPEG Thc method wed is the iiivcrx bnhis and the problem size is constrained 1)Y m 5 128 n I 1792 (4096 word drums) ?a 5 4096 (7168 word drums); where a = number of sources, b = number of destinations and m = a b and ri = ab. Iteration time is approximately 7 n/100 see. Sample problem with a = 35, b = 93, m = 128, n = 3255 took 211 iterations with 139 min computing time and 152 min total time.

+

+

TRAXSTED This code uses the stepping-stone method and the problem size is re128 and n 4096 (either drum). A problcm of size a = 31, stricted by nz

,much of the switching-theory literature is concerned with efficiency.) An existence theorem about automata satisfying certain conditions would be a typical example of work done in this first area. The second main area is the study of the various behavioral descriptions of automata of all varieties. An algorithm converting one kind of behavioral description into another is an example in this area. The reader may be interested in a comparison of this survey with other papers with similar titles. Shannon’s survey [loo] gives, for the most part, a more detailed discussion of some of the material in Section 8. In spite of its early date, it makes rather interesting reading. After a short discussion

THE THEORY OF AUTOMATA, A SURVEY

385

of the human brain, computers, Turing machines, and logic machines, it presents a rather systematic study of machine game playing. It then goes into learning machines and self-reproducing machines. Holland [17] briefly reviews formal systems and logical calculi, recursive functions, Turing machines, stored-program computing machines, the theory of logical nets, information theory, and neurophysiology and psychology. The article [11 by Aizermnii et al. is not a survey of automaton theory but is an original paper mostly in switching theory, according to the definition I have given above. For the most part it is rather abstract switching theory, in that its methods do not merely apply to just one kind of hardware realization but involve structural considerations that are quite general. The first part of this paper does have some material within the limits of this survey, and will be mentioned in a later section.

2. Finite Automata

In general, the output of an automaton reacts a t any time not merely to the input a t that time, but to the input history. Thus, in some way or other, such an automaton must have memory. An important theoretical concept in this connection is the concept of state: an automaton can be in various states a t various times, and it by going from one state to another in response to something that has happened a t the inputs that it remembers what has happened. The total number of different states is an indication of the amount of memory that the automaton has. All the memory is explained in terms of the states, so that the value of each of the outputs, a t any time, depends on just the values of the inputs a t that time and the state; the only influence the past input history can have on the present value of the output is through its determination of the present state. (It is to be noted that, if this point of view is strictly maintained, a Turing machine has an infinite number of states. What is usually referred to by the word “state” in the Turing machine is really the state of part of it, not including the tape.) Another important distinction is the o w between finite-state automata a i d automata with infinitely ninny states. An example of the former is a switchiiig circuit as described in Section 1. The example of the latter is any interesting growing automaton (see Section 5). I n this context the number of states of an automaton is to be understood as the smallest number of states that ran he used to explain its behavior. “State” is used ill a behaviorial and not a structural sense. Thus a device that can be analyzed structurally as having an infinite number of states is a finite-state device, if

386

ROBERT McNAUGHTON

all but a finite number of its states can be dispensed with in explaining its behavior. I n recapitulation, automata have been separated into fixed and growing, into discrete and continuous, into synchronous and nonsynchronous, into deterministic and probabilistic, and into finite-state and infinite-state. These five bifurcations together divide the class of automata into thirty-two possible subclasses, some of which have received attention in the literature and will be discussed in this survey. Some of these classes may be empty. For example, it seems that there are no fixed discrete synchronous deterministic infinite-state automata. But no one has proved this proposition, and it seems clear that it cannot be proved by abstract reasoning from the definitions without the aid of some extra assumptions. The most elementary of the thirty-two classes is the class of fixed discrete synchronous deterministic finite-state automata, in the sense that the study of any of the other classes usually makes use of the fundamental properties of automata in this class. These automata are discussed so often that they have the simple name “finite automata.”

Fro. 1

Given a state and a value for each input of a finite automaton, at any moment of time, the value for each output and the state at the next moment of time is determined. The behavior of the device is characterized completely by a description of these transitions from state to state and output values, presented as a table or a graph. This characterization is quite useful, and is fundamental in the theory. For example, consider the finite automaton characterized by the state graph of Fig. 1. This device has one input and one output each having a range of two values, 0 and 1. The two circles represent the two states and the arrows show the transitions. The “l/0” on the arrow leading from state I means that if the automaton is in state I and the input value is 1 then the output value is 0 and the state a t the next moment of time is 11. Note that, of necessity, there is exactly one arrow leaving each state with any given input value. Many authors assume that an automaton has an initial state, which it assumes at the initial moment of time. The history of the device consists completely of what happened at the initial moment of time and at succeeding moments of time up through the present. The initial moment of time

THE THEORY

OF AUTOMATA, A SURVEY

387

is called “time 0”; the succeeding moments are, in order, time 1, time 2, time 3, etc. Since the automaton invariably starts in the initial state, the succession of stat.es is determined from the histories of all the inputs. Since the output values at any time depend on the state and the input values, the output values at any time are determined from the input histories. Thus with the aid of the initial-state assumption, the state-graph characterization fits with the part of the definition of “automaton” given in Section 1 that says an output value a t any time is determined from the histories of the inputs. Some papers do not assume an initial state in their discussion of finite automata. (Examples are Moore [78] and other papers that have pursued Moore’s problems.) A finite automaton as conceived in that way would be a class of finite automata as conceived here, one for each different choice of an initial state. It is not difficult to discuss the results of those papers in these terms. The advantage of the definition of Section 1 (and the initial-

state assumption) is that an automaton can be thought of as a device for transforming the information of the input histories into the information of the output histories. Convenient and precise behavioral descriptions are then possible. For example, if state I is the initial state of the device whose graph is Fig. 1, then there is another way of characterizing its behavior. That is to say that the output value is 1a t any time if and only if the input value is 1 a t that time and there have been an even number of 1’s in the input history up to and including that time. No similar behavioral description would be available if there were no initial time and initial state. An important concept in connection with state graphs is reduction. This concept is most conveniently described under the initial-state assumption, although it does not depend on it. Two state graphs are equivalent if every input sequence (i.e. input history) yields two identical output sequences (i.e. output histories) when applied to both graphs. Thus, for example, the state graph of Fig. 2 is equivalent to the state graph of Fig. 1 assuming that state I is the initial state in each case. A state graph is reduced if there is no equivalent state graph with fewer states. A funda-

388

ROBERT McNAUGHTON

mental thcorcm is that two rcduced equivalent state graphs are isomorphic, which means that thcrc is a one-to-one correspondence between the states of one and the states of the other such that initial states correspond to each other and transition arrows between states in one graph are the same, and are labeled the same, as the transition arrows between corresponding states in the other. (For further elucidation on state graphs see Huffman [50], Mealy [72], Moore [78], Rubinoff [98], and Caldwell [15].) A state graph certain of whose transition arrows or output specifications are missing is called a “don’t-care state graph.” Such a graph might be drawn by someone who wanted to specify a finite automaton but in the case of certain combinations of state and input value did not care either about the next state or the output value or both. One’s interest in the concept of a don’t-care state graph depends to some extent on how likely it is that he will ever prescribe a finite automaton in this manner. I n my opinion, when one wants to specify a finite automaton it is more likely that he will find one of the languages discussed in Section 4 more suitable than the language of state graphs. If this is so, then the concept of a don’t-care state graph is important, assuming don’t-care conditions as taken into account in those languages give rise to don’t-care state graphs. Ginsburg [35-381 and Paul1 and Unger [89], seek to generalize the concepts and algorithms of fully specified state graphs to don’t-care state graphs. The same is attempted by C. Y. Lee [65, p. 1279ff.1, although Lee’s approach differs from the others. The reader who has never studied any of these papers may begin to understand how difficult the problems are if he tries to formulate a notion for don’t-care state graphs that will serve thc same purpose that the notion of equivalence serves for fully specified state graphs. The concept of cquivalericc is applied to automata themselves as well as to state graphs. Two finite automata are equivalent if every input sequence (applied at the initial state) yields the same output sequence from one as from the other. Moore [78] considers the problem of devising experiments to tell whether machines are equivalent: an experiment is simply an input sequence. Because Moore does not stipulate an initial state, his concepts are more complicated than the corresponding concepts introduced here. Ginsburg [34] continues that part of Moore’s study having to do with how long experiments have to be, and Deuel [28] gives a more exacting analysis of some of Moore’s important concepts, and attempts to reconstruct in formal terms the theory behind experiments on machines. Trakhtenbrot [lo61 is essentially a study of the quantity of memory in a finite automaton. Tf the automaton has n states then the amount of memory is logzn.In these terms Trakhtenbrot characterizes the periodic behavior of the output given the periodic behavior of the input of an automaton


389

(with a given quantity of memory), and gives an indepcndent proof of a theorem of Moore [78] on how short an csperiment can hc that distinguishes two inequivalcnt fiiiitc automata (with a givcii quantity of memory). I n Gill [33] an experiment of length 2” n - 1 is developed for an automaton with a single binary input that will reveal everything there is to know about the automaton given that, for any time t , the output values a t time 2 depend only on the input histories after time t - n. An automaton fulfilling this last condition is called by Gill a “finite-memory automaton.” Srinivasan and Karasimhan [lo51 show how to convert the specification of a finite-memory automaton (which is simply a specification of the output a t time t given each input sequence of length n) into a state graph for the automaton. I n Burks and-Wang [lo] a finite automaton is structurally characterized as a net made up of a finite number of “and” gates, “or” gates, “not” gates, and delay elements connected together in a not too arbitrary manner. (The concept of a logical net goes back to Burks and Wright [ll].) This concept of finite autolr;aton is equivalent to the one defined here in the sense that every finite automaton is behaviorally equivalent to some net. However, a finite automaton need not be structurally similar to any net. Given any two states S1and Szin the state graph, there may or may not be a path of arrows from S1 to AS.If thcre is not then it is impossible for the automaton ever to get into Szafter it has once been in S1. State graphs and states arc classified according to this kind of consideration. If there is no path from the initial state to a state X, then S is a n inadmissible state. An inadmissible state is a state that the automaton can never get into; hence it is regarded as a state of an automaton for structural reasons and not for kehavioral reasons. It represents a conceivable configuration of the automaton that the automaton can never get into. (See Burks [8].) If, for every two states S1 and Sz of a state graph, there is a path from S1to Szand a path from Szto S1then the state graph is strongly connected. An automaton that is described by a strongly connected state graph is fully reversible in the sense that whatever is done to it in the way of a n input sequence can always be undone hy subsequently applying another suitable input sequence. The state graph of Fig. 1 is strongly connected. Not all important finite automata have strongly connected state graphs. For example, the oneinput one-output automaton, whose output value is 1 a t time t if and only if there has been a 1 in the input sequence a t some time preceding time t, is described by the state graph of Fig. 3, which is obviously not strongly connected. One approach to state graphs has becn given here out of several appear-

+

390

ROBERT McNAUGHTON

ing in the literature. In some places the output is a function of the state only, instead of the state and the input. Also, by making some minor conceptual adjustments, state graphs can be applied to the asynchronous circuits of switching theory, which are nonsynchronous but are otherwise like finite automata. Cadden’s papers [13,14], both in the area of switching theory, show how three types of state graph arise in the study of switching circuits, and present algorithms for converting a graph of any one of these types into a graph of any other type that is functionally equivalent to it. If we apply the term “information” to what goes in a t the inputs and what comes out a t the outputs of an automaton, we can think of it as a device for transforming information. (See Section 1.) I n a deterministic automaton the input information must in a sense contain the output information, since the output information is determined given the input information. Thinking in these terms we are led to ask the question, for what automata does the output information also contain the input information? Huffman calls such automata “information-lossless” ;he has made a study of finite automata and has advanced methods of testing these

FIQ.3

automata to determine whether they have this property [51,52]. Note that it is theoretically possible for a person to observe the outputs of an information-lossless automaton and then to tell exactly how the inputs were activated. In general, however, in order to tell what the input values are at a given time, he may have to wait for information about the output values many units of time later. In a series of papers, Huzino studies some of the concepts discussed in this section from an advanced and quite an abstract mathematical point of view. He calls attention [54] to some interesting kinds of machine, based on a state-graph analysis: his uniform machine is in effect a combinational machine, i.e., one whose output reflects the input at that time and is independent of the past history; his Latin-connected machine is like a counter, in that each input serves to rotate the machine a certain distance and direction around the cycle of states, but resembles a Latin square in that for any two states there is an input taking the machine from one state to the other; his reversible machine, if it goes from one state to another, can get back to the fist state again. In his 1958 paper [55] Huzino reformulates the problem of reducibility. Later papers [56, 57, 581 study operators on state graphs, which are in


39 1

effect operators that map machines or pairs of machines into machines. In fact, the introduction and use of operators on state graphs seem t o characterize Huzino’s general approach, the underlying motivation of which is often unclear. Many writers on automata take an abstract set-theoretical point of view. As a typical example, they may define “finite automaton” as an ordered quintuple (S, I, U , f, g) where each of the five items is itself a n abstract mathematical entity: S, I, and U are finite sets of abstract entities, called elements, f is a function that maps S X I into S , and g is a function that maps S X I into U. Thus, for every element s in S and i in I,f(s, i) is an element of S and g(s, i) is an element of U . It is to be noted that,, according to the abstract approach, this definition is complete. A finite automaton is any ordered quintuple satisfying just these conditions. To satisfy the reader’s psychological need, and not to satisfy any theoretical need, the abstract automaton theorist will explain that S can be thought of as a set of states, I a set of input values and U a set of output values; f can be thought of as a transition function, which tells the next state given the input value and present state; and g the output function, which tells the output value given the input value and present state. Given the quintuple, a state graph is determined and given a state graph of a one-input one-output automaton the quintuple is dctermined. (For automata with several inputs or several outputs, minor adjustments can be made.) Perhaps the greatest justification of the abstract approach is that it leads to generalization to other iiiteresting notions by liberalizing the abstract features of the definition. It, can then be shown how certain concepts that are apparently unrelated to the concept of finite automaton are in fact related. Two papers that have achieved such results are those by Burks and Wright [12] and Ginsburg [40].

3. Probabilistic Automata

The physical universe, it is often said, is a statistical one and not a fully deterministic one, with regard to laws of cause and effect. If this is so then it would seem that deterministic automata are fictitious, and that all automata are probabilistic, alt hough the relevant probabilities may be close to 1. It has to be explained, therefore, why a large part of automaton theory is concerned with deterministic devices, and why, indeed, the study of probabilistic automata has, in spite of some noble attempts, not procccdcd far at all.

392

ROBERT McNAUGHTON

The area of application of automaton theory can be divided into manmade devices and natural devices (such as the animal nervous system). The application t o man-made devices has so far been more significant than the application to natural devices. Since man-made devices are usually built to be as deterministic as possible, departures from determinism, inevitable as they are, are regarded as imperfections to be minimized. Cases of useful probabilistic features are relatively rare. For this reason, deterministic automata are studied as the ideal, even though actual machines may never come up to them perfectly. There are cases in which artificial machines do use probabilistic features by design. An example would be any calculation on a digital computer using random numbers. Other examples are machines that are built (or programmed) to imitate natural machines, discussed in Section 8. To throw light on these and on natural machines (animal nervous systems, etc.) a probabilistic theory of automata would certainly be appropriate. Some have felt that a more realistic switching theory would be probabilistic rather than deterministic. Von Neumann’s paper [110] is best regarded as part of switching theory-rather than of the theory of automata proper, by the criteria of Section 1. However, the application of his work is so broad (to machines in general, to the nervous system, etc.) that this classification may seem strange even if technically correct according to my definitions. Von Neumann’s main problem is to show how to construct nets to control error out of elements with a certain error associated with them. This is also the problem in Moore and Shannon [81] and other more recent papers in switching theory. An interesting question is whether there is anything that can be done by a probabilistic automaton but not by a deterministic automaton. Of course, the answer is in the affirmative if the question is interpreted in its broadest sense: a probabilistic automaton can generate a random sequence and a deterministic automaton cannot, in the strictest sense of the word “random.” But this answer seems to be unsatisfactory. Producing a random sequence is a rather indefinite task. We would like to know if there is anything more definite that a probabilistic machine can do. Now the question seems a little unfair to probabilistic machines, since whatever they can do they must do probabilistically. The reader might think that it is impossible to formulate an interesting and precise question here. Actually, in de Leeuw et al. [27], such a precise and interesting quest,ion is formulated. The formulation has to do with enumeration of a set of symbols printed by the output of a machine. For every symbol S that the output can logirally print there will be a degree of probability r that S will eventually be printed by a given probabilistic machine. If, for some probabilistic machine, S is the set of all symbols


393

that is printed with at least the pro1)ahility p , then S is p-enumerable. The precise formulation of thr qurstion, thru, is as follow: arc therc any sets that are p-enunicrahlc Init, not 1-c.iirinicl.al~lc? Tlic clucht ion is aiihwcrcd iii the article in the ncgative. The precise question Eorniulnted by de Lecuw el al., it seems, does not exhaust the meaning of the original question. It would be interesting to formulate othrr precise versions of the question and attempt, to answer them. As far as I know, no other formulations have been made. Indeed, one might say in general that,, before the theory of probabilistic automata can proceed to develop, someone must formulate some interesting and precise problenis. Recently, G. F. Rose has given a probabilistic iiiterprctation of a deterministic machine [9G]. He considers a Turing machine (see Seceion 5) which, when given n numbers, rl,. . . , T,, in order on its tape (for some fixed n ) , prints n other numbers, gl, . . . , gn,in order 011 its tape. If, from every such n-tuple of z’s, there are at, least m of the z’s such thntf(z,) = ys, then the machine is said to compute j with probability m / n . The justification of this definition lies in the fact that, in general, thrre may be no way of knowing for which nz of the n x’s thc function has been correctly computed. Rose leaves open the crucial question, are there any functions that are computable with probability ? n / f i , for any ?n and n, I I ) 5 n, that are not computable with probability 1/1 (which is the same as being computable in the sense discussed in Section 5)? As Myhill points out in correspondence, it will be interesting if it turns out that there are such functions for some probability m / n dose to 1.

4. Behavioral Descriptions

In Kleene [64], a problem is introduced and solved that has turned out to be fundamental in the theory of automata. Consider the class of all finite automata with one binary input and one binary output, with values 0 and 1. Also let us stipulate that each automaton in the class has a n initial state, the state a t time 0. For any finite sequence S of 0’s and 1’s as an input history, a n automaton will react eithrr with an output 1 or an output 0. Every automaton breaks down thr set of all finite sequences of 0’s and 1’s into two disjoint sets, those to which it reacts with an output 1 and those to which it reacts with an output 0; we say that thc automaton rpprescnts that set of sequences to which it reacts with a 1. Kleene’s problem is, what sets of sequences of 0’s and 1’s are representable by finite automata? (The generalization of the problem that stllows

394

ROBERT McNAUGHTON

iiiput values other tlian 0 and 1 turns out to add no further difficult8y.) Kleene’s answer is that a set is so representable if and only if it is “regular.” “Regular set” (or “regular event,,” as he calls it) he defines by recursioii, which I shall paraphrase after giving some preliminary definitions. Let A and B be two sets of sequences. Then the concatenation set A - B of A and B is the set of all sequences which are made up of a member of A followed by a member of B. (Thus, for example, if 00111 is a member of A and 0010 is a member of B then 001110010 is a member of A.B.) The iterate of A on B, A*B, is the smallest set that contains every sequence of B, A.B, A.A.B,A-A.A.B,etc. Inother words, A*Bisthesetofallsequences which are obtained from taking (for all n 2 0 ) n sequences of A (possibly with repetitions) and a sequence of B and joining them together in that order.’ “Regular set” is defined recursively as follows: (1) Every finite set of sequences is regular, i.e., a set with only finitely many members. (Keep in mind that although we are dealing exclusively with finite sequences, the set of all finite sequences is infinite.) (2) The concatenation set of two regular sets is regular. (3) The iterate of a regular set on a regular set is regular. (4) The union of two regular sets is regular. ( 5 ) No set is regular unless its being so follows from (l), (2), (3), and (4). (The reader who is not familiar with recursive definitions is referred to Church [21] or Copi ~31.1 Kleene’s theorem is that a set of finite sequences is representable by a finite automaton if and only if it is regular. Kleene’s work was continued by Myhill [84]. The definition of regularity givcs us a behavioral language known as the “regular-expression laiiguage” for finite automata with one binary output and one binary input. “Regular expression” is defined as follows: (1) “0” and “1” are regular expressions. If a! and p are regular expressions then (2) asp, (3) a*& and (4) a! U p are regular expressions. (5) Nothing is a regular expression unless its being so follows from (l), (2), (3), and (4). An example of a regular expression is ((0-0)*1 U 1*0).(0.0)*1. Rather obviously, a regular expression is a name for a regular set. The behavior of a one-input one-output automaton A is described by a regular expression a! if the following is true: the output value of A a t time t is 1 if and only if the input sequence from time 0 through time t inclusive is a sequence in the class named by the expression. For example, if an automaton’s behavior is described by (10)*1,and if the input sequence is 101010.. . then the output sequence will be 101010. . . . For since 1 (the input sequence up to time 0 ) is a member of (10)*1 the output is 1 at time 0 ; since 10 (the input sequence up t o time 1) is not a member of (10)*1 the output is 0 a t time 1; since 101 (the input sequence up to time 2)


395

is a member of (10)*1 the output is 1 a t time 2; etc. Hence the output sequence is 101010. . . . (See Copi et al. [24].) I n order to use regular expressions to describe the behavior of automata other than the kind that has one binary input and one binary output, adjustments must be made. If the input is not binary or if there is more than one input, the digits 0 and 1 must be supplemented. It is convenient to use a separate digit or special symbol for each combination of input values. For example, if there are three binary inputs then there are eight combinations of input values which are symbolized as 0, 1, 2, 3, 4, 5, 6, 7; regular expressions used in describing this automaton may have these as digits. If there are several outputs, but these are binary, then a separate regular expression must be used for each output. If an output is not binary, and has p possible values, then the following plan can be used. Where z is the least integer such that 2” g p , the output can be coded as if it were x binary outputs, and z regular expressions are used. For example, if, for a given output U,p = 3, then x = 2; U can be thought of as two binary outputs Ul and Uz,where, if the three values of U are 0, 1, 2, then 0 for U can be thought of as 0, 0 (0 for U1,0 for U,), 1 as 0, 1, and 2 as 1, 0. The regular-expression language can be liberalized in several ways. The first is t o allow the star operator to stand alone. Thus a* names the set that differs from the set named by a*a in that the set named by a* also contains the empty sequence, i.e., the sequence of zero length. an p names the set which is the intersection of the set named by a and the set named by p. And -a is a name for the complement (with respect to the set of all finite sequences of 0’s and 1’s) of the sct named by a. It is readily proved that the intersection of two regular sets is regular, and the complement of a regular set is regular; these are consequences of Kleene’s theorem, using elementary facts about automata. The regular-expression language so enlarged is discussed in McNaughton and Yamada [71], which also gives a synthesis algorithm. This language seems suitable to a design engineer’s thinking and for specifying precisely the kind of machine he wants. For example (borrowed from the cited article), suppose he wishes to specify that an automaton is to have an output 1 a t time t if and only if either there have never been three consecutive 0’s in the input sequeiice to time t, or there have been three consecutive 1’s in the input sequence t o tirile t sirice the last three consecutive 0’s. A regular expression can readily be written down specifying the automaton. Noting that (0 U 1)* denotes the set of all finite sequences (of 0’s and l’s), and that (0 LJ, 1)*000(0 U 1)* denotes the set of all finite sequences having three consecutive 0’s somewhere, the desired regular expression is [(O 1)*000(0 1)*] (0 1)”111 [(O 1)*000(0 1)*].

-

u

u

u u

-

u

u

396

ROBERT McNAUGHTON

The state-graph language and the regular-expression language are two languages for describing the behavior of finite automata. Several languages have been proposed using symbolic logic. It will not be possible to present in detail the languages of symbolic logic which have appeared in the literature. A rough outline and references will have to suffice. The reader is possibly familiar with symbolic logic in its use as a behavioral language for combinational automata. (If, in a given automaton, each output value at any time depends only on the input values a t that time and is independent of the history of the device, then the device is known as a combinational automaton. In short, it is an automaton without memory.) For describing these automata, one uses that part of symbolic logic known as the propositional calculus, which contains propositional variables standing for simple sentences, connectives such as “and,” “or,” and “not,” but does not contain quantifiers “for all x” or “for some z.” The notation of Boolean algebra is used more often and is quite similar to the propositional calculus. Automata with memory can be described by symbolic logic by introducing variables standing for instants of time; having introduced these variables it is natural to introduce quantifiers. One of the earliest suggestions to use variables and quantifiers in this way is in Patterson [88]. To illustrate, let “&” stand for “and,” “ V ” for “or,” I ( - ” for “not,” “= - 9 ) for “if and only if,” “(x)” for “for all 2,’’ and “(32)” for “there exists an x.” Let x < y mean that time moment x precedes time moment y. Let Iz mean that the input I is 1 a t time x; -15 will therefore mean that the input I is 0 a t time x. Let Ut mean that output Ii is 1 a t time t. Then the behavior of the one-input one-output automaton discussed above (whose output is 1 a t time t if and only if there have never been three consecutive 0’s in the input sequence to time t, or there have been three consecutive 1’s and no three consecutive 0’s since) is described by the following formula of symbolic logic :

Ut = [-(35)(2+1

Advances in Computers.. v. 2 (AP Inc.,U.S)(

Advances in Computers, Volume 2

Advances in Computers, Vol. 15 (v. 15)

Advances in Computers, Vol. 5 (AP Inc)(ISBN )(415s)