Advances in Computers, Volume 3

Advances in COMPUTERS VOLUME 3 This Page Intentionally Left Blank Advances in COMPUTERS edited b y FRANZ L.ALT N...

Author: Author Unknown

48 downloads 1171 Views 16MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Advances in COMPUTERS VOLUME 3

This Page Intentionally Left Blank

Advances

in

COMPUTERS edited b y

FRANZ L.ALT National Bureau of Standards Washington, D. C.

and

MORRIS RUBINOFF University of Pennsylvania and Pennsylvania Research Associates Philadelphia, Pennsylvania

associate editors A. D. BOOTH

R. E. MEAGHER

VOLUME 3

Academic Press New York London 0 1 9 6 2

COPYRIGHT @ 1962

BY

ACADEMICPRESSINC.

ALL RIGHTS RESERVED

NO PART O F T H I S BOOK MAY B E REPRODUCED I N ANY FORM BY PHOTOSTAT, MICROFILM, OR ANY OTHER MEANS, W I T H O U T WRITTEN PERMISSION FROM T H E PUBLISHERS.

ACADEMIC PRESS INC. 111 FIFTHAVENUE

NEW YORK3, N. Y.

United K-ingdom Edition Published by

ACADEMIC PRESS INC. (LONDON) LTD.

BERKELEY SQUARE HOUSE,BERKELEY SQUARE, LONDON W. 1

Library of Congress Catalog Card Number 59-15761

PRINTED I N THE V S I T E D STATES O F AMERICA

Contributors to Volume 3

GARRETTBIRKHOFF,Department of Mathematics, Harvard University, Cambridge, Massachusetts E. F. CODD,Development Laboratory, Data Systems Division, International Business Machines Corporation, Poughkeepsie, New York SAMUEL D. CONTE,*Computation and Data Reduction Center, Space Technology Laboratories, Inc., Los Angeles, California IiEED c. LAWLOR, t Electronic Data Retrieval Committee of the American Bar Association, Los Angeles, California HAROLD I(.SKRAMSTAD, U . S. Naval Ordnance Laboratory, Corona, California RICHARD S . VARGA, Computing Center, Case Institute of Technology, Cleveland, Ohio PHILIPWOLFE,Mathematics Division, The R A N D Corporation, Santa Monica, California ]]AVID YOUNG, Computation Center, University of Texas, Austin, Texas * Present address: Computation Sciences Center, Purdue University, Lafayette, Indiana. t Mailing address: Park Central Building, 412 West Sixth Street, Los Angeles, California.

V


Preface The editors of Advances in Computers have been joined by Morris Rubinoff. This addition, whose effect will be more fully felt in future volumes, is expected to strengthen still further the tendency to broad coverage of different aspects of the computer field, a tendency which has guided us in the earlier volumes and which is demanding increased emphasis as the field is being split into more and more areas of specialization. In the present volume are represented applications, both scientific and data processing; methods, both of numerical analysis and of computer programming; and engineering considerations in computer selection. Taken together with the articles on artificial intelligence, logical design, components and others appearing in the earlier volumes, these contributions appear to constitute a fully representative sample of computer science and technology. We hope that they will be taken as an antidote to specialization. FRANZL. ALT b’eptember, 1962

vii


Contents CONTRIBUTORS TO VOLUME3 . PREFACE . . . . . . . . CONTENTS OF VOLUMES 1 AND 2 The Computation

. .

. .

. .

. .

. .

. .

. . . . . . . .

.

.

.

.

.

.

.

.

.

V

vii

...

.

Xlll

. . . . . . . .

2 4 18 28 35 48 70 74

. . . . . . . . . . . . . . . . . .

78 81 83 87 104 122 150 152 152

of Satellite Orbit Trajectories

SAMUEL D. CONTE

1. The Problems Posed by Artificial Satellites . . 2. The Equations of Motion . . . . . . . 3 . Methods of Integration . . . . . . . . 4. General Perturbation Methods . . . . . . 5 . Accuracy Tests for Integration Programs . . . 6 . Orbit Determination and Tracking Methods . . 7. Organization of a Tracking and Prediction Program Bibliography . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

Mu1t iprogramming

.

E. F CODD

1 . Introduction . . . . . . . 2. Early Contributions . . . . . .3 . Current Scope of Multiprogramming 4 . Batch Multiprogramming . . . 5. The Optimizing Problem . . . . 6. Multiprogramming with Two or More 7. Concluding Remarks . . . . . 8. Acknowledgments . . . . . . Bibliography . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . . Processing Units . . . . . . . . . . . . . . . . . . .

Recent Developments in Nonlinear Programming PHILIP WOLFE

1. Introduction . . . . . . 2. Differential Gradient Methods . 3 . Large-Step Gradient Methods . IX

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

156 161 165

CONTENTS

X

4. Simplicia1 Methods

. . . . . . . . . . . . . 172

5. Columnar Procedures . . . . 6. The Cutting-plane Method . . 7. Initiating an Algorithm . . . 8. Computer Routines and Literature Bibliography . . . . . . .

.

.

.

.

. .

. .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

175 180 . . . . . . . 183 . . . . . . . 184 . . . . . . . 186

Alternating Direction Implicit Methods

.

GARRETT BIRKHOFF, RICHARD S VARGA. and DAVID YOUNG

INTRODUCTION

1. General Remarks . . 2. The Matrix Problem . 3 . Basic AD1 Operators .

. . . . . . . . . . . . 190 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

PART I: STATIONARY AD1 A1K;THODS (Casem

4. Error Reduction Matrix . . . . . . 5. Norm Reduction . . . . . . . . 6 . Application . . . . . . . . . . 7 . Optimum Parameters . . . . . . . 8. The Function F . . . . . . . . 9 . Helmholtz Equation in a Rectangle . . . 10. Monotonicity Principle . . . . . . 11. Crude Upper Bound . . . . . . . 12. Eigenvalues of H , V . . . . . . .

=

191 192

1)

. . . . . . 194 . . . . . . 195 . . . . . . 196 . . . . . . 198 . . . . . . 199 . . . . . . 200 . . . . . . 202 . . . . . . 203 . . . . . . 204

PART 11: COMMUNICATIVE CASE:

13. Introduction

.

.

.

.

.

.

.

.

.

.

14. Problems Leading to Communicative Matrices 15. The Peaceman-Rachford Method . . . . 16. Methods for Selecting Iteration Parameters for Rachford Method . . . . . . . . . 17. The Douglas-Rachford Method . . . . . 18. Applications to the Helmholtz Equation . .

.

. . . . 205 . . . . . 206 . . . . . 210 the Peacenian. . . . . 211 . . . . . 217 . . . . . 222

PART I11: COMPARISON WITH SUCCESSIVE OVERRELAXATION VARIANTS

19. The Point SOR Method . . . 20 . Helmholtz Equation in a Square 21. Block and Multiline SOR Variants 22. Analogies of AD1 with SOR . .

.

.

.

.

.

.

.

.

. 224

.

.

.

.

.

.

.

.

.

. .

. .

. .

. .

. .

. .

. .

. .

225 . 227 . 229

CONTENTS

xi

PART IV: NUMERICAL EXPERIMENTS 23 . Introduction . . . . . . . . . . . . . . . 24 . Experiments with the Dirichlet Problem . . . . . . . 25. Analysis of Results . . . . . . . . . . . . . 26 . Conclusions . . . . . . . . . . . . . . . 27 Experiments Comparing SOR Variants with AD1 Variants . . Appendix A: The Minimax Problem for One Parameter . . . . Appendix B: The Minimax Problem for m > 1 Parameters . . . Appendix C : Nonuniform Mesh Spacings and Mixed Boundary Conditions . . . . . . . . . . . . . . Appendix D : Necessary Conditions for Commutativity . . . . Bibliography . . . . . . . . . . . . . . . .

.

231 232 242 219 250 254 259 263 266 271

Combined Analog-Digital Techniques in Simulation

.

HAROLD K SKRAMSTAD

1. Comparison of Analog and Digital Computers in Simulation . 2 . Interconnected Analog and Digital Computers . . . . . 3. Example of a Combined Solution . . . . . . . . . 4. Analog-Digital Arithmetic in a Digital Computer . . . . . 5 . Systems Using Analog-Digital Variables . . . . . . . Bibliography . . . . . . . . . . . . . . . .

275 277 281 283 288 296

Information Technology and the l a w REED C. LAWLOR

1 . Introduction . . . . . . . . . 2 . Information Growth . . . . . . . 3 . Mechanization in Lam Practice . . . . 4. Applications of Symbolic Logic to Law . . 5. Information Storage and Retrieval . . . 6. Punched Cards and Notched Cards . . . 7. Prediction of Court Decisions . . . . 8. Thinking Machines . . . . . . . 9 . The Law of Computers . . . . . . 10. Use of Computers in Court . . . . . 11 . New Horizons . . . . . . . . . Bibliography . . . . . . . . . . Exhibit “A” . . . . . . . . . .

. .

. .

. .

. .

. .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . .

. . .

. . .

. . .

. . .

.

.

.

AUTHORINDEX . . SUBJECTINDEX .

. .

. .

. .

. .

. .

. .

. .

. .

. .

. .

. .

.

.

.

.

.

.

.

.

.

.

. .

. .

.

. 299 . 300 . 302 . 305 . 310 . 323 . 324 . 334 . 335 . 339 . 340 . 343 . 317 .

.

.

353 357


Contents of Volume 1

General-Purpose Programming for Business Applications CALVINC. GOTLIEB n’umerical Weather Prediction KORMAN A. PHILLIPS The Present Status of Automatic Translation of Languages YEHOSHUA BAR-HILLEL Programming Computers to Play Games ARTHUR1,. SAMUEL Machine Recognition of Spoken Words RICHARDFATEHCHASD Binary *4rithmetic GEORGEW. REIT~-IESXELL Contents

of Volume 2

A Survey of Numerical Methods for Parabolic Differential Equations JIM DOUGLAS, JR. Advances in Orthonormalizing Computation PHILIPJ. DAVISand PHILIPRABIXOWITZ Microelectronics Using Electron-Beam-l-ictivated Machining Techniques R. SHOULDERS KENNETH Recent Developments in Linear Programming SAULI. GASS The Theory of Automata, a Survey ROBERTMCNAUGHTON xiii


The Computation of Satellite Orbit Trajectories" SAMUEL D. CONTEt Computation and Data Reducfion Center. Space Technology Laboratories. Inc.,

Los Angeles. Calif

.

1. The Problems Posed by Artificial Satellites . . . . . . . . . . . . . . . . . 2. The Equations of Motion 2.1 The Cowell Method . . . . . . . . . . . . 2.2 The Encke Method . . . . . . . . . . . . 2.3 Variation of Parameter Methods . . . . . . . . 3 . Methods of Integration . . . . . . . . . . . . 3.1 Runge-Kutta Method . . . . . . . . . . . 3.2 Multi-step Methods . . . . . . . . . . . . 3.3 Special Methods for Second Order Equations . . . . . 3.4 Accumulated Round-off Error . . . . . . . . . 3.5 Integration in Multirevolution Steps . . . . . . . 4 General Perturbation Methods . . . . . . . . . . 4.1 The Diliberto Theory . . . . . . . . . . . . 4.2 Numerical Results . . . . . . . . . . . . 5. Accuracy Tests for Integration Programs . . . . . . . 5.1 Comparison with Analytic Formulas . . . . . . . 5.2 Consistency Checks . . . . . . . . . . . . 5.3 Double Precision Operations . . . . . . . . . . 5.4 Use of Integrals of the Motion . . . . . . . . . 5.5 Comparison Using Different Methods . . . . . . . 5.6 Estimates of Accumulated Truncation and Round-off Errors 5.7 Numerical Comparison of Special Perturbation Methods . 6 . Orbit Determination and Tracking Methods . . . . . . 6.1 Editing the Observational Data . . . . . . . . . 6.2 The Least Squares Problem . . . . . . . . . . 6.3 Stagewise Differential Corrections and Shifting Parameters . 6.4 The Partial Derivatives . . . . . . . . . . . 6.5 The Choice of Burnout Parameter Coordinates . . . . 6.6 Error Analysis in Differential Corrections . . . . . .

.

.

.

2

. . . .

.

.

4

.

.

.

.

. .

. .

. .

. .

. . . . . . . . .

. . . . .

. . . . .

. . . . .

.

.

.

.

.

.

.

.

6 7 15 18 19 21 24 25 28 28 31 34 35 36 36 37 38 38 38 42 48 49 50 56

.

.

.

.

. .

. .

. .

. .

.

.

.

.

.

.

.

.

. . . .

. . . .

. . . .

. . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

58 64

.

.

.

.

66

* The preparation of this paper was made possible by the support. both direct and indirect. of the Computat.ion and Data Reduction Center a t Space Technology Laboratories. Inc. I n particular. the author is indebted to I . J. Abrams. D. D . Morrison. 0. K . Smith. and R . J . Mercer for many of the techniques and ideas contained in this survey paper . t Present Address: Director. Computer Sciences Center. Purdue University. Lafayette. Indiana

.

1

2

SAMUEL D. CONTE

7. Organization of a Tracking and Prediction 7.1 Input and Conversion Block. . . 7.2 Trajectory Integration Block . . 7.3 Partial Derivative and Residual Block 7.4 The Differential Correction Package 7.5 The Ephemeris Processor. . . . Bibliography

.

.

.

.

.

.

.

.

Program . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

70 70 71 72 72 74 74

1. The Problems Posed by Artificial Satellites

The post-Sputnik era has seen a phenomenal growth in the computation of the orbits of artificial satellites. There are literally dozens of installations actively engaged in computing orbits and in related research. Most of the personnel involved are novices in the field of astrodynamics and must inevitably be unfamiliar with the methods and even the literature of classical astronomy on orbit determination. While familiarity with these methods would be of inestimable value t o the modern astrodynamicist, the problems of artificial satellite theory differ in some important respects from those of classical astronomy and therefore call for new methods of attack. Some of these differences arise from the emphasis on and importance of engineering aspects; others concern the relative importance on near-earth satellite orbits of such perturbative forces as drag and oblateness. Rapidly changing forces such as arise from the near approach of a satellite t o a planet also lead t o new effects and to the need for refined computing techniques. The necessity for real time orbital determination and prediction presents new problems which were not encountered by astronomers. The advent of the high speed computing machine makes possible the consideration of complicated forces and effects which were too laborious to be considered in the days of the desk calculator and a t the same time makes necessary a re-evaluation of the numerical techniques used for orbit determination. The engineer is sometimes prone t o accept machine results on faith. The numerical analyst is aware of the many errors which enter in a complicated way into the computations, and it is his responsibility t o make clear to the engineer to what extent, he can trust machine results. I n this paper we propose to discuss and evaluate methods of orbit prediction and determination on high speed computers, pointing out, whenever possible, the sources and the magnitude of errors which enter into the computation. At the outset i t may be well to distinguish, as some writers have done, between “feasibility orbits” and “precision orbits.” Feasibility studies are concerned with the over-all performance characteristics of a missile configuration, with fuel and payload requirements, with the optimization of orbit parameters, and with optimum guidance requirements. I n such studies extreme accuracy is usually not required, two or three significant digits

SATELLITE ORBIT TRAJECTORIES

3

being normally adequate. In feasibility work it is permissible in some cases to use crude mathematical models as is done, for example, in replacing the true orbit of a planet by an approximating circle or in omitting perturbations due to small perturbative forces such as oblateness on an interplanetary flight. In some cases the n-body problem may be approximated by a succession of two-body problems. In addition such studies are often based on the use of approximate values for the physical constants. Even in such studies, however, proper evaluation of the results requires that the nature and magnitude of the approximation be fully understood. In precision orbit work on the other hand one should strive for the ultimate accuracy possible within the limitations of the information available, the machine being used, and the mission under consideration. Precision orbits are required for accurate navigation, for the proper conduct and interpretation of physical experiments, and for determining improved values of physical and astronomical constants. The number of significant digits required for precision work will generally be on the order of six or more depending on the objective of the study. The mathematical model should include, when applicable, perturbations due to nongravitational as well as gravitational forces. Aerodynamic drag effects, for example, on near-earth satellites are very significant. Even small effects such as those caused by electromagnetic forces, meteoritic drag, radiation pressure and relativistic effects may be significant for some missions. Gravitational anomalies may also introduce appreciable effects. The effect of the earth’s oblateness on near earth satellites is now fairly well understood; for lunar orbits the oblateness of the moon may need to be considered. In addition to the forces, every effort should be made to obtain the best available physical and astronomical constants. Indeed as noted above one of the major uses of precision orbits is to determine better these very constants. The present uncertainty in the solar parallax, for example, assumed to be sec, leads to an uncertainty in a Martian flight of on the order of 1 x some 50,000 miles; the uncertainty in the principal oblateness parameter which is on the order of 4 X 1 0 - 6 leads to appreciable effects on the orbital elements of a near earth satellite. Even the tables of the planetary coordinates introduce uncertainties of several hundred miles in the position of a planet in its orbit. The effect of the inaccuracy in the planetary coordinates is considerably more striking if it is necessary to obtain the velocity of the planet by interpolation. It can be shown that errors in the velocity of the earth of up to 1 fps are encountered if they are obtained by interpolation in the tables. This error when translated into errors in the satellite’s velocity will subsequently lead to errors of several thousand miles on flights to Mars or Venus. Assuming that an adequate mathematical model is being used and that

SAMUEL D. CONTE

4

the physical constants are known exactly, one is faced with the problem of maintaining accuracy in the numerical computations. Computational errors, if not properly controlled, may be as serious as the physical errors. Among the computational errors are those which arise from the truncation of series, from the use of approximating polynomials, from the accumulation of rounding errors, from loss of significance through differencing nearly equal quantities and from small divisors. Techniques for estimating these errors and methods for reducing their effects will be discussed in succeeding sections. 2. The Equations of Motion

No attempt will be made here to derive the equations of motion for a satellite acting under the influence of a central body force and various perturbative forces. This material is well covered in several books [37]. However, the equations will be given here both for reference purposes and to serve as a basis for subsequent discussion. Assuming an inertial rectangular equatorial coordinate system centered a t the earth the equations of motion for a vehicle with respect to the earth center have the form

i: =

-P-

r r3

+ Fa + Fi + Fz

(2.1)

where

r

= (5,y, z )

r

= (x2

is the position vector of the vehicle relative to the central body,

p =

+ + y2

~2)~’2,

Gm where G is the gravitational constant and nz is the

mass of the earth, Fa = (Fa,, Fa,, FOE)is the force due to oblateness, FI = ( F I ~F,,, , FlZ) is the force due to the attraction of planetary bodies other than the central body, Fz = (Fz,, FZ,, Fz.) is the force due to drag.

-p

(5>

=

(-

p

2, -p $ -p E+, 2

(2.2)

is the attraction of the central

body on the vehicle. More specifically, the components of the oblateness force, including only the first three harmonics, are :

- J+R4

6 (3

- 42

2 + 63 22


5

22

where J1, Jz, J3 are the oblateness parameters corresponding to the second, third, and fourth harmonics respectively, and R is the radius of the earth. The components of the perturbations due to other planetary bodies, including the moon, are:

FZ1=

-p

,e

mi

t=l

r T + ”) Pi3

(x + Y,z )

where

xi

=

ri

=

x component of the position of the i t h body with respect t o the origin,

p; =

mi

=

+ (y - yJ2 + (z - z ~ ) ~ ] ” ~ + ya2 +

[(x - x J 2 (.a2

za2)1’2

ratio of mass of i t h body to that of the central body.

The form of the drag vector will depend on the model assumed for the atmosphere. If the vehicle is assumed to be spherical so that lift can be neglected, and if variations of the density with latitude or time of day are ignored, the components take the form

FzZ = -D-X a Va

(x --+ Y, 2 )

(2.5)

where

D = pVa2ACD,

2w

density of atmosphere a t height h above surface of the earth, Va = (ka2 ga2 ia2)1’2 = velocity of the missile with respect to the surrounding air, A = cross sectional area of the vehicle, W = mass of the vehicle, CD = drag coefficient, xa = x We’,

p(h) =

+ +

+

ya = y - w,x, ia= 2,

SAMUEL D. CONTE

6

and where the rotation ratc of the atmosphere is assumed to be equal to that of the earth, we.The density p(h) may be approximated over a limited range in height h by a formula of the type p =

exp (cl

+ c2h + cyh2)

where the coefficients c; (i = 1, 2, 3) which may differ for various height levels are obtained by making a fit of p ( h ) to the latest standard atmosphere table. If the perturbativc forces in (2.1) arc all neglected, the motion reduces t o that of a body moving in an inverse square law field. I n this case the orbit is a conic section and explicit analytic formulas can be given for the motion of the vehicle. Another problem which can be solved explicitly is the restricted three body problem. I n the general case analytic solutions are not available and recourse must be made t o numerical integration, or possibly t o series solutions. Our major concern will be with the various possible forms of the equations of motion and with the methods of numerical integration which will lead to the most efficient computational schemes for solving (2.1). Three mathematical formulations which are in common use today-those of Cowcll and Encke and the variation-of-parameters method-will be described and evaluated for computational efficiency. The two major criteria to be used in this evaluation are overall accuracy and computational efficiency, although other criteria such as simplicity and versatility will he discussed. 2.1 The Cowell Method

The direct integration of (2.1) in rectangular coordinates is referred t o in the literature as Cowell’s method. If the position and velocity components (20,yo, 20, 5 0 , go, 2 0 ) are given at a time to, this system of three secondorder equations can be integrated directly to obtain the total velocity and position a t any subsequent time t. In this form the equations, although nonlinear, are very simple and symmetrical. For the integration itself no conversion from one coordinate system to another is necessary and integration time per step is very nominal. However, because the total accelerations are integrated, the attractions change rapidly with small time changes so that small integration steps are required to maintain accuracy and in addition a large number of significant figures must be carried. Over-all computing time depends upon the computing time per step and on the number of steps required for a given accuracy. For many orbits the Cowell method requires about ten times as many integration steps as the Encke or Variation of Parameter Methods, and although computing time per step is approximately 50% less, over-all computing time may be considerably greater. The Cowcll method also suffers more from accumulation of round-


7

off errors which may lead to serious degradation of accuracy. I n general, round-off errors will accumulace as some power of the number of steps. Since the Cowell method requires more integrating steps for a given accuracy, round-off error will be proportionately larger and over-all accuracy may suffer as a result. I n general, a method which requires the smallest number of integration steps should be preferred because round-off error effects will be minimized. For many orbits-including lunar orbits-and in general for any orbits for which two body motion is a reasonable approximation, the Cowell method is probably the least efficient computationally of the three methods to be discussed. Studies a t Aeronutronics [2] and Republic Aviation [33] confirm these remarks. There are some orbits in which the Cowell method does show up t o good advantage, however, particularly those where the thrust and perturbations are changing rapidly. Moreover, it is universally applicable t o all types of orbits and presents no fundamental difficulties in the exceptional cases of nearly circular, nearly parabolic or even hyperbolic orbits. Because of its simplicity and universality of application, every installation engaged in serious work in space trajectories should have available a routine based on Cowell’s method, this in spite of its apparent deficiencies on many types of orbits. 2.2 The Encke Method

Over a century ago the German astronomer J. E. Encke observed that the motion of a celestial body, such as a planet, deviates only slightly from two-body motion. He proposed that instead of integrating the total accelerations as in the Cowell method one integrates only the deviations of the actual motion from that of a reference conic. Encke’s method has been adapted to the earth satellite problem in recent years with striking success. When the reference orbit is that for two-body motion analytic solutions are available. Since, theoretically, these equations can be solved exactly and since the deviations from the two-body motion are assumed to be small, it seems intuitively evident that much larger integrating steps for the deviations should be possible. Moreover, greater accuracy should be possible when working with a fixed word length because round-off and truncation errors enter only into the deviations. To insure that the deviations are kept small compared to the reference orbit motion, a process of rectification is necessary. Whenever the deviations from the fixed reference orbit become large-on the order of 1000 miles-a new reference orbit is determined. The frequency of rectification will depend upon the type of orbit. The computing time per integration step is a t least 5Ooj, greater than for the Cowell method, particularly when frequent rectification is necessary, but the integrating step may be ten times as large so that the Encke method compares very favorably in over-all computational efficiency with the Cowell method.

SAMUEL D. CONTE

8

The advantages of Encke’s method are particularly evident on lunar flights where the deviations from two-body motion are slight but seem to be less marked on near earth satellites where the effects of oblateness and drag cause frequent rectification. To derive the equations for Encke’s method we rewrite the equations of motion (2.1) in the form r r = -p-fF 1.3 where F is the vector sum of all perturbations ,acting on the vehicle. If the perturbations are neglected, the resulting equations

re r,3 describe a conic section with the center of the earth a t one focus. Analytic formulas are known for re, i, as functions of time. Now letting E represent the deviations of the actual orbit from the reference orbit’, one has

re =

5

=

and

..

..

E=r-re=-p

-p-

r

-

re,

(s

-

3)+

F.

Equations (2.9) may now be integrated for the components of E and iand the components of the actual motion r and r obtained from (2.8). From a computational point of view the solution of the two-body formulas for re and ?, is very critical and is the source of the largest error in the Encke method. Several sources of computational errors will be mentioned in succeeding paragraphs. In particular errors in computing the mean motion n and the mean anomaly M when the time is large are especially serious. To avoid these errors it is recommended that the computation of n and M be done in double precision. If the rectification were exact, the initial conditions for & and 50 would be zero. Due to rounding and loss of significance errors, the conversion from elliptic elements to rectangular coordinates will not be exact and hence the initial displacements E,,) iowill not vanish. Since the reference orbit need not be the true osculating conic section, one can simply consider that the erroneous-elements so obtained define a conic section and that the deviations be based on this. Thus the initial displacements will in general not vanish a t rectification but instead will be given values equal to the difference between the actual coordinates at the time and those computed from the slightly erroneous elliptic elements. I n attempting to carry out numerically the integrations in (2.9) we are immediately faced with one of a number of computational difficulties which


9

arise in the Encke method. At the beginning of the flight r/r3 and re/r? are very nearly equal to each other and significance will be lost in differencing these quantities. To avoid this loss of significance a method involving a series expansion is given in the literature [&$I. However, a simpler exact form more suitable for computation is the following: (2.10)

where p = re/r. In order to start the integration of (2.10) we must have the elements of the reference orbit and the initial values of E, a t time t = to. If the initial conditions are given in rectangular coordinates we must first compute the elements of the osculating conic section (a, e, i, i2, o,M ) (see Fig. 2). The formulas differ for elliptic motion ( a > 0) and for hyperbolic motion ( a < 0). To determine the semimajor axis a, the eccentricity e and the mean anomaly M we proceed as follows:

r

=

(x2

+ y2 + z2)1/2

v

=

(vz”

+ v; +

Tl

=

1 - r/a

v,)1‘2

(2.11)

+ T22)1’2

=

(T?

=

(TI2- T22)1/2

=

tan-l T 2/T 1

>0 for a < 0 for a > 0

=

tanh-I T 2 / T 1

for a

p

=

a(1 - e2)

M

=

E - esinE

a>O

=

e sinh E - E

a

e

E

for a

D,

*

wDK

r%,TPf

TAPE CHANNEL

* *

y

TAPE SET

AT,,

*

*

* WTP

Note: in this table, “*” means “irrelevant.” The disk time fraction r,,,, includes both transmission and that part of the seeking activity which would be experienced if program i were run alone. In addition to this, we require an estimate 6rc,Dxof the extra seeking activity which is included when program i is run concurrently with another program which uses the disk. An approximate measure for this is 6rZ,DK = m,S/t, where (1) m, is the total number of disk reads and writes requested by program i, excluding those for which program i explicitly requests seeks; (2) S is the average seek time for the disk. Whenever a run request for a nonstandby program i arrives, the weights are updated as follows: w’,= w,f r& wLaX= max (w’,) 3

w‘,= w’,/wLax. The end-of-execution phase of a nonstandby program triggers a similar recomputation of weights, the service times for the departing program being subtracted, instead of added. The primes on the w’s are now dropped.

E. F. CODD

120

To rate a program Q as an addition to the set X of scheduled prograins, we tentatively allocate the tape files (see Section 5.5) to determine the channel loads The disk and PU loads are computed

where X' is X augmented by program Q. We also require the induced disk load

C 6r1,vKif more than one program uses disk, 6hDK= fx

0 otherwise. Loads are now normalized as follows:

A,,

max (Apc, AD, max (I, A,,) = Apu/m

AVK

=

A,

=

m

+6 L ,

h i ,

ATCd

=

XDKIm

XTCl/m ATc2/m Finally, the rating of program Q is given by 70

=

ATCl

=

hTC2

=

wpu.A(wpu - Apu)

+ wDK.A(wDK - h v ~ ) + wTP.A(wTP - ~ C I +) wTP.A(WTP - L P ) .

Note that the inclusion of the induced disk load in the determination of together with its exclusion from AD, gives the desired effect of a potential reduction in the normalized load on the disk.

, , ,A

5.10 Queueing

Four requirements for the queue disciplines used in batch multiprogramming are: (1) a tendency to keep all facilities heavily loaded, particularly the bottleneck facility; (2) the ability to handle urgent programs on a high priority basis; (3) the ability t o expedite programs consuming large amounts of space, if space-time is a bottleneck; and (4) simplicity. To observe the first requirement, it is necessary to exploit the special

MULTIPROGRAMMING

121

properties of the various time-shared facilities; for example, the ability of the processing unit to be interrupted. It is also necessary for the queueing scheme t o be responsive to changing patterns of behavior on the part of the programs currently being coexecuted. The following scheme is likely to be found satisfactory in batch multiprogramming. On the processing unit, priority is given at queue selection time t o that ready program which required the shortest burst of service from the PU when it last obtained such service. Each time a program relinquishes the P U or is displaced from it, the value of the length of the last burst is replaced. Consequently, the processing priority of a program is subject t o continual change depending not only upon the variability of its own behavior, but also on that of the other current programs. Queue selection is carried out when any of the following events occur. (1) A program being serviced b y the PU becomes not ready for processing (for example, a WAIT pseudo-op is issued for a n I0 operation which has not been completed). (2) A program, formerly not ready, becomes ready for processing (for example, an I0 operation being awaited is now completed). (3) A program being serviced by the PU has failed to relinquish it and has not been displaced from it for x units of time; where z is a fixed arbitrary limit imposed on the length of continuous P U service to a single program. At queue selection time, all programs ready for processing are considered t o be in the P U queue, even the program serviced most recently (provided it is still ready). This queueing rule tends t o expedite entry into the NOT READY state of those programs which have a tendency to enter this state. When in this state, these programs are utilizing the I0 equipment. Meanwhile, the P U is kept well-loaded by programs which tend to (but are not allowed to) monopolize it. These latter programs may also use the I0 equipment concurrently with the PU. Associated with this queueing rule on the PU, first-come-first-served (FIFO) would be appropriate on each tape channel. On the disk channel, unnecessary delays in seeking may be avoided by selecting that service request which entails the least movement of the seek mechanism, taking its current position into consideration. This rule needs slight modification because it could result in a request for service being ignored for an indefinitely long time interval if an outlying area of the disk were involved. A simple solution is to make sweeps in alternate directions clearing u p all the current requests for one direction before changing direction. E. S. Lowry has proposed the following, more sophisticated scheme. For program P , the quantity

122

E. F. CODD

Y i= C rij 3

where j ranges over those time-shared facilities for which Pi is currently unready, gives a good measure of the capacity of work on Pi to create readiness on ot,her facilities. The values of rij (supplied initially if the space-time scheduled mode is being used) are added to and subtracted from Yi whenever Pi goes unready or ready on a facility j . The initial value of Yi is the sum of rij for all j except the PU. Queue selection for the PU and tape channels entails choosing that program which has the largest value of Y;at this instant, unless two or more programs possess values very close to this maximum. In this case, priority is given to that request (associated with those programs with values of Yi at or near the maximum) which is likely to restore buffer levels to equilibrium most promptly. Note that, if buffer levels are kept in or near a state of equilibrium, all Yiare likely to be very small.

6. Multiprogramming with Two or More Processing Units 6.1 Motivation and Requirements

The incorporation of two or more functionally identical, self-sufficient, in a single computer system is general purpose processing units (PU’S)~ becoming increasingly important as higher processing speeds and higher degrees of reliability are demanded. We focus attention on systems in which the PU’s share the use of memory and input-output channels, because, in such systems, the current workload can be distributed over the various system components in an extremely flexible way, and this will very often yield high overall efficiency. An example of this type of system is illustrated in Fig. 9. The workload is assumed to consist of a number of independent programs (referred to as problem programs or PP’s) which need to be protected from damaging one another when they are concurrently executed. Each of these problem programs may itself be a complex of dependent programs which can be executed concurrently, subject to constraints contained within the programs themselves in the form of special operations to be discussed later. We shall deal with the object form of these programs and omit consideration of their source form and compilation. A fundamental requirement is that the system is to continue in operatio11 if any PU’s become inoperable, so long as at least one PU remains operable. This requirement has many implications ; in particular, the following: Note that the PU’s may differ in their performance characteristics and construction, but are completely interchangeable in all functional rePpcrt,s.

...

----

9

L-----

I0

\

1 -----

DISK CHANNEL I

\

\

/

1

TAPE UNITS

-

-

I'

-

2' -

3'

4'

2

\

L/

3

TAPE CHANNEL

\

..*

-- - - - 3 - - - - -

TAPE CHANNEL

\

-7-

FIG.9. 0rganisat.ion of computer system.

'~NSTRUCTIONS ---L

I

2

\

PROCESSING UNIT

PROCESSING UNIT

3

/

I-

I0 INTERRUPTIONS 7----7----F----

/

---

32-

TAPE UNITS

4-

I-

DISK UNITS

2-

z

w

h)

d

0

3 3

3

F-r 8-u

124

E. F. CODD

(1) The supervisory program must not be permanently associated with any particular PU or subset of PU’s, nor must it require the undivided attention of a whole PU. (2) It must be possible to initiate I0 activities on any channel from any PU. (3) Every PU must be capable of responding to interruptions of all types, including I0 interruptions. Of course, to avoid duplicate handling of I0 interruptions, it is desirable at any instant for only one PU to be designated to receive such interruptions. (4) Every problem program must be in such a form that the correctness of its execution does not depend on which PU’s, and how many PU’s, are available to service the program. A second requirement is that two or more PU’s be permitted to execute concurrently a single set of instructions using different, but possibly intersecting, sets of data.’ This requirement arises when processing large arrays of data. In these circumstances, segmentation of data for concurrent operation is more readily effected and likely to be more profitable than segmentation of instructions. When several concurrent executions of a single set of instructions are to be initiated, the starting address for fetching instructions is insufficient to characterize uniquely each of the corresponding requests for processing service. For each request we require, in addition to the starting address, those index quantities and parameters which are pertinent to this request. All of these items may be made available in a set of consecutive locations. A request may then be uniquely characterized by specifying the base address of this set. This base address is called the jirst base of the corresponding request. Note that this requirement would not arise at all in a system in which all the PU’s were functionally specialized. A third requirement is a means of preventing more than one PU from concurrently modifying an item in memory. Consider, for example, that two processing units are concurrently servicing a common section of code, and the code requires that each PU reduce a certain count in memory by unity. If the first PU fetched the count from memory after the second PU had fetched it but before the second PU had stored its reduced value, then, when both PU’s have completed their decrementing and storing of the count, the value in memory is only one less, instead of two less, than the original value. To overcome this difficulty it is necessary to provide in the system for temporarily blocking references to selected locations in memory or entry by more than one PU into selected sections of code. The latter approach appears more satisfactory and is adopted in this paper. Note that a shared set of instructions may he a large or a small set, and may or may not contain branches and loops.

MULTIPROGRAMMING

125

A final requirement is that there be a priority scheme which permits urgent programs to be processed expeditiously and normal programs to be processed with relative priorities which yield good utilization of the equipment. Priorities must be under the control of the supervisory program. Accordingly, no problem program must be able to alter the priorities allocated by the supervisory program or impede their implementation by holding on t o a processing unit which is needed by a higher priority program. Thus, it is necessary to provide for displacement (from a PU) of one program by another program a t an arbitrary stage of execution of the first. An address is required for dumping (and later restoring) the P U stat,us (arithmetic and index registers, etc.) of the displaced program-this address is called the second base. 6.2 Note on Implementation

I n what follows, the supervisory functions are described as though they were implemented by an ordinary program executable b y any PU. For the sake of simplicity, and because the supervisory activity may be expected to take but a small percentage of time of the PU’s, we shall assume that the supervisory program always operates in the blocked mode; i.e., a t any instant only one PU may be executing it. An alternative implementation would be by a microprogram. However, whether program or microprogram is used, the supervisor is retained in read-only memory because of its critical role in the operation of the whole system. Supervisory tables, on the other hand, are held in regular core storage and are protected by the same means applied to problem programs. 6.3 Rules for Interruption

Interruptions generated within a given processing unit are normally handled by this same unit. Examples are: (1) the interval timer in this PU reaching zero; ( 2 ) program exceptions, such as invalid op code or illegal address; (3) data exceptions, such as overflow, zero divisor, flagged data word, etc.; (4) a CALL SUPERVISOR operation being encountered in the program. I n cases (I), ( 2 ) , and (4) the supervisory program is called in on the same P U in which the interruption occurred. I n case (3) the interruption causes a trap t o a location within the execution area assigned to the problem program. An exceptional case is that of an interruption generated as a result of a malfunction within some PU. The corresponding interruption signal causes the malfunctioning PU to stop with its operable bit turned off (i.e., set to indicate that this PU is not operable in the normal sense for the time being). This signal also interrupts some other P U which

126

E. F. CODD

has its operable bit on (which one does not matter). Thus the supervisory program may be executed by a PU which is still functioning properly and may also take note of the temporary unavailability of the malfunctioning PU. We have already noted that all PU’s must be capable of receiving and acting upon I0 interruptions, and that only one PU at a time is so designated. If a malfunction is detected in the PU designated to receive I0 interruptions, these interruptions must be switched either automatically or under the control of the supervisor to some other PU which is operable. We choose the latter approach because it facilitates implementation of the priority requirement. Thus, whenever the supervisor assigns work to a PU-or attempts to find work and fails-it determines which PU has the lowest priority activity, and selects that one to receive I0 interruptions, at least until the next work assignment for PU’s is considered. Note that idleness is the lowest priority activity of all. 6.3.1 Nondata-Exception Interruptions Suppose the supervisory program is not currently active on any PU. Then, every operable PU is capable of accepting, and acting upon, a nondata-exception interruption generated within itself and is said to be in the normal state. In addition, one of these PU’s is set to accept and act upon an I0 interruption, should it arise. Now, suppose a nondata-exception interruption occurs on Pun.Further nondata-exception interruptions in Pun are not disabled, because, as we shall see, their occurrence means an emergency situation. PU, is, however, placed in the executing supervisor state. In this state nondata-exception interruptions cause a trap to special locations designated for emergency action. Simultaneously with PU, being placed in the executing supervisor state, all other PU’s and all I0 channels are placed on notice that each one may continue to operate concurrently with PU, until such time as a nondata-exception interruption is generated within that I0 channel or PU. This channel or PU must then enter the waiting for supervisor state, in which all of its registers are frozen until the supervisor can be made available to it. In this way any interruptions requiring supervisory treatment which are generated at a time when the supervisor is busy handling some other activity are held in suspension until the supervisor is free to work on them. The supervisory program is constructed so that it does not generate interruptions at all unless a malfunction has occurred in the PU, memory bus, or memory. Therefore, the generation of an interrupt condition in a PU which is executing the supervisor calls for emergency action. Upon completing its activity on PU,, the supervisor places this PU in the on-notice state and issues a release signal to all I0 channels and all

MULTIPROGRAMMING

127

PU’s including itsclf. If none of the PU’s is waiting for the supervisor, all PU’s and all channels are released by this signal from the on-notice state and returned to the normal state. If any of the PU’s are waiting for the supervisor, one of these is selected in an arbitrary way for placement into the executing supervisor state, and all other PU’s and all channels are kept in the on-notice state. Finally, after PU’s waiting for the supervisor have passed one by one through the executing supervisor state, the frozen channels, if any, are selected one by one to submit their waiting interruptions to the PU with the lowest priority activity. Note that the executing supervisor state is not defined for channels, because they only generate interrupts-they do not act upon them. We may now summarize the interruption states of I0 chaniiels and PU’s by means of the state diagrams in Fig. 10. 1/0 CHANNEL

/-\ NORMAL

% s; : : supervisory :state

Release with no PU waiting for supervisor

WAITING FOR SUPERVISOR

ON NOTICE

I/O interruption

generated within this channel, with supervisor busy on some PU

PROCESSING UNIT

1

Nondata -exception interruption, with supervisor not busy

+ EXECUTING SUPERVISOR

NORMAL Other PU begins executing supervisor

visory activity superon

IReIease

F WAITING FOR SUPERVISOR

ON NOTICE

Nondata -exception interruption, with supervisor busy on other PU

FIG.10. Summary of interruption states of I 0 channels and PU’s.

128

E.

F. CODD

When a P U is in the normal or on-notice stsate,it is either executing a problem program or it is idle (i.e., it is not assigned to any stream and is merely marking time). Conversely, when a PU is executing a problem program or it is idle, it must be in the normal or on-notice state. I n the waiting for supervisor state, a PU is assigned to a stream and is therefore not idle in the sense defined above. The handling of nondata-exception interruptions by the supervisor is facilitated by the availability of two pieces of information in the hardware. First, the identifying number n of PU, may be read by the supervisor and used to locate table entries in memory for PU,. Second, the identifying number m of I0 channel m is deposited by hardware in a standard supervisory location whenever an interruption from that channel is taken by the PU designated to receive and act upon I0 interruptions. 6.3.2 Data-Exception Interruptions A data-exception interruption on a given PU disables further dataexception interruptions on that PU only and has no effect on nondataexception interruptions or their associated int,erruption states described above. It is the responsibility of the problem program generating a data-exception interruption to handle this interruption. A hardware operation ENABLE DATA EXCEPTIONS is available for use by problem programs. When one stream is displaced by another on a PU, the data exception state (enabled or disabled) is saved for the former and restored for the latter-along with all the other it,ems such as accumulator contents, index registers, etc. 6.4 Protection and Relocation

Each PU is equipped with its own protection system and, since all PU’s are functionally identical, so are all the protection systems. The protection system of a given P U is automatically disabled when a nondata-exception interruption occurs on that PU, thus permitting the supervisory program to have access to normally protected memory areas and registers, including the protection system registers themselves. Just prior to passing control back to a problem program on some PU, the supervisory program sets up the protection system registers with values appropriate to this problem program. As it passes control back to this problem program, the supervisor re-enables the protection system on this PU. A simple form of relocation and protection will sufIice for the purpose of illustration. Consider any one of the PU’s and suppose its protection system is enabled. All addresses developed within this PU for use in referring t o memory are incremented just prior to use by the contents A of a relocation register, and the incremented values are compared with the

MULTIPROGRAMMING

129

contents B of an upper boundary register. Whenever the addition results in overflow or the comparison indicates that the relocated address exceeds the upper boundary, a program exception interruption occurs and the supervisor is brought into action on this PU. Normally, A is the base address of the problem program. When two PU’s are servicing the same problem program, the values of A and B in effect on one PU are normally equal respectively to the values of A and B in effect on the other PU. Of course, it is certainly possible to have different pairs of values in effect, but the primary need for protection is between, rather than within, problem programs. 6.4.1 Restricted Hardware Operations The majority of operations provided in hardware are freely usable by any program. Certain ones, however, are intended for use by the supervisory program only while others are intended for use by problem programs only. For example, absolute I0 instructions are in the supervisor-only class, while the operation CALL SUPERVISOR is in the problem-program-only class. An attempt by a problem program to issue an operation restricted t o use by the supervisor, or vice versa, is aborted and a nondata-exception interruption occurs. This is realized in hardware by having the PU examine its own interruption state upon receipt of a restricted operation. A table of these restricted operations follow: ~

Supervisor only

Problem program only

CALL PROBLEM PROGRAM

CALL SUPERVISOR

S E T AND WAIT

ENABLE DATA EXCEPTIONS

START OTHER PU SWITCH I 0 INTERRUPTIOh-S

START CHANNEL,

etc.

We now define the first four supervisory operations and the first PP operation in this table. The others need no elaboration. (a) CALL PROBLEM PROGRAM, address p . Let the contents of the specified memory location p be an address b. The executing PU, say PU,, fetches a starting address from memory location b and places it in its instruction counter. PU, then enables its protection system, prepares so that any future nondata-exception interruption will trap t o nonemergency locations, places itself in the on-notice state, and sends a release signal to all PU’s. Prior to issuing this operation, the supervisor fetches the first base

E. F. CODD

130

or second base (whichever is appropriate) of the stream to be started,

and stores this address b in memory location p . It also restores all arithmetic and index registers per b, and sets up the relocation base register and upper boundary register if new values are required. (b) SET AND WAIT.^ This operation places a PU which encounters it in the idle, on-notice state and causes the release signal to be issued to all PU’s including itself. In the idle state, a PU is ready to accept either a start signal from any other PU or an I0 interruption. Of course, the I0 interruption switch must be set to this PU if it is actually to receive an I0 interruption. (c) START OTHER PU, PU number. Let the PU encountering this be PU, and the PU specified by the PU number be PU,. If n = 2, a program exception interruption occurs. If n # z and the operable bit of PU, is off, a program exception interruption occurs. Otherwise, PU, is placed in the waiting-for-supervisor state ready to start execution at a supervisory location associated with leaving the idle state. Actual execution starts on PU, as soon as it is permitted to go into the executing supervisor state by PU,, or some other PU leaving that state. (d) SWITCH 10 INTERRUPTIOXS, PU number. This operation sets the I0 interruption switch to the specified PU. It is permissible, in this case, for the PU encountering this operation and the specified PU to be identical. upon receipt of this operation from a problem (e) CALL SUPERVISOR. program stream, PU, disables its own protection system, traps to a nonemergency location assigned for this kind of interruption, and prepares for any future interruptions to trap to emergency locations. In addition, PU, places itself in the executing supervisor state or the waiting-for-supervisor state according to the availability of the supervisor. 6.5 Special Operations

A small set of special operations is adequate to permit a programmer to express his program so that two or more PU’s can work on it concurrently, if they happen to be available. These special operations, accompanied by appropriate addresses, are essentially instructions to the supervisory program. They enable the problem programmer to start new streams of activity, stop them, regroup them, and whenever necessary, block them. In addition, these operations provide a vehicle of expression which permits An operation similar t o this was defined in 1957 for the IBM STRETCU Computer by F. P. Brooks, Jr.

MULTIPROGRAMMING

131

a program t o be independent of which PU’s, and how many, are available to service the program a t any instant. A list of the operations follows. Processing

Input-Output

START

GET

STOP

PUT

BLOCK UNBLOCK

We now proceed to give an introductory description of these operations, after which we can deal with the queueing system and describe these operations in more detail. A typical format for one of these operations is as follows. Location

m m+l m+2

Contents special op code Slot First Base, Second Base CALL SUPERVISOR,

The slot is a word in which the supervisor can deposit addresses and bits for queueing purposes.

Slot, First Base, Second Base From the user’s viewpoint, this operation calls for continuing the execution of the current stream of instructions and starting the concurrent execution of a second stream a t a start address specified indirectly b y the first base. The index quantities and parameters for this stream are retrievable from a set of consecutive locations starting a t the first base. If status dumping is required a t any time in the execution of the stream now called, i t is to be effected per the second base. From the supervisor’s viewpoint, this operation represents a request for starting concurrent execution of a second stream-a request which may be serviced either now or later, depending on the availability of units and the competing requirements of other programs and of other streams in this program. Accordingly, the supervisor incorporates the request in the queue for processing service. The request is kept in the processing queue while being serviced as well as while waiting for service. Service for this request is not considered complete until the following operation is encountered. 6.5.1

START,

132

E. F.

CODD

6.5.2 STOP No slot or addresses are needed for this operation. The stream in which this operation is encountered is terminated. The supervisory program removes from the processing queue the corresponding request, and scans the queue for new work for the PU which was servicing this request. 6.5.3 BLOCK, Slot No addresses are needed for this operation. From the user’s viewpoint, this operation prevents more than one processing unit at a time from entering the code which immediately follows this operation. Actually, entry of a second PU is only prohibited if it is attempted via this operation itself. This limited form of protectiong is quite adequate for this purpose. The supervisor determines whether entry into the code immediately following is permissible or not. If it is, a bit is set in the slot to prohibit any other entry via this operation until further notice. If this bit (the blocking bit) is already on, indicating that entry is not permissible, the request which was being serviced by this processing unit is re-chained into the processing queue in such a way as to indicate that availability of the code as well as of the unit is awaited. 6.5.4 UNBLOCK, Block Address No slot is needed for this operation. This operation permits a new entry to this section of code at the block address, such entry having been previously prohibited by a BLOCK operation located at this block address. Upon receipt of the UNBLOCK operation, the supervisor ascertains whether there is a request awaiting availability of this section of code. If there is no such request, the blocking bit for this section is turned off. Otherwise, the blocking bit is left on, and that request for this code which is next in line is given a new position in the queue which indicates that availability of a unit only is awaited. 6.5.5 Programming Example Within a section of code denoted A we desire to make two calls for a section B. The resulting executions of B are intended to proceed concurrently with one another and with the continuation of A. Finally, all three calls are to be regrouped on a section C. One method of programming this requirement is shown in Fig, 11. The strings of dots indicate sections of code of any length or complexity. In particular, these sections may contain conditional branches. Another This form of protection should not be confused with interprogram protection.

MULTIPROGRAMMING

133

BLOCK Reduce count by I Test count

BRANCH

UNBLOCK STOP

Reset count to 3 UNBLOCK

J

FIG.11. Three-call programming scheme.

point of interest is that regrouping may be made to depend on any logical combination of states of the three calls by replacing the counting with suitable manipulation of bits.

6.5.6 The I0 Operations: GET a n d PUT As mentioned earlier, problem programs do not issue absolute I0 instructions at all. The compiled (or object) form of these programs contains the pseudo-ops GET and PUT wherever the program is ready to acquire an input record or to dispose of an output record respectively. The I0 unit involved is designated symbolically by a number called the symbolic unit (SU) * A symbolic unit corresponds to an absolute tape unit or an absolute region of disk storage located on one of the disk units. The user specifies which type of storage he needs, but the allocation routine within the supervisory program determines which physical unit and, in the case of disk storage, which absolute region is to be assigned. When the symbolic unit corresponds to a tape unit, the GET operation implies that the next record from tape (i.e., next in arrival sequence) is required, and the PUT operation similarly implies a sequential disposition on tape. When the symbolic unit corresponds to a disk region, the choice between the sequential and nonsequential modes of referencing must be made. For simplicity, we assume that each problem program defines its own buffer areas in core storage and provides a buffer directory for use by the supervisory program. The supervisor attempts to keep read buffers as full as possible and write buffers as empty as possible, but observes an established set of priorities in so doing.

134

E. F. CODD

Thus, when a GET operation is issued from a processing stream belonging to a problem program, the required record may be already available in core storage. In this case, the supervisor plants the base address of this record (relative to the base address of the problem program) in the slot immediately following the GET operation and permits immediate resumption of the stream on the same PU as before. Whenever the supervisor discovers upon receipt of a GET operation that it has not yet acquired the record now needed (this is invariably the case with the nonsequential mode of reading from disk storage), it suspends processing service to the stream in question until that record has been acquired. It determines if retrieval of the record can be initiated without violating established priorities and seeks other work for this processing unit. Eventually, when the record has been read into the appropriate buffer area, the supervisor is activated by the corresponding I0 interruption, and it changes the status of the service request for the suspended stream to indicate that it is merely awaiting availability of a processing unit. If the stream now released for action is of higher processing priority than the lowest priority stream previously being serviced, then the newly released stream obtains service immediately 011 this PU. The PUT operation is treated similarly. 6.6 The Queueing System

Queueing itself has the effect of decoupling timewise the creation of demands by programs on the one hand and their servicing on the other. When symbolic to-absolute translation is incorporated in the queueing system, spacewiselodecoupling is obtained also. The queueing system proposed herein has the following additional properties : (1) Interprogram priorities are assignable by the supervisor independently for all forms of service. ( 2 ) In the case of disk units it is possible for the priorities to reflect the physical location of the assigned regions and, as a result, cause each arm to service the request involving the least movement in a given direction (inward or outward) until all requests in that direction have been serviced and then to change direction. (3) Each problem program may generate any number of requests for processing service, subject only to core storage limitations. (4) While a problem program may through program error mutilate its own request for service, it cannot damage the rest of the queueing structure, unless a malfunction has occurred in the protection system of a P U or in memory itself. 10 In this context, space connotes areas in core and disk storage, also sets of I 0 units of all kinds.

MULTIPROGRAMMING

135

( 5 ) Processing requests which are awaiting availability of (a) a section of code and a PU, (b) a buffer area and a PU, (c) a PU only, and (d) nothing (i.e., the request is being serviced) are all distinguished from one another. The queues to be considered are those for processing service (one queue only), tape channel service (one queue for channel), disk channel service (one queue per channel) and seek service (one queue per disk unit). In describing the structure of the various queues, three core storage areas are of vital interest. The first is the area in which supervisory tables are located. Each queue has its origin or root in this area. The second area is the control area for each problem program. This area contains PP control nodes which contain information necessary to the proper servicing of the corresponding program. Like the supervisory tables, all PP control areas are accessible to the superwisor only. Finally, there is the execution area for each problem program which is freely accessible by that program and by the supervisor, but by no other program. The request nodes for a PP are all located in the corresponding execution area. The queueing structures are illustrated in Figs. 12, 13, and 14. Each queue consists of a set of nodes (marked 0 ) linked together to form a tree. Each node consists of a set of several consecutively located words in core storage. The location of any node is defined to be the location of its first word. A link from node x t o node y is realized by storing in a slot in node x the location of node y.

6.6.1 T h e Processing Service Tree The processing service tree begins in the supervisory tables a t a supervisory node labelled processing service (Fig. 12). Chained from this node is a set of processing control nodes, there being exactly one such node for every problem program. The ordering of these control nodes in the tree determines the priority with which problem programs receive processing service. Attached to each processing control node is a subtree consisting of processing request nodes. Each of these nodes consists of a START operation and its associated slot, first base and second base. Those request nodes which, in the diagram, are aligned horizontally with the corresponding control node represent requests which may be serviced concurrently. For example, in Fig. 12 request nodes tl, ul,v1 belonging to PP2 are aligned horizontally with the PP2 control node for processing. Thus, three distinct processing units may, if available, be assigned t o service these requests concurrently, one PU per request. Those request nodes which are aligiied in a vmtical chain must not be

E. F. CODD

136

PP 4

PP 2

PP 7

PP I

1ICONTROLf-kI 1 ETC

i

EXEC I

I

‘LPROTECTION BOUNDARY P P = PROBLEM PROGRAM

FIG.12. Processing service tree.

serviced concurrently. For example, in Fig. 12, u1 is blocking u2 from using some shared section of code. Accordingly, the stream corresponding to u1 must issue a n UNBLOCK operation before u2 can be serviced. Note that as far as the user is concerned, only two types of precedence constraintsblocked code and unready buffers-are handled within the queueing sys-

137

MULTIPROGRAMMING CHANNEL 3 SERVICE

SUPERVISORY PROGRAM

CHANNEL 3 ASSIGNMENT

p

/

I

P

PP 2

PP I

CONTROL PP 7

EXEC

PP 4

I

EXEC I

I

PROTECTION BOUNDARY

SU = SYMBOLIC UNIT PP = PROBLEM PROGRAM

FIG.13. Tape channel service tree.

tem. More general requirements for precedence constraints between sections of a program may be implemented by means of regrouping, an example of which was illustrated in Paragraph 6.5.6. We have observed that upon receipt of a STOP from some stream the supervisor is required to remove the corresponding request node from the processing queue; also, from time to time the supervisor is required to displace streams temporarily from PU's. Both of these supervisory activities are facilitated by the processing assignment nodes located in the supervisory

E. F. CODD

138

SUPERVISORY PROGRAM

DISK UNIT I SEEK

,/DISK

ARMS

/’

ETC

UNIT 2

SEEK ARMS

SEEK

PP 2

PP 5

I I I

su4

4 I

I

I

L

I

EXEC

I

‘4

su 2

PP 4

I EXEC I

I

L-

PROTECTION BOUNDARY

SU SYMBOLIC UNIT PP = PROBLEM PROGRAM

FIG.14. Disk channel service tree

tables. There is one such node for each PU in the system and it contains two addresses, one pointing to the control node and the other t o the request node currently being serviced by the PU in question. The four types of processing nodes introduced above are now listed with

MULTIPROGRAMMING

139

their contents in Table I; V stands for vertical chain, H for horizontal chain. In Table I, the items marked X in the processing request node are located in the slot associated with every START operation. They are placed in the slot by the supervisory program, not by a problem program. TABLE I Node

Contents

Processing service (one only)

V address V bit

Processing control (one for each PP)

V address V bit H address H bit

Processing request (one for each stream)

Processing assignment (one for each PU)

X X X X X X X

START (Special Operation) V address V bit H address H bit Busy bit Buffer stalled bit Base bit First base Second base

Address of processing control node being serviced Address of processing request node being serviced Operable bit Busy bit

The V and H addresses are in every case the addresses of successor nodes in the service tree, vertically and horizontally respectively. The V and H bits indicate whether or not there is a vertical or horizontal successor respectively. The assigned bit indicates whether or not a PU is currently

assigned, and hence is servicing, this request. The buffer stalled bit is turned on if, in servicing this request, a GET or PUT is encountered and the buffer in question is not ready for use by the problem program. The base bit indicates which base (first or second) is to be used for setting up a PU assigned to service this request. The first set-up following receipt by the supervisor of a START is effected per the first base. All subsequent set-ups and, of course, all dumps for this stream are effected per the second base.

E. F. CODD

1 40

The items in the processing assignment node apply to the corresponding PU and have self-evident definitions.

START on PUn from stream (c,ra) at location rd

F

Obtain c from Pun aksignrnent node and attach request to end of horizontal chain from control node at c.

Entry point 1

Turn off base bit, bdffer-stalled bit and busy bit in this request node.

1

) -

Scan in sequence PUx assignment nodes x = I, 2,3,._. for idle, operable PU

II

None chosen

PUy chosen

ment node

--

Turn on busy bit in request node, issue --START OTHER PU to PUy

-I Re1ea se signal

$.I

Prepare to resume stream (c, ra) on PUn (using second base b 2 of request node for this stream), issue CALL PROBLEM PROGRAM per b2

---------~

I I Prepare to service assigned stream (c, rd)

(see entry point 3 i n STOP)

FIG.15

-F

n -

MULTlPROGRAMMING

141

START

Pun assigned to (c, ro. PUy idle

AFTER

.

C

?

Pun assigned to (c, ro) PUy assigned to (c, rd)

Busy

4 rc

Code Blocked

FIG.16

address of a processing control node address of a processing request node identification of a stream which has a request node a t r chained t o a control node a t c We omit details concerning the manipulation of chain addresses and bits required to couple a node into a chain or uncouple it from a chain, since the reader can with little effort fill in these details for himself. C

r (c, r)

6.6.3 I 0 Service Trees We assume that cross-channel switching is not provided for the tape or disk channels. If it were, only minor changes would be required in the following scheme. The most important of these would be the combination of all trees for individual tape channels into one tree for all tape channel service.

E. F. CODD

142 STOP on P u n from streom (c,rb) at location h

Obtoin c, r b from P u n ossignrnent node and remove corresponding request from processing queue. I f this request node wos the only one ottoched t o the corresponding control node

1

Otherwise

'-{ End of job: remove this control node from the processing queue

Entry point 2

Scan processing queue for new assignment starting from the beginning of the queue ( i . e., per the V address of the processing service node), choose first request found for which both the busy bit ond the buffer -stolled bit ore o f f .

r-----'1 I f none found

Otherwise

P u n ossignment TL f f busy bit node, in

Place c2, rf for chosen request in P u n

issue SET AND WAIT.

ossignment node

Entry point 3

7

1

Prepare to service assigned stream i.e.. load registers of P u n per f i r s t or second base (occording as base bit is off or on), turn base bit on, set relocation and protection addresses per c, issue CALL PROBLEM PROGRAM per first or second base.

FIG.17

Figures 13 and 14 illustrate respectively the tree for a particular tape channel and the trees (one per disk unit) for a disk channel. The channel service node and assignment node are very similar to their processing counterparts in information content and function. The channel control nodes and request nodes are, however, very different from their processing counterparts. We begin by discussing the complete set of control nodes for a single problem program. The single node for processing has already been described. Each of the remaining control nodes corresponds to a symbolic unit (SU) used or ignored by this program. The SU nodes are consecutively

MULTIPROGRAMMING

143

STOP BEFORE

Stalled

Pun assigned

to ( c l r r b )

I.

Code Blocked

c2f----2--2N0, Not Busy

Busy

AFTER

Busy

CI

8

‘d

rC 0 - y

Busy

~3 Buffer Stalled

Pun assigned to ( c 2 , r f )

‘e

bCode Blocked

FIG.18

numbered and are ordered in memory by this number. Thus, the supervisor is able to locate the appropriate SU node quickly whenever it receives a GET or PUT accompanied by a symbolic unit number. The information within the SU node enables the supervisor to identify the absolute unit (and, in the case of disk storage, the region) corresponding

144

E. F. CODD

.

BLOCK on Pun from stream (c,rd) at location h

Pun

Examine blocking bit in BLOCK slot I f off

I

Turn blocking bit bn in BLOCK slot

+

Obtain r d from Pun assignment node and place in BLOCK slot

J.

Resume service to previous stream (c, rd) on Pun

Obtain address rb of request node for blocking stream from BLOCK slot and vertically chain onto i t the current request node on Pun (address rd of this node obtained from Pun assignment node)

J.

Turn off busy bit in request thus vertically chained

t

Scan for new work for Pun (see entry point 2 i n STOP)

to this SU. As usual, this correspondence is set up at loading time by the allocation routine associated with the supervisor. Other items in the SU node identify the mode in which the unit is being operated (sequential or nonsequential) and the state of readiness of the set of buffer areas associated with this SU." The request nodes chained to a given SU node define buffer areas with origins and sizes determined by the programmer (or course, the origins are compiled in relative form). h'ormally, these buffer nodes are chained t o their SU node permanently, i.e., throughout the execution phase of the problem program. I n addition to defining a buffer area and indicating whether its current use is for reading or writing, a buffer node describes the state of this area: Is it ready for use by some stream, is it in use by a stream, is there a stream stalled because this buffer area is not ready and, if so, where is the processing request for this stalled stream? Chain addresses and bits are also included in each buffer node. Their use depends on whether the sequential or nonsequential mode is in effect for this symbolic unit. 6.6.3.1 Sequential Mode. I n the sequential mode a set of records is to be transmitted into or out of core storage preserving the ordering in which l 1 A buffer area is ready for a GET operation if it is full and for a PUT operation if it it3 empty. A set of buffer areas is ready if all members of the set are ready.

MULTIPROGRAMMING

145

BLOCK BEFORE Pun ossigned to (c,rd) BLOCK cf.L+//G Not Busy Busy ‘d

,//

Code Blocked

AFTER IF NOT BLOCKED Note: BLOCK slot changed from 0 to I , rd

Same diagram 0s obove,

AFTER IF BLOCKED BY rb

Block slot unchanged: 1, r b Pun ossigned to (c, ro) Code Blocked ‘d

Code Blocked

FIG.20

they are originally stored or computed. Thus, if a SU has associated with it a buffer consisting of several areas and the sequential mode is in effect, the supervisor services these buffer areas in a fixed, cyclic manner. Further, either all areas of a given buffer are designated as receiving areas for input or all are designated dispatching areas for output. The buffer nodes associated with a SU node in the sequential mode are vertically chained in a cycle. In the special case of a buffer consisting of only one buffer area, this rule requires that the buffer node be chained to itself vertically; i.e., its V address is set equal to its location. As the supervisor completes servicing one buffer area, it prepares to service the next in

E. F. CODD

146

-Pun

Obtain 'b from PU" assignment node, examine V b i t in this node

No other request waiting for this code

f

Turn off blocking bit in BLOCK slot addressed by h

Request waiting far this code

?

Uncouple next waiting request (c, rd) from vertical chain on rb and attoch to end of horizontal chain on c, leaving any other waiting requests [c, re) vertically attached to i t

$.

Place rd in BLOCK slot oddressed by h, look for PU t o service stream (c, rd) (see entry paint I in START)

Resume service to stream (c, rb) on Pun

FIG.21

the cycle by copying the V address of the node just serviced into the SU node. 6.6.3.2 Nonsequential Mode. I n the nonsequential mode the buffer areas belonging to a given buffer may be serviced in any order or concurrently. Concurrent servicefor a given SU is, however, limited to seeking (we assume two or more arms per disk unit, several disk units per transmission channel). Any buffer area associated with a SU in the nonsequential mode may be used for input or output regardless of its immediately previous use and regardless of the use being made of other buffer areas associated with this SU. Each GET or PUT operation issued for a SU in the nonsequential mode is accompanied by a relative disk address. Any stream which issues a GET operation of this type is unconditionally stalled because, even if there is a buffer area immediately available for receiving input, some time must elapse before the desired record has been read into this area. Note that processing activities for this program as a whole are not stalled-other streams may continue to function. Any stream which issues a nonsequential

MULTIPROGRAMMING

147

UNBLOCK, h

Pun assigned to (c, rb ) BEFORE

Buffer Stalled Block Slot: l,rb

f

?ode Blocked

re

Code Blocked

AFTER

Buffer Stalled

Busy

I

Busy

& Code Blocked 'e

operation is stalled only if all buffer areas for the specified SU are now unready for use by the program. The buffer nodes for a SU in the nonsequential mode are chained horizontally from the SU node. The ordering of nodes in this chain has no special significance. If the disk unit to which this SU is allocated has two arms, any two buffer nodes belonging to this SU may be concurrently assigned to the arms. 6.6.3.3 Completion of I 0 Operations. Successful completion of data transmission to or from a tape or disk unit activates the supervisor on some PU, and implies that a buffer area previously unready for program use is now ready. The supervisor finds the buffer node for this area by indirect referencing via the channel assignment node, the arm assignment node (disk PUT

148

E. F. CODD

only), and the SU node, sets the ready bit on, and determines if there is a stream stalled due to this buffer area being unready a t the time a GET or PUT operation was issued. If a stream is waiting for this buffer area, the supervisor obtains from the buffer node the address of the processing request node for the stalled stream, and turns the buffer stalled bit off, thus making that request eligible for further processing service. If the relevant SU node is in the sequential mode, the supervisor now advances an address in this node so that it points to the nest one of its buffer nodes needing supervisory attention. Regardless of mode, the supervisor looks for work for this channel. In the case of a tape channel it can start scanning the corresponding service tree immediately. For a disk channel, however, the completion of a data transmission operation represents an opportunity to put into effect as many outstanding requests for seek service as possible. Accordingly, the supervisor scans the seek service tree for each unit on this channel which has at least one idle arm, and assigns as many of the idle arms as possible to buffer nodes needing service. Then, a disk unit is selected by taking the next unit (in a fixed cycle) which has an arm assignment such that the seeking is complete but the transmission still remains to be done. The channel is then assigned to such a n arm. 6.6.3.4 I 0 Queueing Rules. The queueing rule invoked when a tape channel service tree is scanned is as follows. The first SU node encountered that indicates that there is a stream stalled due to buffer area(s) belonging to this node being unready is selected for service. If no SU node is in this state, the first SU node encountered, which indicates that not all of its buffer areas are ready for program use, is selected. This selection rule can be implemented in such a way that only a single scan is required. The same rule may be applied to selecting a request in a disk seek service tree. However, the incidence of stalled streams may be expected to be higher for disk operations (the majority of which are likely to be nonsequential) than for tape operations (normally sequential). Moreover, the time taken to service a set of seek requests for a given disk unit may be sensitive to the order in which those requests are serviced. Thus it may be profitable to select next that request which entails the least arm movement. If this particular rule were followed, it would be possible for requests for remote areas to be left unserviced for indefinite periods. One simple modification is to service that request which entails the least arm movement in a given direction (inward or outward) until all requests for that direction have been disposed of, and then change the direction. Figure 14 illustrates an implementation of this rule. A bidirectional chain couples together all the SU nodes for a given disk unit in a sequence determined by the physical location of the corresponding

149

MULTIPROGRAMMING

TABLEI1 Node

Contents

Channel service (one per tape channel)

T T

V address V bit

Seek service (one per disk unit)

D D D

Vt address V1 bit

D

Vz bit

SU control (one per SU)

V2 address

V1

D D D D

address

V1 bit. VZaddress V2 bit Absolute channel Absolute unit Disk region base address Disk region upper boundary Mode bit Stream stalled bit Buffer all ready bit, Address of buffer node

D

Chain address (V or H) Chain bit (V or H) Base address of bufler area Stream stalled bit Address of processing request node of stdled stream Ready bit In use bit GET/PUT bit Itelative disk address

Tape channel assignment (one per tape channel)

T T T

Address of SU control node being serviced Operable bit Busy bit

Disk channel assignment (one per disk channel)

D D D

Address of arm assignment node being serviced Operable bit Busy bit

Arm assignment (one per arm)

D D D 11

Address of SU control node being serviced Address of buffer request node being serviced Operable bit Busy bit Ready for transmission bit Inward/outward bit

Buffer request (one per buffer area)

D D

150

E. F. CODD

disk storage regions. This chain is, of course, established and maintained by the allocation routine. When the supervisor is looking for work for an arm, it. can find that outstanding request which is for the nearest region in a given direction by first scanning the buffer nodes belonging to the SU just serviced and, if no work is found, proceeding to the next SU node in the direction currently in force. Should the end of the chain be reached with no work found, the direction of scan is reversed. 6.6.3.5 Summary of I0 Nodes. The seven types of I0 nodes introduced above are now listed in Table I1 with their contents; D and T denote items relevant t o disk only and tape only, respectively. All other items are relevant to both. Two vertical chain addresses and two vertical chain bits are required for the service and control nodes used with disk units. The address and bit denoted VI are used for inward scanning, those denoted Vz for outward. The reader is invited to develop for himself flow charts suitable for the GET and PUT operations, in the sequential and nonsequential modes, and for I0 interruptions which indicate successful completion of data transmission. 7. Concluding Remarks

Very few computer systems in existence today possess two or more processing units sharing a common set of core storage modules and inputoutput channels. The writer’s opinion is that the demand for this type of system will be greater when an adequate solution is found to the problem of exploiting such a system. The proposals in Section 6 represent an attempt to find an adequate solution to this problem. Consider some of the properties of the proposed scheme from the user’s point of view. First, if the user desires to write his program in a conservative way, as if it were to be executed by a machine with only one processing unit, he may do so by creating only one processing stream. This is certainly reasonable if the running time of his program is short. I n fact, whether his single-stream program is short or long, good utilization of the system can very probably be achieved through running other, independent programs concurrently with his-providing his program does not monopolize memory space. Second, if the user’s problem is expected to consume large amounts of processing time or memory space, he may segment his data or instructions or both and create as many streams as he pleases. The special operations for starting, stopping, and blocking these streams are quite simple for the user, even though complicated for the supervisory program. These oper-

MULTIPROGRAMMING

151

ations are, in fact, suitable for use in a problem-oriented source language, since their definition imposes few constraints on the hardware. Note that a user is not required to segment his instructions and data to fit individual memory modules.12 Third, interprogram protection (an example of which was illustrated) provides the individual user with security for his program and data against accidental or fraudulent manipulation. Now, consider some of the properties of the proposed scheme from the iiistallation point of view. First, space and time can be assigned to programs in a n extremely flexible way, thus giving the supervisory program-and hence t,he installation-a large measure of control over the use of the system. I n particular, processing priorities may be determined in any way the installation sees fit, since they are not constrained by the relative speeds of various units. Second, the scheme accommodates real time processing as well as batch processing. (Note, however, that for certain types of real time processing more elaborate protection would be required.) Implementation of the supervisory program does not appear to present any new problems. Most of the logic (and complexity) required in a multiprogramming supervisor for a system with two or more PU’s is already present in a multiprogramming supervisor for a system with but a single

PU. A word of caution is perhaps in order for both the designer and the user. First, the complexities of communication between memory-sharing PU’s are such that this type of system is not suited to low performance PU’s. Second, an over-zealous programmer may over-segment his program. It is importmt to remember that supervisory overhead is incurred in starting and controlling each stream created, no matter whether the supervisor is implemented in hardware or in software. The programmer is therefore advised not to create a new stream unless the set of operations to be performed in this stream is expected to take a t least as long as the average I0 operation. Today, the outstanding problem associated with multiprogramming is that of finding better methods of automatically allocating storage. This problem is now receiving a good deal of attention and its investigation may be expected to lead to a growing body of theory concerning the nature of programming itself. Although a t this time it appears that few computing installations employ anything more than a restricted form of multiprogramming in their daily operations, we can expect a radical change in this situation within the next l2 We assume that (1) the set of memory modules is addressable as a single homogeneous memory, and (2) any location may be used for instructions or data.

152

E. F. CODD

decade. The need to make effective use of machines with vastly increased performance and capacity will lead to widespread adoption of quite general multiprogramming techniques.

8. Acknowledgments

The author wishes to express his appreciation to the many people who have contributed to this article in one form or another. Particular thanks are due to Messrs. E. S. Lowry and R. H. Ramey, Miss E. McDonough, and Messrs. C. A. Scalzi, F. E. Howley and S. F. Grisoff, who have contributed many valuable ideas and suggestions in the course of their experimental work in multiprogramming under the author’s direction. In addition the author wishes to thank Miss E. McDonough for assistance in preparing the flow charts of Section 6 ; Dr. D. S. Henderson for helpful criticisms of this same section; Messrs. J. W. Franklin and P. J. Nelson for their painstaking editing of the first and second halves respectively; and Mr. E. S. T.owry for editorial assistance in the final stages.

Bibliography 1. Alexander, S. N., The National Bureau of Standards Eastern Automatic Computer. PTOC.Joint AIEE-IRE Gonj., Phaladelphia, Pennsylvania. Pp. 84-89, February (1952). 2. Cocke, J., and Kolsky, H. G., The virtual memory in the STRETCH computer. Proc. Eastern Joint Computer Conf., Boston Massachusetts. Pp. 82-93, December (1959). 3. Codd, E. F., Multiprogram scheduling. Communs. Assoc. Computing Machinery 3, 347-350 and 413-418 (1960). 4. Codd, E. F., Lowry, E. S., McDonough, E., and Scalzi, C. A., Multiprogramming STRETCH: feasibility considerations. Communs. Assoc. Computing Machinery 2, 13-17 (1959). 5. Dreyfus, P., System design of GA&iMA 60. Proc. Western Joint Compufer conf., Los Angeles, California. Pp. 130-132, May (1958). 6. Eckert, J. P., Weiner, J. R., Welsh, H. F., and Mitchell, H. F., The UNIVAC system. Proc. Joint AIEE-IRE Conf., Philadelphia, Pennsyleania. pp. 6-14, February (1952). 7. Everett, R. R., Zraket, C. A,, and Benington, H. D., Sage-a data processing system for air defense. Proc. Eastern Joint Computer Conj.,Wnshin&ton,D.C. pp. 148155, December (1957). 8. Frankovich, J. M., and Peterson, H. P., A functional description of the Lincoln TX-2 computer. Proc. Western Joint Compuler Conf., Los Angeles, California. pp. 146-155, February (1957). 9. Gill, S., Parallel programming. Computer J . 1 , 2-10 (1958). 10. Leiner, A. L., System specifications for the DYSEAC. J . Bssoc. Compiiting Machinery 1 , NO.2, 57-81 (1954).

MULTIPROGRAMMING

153

11. Leiner, A. L., and Alexander, S. N., System organization of DYSEAC.IRE Trans. on Electronic Comruters EC-3, 1-10 (1954). 12. Leiner, A. L., Notz, W. A., Smith, J. L., and Weinberger, A., P1LoT-a new multiple computer system. J . Assoc. Computing Machinery 6,313-335 (1959). 13. Lourie, N., Schrimpf, H., Reach, R., and Kahn, W., Arithmetic and control techniques in a multiprogram computer. Proc. Eastern Joint Comp~derConf., Boston, Massachusetts. pp. 75-81, December (1959). 14. Mersel, J., Program interrupt on the UNIVACscientific computer. Proc. Western Joint Computer Conf., S a n Francisco, California. p. 52, February (1956). 15. Plugge, W. R., and Perry, M . N., American Airlines’ “Sabre” electronic reservation system. Proc. Westerns Joint Computer Conf., Los Angeles, California. Pp. 593402, May (1961). 16. Porter, R. E., The RW400-a new polymorphic data system. Datamation pp. 8-14, January-February (1960). 17. Rochester, N., The computer and its peripheral equipment. Proc. Eastern Joint Computer Conf., Boston, Massachusetts. pp. 64-69, November (1955). 18. Teager, H. M., Time-sharing project. M.I.T. Computation Center Semi-Annual Report No. 6 January (1960). 19. Teager, H. M., and McCarthy, J., Time-shared program testing. 14th Nat. Meeting Assoc. Computing Machinery, Cambridge, Massuchuselis. pp. 12-1 and 12-2, September (1959).


Recent Developments in Nonlinear Programming PHILIP WOLFE Mathematics Division The R A N D Corporation S a n t a Monica. California

1. Introduction . . . . . . . . . . . . 1.1 The Motivation for Nonlinear Programming . . 1.2 The Mathematical Programming Problem . . 1.3 Some Definitions . . . . . . . . . . 2 . Differential Gradient Methods . . . . . . . 2.1 The General Approach . . . . . . . . 2.2 The Direct Differential Gradient Method . . . 2.3 The Lagrangian Differential Gradient Method . 3 Large-Step Gradient Methods . . . . . . . 3.1 The General Approach . . . . . . . . 3.2 The Simplex Method for Linear Programming . 3.3 The Simplex-Corrected Gradient Method . . . 3.4 Projected-gradient Procedures . . . . . . 4 . Simplicia1 Methods . . . . . . . . . . 4.1 The Roleof the Simplex Method . . . . . 4.2 The Simplex Method for Linear Programming . 4.3 The Simplex Method for Quadratic Programming 5. Columnar Procedures . . . . . . . . . . 5.1 Columnar Procedures in General . . . . . 5.2 Separable Programming . . . . . . . . 5.3 The Decomposition Procedure . . . . . . 6. The Cutting-plane Method . . . . . . . . 7 Initiating an Algorithm . . . . . . . . . 8 Computer Routines and Literature . . . . . . 8.1 Direct Differential Gradient Methods . . . . 8.2 Lagrangian Differential Gradient Methods . . 8.3 The Simplex-corrected Gradient Method . . . 8.4 Projected-gradient Procedures . . . . . . 8.5 Quadratic Programming . . . . . . . . 8.6 Separable Programming . . . . . . . . 8.7 The Decomposition Procedure . . . . . . 8.8 The Cutting-Plane Method . . . . . . . Bibliography . . . . . . . . . . . .

.

. .

155

.

.

.

.

.

.

.

.

.

.

.

.

. . .

. . .

. . .

. . .

. . .

. . .

.

.

.

.

.

.

. . . . . .

. . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

. .

. .

. .

. .

. .

. .

. . . . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

. . .

. . .

. . .

. . .

. . .

. . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. .

. .

. .

. .

. .

. .

. .

. .

.

.

.

.

.

.

. .

. .

. .

. .

. .

. .

.

.

.

.

.

.

. 156 . 156 . 157 . 159 . 161 . 161 . 162 . 163 . 165 . 165 . 166 . 168 . 170 . 172 . 172 . 172 . 174 . 175 . 175 . 177 . 178 . 180 . 183 . 184 . 185 . 185 . 185 . 185 . 186 . 186 . 186 . 186 . 186

156

PHILIP WOLFE

1. Introduction 1.1 The Motivation for Nonlinear Programming

While nonlinear programming has been a topic of discussion among people concerned with allocation problems about as long as linear programming has, not much was done about it until three or four years ago. The recent past has shown a great increase in the amount of attention devoted to this area, however, and some of the reasons for this new interest are readily ascertained. On the theoretical side, it would seem that a large part of the purely mathematical theory of linear programming is now well in hand [ I d ] . While, of course, the subject will never be exhausted for the mathematician, he perhaps no longer feels justified in expecting sweeping new results in that area; but when the hypothesis of linearity is dismissed, it takes away with it many of the known results and intuitions we have about these problems. The resulting gap in our knowledge, together with the great practical interest the resulting nonlinear problems have, offers the theoretician a continuing challenge. Likewise the subject of linear programming seems very well in hand on the computational side. While we may never have computer routines and algorithms capable of solving the largest problems we would like to formulate, we can now solve linear programming problems involving several thousand variables and about 500 equations, problems whose input data may have taken years to collect; and it is fairly clear what must be done to increase this capacity even further. For nonlinear programming, nothing of the sort can be said. Quite a few algorithms for various kinds of nonlinear problems have been proposed-these notes are devoted to describing the majority of them-but very few have ever been put into practice. Some procedures are demonstrably quite inefficient, while others will require much computer programming and experience to test. As will be seen below, there is no lack of information as to how nonlinear programming might be done, but on the other hand there is almost no information on how nonlinear programming should be done. Here, then, is a challenge t o the computational expert; one with which he is beginning to grapple seriously. Finally, the practical value of the solution of these problems is high. Almost no real problem is linear; linearity represents our compromise between reality and the limitations of our tools for dealing with it. The user who has only linear techniques for handling his mathematical programming problems is confronted with one of two expensive alternatives: He must accept the results of a linearization of his problem and bear the expense of their deviation from reality, or undertake the imprecize and laborious task of making the results better by heuristic methods. During

RECENT DEVELOPMENTS IN NONLINEAR PROGRAMMING

157

the last few years, the largest users of linear programming techniques have become impatient with these restrictions. They have trained themselves in the use and interpretation of these methods, and have set u p the elaborate data-collecting agencies which are required by their mathematical models. As soon as effective procedures in this area are devised, they will be usedwhich was not true when linear programming first appeared. As we have said, most problems are really nonlinear. While the petroleum industry, for example, which constitutes the major user of linear programming in the United States, has had extraordinary success with linear techniques, the parameters of interest in the refining and blending processes do not behave in precisely a linear manner. At least one firm is now using a nonlinear programming procedure in its daily production work, despite the priniitiveness of the particular method being used. One simple type of nonlinear problem has been fairly well in hand for about two years: Problems having linear constraints, but with a convex, quadratic cost function, can be solved with reasonable efficiency. An algorithm for this problem has been applied to the minimization of power losses in electrical distribution networks, to the cdculation of investment portfolios so as to minimize risk, and to the scheduling of investment and importation of scarce commodities. (The nonlinearity in this last problem arose from the nonlinearity of import cost as a function of previous purchases. It represents about the largest nonliiienr problem ever solved, having some 200 variables and 100 constraints.) The present paper attempts a survey of the majority of techniques which have been proposed for nonlinear programming problems. Disparate as they are, they may be grouped as we do under several broad headings in such a way that the techniques belonging to one group are similar in concept, arc aimed at a certain class of problem, and may have similar computational effectiveness. Our grouping of the methods presented here is given a t the end of this paper, where the References are cited in connection with that outline rather than in the text. 1.2 The Mathematical Programming Problem

The general mathematical programming problem can be posed this way: Given the objective formula f(z) = f(zl, . . . , 5 , ) and the m constraint functions g 2 ( x ) = g2(z1, . . . , xJ, determine x = (a,. . . , 2,) so as to

hIaxiinizc J(L) under the constraints g,(x)

50

for i

=

1, .

. . ,?)I.

158

PHILIP WOLFE

Naturally, the functions f and gi will have to be restricted in some way, as will be done below. It is important to note that (1.1) comprises all the constraints put upon x, which is otherwise allowed to be any collection of n real numbers. It would be a very serious matter to require besides, for example, that the components of x be integers; and while this kind of restriction is dealt with elsewhere for the case of linear programming, it is inappropriate to the bulk of our work. Incidentally, while the constraints zj 2 0 (for each j ) are usually imposed in a programming problem, there is no need especially to mention them here, since they are just special cases of the constraints (1.1). Figure 1 illustrates this problem in two dimensions (n = 2 ) . The shaded

FIG.1. Nonlinear programming problem:

2' = (31,22);m =

3.


159

area is the constraint set-the set of all points x satisfying the relations (1.1). Note that its boundaries are just the places where some constraint function vanishes. These boundaries, as well as the vertices of the constraint set (points, of which there are two in the figure, where so many gi vanish that a unique point is defined), usually play an important role in the problem. A contour of the function f is a set of points on which the function has a constant value: a number of these are sketched. Geometrically stated, the programming problem is that of finding the highest-valued contour having some point in common with the constraint set. 1.3 S o m e Definitions

The gradient at x, denoted by Of (x), has been drawn for a point in the constraint set. Defined by

Vf(x)

=

(F, . . . , - '"'>' ax,

i t is always perpendicular to that countour off which passes through x, and points in the direction of steepest ascent-that is, the direction of maximum rate of change off per unit distance. The gradient plays an essential role, although sometimes a well-concealed one, in most mathematical programming procedures, for it is through the gradient that linear approximations are constructed to nonlinear functions, making them numerically tractable. Specifically, when all the derivatives of f (x) exist, so that Of (2)can be computed, and are continuous, then The equation of the plane tangent to the graph of z point (zo, x") is 2 = 20 Of ($0) . .( - x")

+

= f(x)

a t the (1.2)

(see Fig. 2). Since this tangent plane is the best linear approximation to f in the neighborhood of xo, the formula (1.2) will serve to replace (f(x) where linear computations are to be done. One more assumption will be made about the function f : it is to be concave, which means that the graph of t.he function is to lie below any of its tangent planes (as in Fig. 2). Part of the importance of this requirement lies in the fact that any linear approximation to f by functions like (1.2) will then be one-sided-always give values a t least as high as those sought -so that they place bounds on the possible answers. Another consequence of concavity is the fact that for any number K , the set of all x such that f (2)2 K is a convex set-a set which contains the entire line-segment joining any two of its points (see Fig. 1 for a two-dimensional example). The functions g; will all be assumed to be convex. A convex function is the

PHILIP WOLFE

160

/

Z

FIG.2. Concave function of two v:irinhlcs.

negative of a concave function; its graph lies above any of its tangent planes (see Fig. 3). For any K , the set of all x: such that gl(x:) 5 K is convex. I n particular, each of the inequalities of (1.1) defines a convex set, and thus the constraint Pet itself, being the common part of several convex sets, is convex. The assumptions about f and the g? together give the programming problem a n important property ; any local solution is the proper solution of the problem. A local sclution is a point xo satisfying the constraints such that no constrained point can be found in the immediate neighborhood of x o which gives a higher value off. (The point x = a of Fig. 3 is an example of such a local maximum point which would not be the solution of the maximization problem; there thc objective is convex instead of concave.) If x:l is the solution of the programming problem under the assumptions above on f and gz, then for any other point 2" in the constraint set, all points on the segment will have values of f a t least as high as f (z"), so that zocannot be a local solution without being the proper solution as well. While this "local implies global" propcrty is the most important use of


x=a

XO

161

x=b

FIG.3. Convex fiinction of one variable.

convexity in nonlinear programming, and for the simpler procedures its only use, it plays a more subtle, yet vital, role in some of the procedures to be discussed. 2. Differential Gradient Methods 2.1 The General Approach

The contours of the problem of Fig. 1 are commonly thought of as the geodetic contours of a “hill” up which we are trying to move the point 2, as constrained by the “fences” gz(z) 5 0. The ordinary man has no difficulty moving up such a hill, knowing as he does the direction-Vf(x)--of steepest ascent a t the point a t which he is standing. He takes one step a t a time, as quickly as possible, so as always to increase his altitude; when he is constrained by a fence, he moves along it, so long as he keeps going up; and he finajly stops when no step he can take will move him higher. It is simplest to describe his motion in terms of infinitesimal steps dx taken during the elapse of a n infinitesimal time dt, or in terms of his (vector) velocity dxldt. I n the most primitive kind of steepest ascent his motion could be described just by

dx - = Vf(z),

dt

(2.1)

PHILIP WOLFE

162

that is, by

but, of course, this equation takes no account of the constraints of the problem. Unless we are fortunate enough to have a solution of the problem interior to the constraint set, the point x will eventually wander outside it. The two “differential gradient” methods presented here-the “direct” and the “Lagrangian”-adopt different methods for enforcing the problem’s constraints. 2.2 The Direct Differential Gradient Method

If a constraint gi(x) 5 0 is violated, that is so because gi(x) is too big. This function can be reduced by motion in the direction -Vg;(x), just as (2.1) increases f ; and all this can be done at the same time by differential equations dX

- = Vj(x) -

dt

(2.2)

where

K

if gi(x) 5 0 if gi(x) > 0. I n (2.3) the number K is chosen sufficiently large to keep x from leaving the constraint set-for example, larger than the maximum of all lVf(x)l/lVgi(x)[for any x on the boundary. The terms of C& (x)V gi (x) serve to “kick back” x when it tends to leave the constraint set. Of course, the differential equations (2.2) must not be taken too seriously from a n analytic point of view. They will not, in general, have solutions, owing to the discontinuity of the terms &(x). The equations are to be taken rather as a guide t o the formulation of a computational procedure for this problem. I n digital computations, dx and dt would be replaced by finite intervals A x and At; the interval At would be chosen (perhaps) small, and Eq. ( 2 . 2 ) , with dxldt replaced by Ax/At, used to obtain the new point x Ax to replace z in these formulas. Figure 4 sketches the course of x under this procedure for small, nearly uniform steps. Evidently there is a great deal of room for experimentation in the matter of proper step size, decrease of step size as the solution is approached, and in a number of other technical matters. This procedure certainly has the most immediate, intuitive appeal of all those discussed here. While for convex problems it will, of course, find their proper solution, it will also find a local solution for just about any problem, a property that not all the methods share. On the other hand, we suspect

&(x)

+

=


163

FIG.4. Direct differential gradient method.

that it is likely to be far less efficient for convex problems than some of the methods below, which may require no more computation to take one very large step toward the solution of the problem than this procedure does to take one of its small steps. 2.3 The Lagrangian Differential Gradient Method

I n this type of procedure the discontinuities of the corrections (2.3) applied to the steepest ascent path are avoided by taking a more relaxed approach to the satisfaction of the constraints. The process is governed by the differential equations

PHILIP WOLFE

164

dx - = Vf(Z) dt

and

- CiuiVgi(.)

if u;> 0 otherwise.

dt

(2.4)

or gi(r)

> 0,

(2.5)

n'ow the right-hand sides of the differential equations are continuous if the derivatives off and g; are, and the equations will possess solutions, although it will still be impractical to try to give them in closed form-the computation must be done pretty much as before. If the point 2: violates constraint i, the number ui will tend to rise, by equation (2.5); and whenever ui is positive, equations (2.4) tend to nudge it back across the boundary of g;(x) I 0. Note that (2.5) prevents ui from becoming negative: there is no penalty attached to having gi(x) < 0. It can be shown that under suitable conditions the above differential equations have solutions xj(t), ui(t) which converge to values (xIo, . . . , zn0,U I " , . . . , urno)= (xo,u0) as t -+ 00. It can then be argued that x 0 solves the programming problem, as follows. Evidently at the point (20, uo)we must have

Supposing that the differential equations were started with all u, 2 0, by (2.5) no u,will ever go negative, so that uto2 0 (all i).

(2.6) Equation (2.5) also make it impossible that u,O > 0 and gt(xO)> 0 a t the same time. so that u,Oy,(z") = 0 (alli). (2.7) Equation (2.4)yields

(2.8) In order to show that xu solves the programming problem, suppose that some other x satisfying the constraints of the problem is given; we will show that f ( r ) 5 f (xo). The equation of thc tangent plane to the graph off a t the point (xo,f(zo)) is 2 = j-(xO) Vf(x"(x - x0) V.f(z0) = xtu;vgt(zo).

+

[see Eq. (1.!2)]. Since f is concave, its graph lies below this plane, so that for any x we have

+ 2 g4xO) + Vgdx0)(x -

f(x) L f(z0) Vf(xO)(z - 2". For the convex functions gl., the reverse is true: we have gd.>

x0)


165

for any x and i. These two relations, together with (2.8), give

+ Vf(xO)(l: x*) = fbO) + ~zu,ovg,(xo)(x- zO) < jbO) + C%u*“g%(x) = fbO) + - c~u,ogz(xo).

S(z) 6 j(xO)

-

gE(Z0)l

Czuz0g2(Z)

Since now each g2(x) 5 0, and each u,Og,(zo)

=

0, we have at last

f(.> I f b ” . The Lagrangian differential procedure has the odd property that the point x may wander in and out of the constraint set during the computation; the tcrms uI which eventually force it in do not get large until x has been outside for some time. The final values of the multipliers are, however, of great interest, being the ‘(shadow prices” associated with the constraints of the problem, or the “marginal costs” of the resources whose limited availabilities are asserted by the constraints. They are generally found as by-products in the various solution techniques for these problems, and not introduced explicitly into them as in this procedure. The name “Lagrangian” is given to this method because the u,actually play the role of Lagrange multipliers in the computation. Solving the differential equations has been an elaborate way of solving, for the Lagrangian function

).(S

- CLWd.),

the relations O , [ f ( z > - c&gL(z)l = 0.

Owing to the peculiar nature of the constraints g z ( x ) 5 0, being inequalities instead of equations as in the classical kind of constrained-extremum problem, the multipliers have unusual requirements set upon them : effectively, they are required to satisfy (2.6) and (2.7) above. The formulation of mathematical programming problems in terms of such Lagrangians has been extensively developed, and forms a fruitful and unifying point of view for milch of mathematical theory of these problems.

3. large-Step Gradient Methods 3.1 The General Approach

A “large-step gradient” method is conceived as having the same basic motivation as a differential gradient method; an effort is made to move in the direction of the gradient of the objective function. These methods concentrate, however, on taking as large a step -Ax -in that direction as

166

PHILIP WOLFE

possible, the length of the step being limited only by the constraint boundaries, utter misbehavior of the objective function, or some limitation inherent in the procedure other than the “small step” stipulation of the differential methods. It will be noted that this class is not entirely well-distinguished from the previous class nor those which will be taken up later, the “simplicial” procedures. On the one hand, a differential procedure could certainly be performed with large steps in its initial phases, but ultimately a small-step requirement would have to be imposed to assure convergence, and in any case its underlying reasoning is differential. On the other hand, one of the procedures of this section is certainly simplicial, if anything is-the simplex method for linear programming; nevertheless, we shall exhibit it as a sort of gradient method. (It will also be used within one of the two other procedures described here.) Both of the procedures given here are aimed a t problems having a nonlinear objective function but linear constraints. The ability to take a large step in the direction of the gradient of t.he objective seems to require linear constraints. If the same methods are pushed to cases of nonlinear constraints, it appears that they would spend so much effort dealing with those constraints that they would perform in about the same way that differential gradient methods do. 3.2 The Simplex Method for Linear Programming Figure 5 gives a geometric visualization of the linear programming problem. This one has seven constraints: three of them have the form -zj 5 0 and the other four are given by more complicated linear functions of z. Since the constraints are all linear, the faces of the constraint set are planar. The ith face (i = 1, . . . , 7) is the set of all z for which gi(z) = 0; they have been so labeled in the diagram. The vertices of the constraint set are identified by listing the faces which meet in them: for example, the vertex Pz is identified as lying on the planes gl(z) = gb(z) = gT(s) = 0. The simplex method operates this way: let f(s) = Cjcjx,;

thenvjfx)

=

c,

so Vf(z) is constant. Evidently the solution of the problem will be a point of the constraint set located as far as possible in the direction of the vector c. Take any vertex of the constraint set; the direction numbers of all the edges leading out of the point may be calculated, so that those edges making an acute angle with c can be determined. Following such an edge from the current vertex will yield another having a higher value off. The process is repeated until a vertex is reached none of whose edges make an acute angle with c ; that vertex is the solution of the problem.


167

)(3

FIG.5 . Linear progrunlming : the Simples method.

This is sketched in Fig. 5. Beginning at 0, we find the path 01’1P?PZ around the constraint set terminating in a solution. The simplex method constitutes an efficient means for performing the calculations that are necessary for this process, While we shall not discuss it in detail, let us see how it proceeds from the vertex 0 of Fig. 5 to the next vertex. At 0, all the relations of the problem can be described in terms of the three independent variables xl, x2, x3.Moving from 0 to another vertex will be accomplished by increasing one of these variables until a constraint is met. It. is reasonable to choose that variable which will make the greatest increase in the objective: that will be the one for which c j ( j = 1, 2 , 3) is the largest. (In the figure, it is xz.)Retaining z1 = x3 = 0, we move to PI. At PI, it will be again possible to express all the relations of the problem in terms of three independent variables and try again. This time they will be chosen as xl, z3,and, say, x5 = -g5(x), the last choice made so that they will all be zero at P1 and nonnegative inside the constraint set; and the objective function may be re-expressed in terms of them, say as dlxl d3x3 &x5. The cycle may then be repeated. In every case, the simplex method concentrates on one variable in the problem, increasing it when

+

+

PHILIP WOLFE

168

profitable, and altering the others so as to remain on the edges and vertices of the constraint set. It was mentioned above that there is a sense in which the simplex method is a gradient method. Generally speaking, a gradient vector of a function may be defined as a direction in which the directional derivative of the function is maximal: more precisely, as a vector Ax = (Axl, . . . , Ax%)maximizing

and such that \IA.c\l = 1,

where [ IAx[1 denotes the length of Ax. When we take = (Ari)'

4-. . .

+ (AzJ2,

it is easy to show that the vector which does this is indeed proportional to . . . , af/ax,), which is therefore satisfactory to use. The above definition of llAsll is not, however, the only one that can be used: many other notions of "length" are available, and perhaps equally plausible. One of the more iiiteresting would be

Gf(z) = (af/ar,,

IjAzlI

=

IAzII

+ . . . + lAxnl.

If this coiiveiition is used, the maximization problem posed again, a new answer is found: its components must all be zero excepting the jth, where a.f/az, is the maximum of all the partial derivatives, and Axj = sgn (af/ax,). Kow this is precisely the rule of progress of the simplex method, since af/az, = c,; we try only to increase x,. Thus, with this more liberal interpretation of the concept of the gradient, the simplex method is a gradient method. Evidently the bulk of calculation in this method is connected with expressing the data of the problem in terms of a selected set of independent variables, and changing this mode of expression from step to step. Fortunately, because of the linearity of all the relations, this work is of the same kind as is done in handling linear equations; and since the set of independent variables changes in only one member between steps, the work turns out to be very simple. 3.3 The Simplex-Corrected Gradient Method

Figure 6 poses the problem of maximizing a nonlinear objective function under linear constraints; the latter are drawn like those of Fig. 5. The point P of the figure is located outside the constraint set, and the objective function is taken to be the negative of the square of the distance from the point

RECENT DEVELOPMENTS

IN NONLINEAR PROGRAMMING

169

a-P

'X'

Fro. 6. Simplex-corrected gradient method.

x to P. The problem is then that of finding the nearest point to P within the constraint set. The objective is evidently concave, as desired. The simplex-corrected gradient procedure will yield the sequence 20, 21, 2 2 , . . . of points of the constraint set which will converge to the solution of the problem. There will also be formed an auxiliary sequence xo,XI, 22, . . . of vertices of the constraint set which are used in the process; these are produced through use of the simplex method in a certain way. Initially, let xo be any vertex of the constraint set, and set zo = xo. Now suppose that k steps of the procedure have been taken. There will be a t hand a point zk belonging to the const,raint set and a vertex xk of the constraint set. Then Calculate Vf (zk). Using Of(2)as a (temporarily) constant objective vector and xk as an initial extreme point, take one step of the simplex method in the maximization of the linear function Of(zk) obtaining a new extreme point; call this extreme point &+l. Choose the point zk+1 on the segment joining zk to xk+l so as to minimize f on that segment. This sequence of steps defines a recursive process for generating the sequence zol zl, . . . It is assumed that the one-dimensional maximization problem of step (3) is not too difficult to perform. The special simplex method application e x ,

170

PHILIP WOLFE

of steps (1) and ( 2 ) is not difficult; suitably handled, the work of step ( 2 ) is not a great deal more than that involved in performing one step of the simplex method for a problem having permanently fixed costs. It should be mentioned that, in case the constraint set is not bounded, it is possible that the step (2) may result in the taking of an “unbounded” simplex step. Step (3) need only be slightly altered to incorporate this possibility. An estimate of the speed of convergence of the procedure is given by the following result: letting M be the maximum value of f as constrained, there exists a constant K such that

Unfortunately, the procedure is unlikely to terminate in the exact solution of the problem, although it does in the case of a quadratic objective, where other exact methods also exist. On the other hand, this will be no objection if it arrives in the near neighborhood of the solution with sufficient speed. 3.4 Projected-gradient Procedures

Methods of the “projected gradient’’ kind can be viewed as resulting from the attempt to make a differential gradient method take as large steps as possible, while never allowing the point x to leave the constraint set. Figure 7 illustrates such a procedure, beginning a t t,he point zoand generat-

FIG.7. Projected-gradient method.


171

ing the sequence of points x’, x2, . . . . Below, by “plane” is meant the entirety of a single hyperplane of the form gt(x) = 0, whose intersection with the constraint set yields, in general, one of its faces. Starting with the point xk in the constraint set, either one or two successors of xk are determined by the following steps. A particular set of planes is associated with x k a t all times; initially, let this be the set of all planes which pass through x k . (1) Calculate Vf(xk). ( 2 ) Find the projection of Vf(&) onto the intersection of all the planes associated with xk. (In case there are no planes-as when x k is interior t o the constraint set, this intersection is the whole space, and thus the projection is V f ( z k ) itself.) (3) If the projection is different from zero, extend a ray from xk in that direction, and define xk+’ to be the farthest point along the ray belonging to the constraint set. (a) If f(xk++’)> f(xk), then the cycle is complete. (b) Otherwise, choose xk+2 so as to maximize the function f on the segment xkxk+l;this completes the cycle. (4) If the projection is equal to zero, then Vf(xk) may be written = XkrkAk

as a linear combination of normals Ak to the planes associated with xk (the A k are chosen to point away from the constraint set). (a) If all rk 2 0, then xk is the solution of the problem. (b) Otherwise, define a new set of planes to be associated with xk b y deleting from the present set some one plane for which T k < 0, and return to step ( 2 ) . As with the gradient-corrected simplex method, it is assumed here that the one-dimensional maximization problem which may have to be solved in step (3b) is not difficult to do, which is the case. I n Figure 7, the points x4 and x6 have been obtained as the result of minimizing on the segments x2x3and a t these minima Vf is, of course, perpendicular to the segment in question. Convergence of the procedure to a solution of the nonlinear problem is not difficult t o show. Unlike the previous gradient methods, this procedure does not completely reduce to the simplex method for a linear problem; but it does so reduce if the points x k are vertices of the constraint set. I n the same way that the simplex method may be looked on as a gradient method, a procedure very closely related to the above, which need not actually use Vf(xk) in the calculation, can be given. The projection of Vj(xk) onto the face in which x k lies turns out to be precisely the direction of steepest ascent for the function f per unit distance in the Euclidean

z;

PHILIP WOLFE

172

metric, if that direction is chosen so as to keep one in the constraint set. If, on the other hand, some other metric were used-as in the discussion of the gradient aspect of the simplex method above-a somewhat different algorithm would be obtained. The metric !/Ax/\= Max {\Axl! , . . . , \Ax,/), for example, changes the work of step (2) from that of finding the projection of V.f(xk)onto the face of xk to that of determining a point y maximizing Vf(xk) - y under the linear constraints of the original problem augmented by the constraints \xik - yjl 5 1; the direction of motion away from xk is then that of the ray from xk through y. (The first reference cited under the heading of projected-gradient methods employs the projection; the second reference employs this latter procedure.) 4. Simplicia1 Methods

4.1 The Role of the Simplex Method

I n the current practice of the art of nonlinear programming, those computational methods which are not essentially gradient methods use the simplex method for linear programming to do their basic computational work. As will be seen below, there are three ways in which this happens. The first way we have called “simplicial”: it is the solution of a nonlinear problem-specifically, the quadratic programming problem-by what is almost exactly the simplex method itself. Once the problem has been set up, no additional data need be generated. This is not the case with the other two types of method; they depend in an essential way on the process of linearization, and make use of the linear programming process to solve linear problems derived from the nonlinear problems which give approximate answers to the real problems, using the results thus obtained to further improve the approximation. They are essentially infinite methods. I n the sense in which the term is used here, there is only one “simplicial” procedure current, that for quadratic programming. 4.2 The Simplex Method for linear Programming

The simplex method for linear programming problems was described briefly in Section 3.2. We return to it here only to permit the development of those features which are used in the nonlinear programming algorithms described later. The usual statement of the linear programming problem is: c,yj subject to Maximize C;’=l

yj

>, O (allj),

Cja,,yj

3 b , (alli).

(4.1)

RECENT DEVELOPMENTS IN NONLINEAR PROGRAMMING 173 The Lagrange multipliers, or “dual variables” u L(i = 1, . . . , m) described in Section 2.3, play an important role in the forms of the simplex method used in calculation. The duality theorem for linear programming states that

y1, UI,

. . . , gm solve the problem . . . , u, > 0 such that

(4.1) if and only if there exist

C1.ular, 3 c, (allj), and

ct u,b, = c,cjy,.

The main features of the so-called primal simplex method caii be de. . . , u, and some subset J of all m i be d in these terms. Suppose that u,, the indices 1, . . . , n are known for which y, = 0 unless j € J . (It is not difficult to satisfy these starting conditions, if they do not obtain, by emFedding the given problem in a certain larger one, as described in Section 7.) If now it happens that inequality (4.2) holds for all o t h e r j as well, then the problcm is solved-by the duality theorem. If, however, it is not, then it is possible to select some j for which

C r u t a 2, c,

< 0,

(5.6)

so that x satisfies its constraints. How closely j ( x ) approximates the maxi-

mum obtained in the linear problem is determined by the fineness of the grid in general. Rut it does follow from the concavity o f f that so that x gives at least as high a value of the objective function as is iiidicated by the solution of the linear problem.


177

The principal technical problem in using a columnar method is that of dealing with a sufficiently fine grid without having to handle enormous quantities of data. In the sequel, two different approaches to the solution of this problem will be studied. 5.2 Separable Programming

Separable programming is the application of the grid procedure described in Section 5.1 to a problem in which each nonlinear function is separable, that is, may be written as the sum of separate functions of the components x, of the point x:

f(4 = c;=1.f,(.&),

(5.8) (all 2 ) . gdx) = C?=lgt,(x,) In this application, the linearization technique is applied separately to each variable x,. Suppose that for each j a sequence of values x,~,. . . , x , ~ has been chosen (we suppose the same number T chosen for each j ) . Write T, =

Ct A,th(.x,t),

(5.9)

and use the approximatioil h(xj)

=

Ct AJth(xjf)

(5.10)

for any function h of x, alone. When this is done for each variable, the resulting linear programming problem, derived from Eqs. (5.3) and (5.4), is Maximize

C3C tA , " f q t )

subject to

3

C t A,t

(allj, t ) ,

0

and

0. Define the temporary objective function Px(x) = -CrEIcx)Ggi(x) ;

(7.3)

note that F,(x) = F ( x ) . Employing now any algorithm which is applicable, proceed with the solution of the problem fifnximize gL(x) < 0

Fx(.c) subject to (all i not in I ( x ) )

(7.4)

until a point 2 is found satisfying the constraints of (7.4) for which g&) Q 0 for some i in I ( x ) . Now redefine x to be this new x, and return to the problem (7.4). I n effect, this process works toward satisfying all the constraints at once, retaining each for good as soon as it is satisfied. It is effective especially in the case of linear programming, where it is the standard procedure for finding a starting solution; the fact that Fx is linear makes it work very well. I n problems for which the objective function and the constraints are not of the same type, it may be well to use one sort of procedure for obtaining a starting point and another for subsequent maximization. This is particularly true if the constraints are linear and no “nearly feasible” point is a t hand, in which case the simplex met,hod Phase One seems to be by far the most efficient.

8. Computer Routines and literature

This section will present literature citations for the nonlinear programming methods which have been studied and will describe what computational experience has been made available regarding them. There is no single reference which covers the subject of mathematical programming in the broad sense in which it is used here. The existing textbooks in the field deal almost exclusively with linear programming. Of these, that of Gass [I21 has the widest modern coverage. It is brought up to date in Gass [I,%]. I n contrast to the state of affairs iii linear programming, almost nothing is clearly known regarding the relative computational efficiency of various techniques for nonlinear programming. Each method discussed here has been programmed in some form for a large-scale computer; but only one computer routine has been made publicly available; and most of the known routines have iiot been fashioned for general-purpose usage or even properly documented. There seem also to have been no comparative experi-


185

ments run under similar conditions. What follows will thus serve only as an indication of the level of activity and not as a guide to the user with a problem to solve. 8.1 Direct Differential Gradient Methods

There has been a certain amount of activity in this area relating to digital computers for some ten years now. The best comprehensive reference up to 1957 is the monograph of Brown [4],which surveys a number of variants of the basic method. Proposed applications for digital computers since then have been given by Fiacco et al. [9],Carroll [5],and Rosen [f 81. This type of procedure lends itself very nicely to analog computation, a fact which has been exploited by Ablow and Brigham [f], Pyne [ l 6 ] ,and DeLand [7]. There seems to have been a considerable amount of experimentation with direct differential gradient methods, but no reports have been published. One gets the impression that these methods may have worked welt for some particular small problems, but that they have not succeeded in a general way for large problems. 8.2 Lagrangian Differential Gradient Methods

The earliest experience with these procedures seems to be that of Manne [l4].Since then, most of the work in the area is due to Arrow et al. whose collection of papers [2] contains most of what has been done. Manne reports mildly encouraging experience with this sort of method as programmed for the IBM CPC Model 2 (1953). Thomas Marschak [See 21 reports on some experiments with another version programmed for the JOHSNIACcomputer (whose speed is approximately 1/75th that of the IBM 704). Approximately thirty minutes were required for the solution of a five-equation, fourteen-variable linear programming problem. 8.3 The Simplex-corrected Gradient Method

This procedure is described by Frank and Wolfe [ l o ] .No computational experience with it has been reported. 8.4 Projected-gradient Procedures

Several closely-related procedures of this kind have been proposed. One is given in considerable detail by Rosen [17],and another by Zoutendijk [24].Some of the work of Frisch [I11 is based upon this approach. What seems to be a general-purpose routine embodying this procedure for problems having nonlinear objectives and linear constraints has been programmed by Rosen and R. P. Merrill for the IBM 704 and 7090, and some results using this program have been reported [f 71. The routine is not presently available to others.

186

PHILIP WOLFE

It is believed that one of the Frisch procedures has been programmed for a large-scale computer, but no results have appeared. An excellent detailed survey of these procedures has been written by Witzgall [ZO]. His presentation, more precise than some of those above, exhibits them as small variants of one basic scheme. 8.5 Quadratic Programming

The algorithm described above for the quadratic programming problem (convex quadratic objective, linear constraints) is that of Wolfe [2.2].It has been embodied in the SHARE704-7090 routine RSQP1, which will handle problems for which the sum of the numbers of variables and constraints does not exceed 250. Some computational experience is described in ref. [ZZ]. Another procedure is given by Beale [S], which has recently been programmed for the Ferranti Mercury by D. G. Prinz. 8.6 Separable Programming

The basic separable programming procedure, described for computer implementation for problems having quite general objectives and constraints, is Miller’s [15]. This procedure is in regular use on the IBM 7090 by the Standard Oil Company of California. The routine is not being made available at present. 8.7 The Decomposition Procedure

The decomposition procedure is described for the case of linear coilstraints by Dantzig [S] and for the general case by Wolfe [ZI]. An experimental routine was written by Shapiro [I91 for the IBM 704 embodying this procedure for the particular case of the “chemical equilibrium” problem. 8.8 The Cutting-plane Method

The basic cutting-plane method is given by Kelley [IS], and some comments on it by Wolfe [ZS]. Some computational experience has been obtained with it by Dornheim [8] and by Griffith and Stewart [ I 2 b ] , but no detailed report of this experience has been given, and computer routines embodying the method are not available. Bibliography 1. hblow, C. M., and Brigham, G., An analog solution of programming problems. Operations Research 3 , No. 4, 388-394 (1955). 2. Arrow, K. J., Hurwicz, L., and Uzawa, H., Studies in linear and non-linear programming. I n Stanford Mathematical Studies in the Social Sciences, Vol. 11. Stanford Univ. Press, Stanford, California, 1958. 3. Beale, E. M. L., On quadratic programming Naval Research Logist. Quart. 6, 227-243 (1959).


187

4. Brown, R. R., Gradient methods for the computer solution of system optimization problems. MIT Dept. Electrical Engineering, WADC Tech. Note No. 57-159, September (1957). 5. Carroll, C. W., The created response surface technique for optimizing nonlinear restrained systems. Operations Research 9, No. 2, 169-184 (1961). 6. Dantzig, G. B., General convex objective forms. Rand Corp. Papcr No. P-1664, April (1959). 7. DeLand, E. C., Continuous programming methods on an analog computer. Rand Corp. Paper No. P-1815, September (1959). 8. Dornheim, F. R., Optimization subject to nonlinear constraints using the SIMPLEX method and its application to gasoline blending (Sinclair Research Laboratories, Harvey, Illinois). Presented at Optimization Techniques Symposium, New York University, May (1960). 9. Fiacco, A. V., Smith, N. M., and Blackwell, D., A more general method for nonlinear programming. Presented at Seventeenth National Meeting of the Operations Research Society of America, New York, May (1960). 10. Frank, M., and Wolfe, P., An algorithm for quadratic programming. Naoal Research IJogi~t.Quart. 3, NO. 1-2, 95-110 (1956). 11. Frisch, R., The multiplex method for linear programming. Memorandum Univ. SocialokGn. Institut, Oslo, October (1955). 12. Gass, S. I., Linear Programming. McGraw-Hill, New York, 1958. 12a. Gms, S. I., Recent developments in linear programming. Advances in Computers 9,295-377 (1961). 12b. Griffith, R. E., and Stewart, R. A., A nonlinear programming technique for the optimization of continuous processing systems. Management Science 7 , No. 4, 379392 (1961). 13. Kelley, J. E., Jr., The cutting-plane method for solving convex programs. J. SOC. Ind. and Appl. Math. 8 , No. 4, 703 -712 (1960). 14. Manne, A. S., Concave programming for gasoline blends. Rand Corp. Paper No. P-383, April (1953). method for local separable programming. Report of the 15. Miller, C. E., The SIMPLEX Electronic Computer Center, Standard Oil Company of California, San Francisco, August (1960). 16. Pyne, I. B., Linear programming on an electronic analogue computer. AIEE Trans. Ann., Part I pp. 139-143 (1956). 17. Rosen, J. B., The gradient projection method for nonlinear programming. I . Linear constraints. J. Soc. Znd. and Appl. Math. 8, No. 1, 181-217 (1960). 18. Rosen, J. B., The gradient projection method for nonlinear programming. 11. Nonlinear constraints. Shell Development Company, Emeryville, California, 1961. 19. Shapiro, M., and Dantzig, G. B., Solving the chemical equilibrium problem using the decomposition principle. Rand Corp. Paper No. P-2056, August (1960). 20. Witzgall, C., Gradient-projection methods for linear programming. Princeton Univ. and IBM Corp. Report No. 2, August (1960). 21. Wolfe, P., The generalized SIMPLEX method. Rand Corp. Paper No. P-1818, May (1959). 22. Wolfe, P., The simplex method for quadratic programming. Econometrica 27, No. 3, 382-398 (1959). 23. Wolfe, P., Accelerating the cutting-plane method for nonlinear programming. Rand Corp. Paper No. P-2010, June (1960). 24. Zoutendijk, G., Methods of Feasible Directions. Elsevier, Amsterdam, 1960.


Alternating Direction Implicit Methods* GARRETT BIRKHOFF. RICHARD S. VARGA. AND DAVID YOUNG Deparfmenf of Mathematics. Harvord University. Cambridge. Mossachusetts; Compufing Center. Case Institute of Technology. Cleveland. Ohio; a n d Computation Center. University of Texas. Austin. Texas

.

1 General Remarks . . 2 . The Matrix Problem . 3. Basic AD1 Operators .

Introduction

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. 190

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. 191 . 192

Part I: Stationary AD1 Methods (Case m 4 . Error Reduction Matrix . . . . . . . . . . 5 NormReduction . . . . . . . . . . . . 6 . Application . . . . . . . . . . . . . 7 . Optimum Parameters . . . . . . . . . . 8. T h e F u n c t i o n F . . . . . . . . . . . . 9 . Helmholtz Equation in a Rectangle . . . . . . 10. Monotonicity Principle . . . . . . . . . . . 11. Crude Upper Bound . . . . . . . . . . . 12. Eigenvalues of H , V . . . . . . . . . . .

.

= 1)

. . . . . . 194 . . . . . . 195 . . . . . . . . . . . . . .

.

.

.

.

.

.

.

. . . . . . . . . . . . .

. . . . .

. .

.

Part 11: Commutative Case 13. Introduction . . . . . . . . . . . . . . . . . . . 14 Problems Leading t o Commutative Matrices . . . . . . . . . . 15. The Peaceman-Rachford Method . . . . . . . . . . . . . 16. Methods for Selecting Iteration Parameters for the Peaceman-Raehford Method . . . . . . . . . . . . . . . . . . . . 17 The Douglas-Rachford Method . . . . . . . . . . . . . . 18. Applications to the Helmholtz Equation . . . . . . . . . . .

196 198 199 200 202 203 204

.

205 206 210

.

211 217 222

Part 111: Comparison with Successive Overrelaxation 19. The Point SOR Method . . . . . . . . . . . 20 . Helmholtz Equation in a Square . . . . . . . . 21 . Block and Multiline SOR Variants . . . . . . . . 22. Analogies of AD1 with SOR . . . . . . . . . .

Variants . . . . . 224 . . . . . 225 . . . . . 227 . . . . . 229

* Work supported in part by the Office of Naval Research, under Contract Nonr-1866 (34) *

189

190

23. 24. 25. 26.

GARRETT BIRKHOFF, RICHARD S. VARGA, AND DAVID YOUNG

Part IV: Numerical Experiments Introduction . . . . . . . . . . . . . . . . . Experiments with the Dirichlet Problem . . . . . . . Analysis of Results . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . .

Appendix A: The Minimax Problem for One Parameter. . . . . . Appendix B: The Minimax Problem for m > 1 Parameters. . . . . Appendix C: Nonuniform Mesh Spacings and Mixed Boundary Conditions Appendix D: Necessary Conditions for Commutativity . . . . . . Bibliography

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. 231 . 232 .

242

. 249 . . . .

254 259 263 266

. 271

INTRODUCTION

1. General Remarks

Alternating direction implicit methods, or ADI methods as they are called for short, constitute powerful techniques for solving elliptic and parabolic partial difference equations. However, in contrast with systematic overrelaxation methods, their effectiveness is hard to explain rigorously with any generality. Indeed, to provide a rational explanation for their effectiveness must be regarded as a major unsolved problem of linear numerical analysis. The present article attempts to survey the current status of this problem, as regards elliptic partial difference equation in the plane. It is divided into four chapters and four appendices. Part I deals with AD1 methods which iterate a single cycle of alternating directions. I n this case, the theory of convergence is reasonably satisfactory. Part I1 studies the rate of convergence of AD1 methods using m > 1 iteration parameters, in the special case that the basic linear operators H , V , Z in question are all permutable. I n this case, the theory of convergence and of the selection of good iteration parameters is now also satisfactory. Part I11 surveys what is known about the comparative effectiveness of AD1 methods and methods of systematic overrelaxation, from a theoretical standpoint. Part IV analyzes the results of some systematic numerical experiments which were performed to test comparative convergence rates of different methods. The four appendices deal with various technical questions and generalizations. No attempt has been made to survey practical applications of AD1 methods to industrial problems.

ALTERNATING DIRECTION IMPLICIT METHODS

191

2. The Matrix Problem

Consider the self-adjoint partial differential equation

where the function G is nonnegative, while A and C are positive. Let the solution of Eq. (2.1) be sought in the interior of a bounded plane region (R which assumes given values u = u ( x , y) on the boundary CQ of (R. To h d an approximate solution to the preceding Dirichlet boundary value problem, one commonly [8, Section 201 fist covers (R with a square or rectangular mesh having mesh-lengths h, k , approximating the boundary by nearby mesh-points a t which u is approximately known. One then takes the values u(xi, yj) of u on the set (R(h, k ) of interior mesh-points as un, one approximates -hk a[Aau/as]/az by H and -hk knowns. On ( ~ ( hk), a[Cau/ay]/ay by V , where H and V are difference operators of the form

H u h Y)

= -45,

v)u(z

+ h, Y) + 26(x, ?/Y)u(x,Y> - 4 z , Y>u(z- h, Y)

Vu(z,Y)

=

- a h ?J)ub, Y

+ h) + W(5,Y)Ub, Y>

- Y(Z, Y>U(S, Y The most common’ choices for a, b, c, a,P, y are

+

- Ic).

+ +

(2.2) (2.3)

26 = a c, (2.4) G = kA(z - h/2, y)/h, kA(z h/2, y)/h, 26 = (Y 7. (2.5) a: = hC(z, Y k/2)/k, y = hC(z, Y - k/2)/k, These choices make H and V symmetric matrices, acting on the vector space of functions u = u(xi, yj) with domain @(h,k ) . We will normally consider only the case h = k of a square network; general networks will be treated in Appendix C. For any h > 0, the preceding “discretization” defines a n approximate solution of the given Dirichlet boundary value problem for (2.1), as the algebraic solution of a vector equation (system of linear algebraic equations) of the form (H V Z)U = k. (2.6) I n (2.6), Z is the nonnegative diagonal matrix whose lth diagonal entry, associated with the interior mesh-point XI = (xi, yj), is h2G(xi, yj). The vector k is computed by adding to the source terms h2S(xi,yj) the terms in (2.2)-(2.3) associated with points on the boundary of (R, for which one can substitute approximate known values of U. a

=

+

+ +

Other possible choices are discussed in Birkhoff and Varga [ I , Section 21.

192

GARRETT BIRKHOFF, RICHARD 5. VARGA, AND DAVID YOUNG

Our concern here is with the rapid solution of the vector equation (2.6) for large networks.2 For this purpose, it is essential to keep in mind some general properties of the matrices 2, H , and V . As already stated, 2 is a nonnegative diagonal matrix. Moreover H and V have positive diagonal entries and nonpositive off -diagonal entries. Because of the Dirichlet boundary conditions for (2.1), the diagonal dominance of H and V implies that they are positive definite [19, Section 1.41; such real symmetric and positive definite matrices with nonpositive off-diagonal entries are called Stieltjes matrices. of interior mesh-points is connected, then If the network (R(h, h) = H V and H V 2 are also irreducible; it is known3 that if a Stieltjes matrix is irreducible, then its matrix inverse has all positive entries. The matrices H and V are also diagonally dominated, by which we mean that the absolute value of the diagonal entry in any row is greater than or equal t o the sum of the off-diagonal entries. For any 0 >= 0, the same is true a fortiori of H 02, V 02, and for 0,H O2V 0 2 if > 0, > 0. The above matrices are all diagonally dominated Stieltjes matrices. By ordering the mesh-points by rows, one can make H tridiagonal; by ordering them by columns, one can make V tridiagonal. That is, both H and V are similar to tridiagonal matrices, but one cannot in general make them both tridiagonal simultaneously. It can be shown that the approximate solution of (2.1) for mixed boundary conditions on 63, of the form

+

+ + +

+

+

+

+

d > 0 on 03, &/an d(z, y ) u = U ( x , y), (2.7) can be reduced to a matrix problem of the form (2.6) having the same properties. This is also true of rectangular meshes with variable meshlengths hi, ki, as will be shown in Part I1 and Appendix C; see also [lo, 191. 3. Basic AD1 Operators

From now on, we will consider only the iterative solution of the vector equation (2.6). Since it will no longer be necessary to distinguish the approximate solutions u from the exact solution u(z, y), we will cease to use boldface type. Equation (2.6) is clearly equivalent, for any matrices D and E , to each of the two vector equations (H 2 D ) u = lc - (V - D)u, (3.1) (V 2 E)u = lc - ( H - E)u, (3.2)

+ + + +

We will not consider the truncation or roundoff errors. See Varga [19],Chapter 111, Sectioa 3.5; irreducibility is defined there in Chapter I, Section 1.4. 2

3


+ +

+ +

193

provided (H I; 0)and (V I; E ) are nonsingular. This was first observed by Peaceman and Rachford in [16] for the case Z = 0, D = E = p l a scalar matrix. I n this case, (3.1)-(3.2) reduce to

+

+

( H p I ) u = k - (V - p l ) u , (V p l ) u = k - ( H - pI)?J. The generalization to Z # 0 and arbitrary D = E was made by Wachspress and Habetler [24; see also 231. For the case I; = 0, D = E = PI which they considered, Peaceman and Rachford proposed solving (2.6) by choosing an appropriate sequence of positive numbers p,, and calculating the sequence of vectors un, u,++ defined from the sequence of matrices D, = En = p n l , by the formulas

+ 2 + Dn)un+i (V + 2 + En)un+l = (H

=

k - (V - Dn)un

(3.3)

- (H - En)un+i(3.4) Provided the matrices which have to be inverted are similar to positive definite (hence nonsingular) well-conditioned tridiagonal matrices under permutation matrices, each of Eqs. (3.3) and (3.4) can be rapidly solved by Gauss elimination. The aim is to choose the initial trial vector uo and El, DZ,E2, . . . so as to make the sequence {un>converge the matrices D1, rapidly. Peaceman and Rachford considered the iteration of (3.3) and (3.4) when D, and Enare given by D n = p n l and En = &,I. This defines the PeacemanRachford method:

+ z + p , L l ) - l [ x : - (V - P,J)ZLIll (I' + Z: + pnl)-'[k - (V - p n I > ~ n + + ] .

U?L+I =

(I1

(3.3)

(3.6) The rate of convergence will depend strongly on the choice of the iteration parameters pn, fin. An interesting variant of the Peaceman-Rachford method was suggested by Douglas and Rachford [7, p. 422, (2.3)], again for the case Z = 0. It can be defined for general Z 2 0 by un+l

=

+ PnI)-'[k (1'1 +

(HI

-

(3.7) lhn+l = pJ-'[VlUn ~n?~n++l, (3.8) where H1 and Trl are defined as H +Z and Ti $2, respectively. This amounts to setting D, = En = p J - $2 in (3.3) and (3.4), and making some elementary manipulations. Hence (3.7) and (3.8) are also equivalent to (2.6), if U , = u.++ = Un+lun+i =

+

(v-1

+

-

~ n O ~ n 1

+


194

PART I: STATIONARY AD1 METHODS (CASE m = 1)

4. Error Reduction Matrix

I n Part I, we will discuss only the case that D, = D and En = E are independent of n, so that p n = p and f i n = fi in the preceding formulas. I n this case, it was shown by Wachspress and Habetler [24, Theorem 11 that iteration of the Peaceman-Rachford method (3.5)-(3.6) is always concergent for D = E when D $ is positive definite and symmetric and H V is positive definite. This is always the case in the Dirichlet problem of Section 2, if one chooses D = E = p l - $2, where p is a positive number. We now consider the effect of the Peaceman-Rachford and DouglasRachford methods on the error vector, defined as the difference en = u, - u,, between the approximate solution u, obtained after the nth iteration, and the exact solution urnof the vector Eq. (2.6). A straightforward calculation shows that, for the Peaceman-Rachford method, the effect of a single iteration of (3.5)-(3.6)is to multiply the error vector en by the error reduction matrix T , defined by

+

+ +

+ +

+ +

T p = (V Z pI)-'(H - p I ) ( H 2 pl)-'(V - p l ) . (4.1) Likewise, the error reduction matrix for the Douglas-Rachford method (3.7)-(3.8)with all p n = p is given by

Wp = (Vl

+ pl)-l(H1 + pl)-'(H1V1+ [HiVi + p(Vi + HI) + P~II-'(HIVI + p21) =

+

p20.

(4.2)

If one assumes that D, = -+z p l = En also for the generalized Peaceman-Rachford method (3.3)-(3.4),then from (4.1): Tp

=

(Vi

+ d-'(Hi

- d)(Hi

+ d-'(Vi

- PO,

(4.3)

and the matrices W , and T , are related by 2W,

=

I

+ T,.

But other choices are possible. For example, with D, Douglas-Rachford method is

= pl =

+ z+ k - (v(V + z + (V + 3z)un + (tz + The error reduction matrix for is therefore (V + 2 + pl)-l{(2 + P l ) ( H + 2 + d)-lW - V ) (H

pnl)zcn+; =

Pnl)Un+l =

pnl)Un

(4.4) En,the (4.5)

PnI)un+i.

(4.6)

+ V + 32).

(4.7)

p = pn

l i p

=


195

Error reduction matrices for still other AD1 methods of the form (3.3)(3.4) will be studied in Section 7.

5. Norm Reduction

For fixed D, E , the preceding AD1 methods have the form un+l = M u ,

+ b, where M is a fixed real matrix and b a fixed real vector. I n the terminology of Forsythe and Wasow [8],they are stationary iterative methods. For such methods, it is well known [8, p. 2181 that the asymptotic rate of convergence is determined by the spectral radius A ( M ) of the associated (error reduction) matrix M . This is defined as the maximum of the magnitudes of the eigenvalues of M ; thus

A(M)

=

Maxl {JXi(M)I).

(5.1)

Here the subscript 1 refers to the lth eigenvalue. A stationary iterative method is convergent if and only if its spectral radius is less than one. More generally, the spectral radius a! = A(A) is the greatest number such that the asymptotic error after n iterations, for n large, is o ( P n ) for any /3 > a. Hence R= -log A ( M ) measures the rapidity of convergence; R is called the asymptotic rate of convergence of A . In applying the convergence criterion A ( M ) < 1 t o AD1 methods, it is convenient to use the following well-known r e ~ u l t . ~

LEMMA5.1. For the norm 11x11 = ( ~ ’ Q X ) ” ~Q, a n y real positive definite matrix, if, for a fixed real matrix M , llMx11 5 ~llxllfor all real x, then A ( M ) 5 Y. This must be combined with another lemma, which expresses the algebraic content of a theorem of Wachspress and Habetlel.5 [24, Theorem 13. LEMMA 5.2. Let P and X be positive definite real matrices, with S symmetric. T h e n Q = ( P - S ) ( P S)-l i s norm-reducing for real x relative to the norm

llzll

+

= (zS-’x’)’’2.

Proof. For any norm llxll, the statement that Q is norm-reducing is l(S - P ) y / l z < \l(S P)ylj2 for every equivalent to the statement that I nonzero vector y = ( P S)-’x. In turn, this is equivalent for the special Euclidean norm IlzIl = (ZX-’X’)~/~ to the statement that y ( P 8)S-l (P’ 8’)~’> y ( P - X>X-l(P - X)’y’ for all nonzero y . Expanding the bilinear terms, canceling, and dividing by two, this is equivalent to the

+

+

+

+

4See Householder [ I Z a ] , where the general result for complex matrices is given. The phrase “norm-reducing” there refers to Euclidean norm only in special cases.


196

+

condition that y ( P P’)y’ > 0 for all nonzero y. But this is the hypothesis that P is positive definite.6

THEOREM 5.1. A n y stationary ADI process (3.3)-(3.4) with all D, = D and all E, = E i s convergent, provided Z D E i s symmetric and positive de$nite, and 2H + Z D - E and 2 V B E - D are positive de$nite. Proof. It suffices to show that A ( T ) < 1. But since similar matrices have the same eigenvalues and hence the same spectral radius, the error reduction matrix

+ + + +

+

T

=

(V

+ Z + E)-’(H - E)(H + Z + D)-’(Ti

- D)

(5.2)

of (3.3)-(3.4) has the same spectral radius as

+ +

+ Z + I3-l = + + D)-’][(V - D ) ( V + Z + D)-’]. (3.3) By Lemma 5.2, both factors in square brackets reduce the norm [ d ( Z + D + E)-1x]1/2 = 11x11, provided + D + E 2S, Rw [H + $2 + (D- E)/2] and RV = [V + $2 + ( E - D ) / 2 ] are positive definite, and Z + D + E is also symmetric.’ =

(V 2 E )T (V [(H - E ) ( H 2

=

=

6. Application

I t is easy to apply the preceding result to difference equations (2.2)-(2.3) arising from the Dirichlet problem for the self-adjoint elliptic differential equation (2.1). In this case, as stated in Section 2, H and V are diagonally dominated (positive definite) Stieltjes matrices. The same properties hold a fortiori for BIH 82V e3Z if all Oi 2 0 and O1 82 > 0. Hence the hypotheses of Theorem 5.1 are fulfilled for D = PI - 88, E = pI - 82 for any p, p > 0 and 8, 8 with 0 5 6 , 8 5 2. Substituting into (3.3)-(3.4), we get the following COROLLARY 6.1. If p, j? > 0 and 0 5 8, 0 S 2, then the stationary A D I method defined with 0’ = 2 - 8 by

+

+

+

+ e ~ / 2+ pi)u,+: = 1; - (v+ e ’ ~ / a(1’ + ez/2 + pI)u,,+1 I;; - (H + e’z/2 - ,ijI)u,+:, (H

=

(6.1) (6.2)

i s convergent. In fact, it is norm-reducing for the norm defined by 11~1= 1 ~ d ( Z D E ) - k = ~ ’ [ ( p i)1 (8

+ +

+

+ + e)Z/2]-1~.

6 N o t e t h a t P is not assumed t o be symmetric, but only to be such that r‘(P P‘)r > 0 for all real 5 # 0. ‘This result, for D - E = 0, is due t o Wachspress, Sheldon, and Hahetler (see [93,9 4 ) . For the analogous result on W , see Birkhoff and Varga [ I ] .

+


197

COROLLBRY 6.2. The Douglas-Rachford method i s convergent for any $xed p > 0. The proof is immediate from (4.7), with 6 = 0 = 1. This result shows also that, if 8 = = 1 and if the largestS eigenvalue of T, is positive, the rate of convergence is less than half that of T,. The convergence of the Douglas-Rachford method has not yet been established for other values of 6, except when HZ = ZH and VZ = ZV. In a connected network &, this implies that B = aI is a scalar matrix, as has been shown in [ I ] . If HZ = ZH and VZ = f V , then thc two middle terms of (4.3) are permutable, and so we h a w

e’

T,

=

K-’(H - p l ) ( V - p l ) , I
0 and 8, 8 satisfying 0 S 0, 8 6 2, Corollary 6.1 shows that (6.1)-(6.2) is convergent. We now estimate its asymptotic rate of convergence. By Theorem 5.1, this is R = -In [A(T)] = -In [A(p)],where as in (5.3) and in (6.1)-(6.2),

T = [(He - pI)(Hg + pI)-'][(V$ - j I ) ( V v + PI)-'],

(7.1)

with the notational convention

+

+

He = H 8212 and Ve = V 8212. (7.2) Both products in square brackets in (7.1) are symmetric matrices, and hence have real eigenvalues, if 8 = 8 or if Z = a1 is a scalar matrix. For simplicity, we now assume e = 0; we let a be the least and b the largest eigenvalue of He; we let a be the least and p the largest eigenvalue of Vet; and we restrict 8 so that 0 < a 5 b and 0 < a 6 p. Then the first product in square brackets in (7.1) reduces the Euclidean norm by a factor SUP^^^$^, l ( p - p ) / ( p p)l, or less, and the second product reduces it by a factor less than or equal to sup,s,sa I(. - p ) / ( v p ) l . Hence reduces the Euclidean norm by a factor

+

+

or less. By Lemma 5.1, we conclude

THEOREM 7.1. Let a, b and a, p be the least and greatest eigenvalues of He and Ver, respectively. T h e n , for all p, p , A ( F ) 5 #(a, b ; a,P ; p , p ) . It will be shown in Appendix A that there exist o p t i m u m parameters: values p* and p* of p and p such that +(a, b ; a,P ; P*, p*) = Minp,ii+(a, b ; a,P ; P, 6 ) . The following corollary is immediate.

(7.4)

COROLLARY 7.1. Under the hypotheses of Theorem 2, with 8 = 8, the spectral radius of the generalized Peaceman-Rachford method (6.1)-(6.2) with o p t i m u m parameters i s at most G(a, 6 ; 01, PI = Lfin,, +(a, b ; 01, P ; P, 6). (7.5) I n Appendix A, we will discuss the problem of obtaining such optimum parameters p* and p*. But for the present, we will confine our attention to the simpler problem of optimizing p subject to the constraint p = 8: that is, to the problem of determining a single optimum rho. We have COROLLARY 7.2. I n Corollary 7.1, let

p =

p. Let a, b and a, p be the least

and greatest eigenvalues of He and Vs1,respectively. T h e n , for all p ,

199


(7.6) The right member of (7.6) defines a function of the eigenvalue bounds and p which is so important that we shall denote it b y a special symbol.

DEFINITION. The functions

0

< a 5 b and 0 < a: 5 p, by

X(p)

and F(a, b ; a,

0) are dejined, for given

and

F ( a , b ; a, PI = nlin,>o +(a, b ; a, P ; P I . (7.8) Note that F is a minimax of a family of rational functions; its existence will be established in Appendix A. The following restatement of the key inequality (7.6) follows from the definition of F.

COROLLARY 7.3. I n Theorem 7.1, for the optimum p asymptotic rate of convergence R* which satisfies

R*

=

=

p

= p*,

we have the

-In A(Tp*) 2 -In F(a, b ; a, 0).

(7.9) This corollary shows plainly that one can break down the problem of approximating po and bounding A. into two parts: estimating the least and greatest eigenvalues of He and Vet, and knowing the function F. We will discuss the second of these questions first, referring to Appendix A for details.

8. The Function

F

Some properties of the function F follow almost immediately from its definition by (7.8). LEMM-4 8.1. If a’ 5 a, b 5 b‘, a’ 5 a , and P 5 P’, then F ( U , b ; a, p) 5 F(a’, b’; a’, @’).(Monotonicity Principle) For, the range of values of u and 7 in (7.7) is enlarged, independently of p . Hence, for all p,

X(U, b ; a,P ; p ) 5 X(d,b’; a’, P’;

p).

From this inequality and (7.8), Lemma 8.1 follows immediately.

LEMMA 8.2. For all c

> 0,

F(ca, c b ; ca; cP) = F(a, 0 ; a, P).

For, the substitutions a + ca, 1) -+ab, a ---f ca, definition of F nnafffctcd.

(8.1)

. ,P

---f

CP leave the

200


By the symmetry of the definition, we also have (8.2) F(a, b ; a1 P) = F ( a , P ; a, b), and likewise X(a, b ; a, p ; p ) = X(a, p ; a, b; p) for all p. It is easy to show that 0 5 X(p) < 1for all p > 0, and hence that F < 1. The exact value of F can be computed [keeping the symmetry (8.2) in mind] using Appendix A. Theorem A.l asserts that if ab 6 ap, then F is given by (A.lO) as

with p* = 4%in the first case, and p* = in the second case. Note that since p Z >= 6and 6 p 2 62 a, all factors in (8.3) are positive. Using the preceding formula in Corollary 7.3, we obtain the following result.

./G

THEOREM 8.1. By choosing p* as d/abor as dz, we can make the asymptotic rate of convergence of the Peaceman-Rachford method (6.1)-(6.2) at least -In F , where F i s given by (8.3)) with a, b and a, p the least and greatest eigenvalues of He and Vop (or vice-versa, whichever makes ab 6 a@). 9. Helmholtz Equation in a Rectangle

As an example, consider the modified Helmholtz equation Gou- V 2u = S in the rectangle a:0 5 x 5 X , 0 6 y 6 Y . This is the special case A = B = 1, G = Go 2 0 of (2.l), to which one can reduce any elliptic DE (2.1) with constant coefin'ents by elementary transformations. I n this example, the Dirichlet problem has a known basis of orthogonal eigenfunctions upq = sin (rps/a) sin ( r q y l b ) . (9.1) On the set 6th of interior mesh-points of any subdivision of a into squares of side h = X / M = Y/N, these up, for p = 1, . , M - 1 and q = 1, . . . , N - 1 are also a basis of orthogonal eigenvectors for the three operators H, V , L: defined in Section 2. In fact,

..

Hu,, = PPQuPgl VUP, = VPPUPP, ZUP, = a u P 9 , (9.2) where p p q = 4 sin2(rp/2M), vPP = 4 sin2(rq/2N), u = h2Go. These eigenvalues ppq, vPq range from small positive numbers p~ = 4 sin2(r/2M), Y N = 4 sin2(r/2N) to 4 - p ~ 4, - VN. More specifically, we have the inequalities

4 sin2 ( r / 2 M ) 5

ppn (=

4 cos2 (r/2M),

(9.3)


20 1

4 sin2 (?r/2N) 5 vpn 2 4 cos2 (?r/2N). (9.4) Since the three matrices, H , Ti, and 2 have a common set of eigenvectors (9.1), these are also eigenvectors for the error reduction matrices T,, W,, and U pdefined by Eqs. (4.3), (4.2), and (4.7), and their generalizations to arbitrary 0. The associated eigenvalues, which express the factor by which the uPg-componentof the error function is multiplied, are therefore given by

and, by (4.4),

+

(9.7) M W , ) = [I xP,(s,>l/2, where S , denotes the special case of T , obtained by the choice 0 = 0’ = 1, suggested by Sheldon and Wachspress. Using these general results, it is evident from (9.5) that the PeacemanRachford method is convergent for the Helmholtz equation in the rectangle provided p > (1 - 0)s/2; if 0 2 1, it is convergent if p > 0. Hence, by (9.7), the Douglas-Rachford method with 6 = 1 is convergent (in this special case) provided p > 0. It is also convergent, by (9.5), if 0 = 2 . For 0 = 0’ = 1, T, = X,, one can also compute the exact optimum rho and corresponding most rapid asymptotic rate of convergence for the Helmholtz equation in a rect,angle. By formula (9.5), the spectral radius is

For any fixed p, the two factors inside the absolute value signs are monotone, and so the maximum absolute value of each is assumed for one of the extreme values of pP and vq, numbers which are given by (9.3) and (9.4) respectively. As a consequence, we obtain

N T , ) = +(a, b ; P ; P I , (9.9) where a = paw a/2, b = 4 - pM u/2, a = V N u/2, p = 4 - V N u. Note that a b = a p = 4 a, whence Corollary A.l of Appendix A is applicable. It yields the following result, since ab 5 ap if M 2 N . f f ,

+

+

+

+

+

+

+

THEOREM 9.1. For the Helmholtz equation in a rectangle, with M 2 N, the optimum p for the Peaceman-Rachford method with 0 = 0’ = 1 i s p* =

d ff p = [(4 sin2

+ 5c) (4 cos2 A -+

;)I1?

(9.10)

202


The corresponding spectral radius is (9.11)

I n the case u = 0 (of the Laplace equation), the preceding formulas simplify. Then p* = 2 sin ( a / N ) , and the associated spectral radius is sin ( a / N ) - 2 sin2 ( s / 2 M ) sin (a,”) 2 sin2 ( a / 2 M )

+

cos (s/2N) - sin ( a / 2 N ) cos ( s / 2 N ) sin ( a / 2 N )

+

(9.12) 10. Monotonicity Principle

For most regions and most difference equations (i.e., for most choices of H and V ) , the eigenvalues p p of H and v q of V cannot be varied independently to produce an eigenvalue of Tp.As a result, though the spectral radius is bounded above by the right side of (9.11) for the Helmholtz equation with Dirichlet-type boundary conditions, on a n y rectangular mesh (R(h, k ) in which no connected row has more than M 1 and no column 1 ( N 5 M ) consecutive points, one does not know that p* more than N as given by (9.10) is really the optimum rho. In such cases (for arbitrary self-adjoint elliptic difference equations with Dirichlet-type boundary conditions), one can still determine good values of rho by relating the given boundary value problem to the Helmholtz equation in a rectangle, and applying Weyl’s monotonicity principle9 [25a].

+

+

THEOREM 10.1. Let A and B be two real n x n symmetric matrices, with eigenvalues a1 5 . . . 5 a, and 5 . . . 5 Pn, respectively. Let the eigenB be y1 I ... I 7%.T h e n ai P j 5 y h 5 az Pm if values of C = A i+j -I I k I 1 m - n.

+

+

+

+

This principle has many immediate corollaries for the operators H , V , H BE, V BE, and so on. For instance, it shows that if urninis the 0 2 exceed those of smallest eigenvalue of 2 , then the eigenvalues of H H (arranged in descending order) by at least .ni,& Likewise, it shows that the eigenvalues of H and V increase when A (z, y) and C(z, y ) are increased in (2.1), since one adds a diagonally dominated Stieltjes matrix to each, and such matrices are symmetric and positive definite.’O Finally, it shows that if the spectral radius (= Euclidean norm) of B is

+

+

+

We omit the proof. I n general, only nonnegative definite; but, in the present case, they are positive definite if A(z, v) and C(T, y) are increased a t all points. 9

10


203

+ B = C differ from those of

a t most p, then the eigenvalues of A arranged in the same order by at most /3.

A

11. Crude Upper Bound

Using the preceding observations, one can easily obtain a crude upper bound" for A0 = A(T,,*) and in fact a "good" rho p1 such that X(p1) is less than unity by an appreciable amount. One need only combine Theorem 8.1 with the monotonicity principles of Section 10. For simplicity, we consider only the case of constant h, k. First, one observes that the matrices H and V are changed by positive semidefinite matrices when A ( x , y) and C(x, y) are increased in (2.1), and also when 2 is increased. It follows by Theorem 10.1 that if A ( x , y) and C ( x , y) are replaced at all mesh-points by their maximum and minimum values X, C and A, C,respectively, then the spectrum is shifted up resp. down, as regards all spectral values. Second, if the network @(h, k ) is embedded in a larger (rectangular) network 6 by any extension of the coefficient-functionsA ( x , y) and B ( x , y), then the least eigenvalue is decreased (or left unchanged) and the upper one increased (or left unchanged). This is because, on a,the effect of H and V is that of a matrix which is a principal minor of the corresponding matrices H and V on 6.The least and greatest eigenvalues aminand amax of H have such that vHv' = vgv' = aminvv' and eigenfunctions v, w with support WHW' = WGW'= a,,,vv', respectively. Hence u

a,in

=

min,+o[v&v'/vv'] 5

anlirl

5

a,,ar 5

rnax,#o[wITw'/ww']

= Zimaxl

and likewise for V . Combining the two preceding observations, we obtain the following result. = @(h,k ) can be embedded in n rectangle THEOREM 11.1 Suppose that with side of length M h and N k parallel to the axes. Then

(11.1) a

=

u

4A sin2 2M

a = 4~

u

sin2 2N

+ 2s

b

=

4x cos22 M + 5'

+ 22

p

=

4c cos2 2M

-1

1

i

T

T

s

+ -.2 S

(11.2) (11.3)

COROLLARY 11.1. If A ( x , y) = C ( x , v) and 221 2 N in Theorem 1 2 . 1 , then 5 F ( a , b ; a,p), where p1 =

X(pl) i1

dz.

This result was obtained for the Laplace equation in Varga [f 71.

204


+

+

Proof. In this case, a b = a p ; hence the conclusion follows. If A # C, however, in general a b # a p.

+

+

12. Eigenvalues of W,V One can obtain arbitrarily close approximations to the minimum eigenvalues p1 (and vl) of H (and V ) .For any nonzero vector x, the Rayleigh quotient satisfies x'Hx/x'x 2 pl; if y = Hx is any positive vector, then min;[(Hx);/x;] (= pl. Wachspress [W5] has invented an iterative process, based on the Stieltjes property of H and the inverse power method, for computing p1 with arbitrary accuracy. Similar remarks apply to v;. The less crucial maximum eigenvalues of H and V are bounded above by Gerschgorin's Circle Theorem [19], often with sufficient accuracy. For small mesh-length h, accurate asymptotic bounds can be found using the fact that on each connected row (resp. column) of @ h , H (resp. V ) defines a discrete Sturm-Liouville system. Such discrete Sturm-Liouville systems have been thoroughly studied in the literature."" The least eigenvalue of the matrix H , for small fixed h, is approximately h2 times the lowest eigenvalue of the corresponding continuous Sturm-Liouville system, a fact which gives a convenient asymptotic expression for p l ( h ) . The error in this bound is small for h sma!l."b The largest eigenvalue corresponds to an eigenvector, whose components oscillate in sign, and is about equal to 4Z, the maximum being taken over a.The error is ordinarily O(h), but is O(h2)if A = A(y). Similar estimates can be obtained for V . But the fact that the extreme eigenvalues in question can be accurately estimated does not imply that p* or h(T,) can be accurately estimated. As has already been observed in Section 7, p m and v, cannot be varied independently except in special cases (to be treated in Chapter 11). Ilb

See [g], Chapter X; also [ I O U ] . See [12b].


205

PART II: COMMUTATIVE CASE 13. Introduction

It was proved in Birkhoff and Varga [ I ] that, for m > 1, the analysis of the asymptotic convergence rates discussed in Douglas and Rachford was applicable to the self-adjoint elliptic difference equations of Section 1 in a connected plane network @, if and only if the symmetric matrices H , V , and I; of (2.6) were commutative-that is, if and only if

[?'I

HV = VH, H Z

=

ZH,

VZ

= ZV.

(13.1)

In this chapter we study the extension of this observation to matrices generally. Accordingly, we consider the vector equation (13.2) (H 2)u = I,

+v +

where 2 is a nonnegative diagonal matrix and where H singular. As in [I] we make the following assumptions:

HV

=

VH

Z = a1 (u a nonnegative

+ V + Z is non(13.3) (13.4)

We do not assume that H or V is symmetric. Instead, we make the following weaker assumption: H and V are similar to nonnegative diagonal matrices. (13.5) Conditions (13.3)-(13.5) are related to (13.1) through the following:

THEOREM 13.1. If H and V are positive dejinite symmetric matrices, and if H V i s irreducible, then conditions (13.3)-(13.6) are equivalent to the

+

commutativity condition (13.1). The importance of conditions (13.3)-(13.5) for the study of AD1 methods depends on the following theorem of Frobenius:I3 THEOREM 13.2. T h e matrices H and V have a common basis of eigenvectors i f and only i f H V = V H and H and V are similar to diagonal matrices. From this it follows that H and V have a common basis of eigenvectors and nonnegative eigenvalues if and only if (13.3) and (13.5) hold. If (13.3)-(13.5) hold, then for any nonnegative constants O1 and 02 the matrices l2 We remark that by a slight generalization of Lemma 2 of ref. [ I ] one can show that if H V is irreducible then (13.4) is equivalent to the conditions H z = Z V and v2 = zv. l3 See Exercise 1 in Thrall nnd Tornheim [ I & ] , p. 190.

+

206

+


+

H OIZ and V 0 . 2 also have a common basis of eigenvectors and nonnegative eigenvalues. I n Section 14 we exhibit a class of problems involving elliptic partial differential equations which lead to systems of linear algebraic equations of the form (13.2) where the set of matrices H , V , and Z satisfy (13.3)(13.5). In Sections 15-17 we describe how the assumption of conditions (13.3)-(13.5)-leads to effective methods for choosing iteration parameters and for accelerating the convergence of the Peaceman-Rachford and the Douglas-Rachford methods. The application to the Helmholtz equation is given in Section 18. 14. Problems leading to Commutative Matrices

It has already been shown in Section 9 that the Dirichlet problem for the modified Helmholtz equation in a rectangle leads to matrices H and V which have a common basis of eigenvectors and positive eigenvalues. It then follows from the remark after Theorem 13.2 that H and V satisfy (13.3) and (13.5). Since Z = u1, with u 2 0, (13.4) holds also. It was shown in Ref. [I] that if H V = V H , where the matrices H and V arise from a differential equation of the form (2.1) and from the difference approximations (2.2)-(2.3), then the region is a rectangle, and the differential equation is the modified Helmholtz equation. However, as observed by Wachspress,14one can obtain matrices N, V , and Z satisfying (13.3)(13.5) from more general differential equations of the form

in the rectangle a:0 5 x 5 X , 0 5 y 5 Y.The functions El(%),Fl(y), Ez(z), F&) are assumed to be continuous and positive in a,and K is a nonnegative constant. Evidently (14.1) is a special case of (2.1) with A ( x , y) = E I ( ~ ) F I ( C(z, ~ ) , Y) = E2(x)Fdy), and G(z, Y) = KE~(~>FI(Y). A difference equation leading to commutative matrices H , V , and Z is obtained as follows: First, choose mesh sizes h and k such that X / h and Y / k are integers. Next divide (14.1) by Ez(x)F,(y) obtaining

Replacing - hkd[Eldu/dz]/dx and -hkd[F&~/dy]/dy by the expression^'^ given in (2.2) and (2.3), respectively, and substituting in (14.2) we obtain

(H

+ v + 0 4 x ,Y)

= t(z,

(14.3)

Private communication and Ref. [ah]. If one were to use the difference equation of Section 2, one would obtain matrices H and V which, though symmetric, would not in general commute. '1

15


207

where

+ h, y) - A~(z)u(x- h, y),

(14.4)

V u ( x , Y) = C O ( Y ) U (Y~), - Cz(Y)u(Z,Y 4- k ) - C ~ ( ~ ) Uy( Z-, k ) ,

(14.5)

Hu(Z, Y)

=

Ao(Z)u(x,Y) - Al(x)u(x

Z = hkK,

and t(x, Y) = hkfl(x, y)/~%(~)Fi(y), Ai(z) CZ(Y>= hFz(y (kP))/kFl(y), etc. We now prove

+

=

+

kEi(~

(14.6)

(h/2))/hE2(~),

THEOREM 14.1. Let H, V , and Z be the matrices arising from the solution of the Dirichlet problem in a rectangle for the differential equation (i4.2)and using the difference equation (14.3). Then H, V , and Z satisfy conditions (13 3)-( 2 3.5).

Proof. We first prove

LEMMA 14.2. Under the conditions of Theorem 14.1, conditions (13.4) and (13.5) hold whether or not the region i s a rectangle.

Proof.Because of (14.6) the matrix Z satisfies (13.4). To show that H and V satisfy (13.5) we observe that the matrices H ( s ) = FH and V ( S ) = FV, where F is a diagonal matrix with nonnegative diagonal elements corresponding to the function F(z, y) = Ez(z)Fl(y), are the same as the matrices which one obtains by using the difference approximations (2.2) and (2.3) in (14.2). But in Section 2 it was shown that HCS)and VCs) are symmetric and positive definite. It then follows that H F = Fl/ZHF-l/Z = F-l/ZH(S)F-l/Z and V F= FI/ZV$’-l/Z =- F--1/2V(S)F--1/2 are symmetric. Moreover, since for any nonzero vector v we have (HFu,v) = (F-1/2H(s)F-1/2v, v) = (H(s)F-1/2v, F-%) > 0, since F-% # 0, it follows that HF is positive definite. Similarly, V F is positive definite. Hence H F and V F ,and consequently H and V , are similar to diagonal matrices with positive diagonal elements. To complete the proof of Theorem 14.1, it remains to show that H and V commute. This is equivalent to showing that BP = vg, where and p are difference operators which correspond to H and V , respectively. Actually, f7 and 7 are simply the operators H and V defined by (14.4) and (14.5) but restricted to functions defined only on R ( h , k ) . I n order to avoid the necessity of writing special formulas for H u and Vu for points adjacent to the boundary, where certain terms in (14.4) and (14.5) would be omitted, we write

R @ ( Z , y)

= AO(Z)U(R.,

y) -

ZI(.C, y>u(x

+ h, y)

(14.7) - &(x, Y > u (~ h, Y), l6 Theorem 14.1 can be generalized to include problems involving mised boundary conditions and nonuniform mesh sixes, as shown in Appendix C.

208


m x , 3) = Co(y)u(z, Y)

- Cz(z, y>u(x, Y + k )

- k),

- C4(x, y)u(x, y

where

+ h, y),

(14.8)

(14.9) Ay(x)r(x - h, Y), (14.10) = C4(.4r(x, Y - k ) and where r(z, y) = 1 if (z, y) is in @(h,k ) and r(x, y) = 0 otherwise. The use of the “projection operators” A and is especially convenient for the computation of products of operators. We now prove LEMMA 14.3. Let B and 7 be difference operators deJined over the rectangular nelworki7 @(h,k ) b y (14.7) and (14.8). T h e n and commute. Proof. For any u ( x , y) defined on @(h,k ) we seek to show that Bvu(x,y) = PBu(x, y) for all (2, y) in @(h,k ) . Evidently both Qvu(x,y) and PBu(z, y) are linear combinations of u(x, y) and other values of u in @(h,k ) . The coefficient of u(z h, y) for Bru(x,y) is ---K1(x, y)C~(y)= -Al(Z)Co(y)I’(x h, y) which is equal to the coefficient of u(z h, y) for Vf?u(x,y). Moreover, the coefficients of u(x h, y k ) are

Zi(x, y)

=

Ai(x)I’(x

c2(z,7 ~ )= c2(y)r(5,y + k ) ,

Z~(Z, y) -7 c4(x,Y)

=

r

w

+

+

+

+

+ h, y>r(.z:+ h, y + Ic) Al(z)Cz(y)r(z + h, Y + w x z , ‘Y + 1 ~ )

+

Al(z)Cz(y)r(x for i?pu(x, y) 2nd

+

+

for vBu(z,y). If (x h, y k ) does not belong to &(h, k ) both coefficients are zero. Otherwise, since the region is rectangular and since (2,y) is in &(h, k ) i t follows that both (x h, y) and (z, y k ) belong to B ( h , k ) . Thus the two coefficients are equal. Similar arguments hold for the coefficients of u ( x - h, y), u ( x , y k ) , etc., and the lemma is proved. The proof of Theorem 14.1 is now complete. We remark that the matrices H F and V P considered in Lemma 14.2 commute provided H and V commute. For problems to be solved on large automatic computing machines it may be advantageous to use symmetric matrices because of the savings in storage. The operators H F and VFcorresponding to the matrices H F and V F are given by (14.4) and (14.5) where

+

+

+

+ (h/2)1 + - (h/2)1)/J32(z), etc. ~1(x= ) E~(Z + ( h / 2 ) ) / 4 ~ 2 ( x ) ~+z ( h), z

Adz) = (&[x

Theorem 14.1 shows that, with a self-adjoint differential equation of the form (2.1), for there to exist a function P ( x , y) such that the matrices H , V , and Z satisfy (13.3)-(13.5), it is suficient that the differential equation have the form (14.1). Here H , V , and Z arise from the use of the difference *’A network @(h, k ) is “rectangular” if it consists of the points (m nhere i = 0 , 1 , . . . , p n n d j = 0 , 1 , . . . , 1, for some a,YO,h > 0 and k

+ Ih, yo + j k ) , > 0.


209

approximations (2.2) and (2.3) for the differential equation obtained by multiplying both sides of (2.1) by P(x, y) = 1/E2(x)Fl(y). I n Appendix D it is shown that the condition is also necessary. It is natural to ask whether a similar necessary condition might hold for elliptic equations more gencrally. In this vein, Heller [ I d ] has shown t.hat for the equation

+ G(z, y)u = S ( x , y)

(14.17)

it is sufficient that A and ll depend on x, that C and E depend on y, and G is a constant. However, these conditions are not necessary,'* as the following example shows. Example. Consider the problem of solving the equation (14.18) in the unit square 0 < x < 1, 0 < y < 1 with prescribed values on the boundary of the square. Writing the difference operators H and V in the form (14.4)-(14.5) we have A&, y) = G ( x , y) = 2, A l ( x , y) = CZ(Z,y) = (z y h ) ( x y)-l and A&, y) = G ( x , y) = ( x y+, - h ) ( x y1-l. By direct computation one can show that the operators H and 7 commute. Hence so do H and V . To show that the matrices H and V satisfy (13.5) we observe that H is a tridiagonal matrix whose diagonal elements are positive and whose elements on the adjacent diagonals are negative. Replacing the nonzero off-diagonal elements ai,j by dat,,aj,i we get a symmetric matrix which is similar to the original matrix. Thus H has real eigenvalues and is similar to a diagonal matrix. Because of weak diagonal dominance H has nonnegative eigenvalues and is similar to a nonnegative diagonal matrix. Since the same is true of V, condition (13.5) holds. We remark that one could makc (14.18) self-adjoint by multiplying both sides by - ( x y)z obtaining

+ +

+

+

+

a -d X ((z

+ Y I 2 g)- aya + ((1:

-

(14.19)

Since this equation is not of the form (14.1), it follows from the necessary condition for self-adjoint equations stated above that the matrices H and V corresponding to (14.19) based on the difference approximations (9.9)and (2.8) will not commute even if one first multiplies both sides of (14.19) by any nonnegative function. Thus even though (14.19) and (14.18) are equiv'8This contradicts a statement of Heller [ I d , p. 1621. Even the weaker conditions that there exists a nonvanishing function P such that P A and PD depend only on x , that PC and P E depend only on y, and that P G is constant arc not necessary.

2 10


alent equations, by the use of one difference equation we obtain matrices H , V , and I: which satisfy (13.3)-(13.5) while with the other difference equation we do not. The question of how general the differential equations of the form (14.17) can be in order for the associated matrices H , V , and 2 to satisfy (13.3)(13.5) remains to be studied. 15. The Peaceman-Rachford Method

We now consider the Peaceman-Rachford method for solving (13.2) defined by (a, PnI)Un+l,Z = k - ( V , - P n I ) U n (15.1) (Vl P n I ) U n + l = k - (HI - PnI)Un+l!Z +2. Equation (15.1) is derived from where H1 = H +2, V1 = V (6.3)-(6.4) by replacing p’ and p‘ by pn. If the matrices H , V , and Z satisfy (13.3)-(13.5), then there exists a common basis of eigenvectors for H I = H ( a / 2 ) 1 and for V1 = V (a/2)1. Moreover, if v is an eigenvector of such a basis, then Hlv = p ~ , Vlv = V U , (15.2)

+

+ +

+

1

+

+

where p and v are suitable eigenvalues of Hl and V1 respectively. Hence the p)(v p ) , where eigenvalues of T,are all of the form ( p - p ) ( v - p ) / ( p

+ pI)-l(Hi - pI)(Hi + pI)-’(Vi

Tp = (Vi (See (4.1).) Moreover,

m

v, =

2

(P - PJ(. (p

- Pz)

+ p J ( v + Pi)

+

+

- PI).

(15.3) (15.4)

is an eigenvalue of IIE T,,.Evident’ly all eigenvalues of IIE 1 TPiare given by (15.4) for some eigenvalues I.C of H1 and v of VI. Thus we have

(15.5) where p and v range over all eigenvalues of H1and V1, respectively. In cases such as that of Section 14, where the eigenvalues of HI and V1 of a common basis of eigenvectors include all pairs ( p i , V J of eigenvalues pi of H , and vI of V1, one has equality. I n actual practice there are usually so many eigenvalues of HI and V1 that it is not practical to consider them individually even when they are known. It is, however, often practical to estimate upper and lower bounds for the eigenvalues of H1and V1. Thus, having estimated a, b, a,and /3 such b, a _I v 5 fl one seeks to minimize that a 5 p

s


where p = inequality

(PI,

pz,

21 1

. . . , p"). Frequently, it is convenient to use the

b, 00 P, PI 5 [arn(ait6, p)12 where a = Min (a, a),6 = Max (b, p), and where

(15.7)

*rn(at

(15.8) The problems of minimizing \krnand 9, are equivalent to the problems of determining the minimax of the rational functions involved over certain domains. For the case m = 1 the problem of minimizing XPm is solved in Appendix A. The problem of minimizing 9rnfor m = 2', r an integer, has been solved by Wachspress [25]. The solution is sketched in Appendix B, which also contains a general discussion of the problem of minimizing a,,,.

16. Methods for Selecting Iteration Parameters for the Peaceman-Rachford Method

We now consider two choices of the iteration parameters for the Peaceman-Rachford method defined by (15.1). One choice of parameters was presented by Peaceman and Rachford in [ I S ] . The other was given by Wachspress [23, 241. Though neither choice of parameters is optimum, nevertheless, their use makes the Peaceman-Rachford method effective. We choose a and b so that for all eigenvalues p of H1 and v of V1 me have a 5 p, v 5 b and me let c =a -. (16.1) b

By (15.5), (15.6), and (15.7) the spectral radius of

I I E 1

T P sat , is fie^'^ (16.2)

where, by (15.8), @.,(a, b, p ) is given by (16.3)

The parameters of Peaceman and Rachford are (16.4) and those of Wachspress are l9 The exponent 2 a t the end of (16.2) was omitted in [I61 and in Young and Ehrlich ~91.

212

GARRETT BIRKHOFF, RICHARD S. VARGA, AND DAVID YOUNG (i- l ) / ( m - 1) piw’ =

b

(:)

,

m z 2 , i = 1 , 2, . . . , m.

(16.5)

We first consider the average rate of convergence R, for the PeacemanRachford parameters when m is held fixed. We define Rmfor any choice of parameters byzo (16.6)

Moreover, by (16.2) we have

-

R, 2 R,

=

2 -;log

@,(a, b, p ) .

(16.7)

THEOREM 16.1. For fixed m if the iteration parameters are given by (16.4) then

where (16.9)

and x

= cli2m.

(16.10)

Moreover, as c -+0

R, 2 R,

4

= -z

m

+ O(z2).

(16.1 1)

Proof. The inequality (16.8) is proved in Appendix B, (B.14). To prove (16.11) we first note that by (16.7), (16.8), and (16.9)

R, h

2 -;log

4 m

6 = -2

+

O(22).

(16.12)

On the other hand, by (16.3), (16.4), and (16.7) we have

Equation (16.11) follows from (16.12) and (16.13). We next seek to optimize the choice of m for a given c. We estimate the average rate of convergence from (16.6) and (16.8) as 2 ELPI = -(16.14) log 6, m where 6 is given by (16.9). We note that, by (16.7) and (16.8), R , 2 R, 2 R P . *OEvidently for m Section 5.

=

1, R,, is juet the asymptotic rate of convergence a5 defined in


213

Following a method of Douglas [4] we study the behavior of R$') as a function of m, where m is assumed to be a continuous variable. Because the right member of (16.9) is a monotone-decreasing function of m, by (16.10), a one-to-one correspondence between m and 6 is defined. Solving (16.9) and (16.10) for m we obtain (16.15)

Substituting in (16.14) we obtain

Eg)=

-4 log 6 log [(l - 6)/(1

+ S)].

log c

(16.16)

Equating to zero the first derivative of the above expression with respect to 6 we obtain 1-62 1-6 log -= 6 log 6. (16.17) 2 1+6 It is easy to prove LEMMA 16.2. The function Rg) defined by (16.16) i s maximized when

42 - I

6 =

and the corresponding Galue of

R,?

if$'

=

2

0.414,

(1 6.18)

is 4(10gj)~. 3.11 . -1ogc -1ogc

(16.19)

Of course, the value 8 = 0.414 will in general correspond to a noiiintegral value of m, and the actual value of Eg)would in general be less than indicated by (16.19). I n actual practice one would use the following procedure: (1) Estimate a and b, and compute c = a/b. (2) Find the smallest integer m such that ( i ) ? m 5 c, (16.20) where 8 = 4 5 - 1 A 0.414. (3) Determine the iteration parameters by (16.4). (4) The estimated average rate of convergence is given by

where

For the above procedure we prove

THEOREM 16.3. If for given a and b the number of iteration parameters m i s chosen as the smallest integer satisfying (16.20), and if the iteration parameters are chosen by (16.4), then for a n y q > 0 and for suficiently small c


214

R, 2 where 8

=

4 5 - 1A

B,

2

4(10g 8)' - 7 - 3.11 - Q -log c -log c ?

(16.21)

~

0,414. Moreover,

Lim {&,(--log

c)}

2 4(log 8)'

c)}

5 4 I log 2 1 {I log 8 I

A

(16.22)

3.11,

c-0

and r m {%(-log c-0

+ 8)

4.57.

(16.23)

Proof. If ism5 c then z 2 8 by (16.10). Consequently, by (16.7), (16.8), and (16.9), we have

2 log 6- = -- 2 log & m - - l ) / m -m m-1 I c we have m-1 Rm(-log c) 2 )-4( (log $ 2 , -

But since

@(m-l)

(16.22) follows. By (16.7), the inequality and, since m + m as c - 0 , (16.21) holds. On the other hand, by (16.13) we have

Using the formula -(1/2)log ((1 - .)/(1

+ .))

=

we have

.+

53/3

+ z5/5 + . 2x3

(1 -

i=3

i=2

7

2)2

*

*

, (16.24)

and hence (16.25) But by (16.20) we have m 2 (1/2)(log c/log

I ) , and

Because of (16.20) and (16.10) it follows that Lim 2 = Lim c1/Zrn= 1. c-io

c-0

Consequently (16.23) holds, and the proof of Theorem 16.3 is complete. For a given m, we have by (16.7) and (16.25)


215

COROLLARY. If the p , are chosen by (16.4),then

(i 5:)

%(a, b, P ) 2 - exp [-2z3/(l - z ) ~ = ] 6c-223/(1-2)2.(16.27) We now consider the parameters of Wachspress given by (16.5).For the case of fixed m we prove

THEOREM 16.4. For given m, i f the iteration parameters are given by (16.5)

then

(16.28) where

(16.29) (16.30)

(16.31)

Proof. The inequality (16.28)is proved in Appendix B, (B.16).To prove (16.31)we first note that, by (16.7),(16.28),and (16.29)

R,

2 --log 2

+

e =4 y 0(Y2). m m On the other hand, by (16.3),(16.5),and (16.7)we have

(16.32)

(16.33) and hence, by (16.5),

4

R , S --log

(l - yy)

+

-

log

fi ('1 +-

i=3

y2'-3)

yZip3

=

4

-y

m

+ O(y2).

(16.34)

From this (16.31)follows, and the proof of Theorem 16.3 is complete. We now look for an m which will maximize the average convergence rate as estimated by 2 Eg.)= -log e, (16.35) m where E is given by (16.29).We note that by (16.7)and (16.28) R, 2 R, 2 ELrn. As in the case of the Peaceman-Rachford parameters we consider 7?Lm as a function of e, where by (16.29)and (16.30),e and ma re related by (16.29). Because e is a monotone -decreasing function of m,a one-to-one correspond-

216


ence between m and e is defined. If we were to replace m by m - 1 in (16.35), then we would have, by (16.29) and (16.30),

R??

= -8 log

4 log [(l - 4 ) / ( 1 + dill. log c

By Lemma 16.2 the optimum value of 4; would be :, the optimum of el would be

z

= 82 =

(di- 1 ) 2

=

3 - 245

A

(16.36)

4 5 - 1 & 0.414 and 0.172.

(16.37)

Of course the value Z = 0.172 will be inaccurate not only because of the replacement of m by (m - 1) in (16.35) but also because the value of m corresponding to 2 by (16.29)-(16.30) will not be an integer. I n actual practice one would use the following procedure: (1) Estimate a and b, and compute c = a/b. (2) Find the smallest integer m such that jjzcm-1,

5 -c

(16.38)

5 - 1 = 0.414. where 8 = 4 (3) Determine the iteration parameters by (16.5). (4) The estimated average convergence rate is given by 2 RAW,”’= -log €, m where In spite of the fact that the above procedure does not give the best value of m,we can prove

THEOREM 16.5. If for given a and b the number of iteration parameters m i s chosen as the smallest integer satisfying (16.38), and if the iteration parameters are chosen by (16.5), then for any 7) > 0 and for sz@iciently small c R, L R, 2 16(10g 8)’ -log c where 8

=

~

6.22 - 7, -log c

(16.39)

4 5 - 1 & 0.414. Moreover,

Lim {%(-log c)} L 16(log &)z

A

6.22,

(16.40)

c+o

and

Lim {R,(-log c)} 5 8 I log 8 1 (1 c+o

log 81

+ +8}

7.66.

(16.41)

Proof. If 8z(m-1) 6 c, then y 2 8. Consequently, by (16.7), (16.28), and (16.29) we have


217

But since 82(m-2) 2 c we have, by (16.37),

R,(-log

c) 2 8(log $ 2

("?n

___

".

Moreover, since m 03 as c + 0, (16.40) follows. By (16.7), the inequality (16.29) holds. On the other hand, by (16.34) we have 1 - 2i--1 ---f

But, by (16.24) we have (16.42) Thus, from (16.38) it follows that

-

R,( -log c) I8 log I log l - y+8jlog81 l + Y

y3

(1 - YI2

.

Because of (16.30) and (16.38) it follows that Limy c-0

=

lim ~ 1 / 2 ( m - 1 )

=

8.

c+O

Thus (16.41) follows, and the proof of Theorem 16.5 is complete. For given m, we have by (16.7) and (16.42)

COROLLARY. If the pi are chosen by (16.4), then (16.43) Theorems 16.3 and 16.5 show that the Wachspress parameters are superior to the Peaceman-Rachford parameters by a factor of approximately two, provided that the values of m are chosen by (16.38) and (16.20), respectively. Numerical experiments described in Part IV tend to confirm this superiority. 17. The Douglas-Rachford Method

In Sections 3 and 4, two variants of the Douglas-Rachford method are given. The first is defined by (3.7)-(3.8); the second is defined by (4.5)(4.6). Because of the assumptions (13.3)-(13.5) on the matrices H , V , and 2 we can express the eigenvalues of the error-reduction matrices W , and


21 8

Up,defined by (4.4) and (4.7), respectively, in terms of the eigenvalues p‘ of H and Y’ of V . Thus xw =

(p’ (p’

+ u’)(.’ + u’) + P 2 + u’ + P)

+ u’ +

(17.1)

P)(V’

is an eigenvalue of W, where

.>

(17.2)

u’ = u/2,

and xu =

+ d(.’ + + + + + P) (P’ + + Pa

(P’

P>(V’

P2

(17.3)

Q

is an eigenvalue of Up. Both variants of the Douglas-Rachford method are identical if u = 0. We now show that for u > 0 the variant corresponding to W , is superior to the other variant. We show that using p’ = p u’ with the first variant yields an eigenvalue XW which is smaller for all positive p’ and Y’ than the corresponding eigenvalue of Up.This will imply that A(W,) 5 A(U,) and that for any PI, PZ, . . . , pm,

+

Kow, replacing p by

But since [(P’

.>

p

+ u’ in (17.1) yields

+’.(I. + + + PO

P21 - [(P’

+

.’)(Y’

+ u’) + + u’)zl (P

= u’(p’

+ + u”2, Y‘)

which is positive for u > 0, it follows that Xu > Xw for all p’ and d . Hence, for u > 0 the first variant of the Douglas-Rachford method is superior to the second. Henceforth we shall consider only the first variant. From (17.1), if p and v are eigenvalues of H I = H 0’1 and VI = V u’1,respectively, then

+

+

m

i=l

G

crv + Pi2 + PJ(V + Pi)

is an eigenvalue of H E l W,,?. It is convenient to define

and

(17.4)


219

Evidently, we have

5 *E)(a, b, a, P, P ) 5 @,?(a, 6, P ) where a = Min (a, a),6 convergence R, by

=

(17.7)

Max (b, 0).We also define the average rate of (17.8)

Evidently by (17.7) we have

R, 2

R,

1

(D)

= -- log@,

m

- 6 (a, ,p).

(17.9)

The solution to the problem of minimizing for the case m = 1 is given in Appendix A. It is also shown that if a b = a p, then the Peaceman-Rachford method with the optimum single parameter is a t least as effective as the Douglas-Rachford method with the optimum single parameter. We now study the convergence of the Douglas-Rachford method with parameters as given by (16.4). This selection of parameters was used by Douglas and Rachford [7]. We shall assume that the eigenvalues 1.1 of H and v of V all lie in the range a 5 p , v 5 b. We now prove

+

+

THEOREM 17.1. For $xed m, if the

pi

are given by (16.4), then (17.10)

where

+

+(1 a'), and where 6 i s given by (16.9). Moreover, as c 60

B,

=

=2 z

m

(17.11) =

a / b -+0 we have

+ O@),

(17.12)

where z is given by (16.10). Proof. The inequality (17.10) is proved in a manner similar to that used in Appendix B to prove (B.14). To prove (17.12) we first note that by (17.6), (17.8), (17.10), and (17.11),

-

R, 2

1

-- log So =

m

1

--log m

1 + 2 2

+

(1

2)'

On the other hand, by (17.6): (17.7), and (17.9)

and by (16.4),

"~

1

2 m

____ = - z

+

O(2').

220


This completes the proof of Theorem 17.1. We next seek to optimize the choice of m for a given c. We estimate the average rate of convergence from (17.8) and (17.10) as (17.14) Relating m and 6 as in the case of the Peaceman-Rachford method we have, by (17.11)) (16.9), and (16.10), (17.15) Equating to zero the first derivative of

I??,”’ with respect to 6 we obtain

+

1-6 6(1 - 62) log -= (1 6 2 ) log (4 1 + 6 Let 8 be the solutionz1of (17.16) in the range 0 < 6 putations lead to the value 80 A 0.60.

+

$62).

(17.16)

< 1. Numerical com(17.17)

The corresponding value of 6 0 is 0.68. I n actual practice one would use the following procedure: Estimate a and b and compute c = a/b. Fine the smallest integer m such that $0 A

(S) 1-6

SC)

(17.18)

(17.19)

where 8 satisfies (17.16) and (17.17). By (17.17) one would actually use (0.25)2mS c. (17.20) Determine the iteration parameters by (16.4). The estimated average rate of convergence is given by (17.21) where (17.22) For the above procedure we prove The solution is unique in the range 0

< B < 1 since d2EAD’/dP < 0 in that

range.


221

THEOREM 17.2. If for given a and b the number of iteration parameters m i s chosen as the smallest integer such that (17.19) i s satisjied, and if the iteration parameters are chosen by (16.4), then for any q > 0 and for sumiently small c 210g(310g(;+

R, 2

E,

i 8 2 )

-log c

=

where 8 satisfies (17.16) and 8

A

-* -

'*07 - ', -log c

(17.23)

1.16.

(17.25)

0.68. Moreover,

and

Lim {R,(

-log c)} 5 2

c-0

A

Proof. Let x be given by (16.10) and let (17.26)

+ 8').

If m satisfies (17.19), then 9,5 c, z 5 z, 6 5 8, and 60 5 $(l Consequently, by (17.9) and (17.10) we have

Em(-log c) 2 2(-log 2) log

(; + ;P) (A). m-1

Since m --+ m as c -+0, (17.24) follows. By (17. 9), the inequality (17.23) holds. On the other hand, by (17.13) we have 1

112

R, 5 -;logrI i = l

1 + z2(2i-1) (1 2 2 i - 1 ) 2

+

-

1 --logm

1

1 - 2 2

+

I --log-

2 c " 22i-1 +;

m (1 zI2 But by (17.19) we have

-

(1

a=2

R,( -log c) S 2jlog P /

+ z)2 < -

1 - 22 + 2 -. 23 1 --log---m (I z ) ~ m 1 - z2

+

+2

A}*

(17.27)

222


Because of (16.10), (17.19), and (17.26) it follows that Limz

=

1-8

-- 2 A 0.25. +

c-0

Hence (17.25) follows, and the proof of Theorem 17.2 is complete. For given m,we have by (17.9) and (17.27)

COROLLARY. If the p i are chosen by (16.4), then 1 - 22 @ P ( a ,b, p ) 2 exp [-2zS/(1 - z ) ~ ] (1 zIZ = 60 exp [-2z3/(l - z ) ~ ] . (17.28) Theorems 17.1 and 17.2 show that the Douglas-Rachford met,hod with the parameters (16.4) is much less effective than the Peaceman-Rachford method either with fixed m or with m chosen as a function of c = a/b by (17.19) and (16.20) for the respective methods. The Douglas-Rachford method is inferior to an even greater extent to the Peaceman-Rachford method with the Wachspress parameters for the case where m is allowed to depend on c. This does not necessarily imply, of course, that if optimum parameters were used with each method, the Douglas-Rachford method would be inferior to the Peaceman-Rachford method. However, as stated earlier, for the case a b = a p and m = 1, the Peaceman-Rachford method is definitely better. ~

+

+

+

18. Applications to the Helmholtz Equation

I n this section we apply the results of Sections 4 and 5 to the Dirichlet problem for the modified Helmholtz equation, (18.1) where Go is a nonnegative constant, in the rectangle 0 S x S X , 0 5 y 5 Y . As in Section 9, we assume that the mesh size is the same in both coordinate directions, and that for some integers M and N (18.2)

It follows from (9.3)-(9.4) that the eigenvalues p of H I (u/2)1 satisfy the eigenvalues v of V1 = V

+

=

H

+ (u/2)I and (18.3)


where g

=

h2Go.If L a

=

=

223

max ( M , N ) , ? r u + 2 5 p,v 5 4 cos2+ - = b. 2L 2

? F u

4 sin2 2L

(18.4)

Given m, one could determine m iteration parameters for the PeacemanRachford method by (16.4) for the Peaceman-Rachford parameters and by (16.5) for the Wachspress parameters. One would also use (16.4) for the iteration parameters for the Douglas-Rachford method. On the other hand, if one lets m depend on c = a/b, then m can be determined by (16.20) and (16.38) for the Peaceman-Rachford and the Wachspress parameters, respectively, and by (17.19)-(17.20) for the Douglas-Rachford method. We now determine asymptotic formulas for the rates of convergence, with both parameter choices for the Peaceman-Rachford method and for the Douglas-Rachford method as h -+0. Evidently, by (18.4), we have =

where 2 we have

=

($ + %)h2 + O(h4),

(18.5)

max (X, Y ) .By Theorems 16.1, 16.2, 16.4, 16.5, 17.1, and 17.2,

THEOREM 18.1. For the Dirichlet problem for the modified Helmholtz equation (18.1) in the rectangle 0 5 x 5 X,0 S y 5 Y let RE), RAW, and RZ) denote respectively the average rates of convergence of the Peaceman-Rachford method with the Peaceman-Rachford parameters (16.4), the PeacemanRachford method with the Wachspress parameters (16.5), and the DouglasRachford method with the parameters of (16.4). Then as h + 0 ,

i m RZ) 2

4

- ('h)l/m

+ O(h!Vm),

8 (Kh)l/(m-l)+ O(h2/("')) RAW 2 m (18.6)

+

where K 2 = (7r2/4Z2) Go/8. I f the number m of iterations in each case i s chosen by (16.20), (16.38), and (17.19) respectively, then for any 7 > 0 and for sufiiently small h we have

224


PART 111: COMPARISON WITH SUCCESSIVE OVERRELAXATION VARIANTS 19. The Point SOR Method

For the solution of the matrix equation

+ +

Au = ( H V Z)U = k , introduced in Section 2, the n x n matrix A is, by its construction, real, symmetric, and positive definite. We now split the matrix A into A = D - E - E', (19.1) where D is a real diagonal matrix, and E' and E' are respectively strictly lower and upper triangular matrices.22Since A is positive definite, then D is a diagonal matrix with real positive diagonal entries, and is thus also positive definite. It is convenient to denote the strictly lower and upper triangular matrices D-I E and D-I E' respectively by L and U. Thus,

A

D ( I - L - V). (19.2) The point successive overrelaxation (SOR) method of Young [26] and Frankel [lo]is defined by =

+

+

(19.3) ( D - wE)u,+1 = [ ( I - w)D WE'>Un wlc, where uo is again some initial vector approximation of the unique solution of (19.1). The quantity w in (19.3) is called the relaxation factor. Since D - w E is triangular, this procedure is easily carried out and easily programmed. It is convenient to write (19.3) equivalently as

+ ( D - wE)-'l 1. I n this case, i t can be verified that A{D-l(E

where a

=

minA(C,)

- cos (a/N) + E')} = A(B) = 4 4cos+(a/N) Goh2 1 + a/4

(20.1)

Goh2.Thus, from Theorem 19.2, =

h(C,,) =

Wb

-1

0

cos (a/N)

y

+ a/4 + [(l + a/4)2 - cos2(a/N)211/2 (20.2) For the Peaceman-Rachford method, the matrix 2 in this special case is al. The eigenvalues of the matrices H1 H + (l/2)2 and V+ (1/2)2 can be conveniently calculated from (8.3)-(8.4), and all lie in the interval 4 sin2 (a/2N) + a / 2 5 z 5 4 cos2 (7r/2N) + a/2. Appealing to =

{l

=

V1 =

Theorem 8.1, we conclude for this problem that

where

6 - 4 sin2 ( a / 2 N ) 6 4 sin2 ( s / 2 N ) a/2

(20.3) + j3 = ((4 sin2 ( a / 2 N ) + u/2)(4 cos2 (a/2N) + ~ / 2 ) ) l / ~ . (20.4)

minA(T,) P

>o

=

A(T;)

=

+

But it can be verified that the expressions in (20.2) and (20.3) are identical. Thus we obtain the resultz4 THEOREM 20.1. For the Helmholtz equation in the unit square, the optimized one parameter Peaceman-Rachford method and the optimized point SOR method have identical asymptotic rates of convergence for all h > 0.

We point out, however, that numerical requirements for these two methods are different, since the Peaceman-Rachford method requires roughly twice as much arithmetic computations per mesh sweep as does the point SOR method. This will be discussed more in detail in Part IV. We now consider the asymptotic convergence rates of these methods for small h = 1/N. For the point successive overrelaxation method, it follows from (19.9) and (20.1) that

R(T;) = R(C,,) *4

N

2(a2

+ Go)'12h,h -+ 0 ,

In Varga [I71 only the special case GO = 0 was considered.

(20.5)


227

whereas for the point Gauss-Seidel method, the special case w = 1 of (19.3) kzq for pmposes of comparison, an asymptotic rate of convergence

R(&)

- ( + 2) 1r2

h2, h -+0.

(20.6)

Young and Ehrlich [29] have extended the analysis for Laplace's equation (Go = 0) of a single optimized parameter Peaceman-Rachford method to that of a fixed number m 2 1 of optimized parameters used cyclically, and they showed that (20.7)

See also Section 18. If, however, the number of parameters m is allowed t o change as a function of the mesh spacing h, it can be shown (Section 18) that (20.8)

for all h sufficiently small. These results for the Helmholtz equation for the square rest firmly on the fact that the matrices H and V possess a common basis of orthonormal eigenvectors. But for such problems, the results of (20.5) and (20.8) show that m-parameter AD1 methods are superior for m > 1, in terms of asymptotic rates of convergence, to point SOR methods for all sufficiently small mesh spacings h.

21. Block and Multiline SOR Variants

Several extensions of the results of Section 19 are of practical and theore€icaI interest. First, Ostrowski's Theorem 19.1 permits the use of nondiagonal matrices D. This, however, means that the corresponding SOR method of (19.3) requires the direct solution of nondiagonal matrix equations, like those first introduced in the definition of AD1 methods in (3.3)-(3.4). Second, Young's Theorem 19.2 can be similarly rigorously extended to the case where D is not diagonal, and the corresponding method is called the block or multiline XOR method. One can also show, for irreducible Stieltjes matrices A , that the asymptotic rate of convergence is increased as one passes from point to block or multiline SOR methods, which makes these extensions of practical value. See Varga [19, 201 and references given there. It is relevant to point out that multiline SOR methods are theoretically a special case of block SOR methods, but in actual practical computations, the entries of a block correspond to the mesh points of k adjacent horizontal

228


(or vertical) mesh lines, hence the name k-line SOR. Parter [I51 shows that the rate of convergence of k-line Jacobi method for Laplace’s equation in a rectangle is k (21.1) R(B‘”) 5 X2h2,h + 0

-

where X2 is the minimum eigenvalue of the Helmholtz equation vzp

+

xzp =

(21.2)

0.

Theorem 19.2 can be applied, and we conclude from (19.9) that

R(&p’)

-

2& Ah, h -+ 0.

(21.3)

Thus, increasing the number k of lines in SOR methods yields improved asymptotic rates of convergence, but these asymptotic results are always O(h), as h + 0, in contrast with AD1 methods which have asymptotic convergence rates O(hllm)for this problem. See Section 18. Moreover, the arithmetic requirements2s per mesh point of the multiline SOR methods 6) multiplies and ( k 7) increase linearly with k , in that roughly (k additions are needed per mesh point. These combined observations suggest that k = 1 or k = 2 be used in practical problems. Another generalization of Young’s work is based on the concept of weakly cyclic matrices of index p 2 2, an outgrowth of earlier work by Frobenius. We say that a matrix M is weakly cyclic of index p 2 2 if there exists a permutation matrix P for which P M PT has the form

+

+

(21.4)

where the diagonal submatrices are square and null. If we assume that thc U is of this form, then we can state [21] matrix B = L

+

THEOREM 21.1. Let B = L

+

U be weakly cyclic of index p 2 2. If the has nonnegative real eigenvalues less than unity, then (21.5) A(&J > A(&,) = (W - l ) ( p - I), 0 # WE,, where W b i s the unique positive root less than p / ( p - 1) of AP(B)w~” = p”(p - l ) ‘ - P ( ~ b - 1). (21.6) Moreover, R(&) ( 2 p 2 / ( p - 1))1’2{(R(B)}1~2 as h ( B ) increases to unity. matrix

B p

-

25 See [ZO],where some representative arithmetic requirements are given for SOR variants.


229

The case p = 2 is originally due to Young [26]. Other extensions of Young’s work are worth mentioning. First, it is apparent that the successive overrelaxation method is basically a one-parameter iterative method, in that one selects a single optimum relaxation factor. In generalizing this to iterative methods using a sequence of wi’s, Golub and Varga [ I l l make use of a familiar idea of considering Chebyshev polynomials26 in the matrix B. It is shown that use of optimum relaxation factors wi’s is always superior for any number of iterations to the original successive overrelaxation method of Young and Frankel, but asymptotic convergence rates are unaffected. This superiority has been confirmed in numerical experiments (See Ref. [ I l l ) . In Part IV, these improved SOR variants are compared numerically with AD1 methods.

22. Analogies of AD1 with SOR

The theory of successive overrelaxation for weakly cyclic matrices of index p 2 2 can be applied to AD1 methods. First, we write the equations (3.1)-(3.2), leading to the definition of the Peaceman-Rachford method, in the form (with 6’ = 1 - 6 ) : (22.1) u = ( H 68 p I ) - l { k - (V 6’Z - p l ) u } ,

+ + + u = (v + ez + pi)-l{k - ( H + 6’8 - p l ) ~ } .

(22.2)

Dealing with column rectors with 2n componcnts and 2n X 2n matrices, this can be written as (22.3)

where

c, = ( H + 6 8 + pi)-’(pi - 6’8 - IT); D, = (v + ex + pi)-ypi- e’z - H ) .

(22.4)

The 2n x 2n matrix of (22.4) is thus weakly cyclic of index 2, and applying the successive overrelaxation method with w = 1 (called the Gauss-Seidel or single-step method), we obtain

,,p+s

=

C,I,@)

+ gl;

+

up+’) = D p u(“+’) 1 g2.

(22.5)

Except for notation, this is equivalent to (3.3)-(3.4) for a single fixed acceleration parameter p. Similarly, it can be shown [ I 8 1 that the Peaceman-Rachford method, with nz parameters pz used cyclically, is just the successive overrelaxation 26 The use of Chebyshev polynomials in problems of numerical analysis goes back to Flanders and Shortley [ I & ] .

230


method with w = 1 applied to a 2mn x 2mn matrix which is weakly cyclic of index 2m. There is another interesting comparison between SOR methods and AD1 methods. Consider the numerical solution of the Helmholtz equation of Section 2 on a uniform mesh h in a rectangle, and suppose that the initial error vector do) is such that all its components are zero, save one (which we assume is positive). Then, one iteration of the Peaceman-Rachford method distributes this error at a single point over the entire mesh. On the N -2 other hand, if the rectangle is M h x Nh, it could take up to M iterations of the point SOR method to accomplish the same task. See [19]. Intuitively, successive overrelaxation seems less attractive from this point of view. A final analogy [19] between these different methods is that both can be thought of as approximations to the time-dependent parabolic partial differential equation

+

dU

- = -G(x,

at

Y)U

a +ax

with prescribed initial conditions. SOR methods can be viewed as explicit approximations to (22.6) in which the relaxation factor w = At plays the role of the time increment from one step to the next. AD1 methods, on the other hand, can be viewed as implicit approximations (like the CrankNicolson method), in which the iteration parameter p = 2/At plays the role of the reciprocal of the time increment.


231

PART IV: NUMERICAL EXPERIMENTS

23. Introduction I n this chapter we describe some numerical experiments which were conducted to test the theoretical predictions of Part I1 on the convergence of the Peaceman-Rachford method. One set of experiments involved the solution of the Dirichlet problem with Laplace’s equation for the regions shown in Fig. 1. These experiments were run at the University of Texas Region I.

Unit square ( h = -1 1 5

Region 11.

1 L 1 1

’ 10 ’ 20 ’ 40 ’ 80 ’ 160 )

4 X 10

Unit square with

4 - square

10

removed from center

Region 111.

Unit square with

1 5

X

1

- square 5

removed from each corner

1 1 Region IV. Unit square with - X - square 2 2 removed from one corner

(h=L 10

Region V.

’

1 20

’

r ’ 180 -’ 1601 , 40

Right isosceles triangle with two equal sides of length unity ( h = -1 1 1 1 1 -1) 5

’ 10 ’ 20 ’

40

’

80

’ 160

232


and are described in Sections 24-26. Another series involving more general differential equations and boundary conditions were conducted at the Gulf Research and Development Company. They are described in Section 27. 24. Experiments with the Dirichlet Problem

For each of the regions shown in Fig. 1 the five-point finite difference equation analog of the Dirichlet problem with Laplace’s equation was solved27 for a number of mesh sizes using the Peaceman-Rachford method and the successive overrelaxation method. I n every case, the boundary values were assumed to vanish; hence both the exact solution of the Dirichlet problem and the finite difference analog vanish identically. The advantage of this choice is that a t each stage the approximate value at a given point is exactly equal to the error at that point. I n each experiment, starting values of unity were assumed at each interior mesh point, and the iterative process was terminated when the approximate values at all mesh points became less than in absolute value. We remark that the term “successive overrelaxation” means point successive overrelasation as distinguished from block overrelaxation (see Section 21). For the Peaceman-Rachford method three choices of iteration parameters were used: the Peaceman-Rachford parameters of (16.4) ; the Wachspress parameters (16.5);and the optimum parameters. The optimum parameters were chosen by a procedure of Wachspress [25] for m = 1, 2, 4 (see Appendix B). For m = 3 the determination was made numerically on the computer using a successive approximation procedure. One, two, three, and four Peaceman-Rachford parameters and optimum parameters were used, while two, three, four, and five Wachspress parameters were used. Mesh sizes of h = 1/5,1/10,1/20,1,40, 1/80, l/l20, and 1/160 were used,2* though not all mesh sizes were used for every region or every parameter choice. Table I lists the numerical values of the parameters used. For the successive overrelaxation method the optimum relaxation factors were determined analytically for the square region using (19.8) and (20.1) and empirically for other regions to within f0.01, and are given in Table

*’

The following computing machines were used: the Control Data 1604 computers a t the University of Texas, a t the Control Data Corporation, Minneapolis, Minnesota, and a t the National Bureau of Standards, Boulder, Colorado; the IBM 704 and 709 computers a t Texas Agricultural and Mechanical College. The work of L. W. Ehrlich and W. P. Cash of the University of Texas Computation Center is gratefully acknowledged. 28 I n previous experiments by Young and Ehrlich [29],one, two, three, and four Peaceman-Rachford parameters were used with mesh sizes of h = 1/5, 1/10, 1/20, and 1/40.


233

11. Because of the large amount of machine time which would have been required, the mesh sizes of h = 1/160 and h = 1/120 were not used, and h = 1/80 was used only with the square. Tables I1 and I11 give values of N ; for the Peaceman-Rachford method for the values of h and m indicated. Here N ; refers to an actual or estimated number of iterations, and a and p have the following meanings:

1

P Peaceman-Rachford parameters

a = W Wachspress parameters

B optimum parameters

’

=

I

o observed number of iterations v “virtual” number of iterations, as defined below t predicted number of iterations, as defined below c predicted number of iterations, as defined below

For a given m, the virtual number of iterations Nua was determined by Nya = log 106/-1og ha where A“ is the estimated mean spectral radius found by estimating the limiting value of the quantit,iesjn = for n = m 1, m 2, . . , where en denotes the maximum absolute value of the approximate solution (and hence in this case of the error) after n iterations. In the case of the square, the matrices H and V have a common basis of eigenvectors. As long as in the expansion of eo in terms of the common basis of eigenvectors the component of the vector v associated with the largest eigenvalue of nT= T p ,does not vanish, then j,, will approach the mth root of the spectral radius of IIT=1 T,,,.

+

m-,

+

TABLEI

ITERATION PARAMETERS h-1

m

5 5

1 2

5

3

5

4

5

5

Peaceman-Rachford 1.1755705 0.67009548 2.0623419 0.55560485 1.1755705 2.4873180 0.50591866 0.88754970 1.5570576 2.7315972 -

-

Wachspress 0.38196601 3.6180340 0.38196601 I .1755705 3.6180340 0.38196601 0.80817863 1.7099760 3.6180340 0.38196601 0.67009550 1.1755705 2.0623419

Optimum 1.1755705 0.54887621 2.5178099 0.45359594 1.1755705 3.0466903 0.42174787 0.78715591 1.7556445 3.2767586

234


TABLE I. (Continued) h-1

m

Peaceman-Rachford

10 10

1 2

10

3

10

4

10

5

20 20

1 2

20

3

20

4

20

5

0.61803400 0.24596235 1.1529451 0.18092034 0.61803400 2.1112388 0.15516607 0.38988857 0.97967999 2.4616595

-

0.31319083 0.087907193 1.1158188 0.057556742 0.31319083 1.7042051 0.046572773 0.16592687 0.59115497 2.1061338 -

-

40 40

1 2

40

3

40

4

40

5

0.15695853 0 031115900 0.79174897 0.018143240 0.15695853 1.3578601 0.013854188 0.069884949 0.35252200 1.7782335 -

--

Wachspress 3.6180340 0.097886967 3.9021131 0.097886967 0.61803399 3.9021131 0.097886967 0.33438737 1.1422860 3.9021131 0.097886967 0.24596234 0.61803399 I .5529451 3.9021131

-

0.024623319 3.9753768 0.024623319 0.31286893 3.9753768 0.024623319 0.13407789 0.73007542 3.9753768 0.024623319 0.087771701 0.31286893 1.1152452 3.9753768 0.0061653325 3.9938348 0.0061653325 0.15691819 3.9938348 0.0061653325 0.053345898 0.46157849 3.9938348 0.0061653325 0.031103904 0.15691819 0.79164722 3.9938348

Optimum

0.61803400 0.18760957 2.0359623 0.13497175 0.61803400 2.8299802 0.11821609 0.33457893 1.1416320 3.2310830 -

-

0.31319083 0.024623319 3.9753767 0.040456047 0.31319083 2.4245772 0.033125729 0.14039288 0.69723598 2.9550133 -

-

0.15695853 0.022434444 1.0981320 0.012261962 0.15695853 2.0091540 0.0093924997 0.058924690 0.41809268 2.6229420 -

-


235

TABLEI. (Continued) m

Peaceman-Rac hford

80 80

1 2

80

3

80

4

0.078519631 0.011003253 0.56031907 0.0057152520 0.078519631 1.0787508 0.0041190070 0.029393390 0.20975235 1.4968007

80

5

h-1

-

-

120 120

1 2

120

3

120

4

120

5

0.052353896 0.0059900539 0.45758027 0.0029079793 0.052353897 0.94255504 0.0020261500 0.017708830 0.15477761 1.3527777

-

160 I60

1 2

160

3

160

4

160

5

0.039267386 0.0038908000 0.39630090 0.0018004230 0,039267385 0.85642517 0.0012247357 0.012360483 0.12474654 1.2589880

-

-

Wachspress

Optimum

-

0.078519 631 0.0078568620 0.78470673 0.0037654768 0.078520642 1.6373638 0.0026918638 0.024740608 0.249 19891 2.2903583

0.0015419275 3.9984583 0.0015419275 0.078519632 3.9984582 0.0015419275 0.021183944 0.29103799 3.9984582 0.0015419275 0.011003253 0.078519632 0.56031907 3.9984582

-

0.00068535005 3.9993148 0.00068535005 0.052353897 3.9993148 0.00068535005 0.012338721 0.22214056 3.9993148 0.00068535005 0.0059900539 0.052353897 0.45758027 3.9993148 0.00038551904 3.9996147 0.00038651904 0.039267386 3.9996147 0.00038551904 0.0084082046 0.18338369 3.9996147 0.00038551904 0.0038908000 0.039267385 0.39630090 3.9096147

-

0.052353896 0.0042633301 0.64290834 0.0018960170 0.052354899 1.4456572 0.0013022248 0.014899126 0.18396586 2.1048059 0.039267386 0.0027647161 0.55771640 0.0011669595 0.039267903 1.3213529 0.00077925470 0.010397443 0.14829872 1.9787209

-

-

-

TABLE 11. PREDICTED AND OBSERVED NUMBERSOF ITERATIONS FOR THE PEACEMAN-RACHFORD METHOD AND THE SUCCESSIVE OVERRELAXATION METHOD" Region I

Region I1

Region I

m

h-1

5

10 h)

w

0.

20

40

1 2 3 4 5 SOR

11 8 8 8 -

1 2 3 4 5 SOR

22 14 13 12 -

1 2 3 4 5 SOR

44 22 19 17 -

1 2

87 33 26 24 -

3

4 5 SOR

10 7 6 6

10 7 6 6

-

-

22

22 11

11

10

9

9

44 17 14 13

88

25 18 16

9

_

44 16 13 12

_

86 24 17 15

-

10 11 12 14

_

22 16 17 19

_

44 24 23 24

_

86 34 29 29

_

10 8 8 9 22 12 11 12

44 18 15 15

86 26 19.5 18

12 10 9 8 12

5 5 5 5 (w

12 11 8 9 8 8 8 8 = 1.27)(Nt = 15)

23 16 15 15

-

18 8 9 7

28

(w =

46 24 21 20

-

37 14 11 11

53

(w =

91 36 27 27

-

(w

11

22 14 13 12

1.54)(Nt = 32) 46 18 15 12 1.74)(Nt

91 26 18 18 = 1.86)(Nt

73 22 15 14

117

23 12 12

44 22 19 17

=

11

7 3 4 5

7 6 6

20 9 9 6 40 16 12 11

22 11 9 9 -

44 18 13 12 -

66)

85 33 26 24

=

-

136)

85 23 17 14

85 25 18 15 -

_

-

_

_

-

-

_ _

-

16 13

14 11 12 12

11

10 17 37 20 17 16 38 75 28 26 21 70

(w

-

-

-

-

-

-

-

-

-

-

37 22 17 15 = 1.57)(Nt

72 23 20 19

75 26 27 19 = 1.75)(Nt

-

-

19 12 16 11 8 9 12 10 10 10 = 1.25)(Nt = 14)

-

(w

-

16 15

35 17 15 16 (w

-

-

19 13 9 11 -

42 18 17 15 = 35)

39 16 14 12

11 -

85 30 23 20.5 = 60)

79 22 17 15

85 30 25 14 -

42 17 16

80

I 2 3 4 5

175 48 35 30

175 36 24 2"

175 34 23 19

_

175 49 38 35

_

175 37 25 22

SOR 120 h)

w

v 160

1 2 3 4 5

264 59 41 35

1 2 3 4 5

347 69 46 38

264 45 28 23 347 52 31 25

264 42 26 22

264 60 43 39

264 45 29 24

347 49 29 24

347 70 48 42

347 42 32 26

-

-

-

-

183 49 37 31 236 274 61 44 38

_ 71 47 39

-

146 32 21 18 (O

166 36 25 - - 17 = 1.93)(N; = 292)

166 34 24 19

155 43 34 34

269 59 40 34

46 25 21

269

237 53 41 40

70 46 39

50 31 22

41 23 19

_ -

47 27 22

183 36 24 20

166 48 34 31

274 30 24

33 27 -

- - - _

-

27 22 -

29 21

63 47 39

145 37 26 24

-

155 37 36 30

166 46 31 21

-

166 31 20 25

237 -

269 50 37 33

-

-

- -

-

-

74 42 34

For explanation of ATB";

LY

=

P , W , B ; p = t, c,

0,v;

and N , see Section 24.

-

-

_

-

_

~~

a

166 33 31 25

~~

TABLE 111. OBSERVED NUMBERS OF ITERATIONS FOR THE PEACEMAN-RACHFORD METHODAND THE SUCCESSIVE OVERRELAXATION METHOD^ Region I V

Region I11 h-’

m

NOP

N,R

__

~

5

1 2 3 4

5 SOR 10 h)

0 W

1 2 3 4 5

SOR 20

1 2 3 4

5 SOR 40

1 2 3 4 5

SOR

8 8 9 10 11 19 13 12 15 26 36 20 16 16 51 75 30 22 21 108

Region V

9 11 10 c (w =

18 15 16 16 (w =

36 18 20 22 (w = -

80 23 23 27 (w =

.-

-

8 7 8 8 8 10 9 10 10 8 9 J - 8 1.21)(Nt = 13)

7 7 9 9 -

18 13 12 12 1.50)(A7t= 28)

17 13 14 14

18

37 16 15 19 1.71)(Nt

37 17 17 19

36 15 14 16 -

37 18 17 19 41

85 19 18 - 24 1.85)(N, = 126)

74 23 16 14 -

75 28 23 23 85

19 12 14 15

75 23 19 19

36 19 15 12 ~

=

11 12 14

-

58)

74 28 21 19

-

17 12 14 13 20

20 14 13 12 (w = -

35 18 19 19 (w =

73 26 22 25 (w =

17 13 14 13 1.41)(Nt

19 12 11 11 -

=

19 12 11 10

-

79 30 24 18 = 96)

85 23 16 22

75 26 21 24 l.Sl)(iVt

30 17 15 17

7 8

7

19 14 12 11 -

16 11 11 11 17

42 16 15 16 -

33 17 15 15 41

22)

42 20 15 16 1.65)(Nt = 46)

37 17 18 19

8 7

79 24 18 20 -

67 25 19 19 76

9 8 9 8 (w =

19 13 13 13 (w =

8 8 8 8 -

8 7 7 7

(w

20 11 11 11

19 10 11 10 -

39 19 14 13 = 38)

42 15 16 15

39 15 12 15 -

79 29 22 18 = 81)

85 24 19 19

79 22 18 16 -

16 11 11 12

19 11 9 10 1.36)(Nt = 19)

67 22 17 19 = 1.78)(Nt

80 23 19 20

7 -

r

l.lO)(NL = 9)

33 16 14 15 = 1.60)(Nt

-

c

9 7 7 7

-

41 17 17 17 (w

8

-

80

120

160

1 2 3 4 5

150 40 32 27

1 2

226 49 38 32

1 2 3 4

57 42 35

150 34 25 31

150 32 24 24

146 46 31 27

-

166 27 22 27

223 59 34 31

146 31 22 22

162 43 34 28

223

232 53 40 34

-

-

63 45 38

65 40 34

5 0

For explanation of Ng"; a = P, W , B ; fl

=

t , c,

0, v; and

149 35 27 29

162 38 24 28

166 44 32 29

232 -

223 53 37 32

65 40 37 -

N , see Section 24.

166 31 22 27

166 33 20 20 -

136

-

38

160 34 23 24

223

205 46 32 28

-

27

24

54 36 31

136 32 23 23

166 42 31 28

205

223 53 36 32

-

62 42 36

166 33 21 22

166 32 21 22 223 -

2 40


TABLEI V PREDICTED AND OBSERVED NUMBERSOF ITERATIONS USING PEACEMAN-R ACHFORD AND WACHSPRESS PARAMETERS Region I1

Region I m

Nf

N,”

N,‘

N,”

1 2 3 4 5 6 7 8 9 10

82 33 26 23 22 21 21 20 20 20

-

82 34 29 29 30 31 33 34 36

82 26 20 18 18 18 18 17 20

82 25 18 16 15 15 15 15 15

38

NOP N F N,’

N,”

-

-

91 36 27 27 25 24 26 24 26 27

73 22 15 14 14 11 13 14 11

85 33 26 24 22 21 21 20 20 20

N f N,“

85 23 17 14 14 9 10 10 10

75 28 26 21 21 22 20 25 24 26

72 23 20 19 21 23 25 26 26

NF N:V 85 30 23 20 20 18 16 20 22 23

79 22 17 15 18 19 20 21 24

Mesh size: h = 1/40 (For explanation of symbols see Section 24.)

The predicted number of iterations Nta was determined by Nta = log is defined by (16.7) and equals, for the case of the square, 2 R,, = - - log @,,,(u,b, p ) . (24.1)

106/Emwhere

z,,,

m

I n each case the function 4m(u,b, p ) was evaluated numerically to at least four decimal places of accuracy on the computer. The predicted number of iterations, Nca, was determined by N C P= log 106/E(2 and New = log 106/RLm where Eg) and ELm are lower bounds for Ernfor the Peaceman-Rachford parameters and for the Wachspress parameters respectively which are given respectively by (16.14) and (16.36). For the successive overrelaxation method the observed numbers of iterations are given in Tables I1 and I11 on rows labeled “SOR” and in columns headed “Nop.”The corresponding values of w are also indicated. The predicted number of iterations N t was determined by solving the equationz9 4Nt(w - 1)Ns-1 = (24.2) Table I V gives predicted, virtual, and observed numbers of iterations *9 Young [25b] showed that the number of iterations needed to reduce the 1 2 norm of a n initial error vector by factor a, did not exceed n, where 5n(w - l)n-l = a provided o is the optimum relaxation factor. Varga [fg] showed that this equation could be replaced by m ( ~ 1)n-l = 01 for some constant, v, which can be shown to be less than 4.


24 1

using the Peaceman-Rachford parameters and the Wachspress parameters with up to 10 parameters for the Regions I and I1 and for h = 1/40. Figures 2-6 show graphs, with logarithmic scales, of the observed number of iterations versus h-l for the successive overrelaxation method and for

REGION

I

I

5

10

I

20

I

80 I

120 I 1 160

40 h-l

FIG.2.

the Peaceman-Rachford method with the Peaceman-Rachford parameters and with the Wachspress parameters. Reciprocal slopes of straight lines fitted to the data points in each case are given in Table V. These slopes are also given for the case of the optimum parameters, though the corresponding graphs are not included in Figs. 2-6.

242


25. Analysis of Results

For the case of the square, the numbers of iterations Nta for the Peaceman-Rachford method predicted by the theory of Part I1 agree closely with the observed values Noa. I n fact, for m > 1 t,he values of N p differ

REGION

II:

AD1 PARAMETERS PEACEMAN- WACHSPRESS RACHFORD

SOR ---Q-..

-----

~

V 0

m = 3 V m = 4 +

m.5 I

10

I

20

I

40

I

80

I

m

I

120 I60

Frc. 3.

from the corresponding iVoa by at most five iterations and usually by much less. The agreement is especially good in view of the fact that changing the order in which the p i are used sometimes changes the number of iterations by two or three.


243

The close agreement is to be expected since by (16.6) the actual rate of convergence R, is given by

and since, as noted in Section 15, the inequality (15.5) becomes an equality

m

REGION

30

\/o

10

I

0

A V 0 1

I

I

5

10

20

I

1

40 h-l FIG.

80

I

m=

1 m = 2

m = 3

A

' I

m = 4 4

m=

5

m

I

120 160

4.

in the case of the Helmholtz equation. Thus, the only difference between as given by (16.7) and the actual rate of Convergence R, lies in the approximation of

zm

244


REGION

3O(

lo( *+% *\53

*\A

AD1 PARAMETERS PEACEMAN- WACHSPRESS RACHFOR D

I( SOR

------

-it-

0

I

I

10

20

I

I

40 h-l FIG.

80

f

A

m= I m = 2

o

m a 3 m = 4

m.5

A V + m

I

120 160

5.

where y is any eigenvalue of H or V , by &(u, b, p), where

But for small h by (9.2) there will be a large number of eigenvalues of H and V distributed over the interval a 6 y 5 b ; hence the error in the above approximation will be slight. Since for each parameter choice @,(a, b, p ) was evaluated on the computer to a t least four decimal places of accuracy, the discrepancies between the Nta and the N g are primarily due to roundoff. It is expected that closer


REGION

245

'X

SOR

--

AD1 PARAMETERS PEACEMANWACHSPRESS RACHFORO

-----

7 -

0

A

v o

I

I

5

LO

I

I

20

40

I

I

m = l m = 2 A m:3 V m=4 m.5

I

80 I20 I60

h-1

FIG.

6.

agreement would result if a tighter convergence criterion were used, thus minimizing the influence of the particular initial error vector which was present. In support of this, we note that the virtual numbers of iterations Nva agree much closer with the predicted numbers of iterations Ntathan do the observed numbers of iterations No", especially in the case of the Peaceman-Rachford parameters. Thus the actual rate of convergence as measured by the Nva agrees closely with the predicted rate of convergence as measured by Nta. For regions other than the square, the predictions of the numbers of iterations based on the theory of Part I1 are no longer valid. The observed

TABLE V. RECIPROCALS OF SLOPESOF LINES REPRESENTING LOG N ~

~

~

I

Parameters

P

W I1 I11 N

P

0.

IV

v

B P W B P W B P W B P W B

m

=

1

1.05 1.05 0.96 0.96 1.00 1.00 0.96 -

0.93 1.00

-

1.00

m =2

m =3

1.86 1.00 1.80 1.75 0.90 2.27 1.86 1.00 2.05 1.70 0.98 1.82 1.75 1.00 1.97

2.38 1.60 2.67 2.05 1.77 1.84 2.19 2.65 3.49 2.25 2.14 4.32 2.33 2.07 2.86

m

Reciprocal slopes times m5 =

4

2.95 2.54 2.90 2.14 2.62 2.41 2.75 3.09 4.93 2.79 3.55 3.38 2.71 3.80 3.24

m =5

m = l

m = 2

m = 3

m = 4

3.08

1.05 1.05 0.96 0.96 1.00

0.93 1.00 0.90 0.88 0.90 1.13 0.93 1.00 1.02 0.85 0.98 0.92 0.88 1.00 0.98

0.79 0.80 0.89 0.68 0.89 0.61 0.73 1.33 1.16 0.75 1.07 1.43 0.78 1.03 0.95

0.74 0.83 0.72 0.54 0.87 0.60 0.69 1.03 1.23 0.70 1.18 0.84 0.68 1.27 0.81

-

3.32 3.18 3.27

-

1.00 0.96 0.93 1.00 1.00

c

3.32

-

m = number of parameters P: Peaceman-Rachford parameters W : Wachspress parameters B : Optimum parameters 0

LOG h-l ~

Reciprocal slopes Region

VERSUS

~~

For Wachspress parameters, the reciprocal slopes were multiplied by (m

- 1).

m = 5


247

number of iterations is seldom more than, and never more than twice, that for the square. In the case of the successive overrelaxation method, it can be proved30that rate of convergence for a given region is at most that for any region which includes the given region. For the Peaceman-Rachford method with one parameter, it was shown in Sections 10 and 11 that if a region can be embedded in a rectangle, then the rate of convergence of the Peaceman-Rachford method for the rectangle using the best value of p is at most that using the same p for the given region. Since for m = 1 the optimum p was used for the square for each mesh size, this result is applicable here, and is confirmed by the numerical results. For m > 1 the following conjecture is offered:

For a region which can be embedded in a rectangle, the rate of convergence of the Peaceman-Rachford method for a given set of iteration parameters is a t least 0 times that for the rectangle where e is a constant such that 1 2 8 2 1/2. As a consequence of the agreement between the numbers of iterations as predicted by the theory of Part I1 and the actual number, it follows that the Peaceman-Rachford method is extremely effective. Thus, from Theorem 18.1 it follows that for fixed m, the number of iterations is O(h-1'm),31 and that if a good value of m is used,32then the number of iterations is O(J1oghl). Consequently, one would expect that, asymptotically for small h, log NOa would be a linear function of log h-l with slope l l m for the PeacemanRachford parameters and with slope l / ( m - 1) for the Wachspress parameters. Inspection of the graphs of Figs. 2-6 reveals that the observed data points do indeed lie roughly on straight lines. Moreover, as indicated by Table V, the slopes of the lines are close to the predicted values for small m,especially for the square. For other regions where the theory of Part I1 does not apply and for larger m, the agreement is not as close. The discrepancy for the larger m may be explained by the fact that the quantity (a/b)l/2* = ( 7 r h / 2 ) l / m , which is assumed to be small in the derivation of the asymptotic formulas (18.6), is actually rather large for m = 4 even for h as small as 1/160. I n this case the value is 0.315. Presumably, the actual slopes would be closer to the predicted slopes if much smaller values of h were used. Although the values of h used were not small enough to test whether Noa is O(/loghl) if a suitable value of m is used for each h, nevertheless, there See Young [25b]. For the Wachspress parameters this would be O ( / L - ~ / ( ~ - ~ ) ) . Determined by (16.20) and (16.38) for the Peaceman-Rachford parameters and for the Wachspress parameters, respectivaly. 30

31

248


seems no reason to doubt its validity. In any case, both Nod and N P increased very slowly as h decreased; for example, even with h = 1/160, only twenty-two iterations were required using five Wachspress parameters. The main increase in computer time as h decreased was simply due to the presence of more mesh points rather than to the increase in the number of iterations. In comparing the effectiveness of the Peaceman-Rachford method with that of the successive overrelaxation method, one must remember that twice as much machine time was required per iteration with the PeacemanRachford method as with the successive overrelaxation method. For the case m = 1, it was shown in Part I11 that the spectral radii of the two methods are identical for the square provided that the optimum iteration parameters are used in each case. However, since the Jordan normal form of the matrix corresponding to the successive overrelaxation method is not diagonal, the number of iterations is somewhat larger. For this reason, although the spectral radius of the method using the optimum relaxation factor Wb is (wb - l ) , the predicted number of iterations is determined by (24.2). This yields a larger value than if N t had been determined by the usual formula (ub - l)N1 = lo+, which would be valid if the Jordan normal form of the corresponding matrix were diagonal. While the number of iterations for the successive overrelaxation method is slightly larger than for the Peaceman-Rachford method for m = 1, nevertheless, because only half as much time is required per iteration, the successive overrelaxation method is definitely superior to the PeacemanRachford method with one parameter. However, since the number of iterations with the successive overrelaxation method is asymptotically proportional to (2?rh)-’ as compared to (m/4) (2/rh)”m for the PeacemanRachford method with the Peaceman-Rachford parameters, the superiority of the latter method form > 1 is evident. This superiority is amply reflected in the Tables I1 and I11 and in Figs. 2-6, not only for the square but for the other regions as well. Estimating the number of iterations for the successive overrelaxation method as five hundred seventy for the case h = 1/160 and comparing with twenty-two iterations required using the Peaceman-Rachford method with five Wachspress parameters, the latter method is faster by a factor of nearly thirteen to one. We now consider the choice of iteration parameters. Theorems 16.2 and 16.5 indicate that the Wachspress parameters are superior to the PeacemanRachford parameters provided one chooses good values of m by (16.20) and (16.38), respectively. The results of Tables I1 and I11 confirm this superiority for the case of the square. There seems little to choose between the two parameter choices for the other regions. The optimum parameters are not appreciably better than the Wachspress parameters. Because of the


249

theoretical superiority of the Wachspress parameters over the PeacemanRachford method and because the Wachspress parameters are easy to determine as compared with the optimum parameters, their use is recommended. Concerning the choice of the number of iteration parameters, m, the values predicted for the square by (16.20) for the Peaceman-Rachford parameters can be estimated from Tables I1 and I V by observing where N P is smallest. This follows since (16.20) was derived by maximizing EF and since Ncp = -log 10-6/-log @,?. In the case of h = 1/40 the smallest value of Ncpoccurs for rn = 3 or 4,whereas the value of m from (16.20) is 4. By (16.38) the predicted optimum value of m for the Wachspress parameters would be 5. The fact that Ncw is smaller for m = 9 than for m = 5 is a reflection of the inexactness of the approximation used in Section 16 to derive (16.38).33 It is to be noted that for h = 1/80, 1/120, and 1/160, the values of n determined by (16.20) would be 5, 5, and 6, respectively, and those determined by (16.38) would be 6, 6, and 7 respectively. Such values of m were not used, but if they had been, presumably fewer iterations would have been required. Returning now to the case h = 1/40, based on the observed values Nop and N o r , it appears that it would have been better to use larger values of m, say between 13 and 2 times those indicated by (16.20) and (16.38). I n support of this, we note that N t P and Nrw appear to decrease for all m up to 10. Moreover, even with N P and NCwthere is only a small increase for values of m larger than those given by (16.20) and (16.38) respectively. Consequently, it seems safer to use a value of m which is slightly too large than to use one which is too small. 26. Conclusions

The following conclusions and recommendations summarize the results of the preceding experiments; they seem reliable at least for the Laplace equation with given boundary values (the Dirichlet problem) and a square mesh. (1) The rate of convergence of the Peaceman-Rachford method is accurately predicted by the theory of Part 11. (2) For each of the other regions which were embedded in the square, the number of iterations required was usually less than and never 13 On the other hand, we note that Ncw agrees more closely with NtW than NCpdoes with N P . This is as expected because the bound (B.15) of @,,,(a, b, p ) for the Wachspress parameters took into account two factors of (16.3) while the corresponding bound (B.14) for the Peaceman-Rachford parameters uses only one factor.

250


more than twice that required for the square. It is conjectured that this is true in general for any region embedded in any rectangle. (3) The Peaceman-Rachford method is an extremely effective method, and, for small h, is much superior to the successive overrelaxation method. I n fact, by suitable choice of parameters, the number of iterations only increases as llog h(; hence the increase in computer time involved in passing to a smaller mesh size is almost entirely due to the increase in the number of points, and only very slightly due to an increase in the number of iterations. (4) The Wachspress parameters are recommended in preference to the Peaceman-Rachford and to the optimum parameters. Unless other information is available, it is recommended that the number of parameters used be chosen between 13 and 2 times that obtained from (16.38).

27. Experiments Comparing SOR Variants with AD1 Variants

The following is a brief summary of experimental results34obtained on the IBM-704 and 7090 at the Gulf Research Laboratory (Hamarville, Pa.), comparing the latest SOR variant with the Peaceman-Rachford method. The experimental results of the previous sections specifically compared the point SOR method with the Peaceman-Rachford method, and in n o case (Table 111) did the point SOR method with optimum w require fewer iterations than the Peaceman-Rachford method. The situation is however changed when the newer variant of SOR, using the two-line iterative method (Section 21) coupled with the cyclic Chebyshev semi-iterative method (Section 21), is similarly compared. Using the same regions for the Dirichlet problem, the same starting values of unity and the same method for terminating iterations as described in Section 23, the total number of iterations for each method was normalized by the relative amount of arithmetic required by each method per mesh point. Specifically, the arithmetic requirement for the IBM-704 for the following methods were in the proportions Point SOR 2-line cyclic Chebyshev 1.26 7 Peaceman-Rachford 2.05 and the numbers of observed iterations were multiplied by these constants and called normalized iterations ; these normalized iterations are then directly proportional to actual machine time.

{

34

By Harvey S. Price and Richard S. Varga

l"i


25 1

The curves in Figs. 7 and 8 illustrate the basic results of this experimentation. For each mesh spacing, each process was optimized with respect to acceleration parameters. This means that in the case of the 2-line cyclic Chebyshev method, estimates of the spectral radius of the Jacobi matrix were vaned to find fastest convergence. For the Peaceman-Rachford method, the number of parameters, to be used cyclically was similarly varied. From these curves, we see that there is a substantial decrease in iterative time in passing from the point SOR to the 2-line cyclic Chebyshev semi-iterative method. Second, in each of these cases (and in all other cases actually considered) we see that there is a critical value h* of the mesh spacing such that if h > h*, it is better to use the 2-line cyclic Chebyshev iterative method, but for all h < h*, the optimized Peaceman-Rachford 80

Region I

40

20 111

g

.A 42

f-l

.z -0

10

N

.* E M

z0 5

5

10

14

h-' FIG.

7.

20

40

60


252

method is superior in terms of actual machine time. Again, the curves of Figs. 7 and 8 indicate that the Peaceman-Rachford method for small h is vastly superior to any of the SOR variants. These figures also show that there is a great variation in this critical value h* from problem to problem. Also in this experimental program at Gulf were problems of the general form

- (PlUZ(X,

Y>)z

- (P2%(Z,

Y))Y

+

a 4 2 , Y) = f (2,Y), (2,Y)ER,

(27.1)

where R is a bounded connected set with boundary r, subject to boundary conditions of the form (27.2)

Region I1

40

20

10

5

5

10

20

h-' FIG.8.

40

70 80


253

I n particular, cases where PI and PZwere discontinuous (typical of problems occurring in reactor and petroleum engineering) were similarly considered. For one such problem, it was possible to select iwo parameters p1 > pz > 0 such that the spectral radius of the associated PeacemanRachford method was A(TpzTp1) = 13.48. This divergence is complementary to the known convergence of Theorem 5.1 for a fked value of p and should serve to warn the unsuspecting reader of possible divergence in his use of AD1 methods.


254

APPENDIX A THE MINIMAX PROBLEM FOR ONE PARAMETER 1. Peaceman-Rachford Method

In this section, we again examine the minimax function F(a, b ; a, 6 ) defined by (7.8), which arose in connection with the Peaceman-Rachford method. For 0 < a 5 b, 0 < a 5 8, F(a, b ; a,8) is defined as a minimax function by the formulas

d% b, PI

=

nlaxaSSbl(P - P ) / h

and

+ P)i,

(-4.1)

F(a, 6; a,P> = min, @(a,6, P>@(c., P, P ) . (A.2) Clearly C$ < 1 for any p > 0, and C$ > 1 for any p < 0; moreover 4 tends continuously to 1 as p -+ m. Hence the minimum is assumed in (A.2) for at least one finite positive “optimum rho” p*. Again, for fixed p 2 0, the continuous function \ ( p - p ) / ( p p)I is decreasing for 1.1 < p and increasing for p > p ; moreover it has its minimum when p = p . Hence its maximum value occurs at p = a or p = b. Comparing the values there, we obtain if o 4 p 5 dZ, (b - p ) / ( b p) 44%b, PI = (A.3) (p - a)/(p a) if p 2 4. This completes the determination of 4; it is analytic for all nonnegative p # 6, and continuous everywhere. It is also easy to determine the unique “optimum rho” p* which minimizes +(a, b, p ) . Since In @ is an increasing function of 4, p* is that rho which minimizes In 4. By (A.3),

+

+

i

Hence the optimum rho is

+

6, and =

Since 0 4 4

where d

=

tanh [(ln c)/4],

< 1 for 0 < p < + oc, it follows that

min @/a, P / a ) .

c = b/a.


255

It is also evident that (A.7) F(a, b ; a, PI L [Inin, +(a, b, P ) 1 . [mi% +(a,P, P I 1 1 equality holding if and only if +(a, b, p ) has the same “optimum rho’’ p* as 4(a, 0, p ) . Referring back to (A.5), we obtain the following result. LEMMA A . l . F eatisjies the inquality

equality holding i f and only if ab

COROLLARY A . l . If a

= a

= aP.

and b

=

P, we have

We now try to determine F generally. Since In F is an increasing function of F , we have In F = min, Cln +(a, b, PI In 4(a1P, PI]. Moreover by the remark after (A.3), the sum in brackets is continuous everywhere, and analytic for p # 6, Finally, differentiating (A.4) again, we obtain d2(ln+)/dp2 < 0 for all p # dab. A similar result holds for +(a,0, p ) , and so we get

+

Go.

+(a, b, P )

a.

+ In

(a!,

(A.9)

P, p ) l l d p 2 < 0,

for p # G b , Since a minimum cannot occur where the second derivative is negative, we conclude -

LEMMA A.2. I n all cases, p* = dab or p* = d a p . Substituting back into (A.2) and (A.3), we obtain the following definitive result.

THEOREM A . l . If ab 5 ap, then F ( a , b ; a, p) of the following two numbers:

=

F ( q p ; a, b) i s the smaller

-

The first option occurs if p* = dab; the second if p* = d a p . The following condition, which we mention without proof, states which value of p is optimal. a, then p* THEOREM A.2. Let ab 5 ap. If a b 2 p, then p* = 42.If a 5 a! and b 5 8, then p* p* = &$if ap 5 ab.

= =

d/ab;if a S

and 6 i f ap 2 ab and a!

256


COROLLARY A.l. If

+ b = a + /3 and ab S a@, then p* = 42.

a

Caution. I n Theorem A.2, the value of p* is not necessarily unique. For example, if a@= ab, the two values p* = &6 and p* = are both optimal, though they are in general distinct.

42

2. Douglas-Rachford Method

We now determine the minimax function FD(a, b ; a,P) for the DouglasRachford method, defined on the domain 0 < a 5 b, 0 < a S @ by the formula

+

+

+

pz)/(p p)(v p) < 1 if p > 0 for p, v as specified Clearly, 0 < (pv . the minimum is assumed and it tends to 1 continuously as p + 0 , ~Hence in (A.ll) for some finite positive optimum p = p*. One easily verifies the algebraic identity

I.lv

(Lc

+

+

P)(V

+ P) -- 2-1

1 (cl

Pa

+

- P > ( V - PI.

5 (P +

P>(V

+ PI

(A.12)

On the other hand, from (A.l)-(A.2), using the remarks after (A.2), one can derive the following alternative formula for F :

This will be compared with the following consequence of formulas (A.ll) and (A.12). 1 1 (A.14) FD(a,b; a,PI = 5 5 min,>o +D(u, b; a,P ; PI, where

+

We can compute +D by (A.3). If ab 5

afi and

a

S b, then if0 5 p 5

6,

if-sp.

If a > b, then 4~ is negative, and so FD < 1/2; this case is atypical for elliptic difference equations.

257


When a = a and b = p (for example, if H = V as for the Helmholtz problem in a square), +D(u, b ; a, b ; p) = [+(a, b, p)lZby (A.16) and (A.3). Hence, in this special case, FD = (1 F)/2. In general, one merely has the inequality

+

+

FD(a,b ; a,8) 5 [l F(a, b ; a,P)1/2: (A.17) which is evident if one compares (A.13) with (A.14)-(A.15). A complete discussion involves an elaborate analysis of special cases, and so we merely state a partial result without proof. THEOREM A.3. If ab 5 b Douglas-Rachford method i s duction matrix i s XD(6)

a 6 Gfl then , the optimum rho p ~ for * the the spectral radius of the error re-

dolb, and =

2dorb/(a

+b +2dZ).

(A.18)

if ab 5 a0 5 bp, then

+ a)bp - ( b + b)aal/[(b + PI - ( a + 4)11”,

PO* = { [ ( a

(A.19)

and the spectral radius of the error reduction matrix is

(A.20)

+

For the Helmholtz equation in a rectangle, treated in Section 9, a b p and so b 5 a. Hence (A.19)-(A.20) hold, and so FD > F except in trivial cases. =a

+

3. Parameter Translation

As in section 7, we define

and we define G as the minimax function

G(a, b ; a,P )

=

min,,;; #(a, 6; a,P ; P,

PI,

(A.21)

all for 0 < a 5 b and 0 < (Y 5 p. Since the functions whose extrema are sought are continuous, the existence of # for p > -a and p > -a, and hence that of G, follows by simple compactness arguments. Any pair p*, p* minimizing )I will be called optimal, for the reason stated in Section 7. The function # is closely related to the function 4. Indeed, setting A = ( p - p)/2, p1 = p A, v1 = Y - A, and 7 = ( p p)/2, clearly

+

(P

- P>(v - p ) / h

+ b)(v +

+

P ) = (pi - 7)(vi

- 7)/(pi

+

T)(vi

+

T).

258


Substituting into (A.20), we get

+

4 ( ~ A, b Xow taking the minimax, we get =

+ A;

LY

- A,

- A; 7).

+ A, b + A; a - A, /3 - A). (A.23) We will now calculate this expression. One easily verifies that (a + A) (b + A) = (a + A) (0+ A) if and only if A = (a0 - ab)/(a + b + a + a). With this choice of A, both options in (A.lO) assume the same value. Hence we have THEOREM A.4. For A (ap - ab)/(a + b + a + p), G(u, b ; a,0) 6 F ( u + A, b + A; a - A, /3 - A) G(a, b ; a,p) = minA F ( a

=

=(

(A.24)

It is attractive to speculate that the preceding inequality can be reversed, so that one optimizes the iteration parameters by a translation making ab = ap.


259

APPENDIX B: THE MINIMAX PROBLEM FOR m > 1 PARAMETERS 1. Optimum Parameters

For any 0 functions

0. That is, the value of F,(a, b ) depends only on the ratio b/a, and the positive integer m. An optimum m-vector p* = p*(a, b ; m) for given a, b, and m is defined as a real m-vector which minimizes +,(a, b ; p)-that is, such that &(a, b ; p*) = Fm(a, b). The existence and continuity of +*' for fixed a, b, p is evident since the product on the right of (B.1) is continuous and the domain is compact. The existence of p* then follows since +m(a,b ; p ) is decreased if a negative pz is replaced by -pL, since & < 1 if all pz are positive, and since +m -+ 1 as all pz -+ +.o ; this makes the domain where &(u, b ; p ) 4 1 - e compact, and nonvoid for sufficiently small E > 0. The uniqueness of p*, a more difficult question, is also known. It expresses the fact that the family of rational function expressible as products of the form II[(p - p%)/(p p,)] has the following basic property. Chebyshev Property. For given 0 < a < b and m 2 1, there is a unique optimum m-vector p* with a < pl* < pz* < . . . < pm* < b, such that ?',(a, b) = +,(a, b ; p * ) . This vector is determined by the property that the product IIE1 ( p - p,)/(p pz) in (B.1) assumes its maximum absolute 1 points T ~ ,with value F d a , b), with alternating signs, in exactly m a = TO < PI* < TI < . . . < T,-I < pm* < rm = b.

+

+

+


260

For the proof of the fact that the functions in question have the stated property, the reader is referred to Wachspress [ 2 5 ] . It is closely related to pI) is varithe fact that the family of rational functions II ( p - p i ) / ( p solvent36 (unisolvent of variable degree). The following symmetry property is also very helpful:

+

-

-

03.5) 4m(a, b; U ~ / P I , * * 9 ab/pm)This identity is a corollary of the fact that the correspondence p 4 a b / p maps the interval a 6 p 5 b onto itself, combined with the evident algebraic identity +m(a, b ; PI, *

* 7

Pm) =

- (ab/Pi)l/[(ab/P)

[(ablll)

+ bb/Pdl

= (Pi

- d / ( P * + 4.

From (B.5) and the Chebyshev Property, it follows that P%+1-*

(B.6)

= Ub/P%*.

I n particular, for odd m = 2n - 1, it implies pn* = 4, as was proved for n = 1 by elementary methods in Appendix A. From this Symmetry Property and the Chebyshev Property, it follows that for even m = 2n, r n = 4. As shown by Wachspress [ 2 5 ] , one can use the correspondence p 4 ( p a b / p ) / 2 to establish the following sharper result.

+

THEOREM B.l. For any even positive integer m F2n(a,

b) = + ~ n ( a , b ; PI*, . . . =

&(d/ab, (a

pan) =

+ b)/2;

01*,

=

2n,

Pn(dab, (a

+ b)/2)

(B.7)

. . . , an*).

The optimum an-vector p* is related to the optimum n-vector a* by p;-j+i

= u3*-I- 4/(u3*)' -

so that %* = ( P L t l

ab, pi+I

+ ab/pL+1)/2

- d(u3*)'- ab,

= oj*

= (Pi+,

+ Ub/Pi+,)/2.

(B.8)

For n = 1, the optimum parameter for the interval a 5 p 6 b is 6. Hence the case m = 2 can be explicitly calculated from (B.7) and (B.8) as follows.

COROLLARY B.l. For 0

< a 5 b and m

= 2,

we have

+ b ) 4 ] 1 / 2 f [ ( a + b)V% - 2 ~ b ] ' / ~ (B.9) F ~ ( u b) , = (a + b - [ 2 ( ~+ b ) d Z ] l / z } / { ~ + b + [2(a + b ) 4 a b ] 1 1 2 } . &p,*

= [(a

and so

(B.10) Making repeated use of (B.8), one can explicitly compute the optimum m-vector for m = 2? any power of two. One can also compute F,, using 36

See Rice [f 6 b ] .


26 1

(B.7). Specifically, one first computes the nested sequence of values, tending to the arithmetico-geometric mean of a and b :

a, + bi)/2.

b, ai+l = bi+l With these definitions, we obtain from (B.7) = a, bo =

= (Ui

(B.ll)

1

1.0

.8

.6

m=4

.4

.2

m=8

0

-2

-4

-6

>

log1oa

FIG.9.

When m is not a power of two, the optimum parameters can still be computed effectively using an algorithm of Remes [ I B U ] ? ~This method is described and applied to compute numerical values by de Boor and Rice

Pal

*

a6 See

also Stiefel [16d].

262


2. Good Iteration Parameters

For many purposes, one can approximate F,(a, b) sufficiently well by relatively simple explicit formulas for p. Such choices of parameters may be called good parameters, since they give rates of convergence for AD1 methods not too far from the optimum. A very simple and quite good parameter vector for arbitrary a, b, and m was suggested by Peaceman and Rachford [16]. Their suggestion was to use p i p ) = ak2*--l, where k = ( b / ~ ) " ~ " . (B.13)

THEOREM B.2. For the Peaceman-Rachford parameter vector defined by (B.13), we have the inequality 2m am 4 , ( ~ ,b ; p ( P ) ) (= ( k - l ) / ( k 1) = (1 - d u / b ) / ( l d / a / b ) . (B.14)

+

+

Proof. Let I( be given. Since each factor in (B.l) is less than one in magnitude, it suffices to show that one factor is bounded by ( k - l ) / ( k 1). But either p is in (a, ak), or in (bk-l, b ) , or in some interval ( ~ ~ - 1 ,p J . I n the first two cases,

+

+ 11, 0 5 (b - d / ( b + I( k - l ) / ( k + 1). 0

or

s ( P - U ) / ( P + a ) 5 ( k - 1)/(E P)

In the third case,

Comparing these inequalities, (B.14) follows immediately. ~~ Wachspress [23, 241 has pointed out that a better t h e ~ r e t i c a lupper bound is given, for all m > 1, by the choice

pp= ad2"2 ,

where d = ( b / ~ ) l ' ( ~ ~ - ~ ) . THEOREM B.3. The Wachspress parameter vector (B.15) satiq$es

(B.15)

+

+,(a, b ; d W ' ) 5 [ ( d - l ) / ( d 111'. (B.16) Proof. Each factor in ( B . l ) is less than one in magnitude, while for one i, I.L is in the interval (pi+ pi). For this i, the corresponding factor in ( B . l ) satisfies

completing the proof. A still better parameter vector is defined by de Boor and Rice [flu]. 36

Though the bound (B.16) is better than (B.14), the inequality +m(u,b ; pew)) not hold in all cases (remark by J. Rice and C. deBoor).

C $ ~ ( U ,b ; P ( ~ ) does )

, Y), and (z, Y - hs(Y>)Given the problem of solving (14.1) in a rectangle, we let L1 and LZbe arbitrary except that the horizontal sides of the rectangle must belong to L1, and the vertical sides to Lz. We assume that on each side of the rectangle either u is given or else (2.7) holds with d a constant on each such side. The set ciiD = cii~(L1,Lz) consists of the interior mesh points and those points of Q D on the boundary for which the mixed conditions apply. For each interior mesh point on 520 we approximate the differential equation (14.2) by the difference equation defined by (14.3)-(14.6) where

+

CdY) Z = hkK.

=

CdY>

+ CdY),

W.2)

((3.3)

Here hE’ = h ~ / 2h,N r = h~/v/2, etc., and h and k are arbitrary positive numbers which might be chosen as the mesh spacings in the z- and y-directions if these were constant. 3’See [ 5 ] , p. 194.

264


For points of 6 t D which are on the boundary of the rectangle we develop a difference equation based on both the differential equation (14.2) and the mixed boundary condition. Consider, for example, the case of a point (x,y) on the left vertical side. The boundary condition becomes

The formulas for the difference operator V will be the same as for interior points. To represent the differential operator - [a(El(s)au/ax)/ax]/E,(z) we use the approximation

where (au/ax), and (au/ax)l represent values of &/ax at the points and (x hE’, y) respectively. But by (C.4) we have

+

If we use the central difference approximation &‘(u(x for (au/az), we obtain

(2,

y)

+ h E , y) - u(x, y))

and we have

Similar formulas can rectangle. We now seek to show that the matrices H , V ,and I: obtained from the difference equation satisfy conditions (13.3)-(13.5). By (C.l) and (C.7) the coefficientsof the values of u appearing in the expression for Hu(x, y) depend on x alone. Similarly, the coefficients in Vu(x,y) depend on y alone. Hence the coefficients of the projection operators a and P of (14.9)-(14.10) are of the form (14.9)-(14.10). Therefore it follows by Lemma 14.3 that and P commute, and hence the corresponding matrices H and V commute. Hence condition (13.3) holds. Moreover, by (C.3) condition (13.4) holds. To show that the matrices H and V are similar to nonnegative diagonal matrices we note that FH and FV are symmetric, where F is a diagonal matrix with positive diagonal elements which correspond to the

ALTERNATING DIRECTION MPLlClT METHODS

+

+

265

function F(z, y) which equals Ez(z)F1(y)(hg h w ) ( h ~ hs) as points of (RD inside the rectangle. On points of (RD on the left vertical side F(z, y) = E2(z)Fl(y)(h~hs)hE. Similar formulas hold for the other sides of the rectangle. Since the matrices FH and F V are symmetric and have diagonal dominance, it follows that they are nonnegative definite. Also, as in Lemma 14.2 it follows at once that H and V satisfy (13.5). Thus conditions (13.3)(13.5)are satisfied by the matrices H , V , and 2.

+


266

APPENDIX D: NECESSARY CONDITIONS FOR COMMUTATIVITY

I n Section 14 and in Appendix C we have given some sufficient conditions on the differential equation and the region for the matrices H , V, and B to satisfy conditions (13.3)-(13.5). We now present some necessary conditions. We restrict our attention to the Dirichlet problem with the differential equation (2.1),

+

where G ( x , y ) , A ( x , y ) , and C(x, y ) belong to class38 Uz)in (R @ and where G 2 0, A > 0 and C > 0 in 6t a.We assume that a square mesh of length h is used and that (R and @ are such that for an infinite sequence, X, of values of h tending to zero all boundary points of the network belong to @, and moreover for all sufficiently small h in this sequence 6th is connected. We now prove

+

THEOREM D.1. For the Dirichlet problem for the region and the digerential equation (D.l), let there exist a nonvanishing function P ( x , y) such that for all h in X the matrices H , V , and Z satisfy conditions (13.3)-(13.5), where H , V , and B are derived from the equation

fY-6 Y)-u‘1L)

P(., Y)S(J.,Y ) ,

(D.2) using the diflerence approximations (2.2) and (2.3). Then there exists a nonnegative constant K and functions E’,(z),E 2 ( x ) ,Fl(y), F2(y) which are positive and belong to class C(2)in 6t a,such that =

+

A ( x ,Y ) L ( x , Y)

Bi(x)Fi(y),C ( x , y ) = fiz(z)Fz(y), G ( r ,Y ) = KEz(z)Fi(y), (D.3) = c / E ~ ( z ) F ~((c~ i s)a, Constant). =

Proof. The difference operators H and V corresponding to (D.2) are given by

H u h Y)

=

Aob, Y > U ( Z , Y ) - A l b ,

Y)Ub

+ h, Y >

- A d z , Y ) U b - h, Y ) , V U b ,Y)

=

C&,

Y)U(X,Y

)-

C2(2, Y ) U ( Z , Y

0 4 )

+ h)

- C4(2, Y ) U ( Z , Y - h ) , (D.5) where A d z , Y) = Pb, Y ) A ( ~ ( h / 2 ) ~ ) ,A&, Y ) = P ( x , Y ) A ( x- ( h / 2 ) ~ ) ,

+

etc. The so-called ‘Lprojectionoperators” 14 by 38

and

P are defined as in Section

Functions with continuous second partial derivatives are said to be of class C(*).


267

+ h, y) (D.6) Pub,Y ) = Cob, y>u(z,Y> Gb, Y M X , y + h) - E~(X, Y)u(~ y ,- h ) , (D.7) where Xdz, y) A I ~Y)r(z , + h, y), &(z, y ) Adz, y ) r ( z - h, y), etc., and where r(z, y ) 1 or 0 according to whether or not (z, y ) belongs to Rub, 9) = AOb, Y>U(Z,Y) - U

y)u(z

X ,

- Z ( z , YMJ:- h, Y),

-

=

=

=

Rh.

We now prove two lemmas about general difference operators of the form (D.6)-(D.7). LEMMAD.2. If the coeficients A,(z, y), i = 0, 1, 3, and Cz(z, y ) , i = 0, 2, 4, are positive in a h , and if A and P commute, then 6th is rectangular, Ao(z, y) depends only on x and CO(X,y) depends only o n y.

Proof. We first show that for any (z, y) if any three of the four points h, y), (2, y h ) , and (z h, y h) belong to 6 t h , then the fourth does also. This and the assumption that 6 t h is connected will prove that 6this rectangular. Let us assume that the three points (z, y), (z h, y), and (z h, y h) belong to 6th. Equating coefficients of u(z h, y h) in the expressions for gpu(z,y ) and PRu(z,y) we have XI(%,y)c,(z h, y) = Zdz, y h)Cz(z, Y), or h, d r ( z h, Y h) A&, y)Cdz h, = AI(z, Y h ) C d ~y)r(z , h, Y h ) r ( z ,Y h). But since r(z h, y h) = r(z h, y) and since none of the coefficients Ai(z, y) or Cz(z, y ) vanishes, equality is possible only if r(z, y h) = 1. Hence (z, y h) belongs to 6 t h . Since similar arguments hold in other cases it follows that 6th is rectangular. If the rectangular network @, had only one column of points, then CO(X,y) would clearly be independent of x. Otherwise, let (z, y) and (z h, y) be only two points of 6th. Equating coe6cients of u(x h, y) in the exh, y) = pressions for RPu(z, y) and VRu(z,y) we have -Xl(z, y)Co(z -Zl(z, y)Co(z, y), or equivalently (z, y), (z

+

+

+

+

+

+

+

+

+ + +

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+ - A ~ ( z , Y ) C ~ ( X + h, y ) r ( z + h, Y ) -A&, y ) c O ( z ,y)r(J: + h y ) . But since r ( z + h, y ) 1, and since Al(z, y) > 0 we have CO(Z+ h, y) =

=

+

=

CO(z,y). Since this is true for any point ( 2 , y) of 6th such that (x h, y) is also in 6th it follows that Co(z, y) is independent of z. Similarly, Ao(z, y) is

independent of y, and Lemma D.2 is proved. and 7 symmetric if the correWe shall call the difference operators sponding matrices H and V, respectively, are symmetric. Symmetry of Z? implies that the coefficient of u(z h, y) in the expression for g u ( z , y) is the same as the coefficient of u(z,y ) in the expression for Ru(z h, y),

+

+


268

+

assuming that both (2, y) and (z h, Y) are in Ub. One can readily verify that necessary and sufficient conditions for symmetry of If and P are (for (5, Y) and (5 h, Y) in Ed, (D.8) &(x, y) = Z3(z h, y), (for (z, Y) and (2, Y h) in R h h (D.9) C2(z,y) = C&, y h),

+

+ +

+

We now prove LEMMA D.3. Under the hypotheses of Lemma D S if I? and are symmetric, then the nonzero values of Zl(x, y) and Z3(x,y) depend only on x, and the nonzero values of C2(x,y) and C4(x,y) depend only on y. is rectangular, by Lemma D.l. If (z, y) and Proof. The network such that 2'1(z,y) (z, y h) are any two points (5, y) and (z, y h) in and &(z, y h) do not vanish, then I'(z h, y) = r(z h, y h) = 1. Hence (z h, y) and (z h, y h) belong to a h . Equating the coefficients of u(z h, y h) in the expressions for n r u ( z , y) and rRu(z, y) we obtain &(z, y)C& h, Y) = &(z, Y h)Cz(z,Y) or Adz, Y)CZ(S h, Y) = Adz, Y h)Cz(z, Y). (D.lO) Also, equating the coefficients of u(z* - h, y h) in the expressions for Bvu(z*, y) and rflu(z*, y), where z* = z h we obtain Z3(z h, y) G(z, Y) = Z ( 5 h, Y h)C& h, y), or

+

+ + + +

+

+

+ +

+

+

+

+

+

+

+

+

+

+

+

+ h, Y + h)C2(z + h, y), A i b , ~)C2(z,Y) = Ai(z, Y + h)Cz(z + h, Y).

A3(2

and, by (D.81

+

+ h, Y)C2(2, Y) =

A3(X

Combining (D.lO) and (D.ll) we obtain [Al(z,y)I2 = [Al(z, y and since A1 > 0 we have

+

(D.ll)

+ h)I2,

Ai(z, Y) = Ai(z, Y h). Since this is true for any two points (2,y) and (z h, y) in ah,it follows that the nonzero values of &(z, y) are independent of y. Similar arguments can be used to prove this about Z3(x,y) and to show that the nonzero values of C2(x, y) and C4(z, y) are independent of z. Thus Lemma D.3 is proved. In order to apply Lemma D.3 to the proof of Theorem D.l, since I? and P are not in general symmetric, we construct operators HcN) and W N ) which are both symmetric and commutative. We let

+ h, Y) - zP')(z,Y ) u ( ~- h, Y), y) = C P ( z , y)u(z, Y) - CP"3, Y)U(S, y + h)

H Y z , Y) VN)(X,

+

=

Ar)(z, y)u(z, Y) - Z N ) ( zYMz ,

- CY)(z, Y ) u ( ~y, - h),

where

03.12) (D.13)


269

+

Zf")(x,y)

= Af")(x, y)r(x h, y), - h, y), etc. It is easy to see that H w ) and V") are symmetric. To show that they commute we consider the associated matrices H") and V"). Evidently, if F(x, y) = 1/P(x, y) and if the diagonal matrix F corresponds to the function F(z, y) then H") = F1/2HF-1/2and V") = F1/2VF-1/2.Clearly, if H V = V H , then Hw)Vw) = V")H"). Hence, by Lemma D.2 it follows that the nonzero coefficients~ P ) y) (Z and, Plm(x, y) depend only on x and y, respectively. In particular, A:"(%, y) = P'/'((s,y)P'/2(z h, y)A(x (h/2), y) must be independent of y except for points (5, y) of a h such that (x h, y) does not belong to a h . It follows that

Here, as usual,

71hm(z,y)

= A P ( ( s ,y)r(z

+

+

+

for all h in X and for all (x,y) in a h except as noted above. Since P(x, y) is continuous, the limit of both sides of the above equation exists as h + 0 through the sequence X, and we have P(x, y ) A ( x , y)

=

(D.16)

X ( x ) = Lim e(x, y). h+O h€%

Since this is true for all points (x,y) which for some h in X belong to @I, and such that (x h, y) is in ah, and since such points are dense in 6i 63, it follows by continuity that (D.16) holds throughout (R 63. Similarly, we have for some continuous function Y ( y )

+

Substituting (D.16) in (D.15) we have

+

+

270


or

log A(x,y)

- 51 log A(x: + h, ?J)= log Oi(x, y) =

ez(z, h).

(D.19)

+ 6%we have by the mean

Since A belongs to class C(l) and is positive in (R

- 2-1 r ( z + h, y)

(D.20) =

0,

where (D.21)

The general solution of the difference equation (D.20) is

T(z,Y)

+

= 4-l) @(?I)

(D.22)

for suitable functions a ( y ) and P(y). Upon substituting (D.22) in (D.21) and integrat,ing we have

A( 2 , y) = -&(z)Fl(y)exp [ z Y l ( ~ ) l (D.23) for suitable functions El(z)Fl(y) and Yl(y). Similarly for suitable Ez(z), Fz(Y), X&) we have C(z, ?J) = Ez(z)Fdy) exP [Yxl(z)l. (D.24) But by (D.16) and (D.17) we have A ( z , y)/C(z, y) = X ( z ) / Y ( y ) so that (D.25)

But by (D.23) and (D.24), for some constant a

Xl(Z) and hence

A ( z , y)

=

=

Yl(Y) = a,

El(z)Fl(y)eazY,C ( x , y)

=

Ez(z)Fz(y)eaZ”.

(D.26)

Moreover, by (D.16), (D.17), and (D.26) there exists a constant c different from zero such that (D.27)

Since the diagonal matrix 2 which corresponds to the function h2P(z,y)


271

G(x,y) must be a constant times the identity matrix, by (13.4), it follows that for some constant K

G(x,y) = K&(x)Fi(y). (D.28) To determine the constant a we use the fact that AP(x,y) and ChN(z, y) are independent of x and y, respectively. By (D.14), (D.26), and (D.27) we must have

independent of y. But since El(z) is a positive function this is clearly impossible unless a = 0. Therefore (D.3) follows from (D.26), (D.27), and (D.28), and Theorem D.l is proved. Even if the diagonal matrix 2 is not a constant times the identity matrix one might try to obtain matrices H', V' and 2' = 0 satisfying conditions (13.3)-(13.5) by letting H' = H yZ', V' = V (1 - 7)Z' for some constant y. Conceivably H' and V' might commute even though H and V did not. This is clearly not possible, of course, if B = al. It can be shown that if G(z, y), A(x,y), and C(z, y) are of class C(3),then the conditions of Theorem D . l are necessary in order for H' and V' to commute. We omit the proof.

+

+

Bibliography 1. Birkhoff, G., and Varga, R. S., Implicit alternating direction methods. Trans. AMS 92,13-24 (1959). 2. Bruce, G. H., Peaceman, D. W., Rachford, H. H., and Rice, J. D., Calculation of unsteady-state gas flow through porous media. Trans. A I M M E 198,79-91 (1953). 3. Conte, S., and Dames, R. J., An alternating direction scheme for the biharmonic difference equations. Math. Tables Aid Comput. 12, 198-205 (1958). 3a. de Boor, C. M., and Rice, J. R., Tchebycheff approximation by

+

a W ( z - r,)/(z rJ1 and application to AD1 iteration. To appear in J . SOC.Ind. Appl. Math. 4. Douglas, J., Jr., A note on the alternating direction implicit method for the numerical solution of heat flow problems. Proc. A M S 8, 409-411 (1957). a% a% au 5. Douglas, J., Jr., On the numerical integration of - - = - by implicit meth-

ax*

+a g

at

ods, J . Soc. Ind. Appl. Math. 3 , 4 2 4 5 (1955). 6 . Douglas, J., Jr., Alternating direction iteration for mildly nonlinear elliptic differential equations. Numer. Math. 3, 92-98 (1961). 7. Douglas, J., Jr., and Rachford, H., On the numerical solution of heat conduction problems in two and three space variables. Trans. A M S 82,421-439 (1956). 8. Forsythe, G. E., and Wasow, W. R., Finite-difference Methods for Partial Differential Equations. Wiley, New York, 1960. 9. Fort, T., Finite LXfferences.Oxford Univ. Press, London and New York, 1948. 10. Frankel, S., Convergence rates of iterative treatments of partial differential equations. Math. Tables Aid Comput. 4, 65-75 (1950).

272


10a. Gantmakher, F. and Krein, M., Sur les matrices completement non-negatives et oscillatoires. Compositio Math. 4,445-476 (1937). 11. Golub, G. H., and Varga, R. S., Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods, I. Numer. Math. 3,147-156 (1961); Part 11.Nunaer. Math. 3, 157-168 (1962). 12. Heller, J., Simultaneous successive and alternating direction schemes. J. SOC.Ind. and Appl. Math. 8, 150-173 (1960). 12a. Householder, A. S., The approximate solution of matrix problems. J. Assoc. Computing Machinery 5,205-243 (1958). 12b. Kryloff, N., Les mbthodes de solution approchh des problhes de la physique mathbmatique. M h . Sci. Math. No. 49,68 pp. (1931). 13. Lees, M., Alternating direction and semi-explicit difference methods for parabolic partial differential equations. NUWT. Math. 3,398-412 (1962). 14. Ostrowski, A. M., On the linear iterative procedures for symmetric matrices. Rend. mat. appl. [5] 14,140-163 (1954). 15. Parter, S. V., “Multi-line” iterative methods for elliptic difference equations and fundamental frequencies. Numer. Math. 3, 305-319 (1961). 16. Peaceman, D. W., and Rachford, H. H., Jr., The numerical solution of parabolic and elliptic differential equations. J. SOC.Znd. and Appl. Math. 3, 28-41 (1955). 16a. Remes, E., Sur un procbd6 convergent d’approximations successives pour dbterminer les polpomes d’approximation. Compt. rend. acad. sn’. 198,2063-2065 (1934); Sur le calcul effectif des polynomes d’approximation de Tchebichef. Ibid. 199, 337-340 (1934). 16b. Rice. J. R., Tchebycheff approximations by functions unisolvent of variable degree. Trans. A M S 99,298-302 (1961). 16c. Shortley, D., and Flanders, G. A., J. Appl. Phys. 21,1326-1332 (1950). 16d. Stiefel, E. L., Numerical methods of Tchebycheff approximation. In On Numerical Approximatiom (R. Langer, ed.), pp. 217-233. Univ. of Wisconsin Press, Madison, Wisconsin, 1959. 16e. Thrall, R. M., and Tornheim, L., Vector Spaces and Matrices, p. 190. Wiley, New York, 1957. 17. Varga, R. S., Overrelaxation applied to implicit alternating direction methods. Proc. Intern. Congr. on Information Processing, Paris, pp. 85-90, June (1958). 18. Varga, R. S., p-cyclic matrices: A generalization of the Young-Frankel successive overrelaxation scheme. Pacific J. Math. 9, 617-628 (1959). 19. Varga, R. S., “Matrix Iterative Analysis.” Prentice-HaI1, Englewood Cliffs, New Jersey, 1962. 20. Varga, R. S., Factorization and normalized iterative methods. In Boundary Problems i n Bfferential Equations, pp. 121-141. Univ. of Wisconsin Press, Madison, Wisconsin, 1960. 21. Varga, R. S., Orderings of the successive overrelaxation scheme. Pmijic J. Math. 9, 925-939 (1959). 22. Varga, R. S., Higher order stable implicit methods for solving parabolic partial differential equations. J. Math. and Phys. 40, 220-231 (1961). 23. Wachspress, E. L., CURE:a generalized two-space-dimension multigroup coding for the IBM 704. Knolls Atomic Power Laboratory Report No. KAPL 1724, General Electric Co., Schenectady, New York, April (1957). 24. Wachspress, E. L., and Habetler, G. J., An alternatingdirection-implicit iteration technique. J. SOC.Ind. and Appl. Math. 8,403424 (1960).


273

25. Wachspress, E. L., Optimum alternating-direction-implicititeration parameters for a model problem. J. SOC.Ind. and Appl. Math. 10, 339-50 (1962). 25a. Weyl, H., Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen.Math. Ann. (Leipzig) 71,441479 (1912). 25b. Young, D., Iterative methods for solving partial difference equations of elliptic type. Ph.D. Thesis, Harvard (1950). 26. Young, D., Iterative methods for solving partial difference equations of elliptic type. Trans. AMS 76,92-111 (1954). 27. Young, D., Ordvac solutions of the Dirichlet problem, J. Assoe. Computing Machinery 2,137-161 (1955). 28. Young, D., On the solution of linear systems by iteration. AMS Symposium on Numer. Anal. 6 (1956). 29. Young, D., and Ehrlich, L., Numerical Experimenls Involving Boundary Problems in Differential Equations, pp. 143-162. Univ. of Wisconsin Press, Madison, Wisconsin, 1960.


Combined Analog-Digital Techniques in Simulation HAROLD

K. SKRAMSTAD

U. S. Navol Ordnance Laboratory Corona, California*

1. Comparison of Analog and Digital Computers in Simulation 2 . Interconnected Analog and Digital Computers . . . . 3. Example of a Combined Solution . . . . . . . . 4. Analog-Digital Arithmetic in a Digital Computer . . . . 5. Systems Using Analog-Digital Variables . . . . . . Bibliography . . . . . . . . . . . . . .

. .

. . .

.

.

.

.

. 277

. .

. .

.

.

. .

. .

. 281 283 . 288 . 296

.

.

.

275

.

1. Comparison of Analog and Digital Computers in Simulation

One of the most common types of problems to which computers have been applied is the study of complex nonlinear dynamic systems subjected to external disturbances. Such systems are represented mathematically by nonlinear systems of differential equations with time as the independent variable. Computers used to solve problems of this type are commonly called simulators, since often the passage of time in the computer solution is proportional to time in the system under study, and thus the system under study is “simulated” by the computer. Examples of typical types of dynamic problems to which computers (or simulators) have been applied are shown in Table I. The comparative characteristics of analog and digital computers when applied to the solution of dynamic systems problems are shown in Table II. It is evident that the electronic analog computer has features which make it particularly adaptable to this type of problem. The more important of these features are continuous representation of variables, high speed of operation, and ease of incorporating actual systems hardware. The fact that all mathematical operations are going on in parallel in separate analog computing elements gives this technique a decided speed advantage over digital machines in which most operations are carried out in sequence in a

* Former affiliation:Xational Bureau

of Standards, Washington, D.C. 275

HAROLD K. SKRAMSTAD

276

TABLE I. EXAMPLES OF DYNAMICSYSTEMS SIMULATION PROBLEMS Aircraft dynamic response Aircraft landing shocks Automatic control systems Automatic pilots Automobile suspension systems Chemical equilibrium Chemical kinetics Chemical process control Economic systems Electron optics Exothermic reactor control Feedback amplifiers Geophysics Heat exchangers Helicopter vibrations Hydraulic transmissions Insect population dynamics Linear and nonlinear circuits

Missile guidance and control systems Modulation systems Noise effects Nuclear physics Nuclear reactor control Operations research Particle injection into accelerator Radar tracking circuits Servomechanisms Spring systems Seismometer systems Transducers Transient heat conduction Torsional vibrations Transmission lines Transistor performance Vibration of structures Vibrometers

TABLE 11. COMPARATIVE CHARACTERISTICS OF ANALOG AND DIGITAL COMPUTERS FOR SIMULATION OF DYNAMIC SYSTEMS Analog

Digital

Continuous representation of variables Parallel operation of components High operation speed Increasing problem size requires proportional increase in computer size a t no increase in solution time Accuracy and dynamic range limited by physical measurement capability Continuous integration Difficult t o handle and storc discretc data Solution progresses at rate proportional t o rate of progression of system studied Easy to include parts of actual system in simulation

Variables represented by discrete numbers Operations performed in sequence Slower operation speed Increasing problem size requires increase in memory size and time of solution

Behavior of system easily visualized as simulation proceeds

Accuracy and dynamic range extendablc t o any desired degree Integration by finite difference calculus Easy to handle and store discrete data Ordinarily no correspondence between rate of solution and rate of system studied Requires analog-digital converters and real time computing speeds to include system parts Output data ordinarily not available as solution proceeds

single time shared arithmetic unit. Also many operations such as integration, trigonometric resolution, and function generation are performed on a digital computer by approximate numerical techniques requiring many computing steps or iterations. The only limitation on the highest natural

ANALOG-DIGITAL TECHNIQUES IN SIMULATION

277

frequencies that can be simulated with fidelity in an analog computer is the bandwidth or frequency response of the individual analog elements employed. The limitations on the speed of a digital computer are such factors as memory cycle time, order code, the time required to perform common operations such as add, subtract, multiply, and divide, and input/output equipment speed. In real-time simulation the digital computer must carry out all the calculations called for in the mathematical model a sufficient number of times per second to follow accurately the highest natural frequencies present in the solution. This usually demands an iteration rate for integration of the order of ten times the highest natural frequency involved in the problem, dependent on the numerical algorithm chosen. If real-time operation is not required, an increase in the time of solution can relieve the dynamic requirements on the individual components of an analog computer, but any increase in problem size will call for a corresponding increase in the size of the computer. I n the digital case however, increasing the time of solution automatically increases the size of the problem that can be solved. If the memory capacity is sufficient, no increase in equipment is needed to handle a larger problem. Thus a digital computer may be said to be time expandable as opposed to the hardware expandable analog computer. The biggest deficiency of the analog computer is that its precision and dynamic range are limited by physical measurement capnbility, and thus it cannot be applied to problems where the precision and dynamic range required is beyond its capabilities.

2. Interconnected Analog and Digital Computers

Many simulation type problems may be solved almost equally well on either an analog or a digital computer. There are many type. of problems however that are not handled well on either type alone, and consideration has been given to the advantages to be derived from interconnecting analog and digital machines through appropriate analog-to-digital and digital-toanalog converter equipment. In such a system, part of the problem can be solved on the digital computer and part on the analog computer, with exchange of data taking place through interconnecting data channels. Primarily, combined systems of this type are needed to handle problems which involve highly accurate calculations on low frequency data as well as lower accuracy calculations on high frequency data. These problems require the high bandwidth of analog computers as well as the high precision and dynamic range of the digital computer. For example, missile guidance and control systems often involve high frequencies in the dynamic equa-

278

HAROLD

K. SKUAMSTAD

tions of motion of the vehicle and a t the same time low frequency large dynamic range variables, such as position, which need to be known to an accuracy of perhaps a few feet in several hundred thousand feet. Solving problems of this nature on a digital computer alone would require very large size computers or excessive running time due to their low bandwidth capacity. On the other hand, analog computers would not be capable of yielding the required precision. Other examples of problems which might better be solved by interconnection of analog and digital computers are: (1) Simulation of hybrid systems, such as a digitally controlled physical process. The analog computer might be used to simulate the process and the digital computer to simulate the digital controller. (2) Problems handled adequately on analog computers except for requiring the generation of arbitrary functions of two or more variables, generation of pure time delays, axis transformation or long-time integration-jobs which frequently are more easily accomplished on digital computers. (3) Very large dynamic problems which would require excessive time on a digital computer or excessive equipment on the analog, but which, by proper division of the problem, could be solved in a reasonable time and with a reasonable amount of equipment on a combined facility. Work in interconnecting analog and digital computers began in 1954 at Convair, Xan Diego [.@?I, and involved interconnecting an ERA 1103 digital computer to a large analog facility. In 1956, a combined system was put into operation simulating the flight of an Atlas missile. Shortly thereafter, a t a new location, an IBM 704 digital computer was linked to a large Electronic Associates analog computer through Epsco “Add-a-Verter” conversion equipment. At about the same time, Ramo-Wooldridge linked together an 1103 and an Electronic Associates analog computer through similar “Add-a-Verter” equipment [ I ] . This laboratory (now under Space Technology Laboratories) is now linking a Packard-Bell PB 250 digital computer to their analog computer through this same Add-a-Verter equipment [ I 4 241.In 1957, the National Bureau of Standards interconnected their SEACdigital computer, a Mid-Century analog computer, and appropriate display and control equipment for simulation of complex manmachine systems, such as a ground controlled intercept system [SS, EL$]. The analog computer was used to simulate the dynamics of the interceptor aircraft being flown by a pilot in a simulated cockpit. The velocity components of the interceptor were converted to digital form, and used as inputs to the digital computer. The digital computer was used to keep track of the interceptor’s position as well as that of all other aircraft in an


279

area under study. This facility has also been used for studies of air traffic control. IBM Research Laboratories in Yorktown Heights have interconnected analog computer to an IBM 704 through an Add-a-Link converter a PACE unit. This system has been used for the simulation of chemical processes [SI].At IBM, Owego, a similar combined facility has been applied to the simulation of automatic navigation systems. At General Electric Missile & Space Vehicles Department, an IBM 7070 and a PACEanalog computer with automatic digital input and output equipment have been linked by a special interconnecting link, called HYCOL,which includes analog-to-digital and digital-to-analog conversion equipment built by Packard-Bell [W7]. In this system the analog computer is being put completely (except for patching) under the control of the digital computer. This facility is intended for use in exploring some of the increasingly complex problems of space navigation and control. At the University of Minnesota, an 1103 digital computer and a Reeves analog computer have been linked with Add-a-Link equipment [%I. This combined facility is designed as an educational tool for the study of control systems, particularly in the field of chemical engineering. It is also being used to investigate the ability of the combined system to function as a general purpose mathematical computing machine. The Armament & Flight Control Division of Autonetics has interconnected a large PACEanalog computer with a G-15 digital computer with Packard-Bell conversion equipment. This combination is used primarily in the simulation of complex weapon control systems. At Grumman Aircraft Engineering Corporation, a 704 digital computer and a large analog facility have been interconnected [8, 91 with a data-link manufactured by Adage Incorporated. Presently they have an IBM 7090 linked with their analog facility. The FAA has set up, a t the n’ational Air Facility Experimental Center a t Atlantic City, a large scale man-machine simulator for the study of terminal area air traffic control systems [18].Their simulation effort involves the interconnection of digital computers with special purpose analog aircraft generators and displays. Combined facilities are also a t Convair Astronautics where an IBM 7090 is linked to an analog computer with the Add-a-Verter equipment [@I, and at Douglas where their analog computer is linked to a Bendix G-15 computer. North American Aviation and the Naval Ordnance Laboratory, Corona are interconnecting digital differential analytheir analog facilities with Packard-Bell TRICE zer equipment, which will provide the capability of precise long term integration and extremely rapid and precise axis rotation and other nonlinear transformations. This list is not exhaustive, and there are many

HAROLD K. SKRAMSTAD

280

others not mentioned here which are now in operation. I t is prcscntcd only to show the rapidly increasing use of combined facilities. Figure 1 shows a block diagram of a combined analogdigital facility. Analog-to-digital converters are required to convert analog voltages to digital quantities for introduction into the digital c*omputcr, and digital-

L

4

ANALOG COMPUTER

ANALOG TO DIGITAL CONVERTERS

-

DIGITAL TO ANALOG CONVERTERS

c

c

DIGITAL COMPUTER

1 4

p~

CONVERSION CONTROL

4

c

EQUIPMENT

Fic. 1 Combined urinlog-digital computrr farility

to-analog converters to convert digital numbers to analog voltages. Special conversion control equipment is rcquired to control the timing of the conversion in either direction. In a typical simulation, the digital computer will receive information sampled from the analog, use this data in a computation, and transmit the results back to the analog computer. It is generally required that the continuous data from the analog computer be sampled at a definite frequency, and it is ordinarily preferable to control the sampling from a timing source cxternal to the digital computcr rathcr than from the computcr itself. It is necessary, of course, to sytwhronizc the input and output of the digital computer with this timing soiircc.


28 1

A special problem which occurs when combining analog and digital computations is the time lag introduced into the problem by the time required for the analog voltage to be sampled, converted to digital form, introduced into the digital computer, digital calculations made, and then returned again as a voltage to the analog computer. This is equivalent to a transport delay. If this delay is objectionable, some type of extrapolation must be performed in the digital computer to update the data before conversion to analog form. The necessity for the analog computer to receive inputs in continuous form produces another problem. Usually some type of interpolation of the digital data is used, such as to hold the converted data from the digital computer constant until each new value is received. More sophisticated methods involve proper interconnection of analog components using higher order derivatives from the digital computer. I n applying a combined system to a particular problem, certain decisions must be made, such as: (1) which calculations to do on the analog and which on the digital computer, (2) what precision of analog-to-digital and digitalto-analog conversion is necessary, and (3) what is the optimum sampling period for various variables. As mentioned previously, high frequency low accuracy computations should be handled on the analog, while the low frequency high precision calculations should be given to the digital. The precision of conversion is usually made five to ten times greater than that of the information it handles. The sampling period for any variable needs to be long enough so that the digital calculations required during each period can be completed, and yet short enough so that several samples will be obtained during each cycle of the highest frequency present in the analog input. Thus the time scale that can be used for a particular simulation is dependent upon the speed of the converters and speed of calculations of the digital computer, as well as the highest frequencies present in the analog computer variables. 3. Example of a Combined Solution

To illustrate the limitations of either an all analog or an all digital solution, and the advantages of a combined analog-digital solution for certain types of problems, let us consider the terminal phase of the missile intercept problem in two dimensions considered by Grumman Aircraft, and illustrated in Fig. 2 [8, 91. Equating velocities normal and transverse to the line of sight give the following equations:

+ V,sinp + vt cos p

R& = - V m s i n a

-k = v,

cos

(3.1)

(3.2)

282

HAROLD K. SKRAMSTAD Y

FIG.2. Illustrative missile intercept problem.

From the geomet,ry,the angles CY and 0are given by the following equations:

cr=e-d

(3.3)

P=d-r+r (3.4) Let us assume a control equation with a second order time lag given by 9’8 276 6 = N$* (3.5) where 6 is a control surface deflection of the missile, d* is an amplitudelimited value of $, N is the proportional navigation constant, and 7 is the control system time constant. The missile transfer function is assumed of the second order giving the following relationship between e and 6 where w and 5- are the natural angular frequency of the missile and the damping coefficient, respectively : 1 25- .. -e+-e+e=k6

+

+

a*.

w2

w

A basic difficulty with an all analog solution of this problem is the scaling. I n this case, R is initially of the order of magnitude of 50,000 f t and ap-


28 3

proaches zero for a direct hit. Miss distances in the order of plus or minus two feet are desired. If we scale so that 100 volts represents R = 50,000 ft, R = 2 ft would correspond to 4 mv, which is well below the probable noise level in practical operation of the analog computer. Another variable which presents a scaling problem is 4,the rate of change of the line of sight angle. Initially, when R is large, 4is small, but it approaches a large value when R approaches zero. This does not offerany particular problem in the guidance equation loop as this signal is limited. However, the true 6 is required to compute the kinematics of the problem. A disadvantage of a digital method of solution lies in the relatively longer amount of computing time required for each solution. If a parametric study or statistical analysis of the system is made, the number of solutions required for any one analysis may be very large. Figure 3 shows how this problem would be solved in a combined system. The airframe and control equations are solved on the analog computer since it can easily meet the precision and frequency requirements of these equations. The digital computer handles the equations involving R and 4 since the dynamic range required here is large. To handle the high frequencies involved in the control and aerodynamic equations, an all digital solution would have to use about 0.025 sec time interval for numerical integration until a range of about 100 ft is reached, and a sampling interval of 0.001 sec for the remainder of the flight. This would result in a running time of about eight times real time. However, in the combined analog digital solution, since the dynamic equations are solved by the analog computer and the equations solved by the digital computer have parameters which do not vary rapidly, it is estimated that the solution could be run a t about one-tenth real time. Thus a considerable time saving can be realized over an all digital solution.

4. Analog-Digital Arithmetic in a Digital Computer

Combined analog-digital techniques have been devised to perform the arithmetic operations in a digital computer. One technique is based on the fact that a digital-to-analog converter is inherently a multiplier, and the analog-to-digital converter a divider. For example, a digital-to-analog converter, shown diagrammat)ically in Fig. 4(a), multiplies the analog reference voltage ( E ) by the digital input ( X ) to produce an analog output signal ( X E ) . Also, an analog-to-digital converter, shown diagrammatically in Fig. 4(b), divides the analog input ( X E ) by the analog reference signal ( E ) to produce the digital quotient ( X ) . Figures 4(c) and 4(d) show

284

P

c

2

R 4 -

i*

9

D-A CONVERTERS

FOR PLOTTING

f

-

ANALOG COMPUTER

D I G I T A L COMPUTER R;

=

-vM

=

vM

0

-R

sin a t

vT sin

c o s a t vT c o s p 0

0

SYSTEM CONTROL 000

8

O0

Kw28 - 2SwO-

w

2

0

8

0

0

-9m 9 < - 9 m a=8-9

p= 9 Y=

FIG.3. Block diagrnm for combined solution.

- y t

r,+9

7

HAROLD K. SKRAMSTAD

1 CONVERSION


(0)

DIGITAL

285

TO ANALOG CONVERTER

E

XE

( b ) ANALOG TO DIGITAL CONVERTER

E

D/A

I

E

(c

MULTIPLIER

FIG.4. Hybrid arithmetic.

how converters might be interconnected to perform digital multiplication and division, respectively. Hybrid arithmetic units based on converters, particularly as applied in control computers, is discussed in detail in references [4, 5, 101. A system using hybrid arithmetic has been studied a t IBM Laboratories, Yorktown Heights, N. Y. [@I, and an arithmetic unit has been built to demonstrate the feasibility of the system. If scaling is arranged properly so as to operate a,s close to full scale as possible on the converters, preciaions cJf 1 part in lo4 to 1 part in lo5 can be achieved-sufficient for many problems. The speed of calculation depends primarily on the speed of the

286

DECODER ( D I G I T A L TO ANALOG 1

DIGITAL COMPUTER

~AE"I\cLOoDGERTO DIG

I 2

M U LT.

-(AX +Bx)

--c

+ A ~ Z +B X + C

I

FIG.5 . Use of pulsed-analog cornporierits t o evsluate the polynomial ( A 3

+ Ux* + Cx + D).

HAROLD K. SKRAMSTAD

I1 .


287

conversion equipment, and an arithmetic unit of this hybrid type would have a speed advantage over a purely digital unit mainly in cases in which the arithmetic unit has many digital-analog components operating in parallel. Another method of combining analog and digital techniques in the arithmetic unit of a digital computer, known as the “pulsed analog” technique, is under development a t the MIT Electronic Systems Laboratory [ I l l 15, 2’2.1. Figure 5 shows how the third degree polynomial y = Ax3

+ Bx2 + CX + D

can be evaluated using a digital-to-analog converter, an analog-todigital converter, three summing amplifiers, three multipliers, and four coefficient potentiometers. In this system the constants A , B, C , and D DECODER (DIGITAL TO ANALOG

-

SIGNALS

1

GATES ACTIVATED

PROGRAM

I . DECODE AND STORE

X

GI G 3 GI G4

2. DECODE AND STORE A 3. DECODE AND STORE 8 , STORE A x + B 4. TRANSFER Ax t B 5. DECODE AND STORE C, STORE A x z t Bx t C 6. TRANSFER Ax2+ Bx + C z DECODE AND STORE D, STORE y

I

y:

GI G5G6 G2 G4 GI G5 G6 G2G4 G5G6

Ax3tBx2+Cx+D

SEVEN STEPS

- 70

I

MICROSECONDS

FIG.6 . Alternative pulsed-analog configuration for polynomial expansion.

288

HAROLD K. SKRAMSTAD

would have to be set up on analog coefficient potentiometers. Using typical components, the evaluation of the polynomial would require about 10 psec. On the basis of a 10 psec add execute time and 25 psec multiply execute time, the evaluation of the polynomial on a digha1 computer would require 115 psec. A second, more general pulsed-analog configuration is shown in Fig 6 . GI and Gz are sample gates which sample their input voltages on command. Ga to Gs are storage gates which sample and hold their input voltages after disconnection from their inputs. I n this case, the variable x and the constants A , B, C , and D must be obtained from the digital memory. The sequence in which the various sample and storage gates are activated and deactivated under control of the digital program is given in the figure. The computation time saved with the pulsed analog technique depends, of course, on the amount of special purpose analog equipment one is willing to use, the frequency of oft repeated computational sequences, and the time consumed in carrying out these sequences by digital means. The technique is expected to increase the effective computation speed of a digital machine by a factor of two or more, and is currently being applied in the design of a flexible operational flight trainer. 5. Systems Using Analog-Digital Variables

A somewhat different method of combining analog and digital techniques is to represent quantities not by numbers as in the digital computer or by electrical voltages as in the electronic analog, but by the sum of a number and an electrical voltage where full scale on the electrical voltage is equivalent to one in the least significant digit of the digital part of the number. For example, if we assume 100 volts in the analog part is equal to 0.001 in the digital part, a quantity which is represented digitally as the number 0.846753 would be represented in this combined system by the digital number 0.846 plus an analog volt,age equal to 75.3 volts. An alternate representation would be a digital part 0.847 and an analog voltage of a - 24.7 vo1t.s. Figure 7 shows how a sine wave of angular frequency w would be represented in this hybrid system if a factor of 10 is carried digitally. It is obvious that to gain a given factor in precision, the rate of change of the analog voltage is increased by the same factor and thus the basic limitation on such a combined system will be the bandwidth of the analog components. A computer based on this type of numerical representation is described by the author [55, 581, and some additional computing elements are described in reference [SO]. It is similar in organization to an analog computer in that it consists of computer components such as integrators, multipliers,

ANALOG-DIGITAL TECHNIQUES IN SIMULATION 289

290

HAROLD K. SKRAMSTAD

summers, etc., interconnected in open- or closed-loops to solve a problem. Let us consider what form some of the computer components such as an integrator and a multiplier take in such a combined system. Assume we wish to obtain the following: 1 y = xdt

-It

T o

where x and y are functions of the time and T is the time constant of the integration. Let us also assume that the problem has been scaled so that the maximum value of all dependent variables will not exceed unity. Let each of the two dependent variables x and y consist of a digital part and an analog part, denoted by the subscripts D and A , respectively. Thus, we have = XD %A (5.2)

+

y =

+ +

y = yD YA 1 (ZD Z A ) dt.

(5.3) (5.4)

Let us assume time to be divided into discrete equal intervals of duration At, and that the digital parts of x and y can change only at times which are integral multiples of At. We may then write for the value of 9 a t a time t somewhere in the nth interval :

where ( z D ) is ~ the value of XD during the ith interval At. Figure 8 shows a curve of x as an arbitrary function of t. The area under this curve from 2 = 0 to any arbitrary t would equal the integral in Eq. (5.4) or the bracketed expression in Eq. (5.5). The first term in the bracketed expression, represented by area 1, is the integral of the digital part of z up to the time (n - 1) At. The second term, represented by area 2, is the integral of the digital part of x between (n - 1) At and t. The third term, represented by area 3, is the integral of the analog part of x from t = 0 to t. Figure 9 is a block diagram of an integrator unit. It contains an input digital register xD, a digital register R, two digital-to-analog converters, a conventional analog integrator, a special resettable analog integrator, an analog summer, and a comparator unit. The register YD shown on the far right of the figure is the input register of the next component to which this unit might be connected in solving a problem. E is the analog reference voltage supplied to the digital-to-analog converters and a is the digital equivalent of the reference voltage E. At the beginning of each At period, the values XD and R are sampled and converted to analog voltages which are held constant during the period,

ANALOG-DIGITAL TECHNIQUES IN SIMULATION 291

h)

0

CLOCK PERIOD A t

CLOCK PERIOD A t

I I

I I

REGISTER

REGISTER

h)

CLOCK PERIOD A t I I

ka

_ _INCREMENTS _ _ _---_-

FROM OTHER UNITS

REGISTER,

L----

E CONVERTER

RESETTABLE ANALOG INTEGRATOR TIME CONSTANT = A t

rt

I

Jo

a

ANALOG VOLTAGE FROM OTHER UNITS T I M E CONSTANT

=

FIG.9. Integrator unit.

I


293

unaffected by future changes in xD or R which occur during the period. The value of X D is then algebraically added to the R register. The voltage VI, which represents that portion of the prior summation of ( x D ) ~At which is of analog magnitude is given during the nth interval At by: V1 = -ER, (5.6) , provides integration of the current X D value within The voltage V Z which the nth interval At? and which is reset to zero a t the end of this interval, is given by : -E(xD)x{t - ( n - 1) At}. V2 = (5.7) At

The voltage V3, which results from the purely analog integration of the continuously varying analog part of x , is given by:

These three voltages are added in the analog summer to give voltage V . The analog part of the output of the integrator is equal to:

h a = V = -(V

1

+ Vz + Vd.

(5.9)

If, a t any time during a period At, the voltage V a t the output of the analog summer exceeds a predetermined upper threshold, this is sensed by the comparator and, during the next At interval, immediately following the addition of XD to R, unity is subtracted from the R register and a is added to the input register of the following unit (YD in Fig. 9). Conversely, if the voltage V falls below a predetermined lower threshold, unity is added to the R register and a is subtracted from the input register of the following unit. It is easily shown that the time constant T of this integrator is equal to Atla.

A multiplier unit operating in this combined representation is shown in Fig. 10. Suppose we wish to obtain the product z = xy. Assuming as before that each variable consists of a digital part and an analog part, we have ZD Z A = XDYD XDYA XAYD XAYA (5.10) where the subscripts D and A signify digital and analog parts, respectively. As seen from the figure, a multiplier unit has three digital registers for xD, YD and R ;three digital-to-analog converters; an analog summer; an analog multiplier; and a comparator unit. At the beginning of each period At, the values of XD, YO, and R are sampled and converted to voltages which are held constant during the period. If, during the period, XD receives an increment (or decrement) a from another unit, YD is added to (or subtracted from) R ; and if YD receives an increment (or decrement) a from another unit, XD is added to (or subtracted from) R. If both X D and YD change during At, the additions to R must either

+

+

+

+

h)

'0 P

CLOCK PERIOD A t I

D: , -4 -C a INCREMENTSFROM OTHER UNITS

-

'DT0 CH INCREMENT

---____

?EGISTER L

I IN

.--

7

Yn

I CLOCK IPERIOD A t

I CLOCK I PERIOD A t

Eyp, a

I/

UPPER - - LOWER THRESHOLD THRESHOLD E VOLTAGE V 0 LTA G E

b ?F

I

ANALOG VOLTAGES FROM OTHER UNITS

\ a

T

I

i L

ANALOG MULTIPLIER

I

V, = E R

n

C

DIGITAL TO ANALOG CONVERTER

------t

T a INCREMENTS

FROM OTHER UNITS

REGISTER

,

___

I'

~ a" 4 ' -

I

Jv3=-

U

€ Y D x~

'

I

EACH ADD YINCREMENT D TO R a

I

IN X D

CLOCK PERIOD A t

FIG.10. Multiplier unit.

V=-(V, t v 2 t v 3 t v 4 ) = - -

ANALOG SUMMER

EZA


295

be done serially, using the new XD or yo obtained after each addition to R, for the next addition to R, or some other system must be used to obtain a true digital product xDYD. The quantity XDYD can contain twice as many digits as X D or Y D ;the more significant part will be of digital magnitude, and appear in Z D , the input register of the following unit; and the less significant part will be of analog magnitude, and remain in the R register. The reference voltage E is applied to the digital-to-analog converter connected to register R, producing an output voltage V1 = E R ; the input voltage yAE/a is applied to the converter connected to the register XD producing a n output and the input voltage E x A / ~ is applied to the voltage VZ= EXDYA/CY, converter connected to register YD producing a n output voltage V3 = E y ~ x ~ /An a .analog multiplier is connected to the two analog inputs EyA/a and E x A / ~ Its. output, attenuated by a,produces a voltage V4 = ExAy.k/a. An analog summer sums the voltages V1, V 2 ,V 3 ,and V4 to produce a voltage V equal to -E z A / ~ . During the next At after the voltage V exceeds (or falls below) predetermined threshold voltages, unity is subtracted from (or added to) the R register and the number a is added to (or subtracted from) the input register of the following unit ( Z D in Fig. 10). It should be noted that for small values of CY the analog multiplier may be omitted, producing a maximum error of a. For values of a less than the resolution of the analog components, say 0.001 or less, this error is negligible. If one of the factors to be multiplied is a constant, the equipment required is simplified, since only one digital register needs to be capable of accepting increments, and the R register receives additions from only one other register. If the factor is a purely digital quantity, one of the digitalto-analog converters and the analog multiplier may be omitted. Summing may most easily be done by permitting each integrator or multiplier unit to accept digital increments and analog voltages from several units. For example, in the integrator of Fig. 9, if the f a increments from a number of other units are connected to its X D register, and if the sum of the increments put, out by these units is N a during any period, the increment in X D would equal N a . The analog outputs from the other units would each be connected to an input summing resistor in the analog integrator. I n the case of the multiplier unit, if the f a increments from a number of other units are connected to its x register, and if the sum of the increments put out by these units is NCY,yD would be summed into the R register N times. The analog outputs from the other units would be connected to inputs of a n analog summer whose output would form the analog input XAE/ato the multiplier.

296

HAROLD K. SKRAMSTAD

For a maximum speed-precision product to be obtained, the value of At should be as small as possible consistent with hardware limitations. The smaller At is made, however, the greater the bandwidth required in the

operational amplifiers, since the analog voltages must be capable of a full scale voltage excursion E during the time At. For any particular problem, the part of the variable to be carried digitally depends u p m the particular compromise between precision and speed of solution desired. For example, consider integration of the function x = sin wt and assume At = 0.001 sec and a = 0.001. Since the maximum rate of change of this function should not exceed a / A t , the highest frequency representable a t full scale amplitude would be one radian per second and the precision (assuming an analog precision of 0.001 of full scale) would be one part in one million. If we chose to carry only a factor of ten digitally (a = O.l), the highest frequency representable a t full scale amplitude would be 100 radians/sec and the precision would be one part in ten thousand. It is seen that if analog components of sufficient bandwidth could be produced, a precision speed product of this combined system should be greater than that possible with a parallel digital differential analyzer having equal length digital registers and equal iteration rate by a factor equal to the resolution of the analog components, perhaps a factor of one thousand. It is believed that the greatest usefulness of such a combined system would be on simulation problems where the precision required is somewhat greater than that obtainable by analog methods and that also require the real-time speed of the analog computer. If the precision required is the order of ten to a hundred times that obtainable by analog methods, integrators and multipliers of the combined system would contain short digital registers and only moderate requirements would be put on the speed of switching circuits and the bandwidth of the analog components. Bibliography 1 . Bauer, W. F., and West, 0. P., A system for general-purpose analog-digital computation. J . Assoc. Computing Machinery 4, No. 1 (1957). 2. Bauer, W. F., Aspects of real-time simulation. IRE Trans. on Electronic Computers EC7,134-136 (1958). 3 . Baxter, D. C., and Milsuni, J. H., Requirements for a hybrid analog-digital computer. ASME Paper No. 59-A-304, October (1959). 4. Birkel, G., Jr., Mathematical approach to hybrid computing. Proc. Nat. Symposium on Space Electronics and Telemetry, Sari Francisco, California, Paper No. 2.1, September (1959). 5. Birkel, G., Jr., Hybrid computers for process control. AIEE Transaction Paper No. 6@978, presented at the Joint Automatic Control Conf., M I T , Cambridge, Massachusetts pp. 726-734, September (1960). 6 . Birkel, G., Jr., Scaling and informationtransfer in combined analog-digital computer


297

systems. Proc. Combined Analog-Digital Computer Systems Symposium, Philadelphia, Pennsylvania, December (1960). 7. Blanyer, C. G., and Mori, H., Analog, digital, and combined analog-digital computers for real-time simulation. Proc. Eastern Joint Computer Conf ., Washington, D.C. pp. 104-110, December (1957). 8. Burns, A. J., and Kopp, R. E., A communication link between a n analog and a digital computer (data-link). Grumman Aircraft Engineering Corp. Research Report RE-142, October (1960). 9. Burns, A. J., and Kopp, R. E., Combined analog-digital simulation. Proc. Eastern Joint Computer Conf., Washington, D.C. pp. 114-123, December (1961). 10. Burns, M. C., High-speed hybrid computer. Proc. Natl. Symposium on Space Electronics and Telemetry, S a n Francisco, California, Paper No. 2.2, September (1959). 11. Connelly, M. E., Analog-digital computers for real-time simulation. MIT Final Report No. ESLFR-110, June (1961). 12. Greenstein, J. L., Application of AD-DA verter system in combined analog-digital computer operation. Presented a t the Pacific General Meeting of AIEE. AIEE C P No. 56-842, June (1956). 13. Greenstein, J. L., A two-channel data link for combined analog-digital simulation. Presented at the A I E E Summer General Meeting, Montreal, Canada. AIEE CP No. 57-856, June (1957). 14. Hartsfield, E., Timing consideration in a combined simulation system. Proc. Combined Analog-Digital Computer Systems Symposium, Philadelphia, Pennsylvania. December (1960). 15. Herzog, A. W., Pulsed analog computer for simulation of aircraft. Proc. I R E 47, No. 5 (1959). 16. Horwitz, R. D., Testing systems by combined analog and digital simulation. Control Eng. 6, No. 9 (1959). 17. Hurney, P. A., Jr., Combined analogue and digital computing techniques for the solution of differential equations. Proc. Western Joint Computer Conf ., San Francisco, California pp. 64-68, February (1956). 18. Jackson, A., and Ottoson, H., Air traffic control system simulator. Proc. Combined Analog-Digital Computer Systems Symposium, Philadelphia, Pennsyluania. December (1960). 19. Leger, R. M., Specifications for analog-digital-analog converting equipment for simulation use. AIEE Paper No. 56-860, June (1956). 20. Leger, R. M., and Greenstein, J. L., Simulate digitally, or by combining analog and digital computing facilities. Control. Eng. pp. 145-153, September (1956). 21. Leger, R. M., Requirements for simulation of complex control systems, Proc. First Flight Simulation Symposium. White Sands Proving Ground Special Report No. 9, September (1957). 22. Lee, R. C., and Cox, F. B., A high-speed analog-digital computer for simulation. IRE Trans. on Electronic Computers EC-8, No. 2 (1959). 23. McLeod, J. H., and Leger, R. M., Combined analog and digital systems-why, when, and how. Instr. & Automation 30,1126-1130, June (1957). 24. Neustadt, L. W., and Bekey, G. A., Combined simulation at STL. Proc. Combined Analog Digital Computer S p t e m s Symposium, Philadelphia, Pennsylvania. December (1960). 25. Nothman, M. H., Combined analog-digital control systems. Elec. M.fg. June (1958). 26. Palevsky, M., Hybrid analog-digital computing systems. Znstr. R. Automation October (1957).

298

HAROLD K. SKRAMSTAD

27. Paskman, M., and Heid, J., The combined analog-digital computer system. Proc. Combined Analog Cigital Ccmputer Systems Symposium, Philadelphia, Pennsylvania. December (1960). 28. Paul, R. J. A., and Maxwell, M. E., The genera! trend towards digital analogue techniques. Proc. Secondes Journ6es Intern. de Calcul Analogique, StrasbouTg pp. 403408, September (1958). 29. Peet, W. J., 11, Some aspects of computer linkage system design. Proc. Combined Analog Digital Computer Systems Symposium, Philadelphia, Pennsylvania. December (1960). 30. Schmid, H., Combined analog-digital computing elements. Proc. Western Joint Computer Conf., Los Angeles, California pp. 299-314, May (1961). 31. Shapiro, S., and Lapides, L., A combined analog-digital computer for simulation of chemical processes. Proc. Combined Analog-Digital Computer Systems Symposium, Philadelphia, Pennsylvania. December (1960). 32. Shumate, M. S., Simulation of sampled-data systems using analog-to-digital computers. Proc. Western Joint Computer Conf., S a n Francisco, California pp. 331-338, March (1959). 33. Skramstad, H. K., Combined andog-digital simulation of sampled data systems. Presented a t the AIEE Summer General Meeting, Montreal, Canada, June (1957). 34. Skramstad, H. K., Ernst, A. A., and Nigro, J. P., An analog-digital simulator for the design and improvement of man-machine systems. PTOC. Eastern Joint Computer Conf., Washington, D.C. pp. 9(t96, December (1957). 35. Skramstad, H. K., A combined analog-digital differential analyzer. Proc. Eastern Joint Computer Conf., Boston, Massachusetts pp. 94-100, December (1959). 36. Stein, M. L., A general-purpose analog-digital computer system. Proc. Combined Analog-Digital Computer S y s t e m Symposium, Philadelphia, Pennsylvania, December (1960). 37. Susskind, A. K., Notes on Analog-Digital Conoersion Techniques. Technology Press (M.I.T.) and Wiley, New York, 1957. 38. Urban, W. D., Hahn, W. R., Jr., and Skramstad, H. K., Combined analog-digital differential analyzer (CADDA). Proc. Combined Analog-Digital Computer Systems Symposium, Philadelphia, Pennsylvania. December (1960). 39. West, G. P., Computer control experience gained from operation of a large combined analog digital computation system. Proc. Joirt A I E E - I R E Symposium on “Computers in Control” pp. 95-97, May (1958). 40. West, G . P., Combined analog-digital computing system. Handbook of Automation, Computation, and Control, Vol. 2, Chapter 30. Wiley, New York, 1959. 41. Wilson, A. N., Recent experiments in missile flight dynamics simulation with the Convair “Addaverter” system. Proc. Combined Analog-Digatal Computer Systems Symposium, Philadelphia, Pennsylvania. December (1960). 42. Wilson, A. N., Use of a combined analog-digital system for re-entry vehicle flight simulation. Proc. Eastern Joint Computer Conf., Washington, D.C. pp. 105-113, December (1961). 43. Wortzman, D., Use of a digital-analog arithmetic unit within a digital computer. Proc. Eastern. Joint Computer Conf., N e w York, pp. 269-282, December (1960). 44. Wright, R. E., Analog-digital computer linkage system for Bendix G-15. Proc. Combined Analog-Digital Computer Systems Symposium, Philadelphia, Pennsylvania. December (1960).

Information Technology and the Law REED

C. LAWLOR

Potent lawyer, 10s Angeles, California Chairman, Electronic Data Retrieval Committee

of the American Bar Association

1. 2. 3. 4. 5. 6.

7. 8. 9. 10. 11.

Introduction . . . . . . . . Information Growth . . . . . . Mechanization in Law Practice . . Applications of Symbolic Logic to Law Information Storage and Retrieval . Punched and Notched Cards . . . Prediction of Court Decisions . . . Thinking Machines . . . . . . The Law of Computers . . . . . Use of Computers in Court . . . . New Horizons . . . . . . . Bibliography . . . . . . . Exhibit “A” . . . . . . . .

.

.

.

.

.

. .

. .

. .

. .

. .

. .

. .

.

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

.

.

.

.

.

.

.

.

.

.

. . .

. . .

. . .

. . .

. . .

. . .

. .

. .

. .

. .

. .

. .

. .

. .

. .

. .

. .

. . .

. . .

. . .

. 299 300 . 302 . 305 . 3 10 . 323 . 324 . 334 . 335 . 339 . 340 . 343 . 347 .

1. Introduction

The information explosion that,is occurring in other fields is also occurring in law. Today the lawyer must cope with more laws, more regulations, more problems, and more judicial and administrative bodies than ever before. His old methods of work are inadequate. He needs new tools. Many of the tools are here. But scientists and engineers must help him learn to use these tools. The conjunction, “Information Technology and the Law,” is commutative and therefore symmetrical. When the relation between information technology and the law is examined a little more closely, it is soon recognized that two oppositely polarized relations exist. We are not only concerned with how the new technology may be used in law, but also with how law applies to the new technology. Lawyers today are interested in making use of information technology in recording, classifying, analyzing, utilizing. and applying the law. In addit,ion, lawyers are becoming con299

300

REED C. LAWLOR

cerned with the application of the common law, which has evolved through centuries of trial and error, and also modern statutory law, to problems that are being created by information technology. Some lawyers are turning to the computer and to modern mathematical science to help them in the solution of all their problems, both old and new. At the same time, the use of computers is creating new legal problems which can be solved best by an understanding of computers as well as an understanding of established legal principles. The purpose of this paper is to provide an overview of the activity occurring a t the interface between information technology and law and to suggest which problems may be most significant from the standpoint of the lawyer, the courts, and the public as a whole. I n this paper we will take some short excursions into a number of technical fields, including the fields of symbolic logic, information retrieval, prediction of decisions, and the law of computers.

2. Information Growth

The rate of growth of information is very similar in many fields. A survey of the magnitude of the task that confronts the legal profession has been discussed by Layman E. Allen et al. [4, 51. That the growth of information in the field of law is similar to that in the fields of medicine and biology is apparent from Fig. 1. I n this figure, the number of new periodicals first published in the respective fields in various years is indicated. I n Fig. 2, the number of words in the federal statutes in force in various years are indicated [fir].It is interesting to replot this data to semilogarithmic scales to ascertain to what extent the exponential law of growth applies. This has been done in Fig. 3. From Fig. 3 i t is apparent that the growth of the number of periodicals is regular in the fields of medicine and biology; but it is erratic in the field of law. Nevertheless, the rate of growth in the number of periodicals is about the same in all three fields, and constant over long periods, i t . , growth in all three fields is truly very nearly exponential. But when we examine the number of words in the federal statutes, we find that the rate of growth is not constant but is increasing. We leave it to others to determine whether this is good or bad. But we wish to suggest that what America needs today may be heroic legislators and legal draftsmen who will spend their time repealing laws and reducing them to fewer words. For example, it is suggested that the number of words in the Internal Revenue Code could be reduced by at least 70y0without loss of revenue or fairness. There is, of course, a danger that when computer technology and other

INFORMATION TECHNOLOGY AND THE LAW

30 1

phases of information technology are applied extensively to the field of law, instead of making matters better, they may make them worse as far as quantity is concerned. Just as freeways result in inflation of traffic problems, there is danger that computers may inflate the information problem. 900

-

825

-

750

-

675

-

600

-

525

-

450

-

375

-

300

-

225

-

150

-

FIG.1. Rates of growth of periodical literature in medicine, biology, and law.

This might arise because of the reliance that legislators and lawyers, and others, too, will place on the high speed and large storage capacity of computers. It is important, therefore, when considering the application of information technology to law, that some effort be made to restrain the tendency that one might otherwise have to use many words and to cover many contingencies unnecessarily. Unless this is done, the rate of growth of legal information may increase to such a n extent that even the largest

302

REED C. LAWLOR

FIG.2. Growth of federal statutes in force in the United States.

and highest-speed computers will not be able to cope with the information problems that computers themselves have encouraged. This warning against information inflation also applies just as well to fields other than law.

3. Mechanization in l a w Practice

Though he was not the first to suggest the application of modern information technology to law, Dr. Lucien Mehl, the French jurist, set forth a nice prognostic summary of the possible use of automatic equipment in law in November, 1958, in the following words [5‘7]: “It may seem an ambitious step to try to apply mechanization or automation to the legal sciences. However, a machine for processing information can be an effective aid in searching for sources of legal information, in developing legal argument, in preparing the decision of the Administrator or Judge, and, finally, in checking the coherence of solutions arrivcd at.”


I

303

I

FIG.3. Growth of information in medicine, biology, and law.

Mehl suggested that two types of “law machines” might be developed: (1) “The documentary or information machine” and (2) “The consultation or judgment machine.” We will have occasion herein to refer to the application of computers to some of the problems mentioned by Mehl and

304

REED C. LAWLOR

also to some of the more prosaic applications of modern technology to law. First we will consider the most simple. There was once a time when it was undignified to prepare legal documents with a typewriter. Greater respect was given to documents written in longhand than to documents written with the typewriter. This is still true about wills. The problems with which we are primarily concerned relate to the use of more modern mechanical devices to aid lawyers in service to their clients. Today the lawyer receives many of the benefits of the first industrial revolution in that mechanical means, including typewriters, dictating machines, and telephones, are accepted as ordinary tools of the profession. But few lawyers have utilized the fruits of the second industrial revolution in their practice. Today a few law offices have turned to data-processing systems to help them in the analysis of internal activities of a law office and, more particularly, to prepare records of the time spent on legal research, in conference with clients, and in the study, analysis, and preparation of documents in order to determine what is a fair charge to make for services rendered and also to increase the efficiency of operation of a law office [55]. Shell Development (Emeryville, California) utilizes computers to schedule work of its Patent Department. Johnson and Johnson (New Brunswick, New Jersey) utilizes computers to evaluate and preserve its trademarks in various countries and to monitor its contracts respecting trademarks. Carl G. Paffendorf (New York) has developed computer programs which can prepare plans for estates, taking into account numerous aspects of the law and a client’s financial and personal problems. Kenneth L. Black, of Auto-Typist Institute (Gainesville, Florida), now offers a service utilizing Auto-Typist technique for composing legal documents that utilize standard paragraphs. A few lawyers have turned to the use of automatic typewriters which use punched tape, either to reduce the expense of rewriting documents or to aid them in the composition of documents that make use of standard paragraphs. The list includes Homer Montague (Washington, D.C.), Scott D. Kellogg (Oakland, California), and Harold I. Boucher (San Francisco, California). Such methods will go into wider use when inexpensive silent typewriters capable of preparing machine-readable records are available. To the best of the author’s knowledge, no lawyer has yet used punched tapes or other machine-readable records of documents prepared by or for him for the production of indices to those documents. There is an unrecognized need for an inexpensive technique which can be used to index automatically, in alphabetical order, the significant terms appearing in legal


305

documents of all kinds, whether they be contracts, statutes, articles, depositions, or trial records. Very few lawyers realize that automatic indexing is even possible.

4. Applications of Symbolic logic to law

The application of symbolic logic to law is not new. The very first problem solved by George Boole in his work, Laws of Thou& is a problem in Jewish dietary law. By means of the class calculus originated by him, George Boole solved the following problem [ I S ]: “Define the classes of foods that exist and those that do not exist under a law that permits a person to eat only clean beasts, which are defined as those which both divide the hoof and chew the cud.”

In this work, Boole also showed that there is an interrelationship between his symbolic logic and the theory of probability. He applied his logic to the analysis of jury verdicts in the chapter entitled, “On Judgments. ” Edmund C. Berkeley, in an article published in 1937 dealing with ‘%oolean Algebra . . . and Applications to Insurance” [ I I ] , employed the class calculus to analyze problems involving the crediting of last payments made under inconsistent rules of an insurance company. I n about 1952, members of various committees of the American Bar Association began to exchange communications on the applications of symbolic logic to law. More particularly, early attempts were made by Miss Elizabeth Freret (Washington, D.C.) , John E. Scheifly (Los Angeles, California), and Vincent P. Biunno (Xewark, Kew Jersey) to apply both the class calculus and the propositional calculus to problems arising in the application of various sections of the Internal Revenue Code. Without doubt, others also have tried their hands a t the art. This interest may have been stimulated, in part a t least, by the work of Edmund C. Berkeley [ I l l and also by an article on the subject of symbolic logic by John E. Pfeiffer, published in Scientific American [ H I . In that article, Pfeiffer reported that symbolic logic had been used by a number of insurance companies in the analysis of war damage clauses and group insurance contracts. However, attempts to identify this earlier work have not borne fruit. In 1956, Ilmar Tammelo proposed that the first-order functional calculus be applied to law [65, I S ] . In 1961 Ward Waddell, a lawyer of San Diego, California, who never went past calculus in college mathematics, authored a fine work that also relates to the application of the first-order functional calculus to law, with particular emphasis on the Hohfeldian theory of

306

REED C. LAWLOR

rights and duties [68]. So far as is known to the author, however, no one has succeeded in making use of the first-order functional calculus in the development of any new legal theory or in the solution of any legal problems. Commencing in 1957, Layman E. Allen, a mathemat,ician and Professor of Law at Yale University, published a series of articles [ 2 4 ] showing how symbolic logic could be applied to various problems, including the analysis of contracts, the organization of arguments, and legislative draftsmanship. He has concentrated (but not exclusively) on the development of techniques that avoid mathematical symbols, thus opening the door to the use of symbolic logic techniques to lawyers having a minimum background in mathematics. Some of his methods distribute unabbreviated naturallanguage sentences vertically on a page and use the expressions “or” and “and” to express logical connections, and horizontal lines extending across the page to express implication and coimplication relations. Allen has also described new techniques for diagramming the relationships of alternative subsidiary propositions that apply under the terms of a compound legal proposition [S]. Figure 4a represents a compound sentence that expresses many alternative subsidiary sentences, each of equal apparent validity, while Fig. 4b represents a compact diagrammatic representation of the various parts of that compound sentence arranged in such a way as to facilitate the recognition of different subsidiary propositions (an ambiguity that requires a more complex symbolism has been omitted in this presentation). Notice the switch-circuit character of the diagram. Every possible set of connections that completes a circuit from one side of the diagram to the other corresponds to a different subsidiary proposition that may be valid under the rule of law expressed by the compound sentence. I n this particular case, the single compound sentence is capable of leading to at least twenty-four correct interpretations. Allen has also utilized similar diagramming techniques for analyzing syntactic ambiguities that arise because of the various meanings of the words (‘and” and and the difficulties of interpretation created by uncertain association of modifiers with the terms modified [S]. A typical example involves the question as to what is meant by the following statement: “The general purposes of the provisions governing the definitions of offenses are: (a) To forbid and prevent conduct that unjustifiably and inexcusably inflicts or threatens substantial harm to individual and public interests; .”

..

I n a survey made by Allen he found great disparity in the opinions of both lawyers and law professors as to what subsidiary purposes that can be composed from the foregoing predicate were purposes that were intended by the author of the statement.

"In all that concerns the riglit of acquiring, possessing or disposing of every kind of property, real or personal, citizens of the United States in Serbia and Serbian subjects in the United States, shall enjoy the rights which tile respective l a w grant or shall grant in each of these states to the subjects of tlie most favored nation." (it)

-

w

0

acquiring

v

I n all that concerns the right of

-

-4

or posse+-

or

+ disposing

-

of every kind of

property,

citizens of tlie United States --. in Serbia or

Serbian subL jects in the United States

-

,

shall enjoy the rights whicli tlie respective laWS

-

in eacli of these states to tlie subjects of the most favored nation.

(b)

FIG.4. Analysis of compound proposition: (a) compound proposition; (b) switch path equivalent with test.

'

-

308

REED C. LAWLOR

In July of 1958, Bryant M. Smith, writing anonymously, published an article in the Stanford Law Review [64a] on the subject of "Mens Rea and Murder by Torture in California." Though the body of this article was written in plain legal English, the footnotes were written in symbolic logic. The author has developed a mathematical theory of patent claims [49]. He has pointed out that patent claims define classes of objects, processes, or compositions of matter. I n his method the claims are rewritten, however, as logical products of propositions; and then the propositional calculus is employed to determine whether an object described in terms of the same propositions do or do not fall within the scope of the patent. This work has led to the development of a computer program for designing around a patent, whenever possible, while still complying with the specifications of a manufacturer. A committee in the United States Patent Office has under consideration the adoption of a rule permitting claims to be rewritten as strings of sentences, instead of as long predicates composed of strings of clauses. The 1att.er style of writing, which has been in use for many, many decades, has been condemned by Judge John R. Brown, of the Fifth Circuit Court of Appeals [1 4 . The relative understandability of claims written in the current style compared with claims written as strings of sentences can easily be appreciated by the reader by the following examples. Under current practice, a patent on a simple, fictitious electric light (invented many years ago) might be written as follows:

I CLAIM: 1. An electric light including: a base having two mutually insulated electrical contacts mounted thereon; a transparent envelope rigidly connected to said base; and a coiled metal filament within said envelope, the two ends of said filament being electrically connected to said contacts. 2. An electric light including: a base, a transparent envelope sealed to said base , two electrical conductors extending through said base into the space enclosed by said envelope, and a coiled filament within said envelope. The same invention could be just as adequately claimed in simple Rudolph-Flesch grammar-school English, as follows: I claim as my invention an object described in one of the following paragraphs: 1. The object is an electric light. The light includes a base. Two mutually insulated electrical contacts are mounted on the base. The light includes a transparent envelope. The envelope is rigidly connected to the


309

base. The light includes a filament. The filament is composed of metal. The filament is coiled. The two ends of the filament are connected to the contacts. The filament is within the envelope. 2. The object is an electric light. The light includes a base. Two electrical conductors extend through the base. The conductors extend into the space enclosed by the envelope. The light includes a transparent envelope. The envelope is sealed to the base. The light includes a filament. The filament is coiled. The filament is within the envelope.

To the untrained eye of the layman-whether he be a non-patent lawyer, scientist, or judge-which type of claim conveys more clearly what is covered by the patent? Which is better: a short claim that is a complex predicate of a long sentence, or a long claim made up of a string of simple sentences? The practical application of symbolic logic to law has been retarded partly because of the lack of training of lawyers in this field, partly because of the Tower-of-Babel, jungle-like, and sometimes contradictory varieties of notations employed in symbolic logic, which impede the study and learning of symbolic logic by nonmathemnticians , and partly because the symbolic logicians, being primarily pure scientists, are more interested in extending and developing theories than in finding everyday uses for them. Lawyers need the help of logicians in pointing the way to learning the symbolic logic methods of solving their problems. To be most helpful, the logicians need to improve their methods of communication with the outside world. As a step in this direction, Layman E. Allen has invented a dice game that is used for teaching logic in Polish notation to six-year-olds and up [7], thus outdoing Lewis (Alice-in-Wonderland) Carroll [I 71, who aimed to make symbolic logic understandable to intelligent fourteen-year-olds and up. It also helps when authors illustrate principles of logic with law problems [ l l ,651, however simple they may be. We have little knowledge as to the extent to which symbolic logic is being applied to law in various foreign countries. However, we do have information that the study of symbolic logic is required in the law schools of Poland [70].I n America, extensive courses in symbolic logic have been offered, so far as I know, only under the leadership of Layman E. Allen a t the Yale Law School. Without doubt, symbolic logic is being studied by law professors and law students in many other law schools. It is to be hoped that scientists and engineers will encourage interest in this field whenever the opportunity to do so is presented and that they will help lawyers who are eager to learn, but who lack advanced mathematical backgrounds. I n this way, the disciplines of law and modern logic can join hands for the benefit of the public. But Boolean algebra is not enough. It is only a n intro-

310

REED C. LAWLOR

duction. In order to apply symbolic logic effectively to low, one must go beyond Boolean algebra. One must utilize propositional calculus and modal logic, particularly deontic and alethic logic. A great opportunity awaits the logician who can translate advanced logic into a language which can be used by ordinary people for solving everyday complex problems. 5. Information Storage and Retrieval

There is a great need for the use of computers for storing legal information and for finding relevant stored material and for presenting it in a form most useful for lawyers. This need was first pointed out by Lewis 0. Kelso [@I in 1946. He said: “Today the lawyer works substantially as he worked before the industrial revolution. Only automatic legal research will save him from plying one of the most confused, ill-paid, and unsatisfactory professions in the world of tomorrow.”

In law, research means finding and analyzing the law applicable to a problem. The most important immediate problem which is of interest to the bar, to clients, and to courts and which might be solved by computer technology involves the development of more efficient methods for finding published decisions and statutes which are relevant to a particular set of facts, so that a lawyer can analyze these decisions and statutes and present them in an orderly fashion for a judge to use as a basis for arriving at a just decision in the particular case before him. I n addition, our legislators are exceedingly interested in knowing what laws have been enacted which relate to different subjects, so that better laws can be prepared and so that superseded, inconsistent laws can be removed from the books. I n the United States there are fifty-two main governments-the federal government, the government of the District of Columbia, and fifty state governments. Under most of these, we have various county governments, city governments, boroughs, and others. In addition, each of these governments has many administrative agencies. The number of statutes and regulations to which individual citizens are subject, and to which professional men in various fields are subject, is increasing more than exponentially. Perhaps we can improve our legislative method some day so as to reduce the number of statutes and regulations. But for the present, it looks as if there will be continuous growth of such statutes and regulations. For this reason today’s problem is to develop improved techniques for finding which statutes, regulations, and cases are applicable to particular sets of facts or particular individuals or groups. There are approximately 29 million Appellate decisions published in


31 1

the United States. Some of these are Supreme Court decisions which apply to all of us. Some of them are decisions of local courts-either federal, state, county, or city-that apply to some of us in particular geographical areas. Because of the origin of laws in various areas of the country and because of the precedents which are at the foundation of the decisions in different areas (such as the New England area, the South and West), the rules of law applicable in these different areas to the same situation are sometimes very different; in some cases, they are almost contradictory. To eliminate the apparent contradiction between the rules of law in different jurisdictions may be impossible. But knowledge of the laws of other jurisdictions is helpful to courts in applying principles of law to particular sets of facts. Today it is often like looking for a pearl on a pebbly beach when a lawyer goes to the law library to find a case which would be recognized as a precedent in a particular jurisdiction and which applies to the particular set of facts in his client’s case. Often the lawyer can’t even tell whether there is a pearl among the pebbles. He is forced to sample the pebbles manually by picking them up in small batches and to study many of them carefully with his legal eye before he dares conclude whether or not there is a pearl and, if so, whether it is black, or white, or grey. I n practice, the presentation of legal arguments involves the citation of cascs which it is believed by the lawyers should be considered by a judge in arriving at a decision. Finding all applicable cases and finding all applicable statutes by manual methods and selecting representative cases to cite is difficult and costly. A properly designed and operatcd machine-processing system can locate a pearl on a pebbly beach with greater certainty than can a lawyer, and with the expenditure of less lawyer effort per pearl, and for fewer client dollars per pearl. This is especially true when many different searchers are trying to locate different pearls on the same beach and where every searcher can make use of the same pearl if it applies to his problem. The first suggestion to apply mechanical searching systems to the field of law seems to have been made by Lewis 0. Kelso [@I. Stimulated by the earlier suggestions of Dr. Vannevar Bush to mechanize searching methods in scientific fields, Mr. Kelso proposed the use of a “Law-dex” system similar to the present Minicard system and Filesearch as early as 1946. In about 1955 Vincent P. Biunno, of the New Jersey Law Institute, proposed the use of a continuous moving tape for making simultaneous searches in the field of law. I n the system suggested by Biunno, all information that someone might want to retrieve would be recorded on tape, and this tape would be moved continuously past a number of readout stations. This would make possible the simultaneous solution of a number of law problems for different lawyers, possibly greatly reducing the cost per

31 2

REED C. LAWLOR

search. Chernerin [see 441 suggested a number of years ago that perhaps a good way to distribute and retrieve information might be to transmit this information continuously on a radio beam, so that selected parts of the information could be retrieved by those in need of the information at various receiving stations. While the suggestion was made in connecton with other problems, this method could also be applied to law, at least theoretically. The efficiency and effectiveness of any such system could be increased by employing a large number of carrier waves simultaneously, so that the cycling period for the broadcast of all decisions could be shortened. But it may be more economical to duplicate magnetic tapes or other recordings bearing legal information and flying them to various computer centers. The problem of storage and retrieval of legal information divided itself into a number of steps: The selection and analysis of the information to be stored. Selection and application of the coding process, if any is to be used. Selection and application of the storage method. Analysis and preparation of questions to be answered in machine language. (5) Retrieval of the desired information. (6) Display of the retrieved information. (7) Marketing of services. (1) (2) (3) (4)

The preparation of the legal information that is to be stored involves the selection of the material to be stored and also involves the analysis of the material in preparation for storage. One basic question that needs to be answered when one considers how to store legal information involves the question as to whether or not the full text should be stored verbatim. By full-text verbatim storage is meant placing every word of a decision or statute on tape in natural language, letter for letter, in alphanumeric code. There is a great advantage of verbatim storage. If the full text is stored verbatim, mistakes of omission and mistakes of commission-prejudice, if you please-that might otherwise be made by predigestion of the law prior to storage are avoided. On the other hand, storage of full texts requires greater storage capacity and longer search time. Full text storage is similar to color photography. It contains much more information than precoded analysis of the same material. For many purposes color photography is superior to black and white photography, especially if definition has been lost in a coarse process such as stippling. Besides verbatim storage of full text, it is also possible to store digests verbatim, whether they are prepared by professional digesters or whether they are prepared by automatic abstracting methods. Another method is


313

to prepare lists of key words which characterize the material in question. Another method is to prepare concept profiles based upon a limited selected list of words [&?I. The application of electronic data-processing systems to legal research was successfully demonstrated publicly for the first time in the summer of 1960 [37]. These demonstrations, which were conducted a t the American Bar Association Convention, were sponsored by the Electronic Data Retrieval Committee of the Bar Activities Section of the American Bar Association, the Health Law Center of the University of Pittsburgh, and the United States Patent Office. These demonstrations fulfilled a n eightyear-old dream of the Electronic Data RetrievaI Committee of the American Bar Association. At these demonstrations, two types of IBM computers were employed for storage and retrieval of legal material. For these demonstrations information of four different kinds was stored. On the IBM 650, the following three types of information were stored: (1) The full texts of all of the hospital law statutes of all the states were stored verbatim and also some labor law material furnished by Lawyers’ Cooperative. (2) Key words characterizing the facts present in all decided foodproduct liability cases were stored. (3) The digests of all the oil and gas law cases of the past two years were stored verbatim.

This application of the IBM 650 was developed a t the Health Law Center of the University of Pittsburgh. John F. Horty, Director of the Health Law Center, and his associates a t the University of Pittsburgh deserve credit for being the first to apply electronic data-processing systems to law problems successfully and for demonstrating this work publicly. They applied it t o the hospital law statutes. The digests of the food-product liability cases were provided by F. Reed Dickerson, Professor of Law, Indiana University School of Law. The digests of the oil and gas cases were provided by Robert A. Wilson, Vice-president of the Southwestern Legal Foundation, of Dallas, Texas [S7]. In addition, a t the ABA Convention key word digests of headnotes of all of the design patent law cases which had been decided in the previous 305. These twenty years were stored on magnetic discs of the IB M RAMAC digests were supplied by the Bureau of National Affairs. The preparation of the material and the demonstrations were made under the direction of Donald D. Andrews, then Director of Research and Development of the United States Patent Office [s]. The material thus stored on magnetic tapes and magnetic discs was

REED C. LAWLOR

314

retrieved in response to logic equations in which the terms represented different English words that had been stored. The stored material was also employed to prepare indices of the text. There are approximately 430 hospital law statutes. Each of these hospital law statutes was assigned a different document number. The computer indexed the stored material automatically. Table I represents a portion of the word index, or dictionary, produced automatically. Here each word is listed in alphabetical order, and opposite each word is a list of document numbers that had been assigned to the different statutes. Words such as TABLE I. PARTIAL AUTOMATIC INDEXTO HOSPITAL LAWSTATUTES BY JOHNF. HORTY IMPRISONtD -Lkt?RLSONMENTIMPROPER _.IMPROVED IMPROVEMENT IMPROVEMtNTS IMPROVING 1N C .. INCAPABLE INCAPACITY INCIDENT INCIDkNTAL I NCLlJDE

_.

INCLUDED INCLUDES INCLUDING __ I N C L U s I ON-

_ _I N C L U S I V E INCOME - - -- __ . _.

.

3 .

INCUMPtTtNT INCONSISTENT INCONVENIENCE INCORPORATtD

__ INCORPORATION I_ N_ C O R P O H A T O K S INCREASE -_II NN CC UU MR BR EE RD t D _ _I_N_C U R R I N G IND -

-.

77

351 51 428 11 242 352 428 337 170

-.

--

59 144 240 399 87 339 35 1 410 102 102 146 330

323 361 406 97 196 2 46 356 397 57 153 360 141 406 355 84 86 10 279

-

61 232

..

227 389

386 393

148 351 140 376 383

431 175 412 404

20 24) 354 42Y 345 32 339 368 407 98 216 4 129 366 403 130 154 383

260 424 431

?52 43 1

31

L74

1 10 L IY

393

344

r l 344 314 409 116 244

133 348 376 415 118

134 350

136 36 7 40 7 132 155

246 368 413 138 156

249 369

98

123 359

26 220 5Y 1

38r

42 1 122

146

15 7

88

11 280

12

358


315

llthc,” “of,” “and,” and the like, which have no index value, were cscludcd automatically. Table I1 represents a list of statutes which was produced by thc IBM 650 in response to the question:

“Please supply me with a list of statutes and copies of statutes dealing with restrictions upon remuneration for officials of charitable corporations.” Table I11 shows the text of several of the statutes actually supplied in response to this same question. I n order to retrieve such information, it is necessary to prepare a logical expression relating the various terms that are in the dictionary (Table I) with the terms that are in the question. Such an expression is represented in Fig. 5 . Here it will be noted that the question is in the form that reLAWYER’S QUESTION Please supply me with a list of statutes and copies of statutes dealing with restrictions upon remuneration for ofEciaIs of charitable corporations. COMPUTER QUESTION (a

+ b + + d + e + I N S + h + i +j + k + 0 c

Q

6 c

d e f

trustee = trustees = officer = officers = director = directors =

compensation h = compensations i = salary j = salaries k = remuneration I = wages g =

FIG.5. Type of question employed to retrieve hospital law statutes.

sembles an expression of a formula of symbolic logic. Actually, the expression is the logical product of two terms, each of which is a logical sum of several key words. The sign means “or”; the juxtaposition of two groups of parentheses represents “and.” The computer is programmed to identify and retrieve each document that contains one or more words in the first parenthetical expression and one or more words in the second parenthetical expression. The preparation of the logical expression depends, of course, upon the type of retrieval technique employed. While methods could be employed for retrieving information according to word roots or by coded terms, the method employed here was one of selecting statutes which included words identical with those in the logical expression. Whenever such a matching technique is required, as in this case, it is necessary to take this fact into account in the preparation of questions, EO as t o include in cach logical

“+”

CONN. __.. GENo STAT. =S 12-81__ /1958/. ME. REV. STAT. ANN. C * 91-A* = S 10 /SUPP. 1957/* -_....N ?..- TAX -. .. ....LAW .. = S 4.___ _. .. ...~. .. TENN. CODE ANN. = S 67-502 / 1 9 5 5 / . . w 1.5...ST *.. .*N N 0.. .=?. .7 0*.!?- !.-I ? ?.I.9 .. AS: . .AMEN PEP. 3 U!!!?r-

*-v

*

I I

338.-.

366 -.... ..... -- .. 3 8P -. 415 -1.95.9-l e ... ... .. __- - _.... ...... . ._._ .. .... ._... ..43.1.. .

__

-_ _ _ _ _ _ _ _---. . ----

TEXTOF HOSPITAL LAWSTATUTES RETRIEVED IN RESPONSE TO QUESTION IN FIG.5 TABLE111. PARTIAL ____I________I

.....

NEB. REV. STAT. SS 21-1503 / R E I S S U E 19541. .- - -=-S- -2_1-1503. - ....... -5 T.OCK- *...D-I-V I DE.NDS~I-.-SA.LARL.E.S_I..-PROHI13 I J E D . . . . . . . . . SUCH / N O N P R O F I T / C O R P O H A T I O N S H A L L H A V E NO C A P I T A L S T O C K 9 AND SHALL P A Y N.0 D I V I.D.ENDS-OR- S.ALAR_I.ESf.Q_IJ~S.__1NCDRF(ORATORS . OR . BOARD OF DIRECTORS. .- - - - - - - - - - - _ - - - - ..- _ .... - - - - - .. - - .. - . - - - - - .......................................... NEV. REV. STAT. -5 81.310 /1957/. . - . ..... - ....... - . . . . . . . . . . . . . --=-S--81.*.31?- .POW SRS ..OF. CORPORA? I QE! *- .... THE /NONPROFIT/ CORPORATION SHALL, A S A N INCID~NT OF ITS PURPOSE AND WITHOUT A.NY_NECE~.S.I~_Y_FI!R_E_XPRE.SS-I_N.G__LH.E_.SAME..IN ..ITS . A R T I C L E S OF . I N C O R P O R A T I O N * H A V E THE F O L L O W I N G POWERS9 W H I C H I T MAY E X E R C I S E I N .- - - - -F-U-L....... L MEASUKE ~ W . I T ~ H O U T ~ ~ ~ T H _ E _ N C E S S ~ I ~ T Y . ~ O F . ~ ~,ANY,.OKDER O ~ T A I . N I . N GO F COURT B Y A U T H O R I Z A T I O N , APPROVAL OH C O N F I R M A T I O N * * ...................... AND -._ P A Y ...OFFICERS_-AND.-AGE.NTS. . - ....... .T.o..CONDUCT AND ..1-5 ..... 1 . T-O APPOINT . A D M I N I S T E R THE . A F F A I R S OF THE CORPORATIONI BUT NO MEMBEK O F T H E BOARD 0F f RU S f E , E ~ ~ ~ ~ H ~ ~SA?.IP.N Pe.................................... . , E N

147

NEV. REV. STAT. =S 85.050 /1957/r ..................... . ... . = s t35.050 T R U S T E E S NOT T O RECEIVE COMPENSATION.* EXCEPTION. .- - - - -NO - - ..TRUSTEES - - ... - . -O. F ~ ~ T H E ~ ~ / N O ~ N ~ P ~ R ~ O F I ~ T ~ / ~ . ~ C ~ OSRHPA~LOL . RBE A T IEONNT I T L E D TO ANY COMPENSATION E X C E P T UNDER SOME SPECIAL EMPLOYMENT OF THE B O A R D * OR A U T H O R I T Y EXPRESSED I N THE O R I G I N A L DEE-D--~.K~-I,N,ST.RU,ME,NT..OF TRUST. . ...

149

-

__

_ _ - _ ***

U

-

4

I,

-

WYO. COMPI STAT. ANN. =S ... 44-1009 ................................ 119451. . . . = S 44-1009 . O F F - I C E R S R E C E I V E NO S A L A R I E S . NO O F. F I C E R OF.................................... ANY C O R P O R A T I O N FORMED.,,UNl3ER . T H E . P R O V I S I O N S .OF., .THE. 9 T H S U B - D I V I S I O N OF = S E C T I O N 1 / = S 44-1001 = H O S P I T A L S AND A H~I~ ES/ ANY cI NoF~I R~MR ~ - SNH~A LRL - R- E~ C~E-I V~E- ~ - ~S - A~ L"A-R~Y ~OR ~F i- REMUNERAT s ~ - ~ - u - !O,N,,,F.K,OM..,SUCH. ~ - H - o - F

,

150 e.0

_,_,

_,

$

318

REED C. LAWLOR

sum (. . .) all different spellings of a word found in the dictionary and, likewise, all different grammatical forms and inflections-such as singular and plural forms, variations due to differences in case endings, declension, conjugation, and the like, and even different parts of speech, involving the same concept. Furthermore, antonyms and synonyms must be grouped together. The reason for this is that negatives and double negatives might otherwise cause the retrieval system to overlook important documents. For example, to search all statutes that include the term “sanitary” would fail to recover documents that include the phrase “not unsanitary.” For this reason the search must be made for the logical sum of the two words “sanitary” and “unsanitary”. Richard F. C. Hayden (former chairman of the Electronic Data Retrieval Committee of the American Bar Association and now judge of the Superior Court in Los Angeles, California) has suggested that words that must be considered equivalent for search purposes be called “searchonyms,” regardless of whether they are synonyms, anyonyms, homonyms, or misspellings. One advantage of recording the full text without predigestion by an abstracter involves the fact that the stored record of the full text may then be examined at any later time by a person who has a different interest or a different point of view from the one who prepared the abstract. This has now been done with all the statutes of Pennsylvania. Table IV, for example, shows a small portion of a list of the Pennsylvania statutes which was retrieved in response to the question: “Where does the word ‘patent’ appear in the Pennsylvania statutes?” A study of these statutes reveals some that deal only with land patents, some that deal only with patents on inventions, and some that deal with so-called patent medicines. This simple question could be answered by reference to an automatically generated dictionary. But more advanced computer programs now undergoing development at the University of Pittsburgh may be used to select one of these groups of laws in preference to the others. But since the machine has difficulty with negates, it may be difficult to search for patents that are not land patents. How soon searching techniques can be developed that are free of this limitation can only be guessed. A question seeking hospital law statutes dealing with copyrights quickly brought forth the answer, “None.” How long would it have taken a lawyer interested in this question to convince himself that there was no pearl on the beach if he had worked without a computer? A telephone call to a law computer center could give him the answer with little effort on his part and at much lower cost to a client. Some comparative tests of the speed and reliability of lawyers and computers in finding the law would be desirable. In the demonstrations that were made at the American Bar Association

LISTOF PENNSYLVANIA STATUTES INCLUDING THE WORD“PATENT” TABLEIV. PARTIAL U i l l V E Y S l T Y O F PITTSdtldW CH A 5 j O C I A T l O N ANC THE L E T T E H S C H ASSOCIATION AND THE L E T T E H S ROOFS C F A 9 V E R T I S E k ’ E M T t H A Y F A THE CO@C43hrEALTHr A:YD L € T T E Y S UHOC( ONE S H A L L U E A Y EXPFKT I N LETlEN5 ARY R E k i E D l t S r P d P U L A H L Y C A L L L D L E OF P R O P R I C T A H Y OR $ + C A L L E D WAYUFACTUREH5 AND VENDOR‘> OF C f U h l N c j 0.4 V E N O I d G NtiSTi<xJ:NS OH NY Y A h U F A C T U K f U 3F H0ST)i:ltYS OH H E H C A H T I L E LICENSES ACT* MERCANTILE LICENSE5 ACT* C E R T A I W A C T 5 t‘XTFNDFi? TO PLKANTILE L I C C N S F . S ACT. ON OF THE A R T I C L E S AND L L T T L R S ihy OF THt A h T I C L C S AYG L t T l r ’ k 5 I O f O FUHTtiiRr 1 H A T NEW L E T T F X S I D E D FURTHCRo TWAT l l E U L C T T E H S NEW LETTEHS COLON OF A CHARTFR OR L F T T E K S I N TMF. SAYE CHARTER .if7 LFTTERS OLOR O f AYY CMAHTEH 04 L E T T C A S VAL or THE G U V E R : ~ . uwi rHE W N 1 2 E D F 3 H AE4Y L A N D AF1f.R THL COLOR OF A C H A k f E H OF LETTI.HS L H PERSl)!i I f 4 TME * S N I l L n :.TATE:> VICE-NARK I * THE dPi1TFiJ ‘ . T A T F S T O W A N T .sQcn PEE5 FOR

KEY.*WO-IN-Cl)kTfXT PATLNT PATENT PATEYT PATiNT PATENT CATEnT PATEhT PArmr PAIEkT PATENT PATENT PATENT PATFNT PATENT PATENT P9TENT PATENT PATENT PATENT YATEHT PAlENT PATENT PATEN I PATENT PATtNT PATEYT PATCNT PATE”r1 PA T .E:+ T P A r w

19/1&/

I S S U E D T O SUCd A S S O C l A T l O N r W E ISSUED 10 S U C H ASSOCIATlONr Yt+€ I ~ . ! ~ . ; l € O TtiEREON TO T M NANLD A P P I 5 S f J E 3 IHCHEr”N* U N T I L A T L E A S T LAW. A h 0 THE OTHER TNiJ SHALL B E ,NAY OE l S 5 t J E D T J C E d T A l N COUPOH * L D I C l Y E S r HOW PkEVEYT STOIIELEE f l E D I C I idE 5 .YEDICI~ES ro LICENSED * E D I C I ‘ 4 L S s OF dHATEVEP C L A S S OU aEDICINKS F k M TdE P A Y W E h l OF T HE D 1C I NE 5 HE D I C I NE 5 MEDICINC LICEN5PS

i x

ME 0 1C I N C 5 OF 5 : l C i i A 5 S O C l A T I O N .

OF SUCH A S S 5 C I A T I O N .

OF w c t i CU%SOLI~ATED c,wPoRArii) UF SUCH C O r i S O L l O A T E D COHPMATIO OF : d C H C J N 5 i ) L I b A T E D C U R P O H A T I O OF I H F C O ~ V P ~ N ~ E A L T H AND I *H€THE OF THE j 4 I D KIk6 CI1ARLES T H E SE O F THE COMWONJEALTHr UPON ACCEP

3 F THE C O W O N * E A L T H . i i P 114E C W M ~ ) N ~ ( E A L T I I! i A S B E E N GR Q F T Y I S COrt;4ONWCALTdr ISSUED US OFFICE w i * rIiE O A T € s TM U F F I C F D C D V E S I Y S An4 A k f A I k C L U L i Oh THE d 5 U A L PtiJUF 3 F SETTLEhE34 GN r w i LJTS

PA. PA.

STAT. STAT.

PA. PA.

STAT.

PA. PA. PA. PA. PA. PA. PA.

STAT. STAT. aTAT. STAT. STAT.

STAT. STAT. STAT.

PAS STAT. PA. \TAT. PA. $141. PA. STAT. PA. ‘nTAT. PA. \ T A T . PA. STAT. PA. S T A T . PA. STAT. PA. STAT. PA. STAT. PA. S T A T . PA. STAT. PA. ITAT. PAD STAT. PA. C,TAT. PA. T A T .

ANN. ANN. ANY. ANN. ANII. ANN. ANN. ANN. ANN. AHNO ANN. AM. AI~N. ANN. ANN. ANN. ANN. ANN. ANN. ANN. ANN. ANN. AWN. AHN. ANN.

1 4 r SEC. 14r SEC. 6 4 r SEC. 6 7 ~SCC. 111. 2 5 r SEC. 111. 1 5 s SEC. T I T . 6 3 r SEC. T I T . b 1 r JEC. TIT. 7.21 SEC. T I T , 7.21 SFC. T I T . 7 2 0 SFC. T I T . 7 2 . SCC. T I T . 721 SEC. T I T . 721 SEC. T I T . 721 SECe TIT. 1 4 s SEC. T I T . 14. SLC. T I T . 1 5 1 SEC. T I T . 159 SEC. 111. ZOr SEC. T I T . 1 5 s SFCI

TIT. TIT. TIT. TIT.

5

1112

0

3006 27

E

311 361 3151 3151 3151 3151 3152 3153 3153 58

97 1203 1963 457

3 --.I

8

I

?

0

2 * 5

ZOll

2

T I T . 64r SEC. 1 111. 6 7 r SEC- 1114 T I T . 6 4 r SEC. 6 1 2

5

TIT. ANN. T I T . At440 T I T . ANN. T I T . ANN. T I T . ANN.

56 97 604

rIr.

b4r IEC. 15s SFC.

73.

arc.

7 3 9 SEC. b 4 r SLC.

64.

615 1191 23 20 162

’ ~ E c .4 1 2

ria,>, LLis ,7, and the like, which are of little indexing value. A total of only 26,000 different English words, including all inflections thereof, are found in those statutes [MI. As many as 500 terms may be used for searching simultaneously, whether the 500 words be used to form one logical question for one search or several logical questions for several searches. The time required for searching the entire statute with respect to a single question is approximately one-half hour. In that half hour all the sections of the statute referring to a particular subject may be located. The average time required per search is reduced somewhat as the number of searches being made simultaneously is increased. Investigations are being made under the direction of Robert A. Wilson a t the Southwestern Legal Foundation for methods for storing and retrieving legal decisions on a large scale. Additional studies of all possible methods of retrieval of statutory law are also being investigated at the American Bar Foundation under the direction of William B. Eldridge. Other projects are under way in other sections of the country. At the present time the bottleneck in legal retrieval involves the manual keypunching of the cards that are fed to machines during the storage process. It is to be hoped that in the near future someone will supply the legal profession with a machine which will read its old books photoelectrically and record the information directly onto a storage medium which can then be read automatically for retrieval purposes. As presently contemplated, the recording will be in alphanumeric form. I n other words, the records will be in the form of dots and blanks. But there is no technical reason why images of the type cannot be recorded directly, albeit in microscopic form, and then searched directly by means of pattern-recognizing machines. At the present time the plan is to present questions in alphanumeric form. But there is no technical reason why a machine could not be built that would respond to an oral question. Though it may sound fantastic in the present state of the art, the day may even come when a machine will print out the text of all relevant material in response to an oral question

322

REED C. LAWLOR

asked by a man with a n accent, and the answer may even bc supplied orally with the same accent. I n any event, it is suggested that for the immediate future it may be far more economical to record printed information in full text rather than to spend time abstracting or digesting cases in order to abbreviate the material prior to storage. Besides recording English words or coded expressions corresponding to those words, it may also be possible to describe the concepts of various cases or parts of cases by means of a new language that has been developed by Martin Kalikow of General Electric. Kalikow has developed a list of terms which can be used to express the concept of any word in a limited dictionary. The total number of terms employed by him is only about 450. Each word in the dictionary is, in effect, represented by the logical product of a selected number of 450 words. Kalikow has used this dictionary in the indexing of the broadest claims of 6500 patents. I n effect, therefore, since there are only 450 terms involved and the concept profile of each claim is represented by the logical product of the equivalent terms in his language, the concept of each of these broadest claims can be punched out onto a single 90-column card with room to spare; and these cards may then be searched t o locate patents that should be studied to determine which patents are infringed by a particular device. Thc probability of false dropouts when using this system is negligible. The Patent Office searching process is by far the largest information retrieval system in daily use in the world. At the present time, this work is done almost entirely by manual techniques. The Research and Development Division of the United States Patent Office is in the process of developing and testing various techniques which can be employed to mechanize a large part of this work. Whatever can be done in the field of patents can also be done in the field of law. There are about 2,500,000 published decisions. There are approximately 3,000,000 patents. The bulk of the material involved in the two fields is of the same order. While the Patent Office is experimenting with the recording of full text, most of its efforts in the past have involved the use of patent examiners to prepare digests of patents in terms of coded language. At the International Patent Office Symposium on Information Retrieval held in Washington, D.C., in October, 1961, it was pointed out that approximately five or six hours are required to analyze the average patent in this way. Dr. C. S. de Haan, of The Netherlands Patent Office, pointed out that, a t the present rate, all of the United States patents could be analyzed in twenty years, provided that the Patent Office continued with its present staff but did no other work during that period [%'O]. This over-all analysis alone demonstrates the importance of developing automatic methods for recording printed text in machine-readable form without predigestion of the text

[@?I,


323

material. Kot only would such an automatic recording system accelerate the work of translating printed law libraries into machine-readable form expeditiously and, perhaps, economically, but it would also record the text in such a way that any part of the material could be located and reproduced automatically by a skillful programmer. As all of these retrieval techniques are developed and improved, and as their practicability is demonstrated in other fields, it is to be hoped that those who work in these fields will bear in mind the fact that lawyers as a whole probably spend more time retrieving information from their library than the members of any other single profession except patent examiners.

6. Punched Cards and Notched Cards

The British Patent Office has used punched cards for searching since 1905. These punched cards are about 12” x 18” and have 4800 punch positions. More recently representatives of many other patent offices have been making use of punched cards in connection with the searching of patents [go]. For many years Universal Studios has indexed the plots to its stories on punched cards. I n 1957, F . B. McKinnon and his associates a t the American Bar Foundation experimented with the use of punched cards to compare statutes of various states with corresponding statutes of the State of Illinois [56]. Richard F. C. Hayden, attorney of Los Angeles, demonstrated the application of edge-notched cards to the indexing of depositions a t the 1958 convention of the American Bar Association. Roy N. Freed [32] has described how punched cards may be employed to the analysis of cases involving a large number of exhibits, such as occurs in antitrust cases. William Cochran, one of the law examiners of the United States Patent Office, has used edge-notched cards to index decisions in the field of patent law. The author has used edge-notched cards to index cases in the preparation of briefs and in the analysis of decisions. It is thus seen that some storage and retrieval methods are available which can be of help to lawyers now. Lawyers do not need to wait for large computer centers to come to their aid. And the experience they gain today with such a simple system as edge-notched cards will help them tomorrow in their use of more advanced systems. More sophisticated peekaboo systems are available. At the National Bureau of Standards a novel peekahoo system has been

324

REED C. LAWLOR

developed. In this system, peekaboo cards are punched that correspond respectively to terms that characterize individual documents. Copies of the documents are recorded on microfilm and the microfilm images mounted on a drum. When documents referring to particular combinations of terms are to be located, the corresponding peekaboo cards are placed on a viewing plate in front of the drum. Two cross-hairs are manually brought to intersect at a point where holes of the cards coincide. This automatically rotates the drum to bring the image of the corresponding document into position in a projector which causes an enlarged view of the document to appear on a screen in front of the user, If the document is of interest, a copy of it is produced automatically merely by operation of a lever. Since this system can record as many as 50,000 pages of documents, it offers possibilities for the recording of about one hundred volumes of 500-page law books. However, this system suffers from the disadvantage that it would be necessary to analyze the text in terms of a code. Jonkers Business Machines (Gaithersberg, Md.) is also developing an application of a peekaboo system to a restricted field of law. The Jonkers system employs a series of peekaboo cards that are placed one upon another on a light table. The points of coincidence of holes on all the cards are used in a conventional manner to identify the corresponding decisions that involve all of the factors represented by the stack of cards.

7. Prediction of Court Decisions

When a client is involved in a lawsuit, he asks the perennial question, “What are my chances of winning?” Strangely, clients seldom ask, “Will justice prevail?” Lawyers, however, are not mere advocates. They are counselors. They are also officers of the court. Lawyers do not merely engage in surgical law requiring a court operation; they also engage in prophylactic law [16].Frequently lawyers counsel their clients to the effect that “Your cause is unjust” or “You have little chance of winning” or “You should settle the case” or “You had better do so-and-so to avoid a collision with the law.” For example, a lawyer often counsels his client in such a way that the client can avoid making mistakes which will entangle him with expensive, time-consuming legal proceedings. Clients want black or white answers, but lawyers can seldom give them. This is not because lawyers don’t know what they are talking about. In fact, the lawyer who frequently says “I am not sure” probably understands the law a great deal better than the lawyer who says “NO,it is impossible” or “Yes, you are absolutely right” most of the time. In the field of medicine,


325

Professor Haynes, of Rush Medical, used to say: I‘ ‘Always,’ ‘never,’ and (must be’ are words that must never be used in medicine.” The same principle always (!) applies in law. We all must recognize that the degree of reliability of the opinion of any lawyer or any layman or any judge is impaired by incomplete information, by undiscovered misrepresentation of facts as they are presented by clients and witnesses to the attorneys and to the courts, and by the frailties of human nature-including lack of understanding, bias, lack of knowledge of what rules to apply, and even by lack of attention. While many of these factors may not be subject to precise analysis, nevertheless mathematical methods should enable us to analyze law problems, to make the legal system more efficient, and to help it attain the ends of justice more effectively. The question arises, of course, as to whether one can predict the decision of the Court and, if so, how and with what degree of reliability. If we attempt to apply mathematical methods to the prediction of law decisions, we are confronted with a basic question: Do the laws of chaos apply or the laws of order; or do both kinds apply, depending on conditions? Are decisions a matter of chance, like throwing a seven at dice; or are decisions a matter of regular, predictable action within limits that narrow with time, like the launching of a satellite? De Laplace [21],Cournot [19], Boole [13],Molina [59], and others have long suggested the applicability of the theory of probability to the study of the legislative process, the judicial process, and the jury system. I n spite of all of this fine theoretical work, very little has been done to apply the principles of probability and statistical analysis to the practical problems with which a lawyer is concerned. Lee Loevinger has presented a review of this and related work [53, 541. Fred Kort, a political scientist at the University of Connecticut, has developed two different methods for predicting decisions of various courts. His methods have been applied particularly to certain classes of civil rights cases which reach the United States Supreme Court. I n his f i s t method [45, 24, 25, 461, Fred Kort developed a linear weighting technique based upon the following assumptions : (1) Figures of merit can be calculated for a set of court decisions related to the same

subject. The figures of merit have the following properties: (a) If the figure of merit (cv) for a case exceeds a predetermined value, then a particular party will win. (b) If the figure of merit is less than that amount, the party will lose. (2) The figure of merit is the sum of positive numerical constants (fu) that correspond to different pivotal factors or factual situations of the case in any given field of law. The numerical constant of each factor is omitted from the sum if the factor is absent from the case, but is included in the sum if the factor is present in the case.

326

.

REED C LAWLOR

I n Table V there is shown a table of pivotal factors involved in the right-to-counsel cases decided by the United States Supreme Court. as identified by Kort. and the weights of those factors. as calculated by Kort . These cases involve the questions as to whether a person charged with a TABLE V . WEIGHTSOF FACTS IN RIGHTTO COUNSELCASES ACCORDING TO KORT’SFIRSTMETHOD Pivotal factors:

fu

Crime subject to capital punishment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Crime subject t o life imprisonment, etc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Crime subject to twenty or thirty years imprisonment, etc . . . . . . . . . . . . . . . . . . Crime subject to five or ten years imprisonment. etc . . . . . . . . . . . . . . . . . . . . . . .

51.8 29.9 25.6 19.2

KO previous experience in court, ctc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arraignment without the assistance of counsol . . . . . . . . . . . . . . . . . . . . . . . . . . . . No assistance of counsel between arraignment and trial, ctc . . . . . . . . . . . . . . . . . No assistance of counsel a t the trial, etc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . No assistance of counsel at the time of sentencing . . . . . . . . . . . . . . . . . . . . . . . . . No assistance of counsel a t any other phase of the proceeding* . . . . . . . . . . . No advice of the “right to counsel,” etc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Request of assigned counsel denied . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Opportunity of consultation with own counsel denied . . . . . . . . . . . . . . . . . . . . . . No explicit waiver of the “riglit to counsel”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Detention incommunicado, etc . . . . . . . . . . . . . . ......................... Detention and trial in a hostile cnvironmcnt., ......................... Deception of the defendant, etc . . . . . . . . . . . . . ......................... No explicit presentation of charges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Coercion or intimidation to plead guilty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Consequences of the plea of guilty not rsplained . . . . . . . . . . . . . . . . . . . . . . . . . . Request of additional time, etc., denied . . . . Accelerated trial . . . . . . . . . . . . . . . . . . . . . . . . Procedural or substitntive error, c-t (. .* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jurisdictional issue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

50.1 51.8 59.4 40.9 38.0 5.2 43.7 43.7 31.2 56.5 32.5 53.9 31.5 31.5 68.3 18.9 14.2

25.2

crime was deprived of the right to be represented by a lawyer at any time of the criminal proceeding. in violation of his constitutional rights . Associated with each pivotal factor is a numerical constant (fv) . According to Kort’s principle. the figure of merit. or case value (cv). is the sum of the constants (fv) corresponding to the factual situations present in the case . I n other words. the figure of merit or case value (cv) for a particular case is given by the equation (cv) =

c a

(fV)i

Ti


327

where Ti = 1 or 0, depending upon whether the ith factor is present or absent. In effect, each of the numerical constants (fv) is a weight of the corresponding pivotal factor or fact. The weights of the facts that are present in the case are simply added up; and then the total weight for the case is compared with a critical borderline value, which lies somewhere between 370 and 378, to determine whether the case has sufficient weight for it to be decided in favor of the alleged criminal or against him. Table VI shows a list of cases for which such figures of merit have been TABLE VI. WEIGHTSOF RIGHTTO COUNSELCASES ACCORDING TO KORT’SFIRST METHOD Pro cases Herman v. Claudy, 350 U.S. 155 *Smith v. O’Grady, 312 U.S. 329 (1941) *White v. Ragen, 324 U.S. 760 (1945) *House v. Mayo, 324 U.S. 42 (1945) *De Meerleer v. Michigan, 329 U.S. 663 (1947) Marino v. Ragen, 332 U.S. 561 (1947) Chandler v. Fretag, 348 US. 3 (1954) Palmer v. Ashe, 342 U.S. 134 (1951) *Williams v. Kaiser, 323 U.S. 329 (1945) *Rice v. Olson, 324 U.S. 786 (1945) Uveges v. Pennsylvania, 335 U.S. 437 (1948) *Powell v. Alabama, 287 U.S. 45 (1932) Wade v. Mayo, 334 US. 672 (1948) Townsend v. Burke, 334 U.S. 736 (1948) *Tomkins v. Missouri, 323 U.S. 471 (1945) *Hawk v. Olson, 326 U.S. 271 (1945) Gibbs v. Burke, 337 U.S. 773 (1949) Massey v. Moore, 348 U.S. 105 (1954)

591.5 491.3 490.7 487.7 478.2 457.0 448.4 443.7 435.9 430.1 422.9 412.7 399.7 399.3 392.2 389.1 380.5 378.2

Con cases *Foster v. Illinois, 332 U.S. 134 (1947) Bute v. Illinois, 333 U.S. 640 (1948) *Betts v. Brady, 316 U.S. 455 (1942) Gryger v. Burke, 334 U.S. 728 (1948) Quicksall v. Michigan, 339 U.S. 660 (1950) Gayes, v. New York, 332 U.S. 145 (1947) *Canizio v. New York, 327 U.S. 82 (1946) *Carter v. Illinois, 329 U.S. 173 (1946) *Avery v. Alabama, 308 U.S. 444 (1940) Stroble v. California, 343 U.S. 181 (1952)

370.4 366.0 340.6 334.4 320.2 299.5 292.5 260.5 153.1 116.1

328

REED C. LAWLOR

calculated. The cases which are decided in favor of the alleged criminal all lie above the boundary line. Those that were decided against him all lie below the boundary line. Notice that the figures of merit for the cases above the line exceed 378, while all of those below the line are less than 370. Only by the analysis of further cases to be decided in the future can it be ascertained whether there is a sharp critical value and, if so, what its exact value may be, or whether the boundary is fuzzy with both pro cases and con cases on opposite sides. The twenty-six constants (fv) of Table V were actually calculated by Fred Kort by a very involved but ingenious technique from the analysis of only fourteen cases [47]. It might seem that twenty-six cases would be required to ascertain the values of twenty-six constants. This, however, is not true. Kort used only fourteen cases, and his method works! The fourteen cases used for calculating the twenty-six constants are indicated with asterisks. The cases which were tested by use of the constants are shown without asterisks. Of the fourteen cases tested, twelve fell in the same areas as the original fourteen, according to whether they were “pro” cases or “con” cases. The two remaining new cases helped to narrow the threshold value that Kort assumed separates all “pro” cases from all “con” cases in the decisions of the United States Supreme Court in this field of law. Fred Kort’s theory has been criticized as being unsound by Franklin M. Fisher [24, .25]. But Glendon A. Schubert and Elmer T. Prosper [64] approached Kort’s work differently. They asked: “Since Kort’s method is reliable, how can we explain it?” They used a ‘iscalogram” technique. Schubert and Prosper showed that the Supreme Court judges who have decided the right-of-counsel cases can be arranged-that is, ranked-in a particular order along one axis, and that the cases themselves can be arranged, or ranked, along another axis; and that when the votes of all the justices are recorded on the chart, a line of separation can then be drawn between almost all the votes in favor of the accused and almost all the votes against the accused. The results of this analysis are represented in Table VII. Here you will note that the double vertical line is used to separate the favorable cases from the unfavorable cases. The ‘ I + ” signs indicate favorable (pro) votes, while the “ - ” signs indicate unfavorable (con) votes of the individual judges. More particularly, you will note that certain judges favor the accused criminal almost invariably, while other judges are more evenly divided, and still others usually voted against the accused-all as indicated by the various steps in the broken line which separates the favorable decisions from the unfavorable decisions. Such scalogram analysis has great merit in that it indicates relations between the reactions of the justices to the cases in a single field of law. But while the scalogram technique can be employed

I

I 1

1

I

I

I

I

~

I

I

+++ +++

++ + ++++

+++f + +++ + +++ +++ +++ + + + 4-

10. Masseg (1954)

9. Chandler (1954)

8. Hawk (1945)

7. Smith (1941)

6. Gibbs (1949)

5. Herman (1956)

4. Reece (1955)

3. Marino (1947)

2. White (1945)

1. De Meerleer (1947)

+ I I I

I I I I I I I I I I

I

I I

I

I

I

I I I I I I

I

I

I l l 1

I l l 1

I I l l 1

I I

I

I I I I

I I

I + I + l

I I I

I I

++ +++++

I

I

I

-

t

31. Auery (1940)

30. Canizio (1946)

29. Quicksall (1950)

28. Stroble (1952)

++

27. Befts (1942)

++

26. Carter (1946)

25. Foster (1947)

24. Groban (1957)

23. Gages (1947)

22. Bute (1948)

21. Gwger (1948)

20. Uveges (1948)

19. Rice (1945)

18. Moore (1957)

17. Palmer (1951)

16. Wade (1948)

15. Townsend (1948)

14. Olium (1948)

13. Tomkins (1945)

12. Williams (1945)

11. House (1945)

++++ ++ +

++++

.+++

++++ ++++ ++++ + + 4-+ ++++

I

+ + ++

I

+ I

I

1 1 + 1 I

I I

I I I +

+++I

+ + + 4++++ ++++ ++++ ++++ ++++

I = I'

++++ + k ++ + + ++++ + ++ ++ + + ++ + + + + ++++ + !+ ++ + ++ + + + + + ++ + + + ++

REED C. LAWLOR

330

to scale the attitudes of the justices in general, it does not appear to bc suitable for the prediction of the outcome of individual cases. Kort’s methods, however, can be used to some extent, at least, for prediction. Fred Kort has now gone further and has applied the statistical methods of Hotelling to the analysis of decisions [47, 481. The Hotelling technique involves the calculation of cross-correlation coefficients. The results obtained by applying this method to the same right-of-counsel cases are represented by Table VIII. TABLEVIII.

WEIGHTS OF RIGHT TO COUNSEL CASES

ACCORDING TO KORT’SSECOND METHOD Decisions in favor of the petitioner

Powell u. Alabama, 287 U.S. 45 (1932) Smith u. O’G-rady, 312 U.S. 329 (1941) Williams v. Kaiser, 323 U.S. 471 (1945) Tomkins u. Missouri, 323 U.S. 485 (1945) House u. Mayo, 324 U.S. 42 (1945) White u. Ragen, 324 U.S. 760 (1945) Rice u. Olson, 324 U.S.786 (1945) Hawk u. Olson, 326 U.S. 271 (1945) De Meerleer v. Michigan, 329 U.S. 663 (1947) Marino u. Ragen, 332 U.S. 561 (1947) Wade u. Mayo, 334 U.S. 672 (1948) Townsend v. Burke, 334 U S . 736 (1948) Uveges v. Pennsylvania, 335 U.S. 437 (1948) Gibbs u. Burke, 337 U.S. 773 (1948) Palmer u. Ashe, 342 U.S. 134 (1951) Chandler v. Fretag, 348 U.S. 3 (1954) Massey v. Moore, 348 U.S. 105 (1954) H e m n u. Claudy, 350 U.S. 116 (1956) Moore v. Michigan, 355 U.S. 155 (1957)

6.87 7.73 5.97 5.68 6.05 7.29 6.25 7.11 8.27 7.84 5.62 5.49 4.76 6.18 6.94 6.63 5.09 8.02 7.80

Decisions against the petitioner

Avery v. Alabama, 308 U.S. 444 (1940) Betts u. Brady, 316 U.S. 455 (1942) Canizio u. New York, 327 U.S. 82 (1946) Carter u. IZlinois, 329 U.S. 173 (1946) Foster u. Illinois, 332 U.S. 134 (1947) Gayes v. New York, 332 U.S. 145 (1947) Bute v. Illinois, 333 U S . 640 (1948) Gryger v. Burke, 334 U.S. 728 (1948) Quicksall v. Michigan, 339 U.S. 660 (1950) Stroble v. California, 343 U.S. 181 (1952) I n re Groban, 352 U.S. 330 (1957) Crooker u. California, 357 U.S. 433 (1958) Cicenia u. Lagay, 357 U.S. 504 (1958)

2.51 3.64 2.02 1.20 3.06 2.79 3.07 3.68 3.08 1.99 .51 2.87 2.87


33 1

As shown in Table VIII, with this method a very wide difference exists between the figures of merit for the favorable cases, which are above the line, and the figures of merit for the unfavorable cases, which are below the line. You will recall that in Fred Kort’s first approach there was no such wide separation between the figures of merit of favorable cases and those of the unfavorable cases. I n the first method, the gap between the pro cases was about 8 parts in 480 (Table VI). But in the second method, the gap is about 100 parts in 800 (Table VIII). Nagel has also suggested the application of correlation techniques to the prediction of court decisions [ G I , 6.21. The author has also developed a mathematical technique for predicting United States Supreme Court decisions [50].This method is based, in part, upon the idea that the U.S. Supreme Court and each individual justice on the Court are self-consistent. A mathematical theory of stare decisis has been developed that is applicable where such a rule of consistency applies. This method has been applied to the right-to-counsel cases that had been previously studied by Kort and Schubert. With this method, Boolean equations were developed which describe how the United States Supreme Court as a whole and how each justice on it votes when presented with any arbitrarily selected subset of the set of facts that has appeared in prior right-to-counsel cases. In applying this technique, twenty-seven cases were employed for the development of the equations, and the equations were tested on the remaining ten cases. All ten calculated votes for the Court as a whole proved to be correct. Of the thirty-seven votes that were calculated for individual justices, thirty-five proved to be correct. This is fairly good, since only a tiny sample of twenty-seven subsets (cases) out of the 100,000,000,000possible subsets of facts were used in the development of the Boolean equations. By assuming order instead of chance, the probable error of prediction seems to have been reduced. language was then developed by A computer program utilizing FORTRAN the author for predicting decisions, based upon the Boolean equations mentioned above and the author’s mathematical theory of stare decisis. Examples of predictions made with the computer program during the preliminary stages of this development are shown in Appendix A.* I n this appendix, typical examples of the output obtained with this program are shown for two cases. In this treatment a pro decision is a decision in favor of the accused, while a con decision is a vote that is unfavorable to him. It will be noted that, in these computer-composed analyses, prediction is

* I wish to thank Edgar A. Jones, Jr., Professor of Law, Law School a t the University of California a t Los Angeles and Chairman of UCLA Committee for interdisciplinary studies of Law and the Administration of Justice, for sponsoring my use of the IBM 7090 a t the Weetern Data Processing Center of the Graduatc School of Business Administration for this work.

332

REED C. LAWLOR

made in accordance with four different techniques, one being based on Kort’s first method and three being based on the author’s work. The general nature of t.he four techniques is described below: First Technique I n this analysis a pro cranberry case is a potential precedent that meets the following requirements: (1) All of the facts of the pro cranberry case are present in the new case under

consideration. ( 2 ) No other facts are present in the pro cranbcrry case. A case is a pro cranberry case only if it meets the foregoing requirements and only if at least one of the justices voted pro in that case. A justice who voted pro on the pro cranberry case should also vote pro in the case under consideration, if he is consistent. Similarly, a con cranberry case is a potential precedent that meets the following requirements : (1) All of the facts of the new case are present in the con cranberry case.

( 2 ) No other facts are present in the new case. A case is a con cranberry case only if it meets the foregoing requiremcnts and only if at least one of the justires voted con in that case. A justice who voted con in a con cranberry case should also vote con in thc case under consideration, if he is consistent.

Second Technique I n this technique the computer proceeds to analyze the facts of the new case in the light of the Boolean equation that has been developed for predicting how the court as a whole will vote. Third Technique Kow, the Boolean formulas that have been developed for various individual justices are applied to the facts of the case under consideration. The votes of the individual justices are then printed out. It is to be noted that in Case 36 of Appendix A the vote of Justice Frankfurter was not correctly predicted. All the other predicted votes in the two cases in Appendix A arc correct. Fourth Technique The computer then adds up the weights of the facts present in the case, using the weights calculated by Bort. It then compares the total weight of the facts of the case with an arbitrarily selected value of 375.0 (between 370 and 378) and then predicts that the decision will be a pro decision if the weight of the case is more than 375, but a con decision if it is less than 375.

The important point about the prediction technique exemplified in Exhibit A is that it is based upon the assumption that the decisions of the courts and the individual justices are self-consistent and not a matter of chance or whim. It is also important to note that with this prediction technique the votes of individual justices are predictable even when no cranberry case exists which forms a precedent that compels the justice to reach any particular decision.


333

It can be shown that the linear equation employed by Kort applies if the occurrences of the factors are statistically independent of each other for both the pro cases and the con cases. This assumes that the votes of the justices are a matter of chance. However, it can also be shown that the linear equation of Kort applies if, in fact, certain types of logical relations exist between the facts of the cases and the vote of the court [50]. Both Kort’s first method and the author’s method have proved to be reliable in the cases to which they are applicable. Theoretically, at least, the prediction by Kort’s method and the author’s method will be contradictory under certain circumstances. So far such circumstances have not arisen. It is conceivable that in the not-too-distant future it will be possible to record decisions, statutes, regulations, and other rules of law in dataprocessing systems and to feed sets of facts to those machines and to draw from them automatically a prediction as to the likelihood that a decision made by a particular court on the particular set of facts will be favorable or unfavorable. Computer-operated probability prediction methods have been used effectively in other fields, such as medicine [51, 691, and have been proposed for use in evaluation of the reliability of international intelligence information [ I ] . The reliability with which it will be possible to predict court decisions is still to be ascertained. The work to date has been too meager and too ineffective to use as a basis for calculating probable error with any substantial degree of reliability. One thing is certain. Regardless of what mathematicians and computers say, the courts will have the last word, and it is anticipated that, for many, many years to come, at the very most computers will be used only to prepare outlines of decisions for consideration by lawyers and the courts. Furthermore, by virtue of the fact that the lawyers and the courts are relieved of some of the more tedious work, they will be able to perceive more easily the solution to more difficult problems and will often reach conclusions far different from those that would be attained by merely following the dictates of a computer. In this connection, it will be well to remember the words of Theodore H. Lassagne, general counsel for Librascope, Inc., which he prepared at the time that the application of information retrieval techniques to law was demonstrated on the IBM 1401 at St. Louis in August, 1961 : “This is the tale of the 1401, The law clerk that was nobody’s son. It spent its days in a furious hunt For authorities, dictum, and argumunt. But after it found them, it burned with shame; The Supreme Court reversed it just the same.”

334

REED C. LAWLOR

8. Thinking Machines

There is considerable debate as to whether or not machines can really think. The answer to this question depends, of course, on how thinking is defined. Ashby [Q]has defined the intellectual power of a machine in terms of the ratio of intelligence output to intelligence input, and he has defined thinking in terms of selection or choice. As a matter of fact, a great deal of our educational system is designed to teach children to make choices between the spaces provided at the ends of questions in true-false or multiplechoice tests. The fact that machines may some day be able to make such choices almost perfectly, while our children will continue to make such choices very imperfectly, does not demonstrate that such machines lack all ability to think. Ashby has pointed out that it has not yet been demonstrated that men cannot possibly construct machines which have more intellectual power than the men who built them. To those who abhor the idea that a machine can think, let me offer this definition: “Thinking is any process by means of which correct conclusions cannot be drawn from premises by a machine unless a man programs the machine and arranges for data to be fed to it.” But to those who can accept the idea that a machine can think in the sense that it can simulate the thinking of the human mind very closely, let me offer the definition: “Thinking is any process by means of which relations between propositions can be tested or derived and described in language understandable to a man, other than those relations which can be recognized instantly by man.” Gelernter et al. [I531 have denionstrated how to solve problems in the field of plane geometry by means of the IBM 704. It is questionable whether law problems are any more difficult. We can look forward to seeing proofs of geometrical problems written in plain English for the benefit of us ordinary people some day. Such logical machine processes serve best when they serve most. They will serve most when they present their results in ordinary language that does not require decoding by the reader. These and other techniques can be extended to law. Without doubt the day will come when legal propositions can be fed to a machine and factual statements can also be fed to a machine; and then the machine will write out what conclusions necessarily follow, which are impossible, and which are uncertain, and will set forth the probability of the truth of the uncertain conclusions and will also write out all of the steps of the reasoning process. Lawyers and judges will then be able to study these machine-made arguments to ascertain the accuracy of the instructions given to the machine and from this the reliability and applicability of thc conclusions to the rases at hand.


335

When such a machine is first used, I can well imagine many lawyers saying, “What a revolting development this is!” Others will use it as a crutch. But still others will welcome the help of the machine and employ it with the same care that an engineer or scientist uses with a computer. When that day comes, many persons will disagree as to whether the results of computer programs represent thinking. I n any event, computer processes based on programs that compose reliable solutions out of millions of possible combinations simulate thinking of a rather laborious and advanced kind. Furthermore, such computer processes may actually be more reliable than human thinking. The effectiveness of legal reasoning will be improved when a lawyer’s thinking and a judge’s thinking is commonly supplemented by computer processes that solve complex problems, such as selection of pertinent references [5, 8, 37, 421, designing around patent claims [49], or predicting court decisions [24, 45-48, 50, 61, 621. When that day comes, lawyers and judges will be able to relieve their minds of the more menial but laborious thinking tasks which now consume a large part of their time and energy and, like the scientists and engineers of today, will be able to devote more of their attention to more difficult, more serious problems.

9. The law of Computers

Roy 3.Freed, of Philadelphia, has made a study of the law of computers [27, 69, 301. It is his general conclusion that very few changes in the law

will be required in order for lawyers, the courts, and people as a whole to cope with the legal problems involved in the manufacture, sale, and use of computers. Courts that learned about wagon wheels were able to adapt themselves to automobile wheels. Courts that were able to solve problems of explosions in seagoing vessels have been able to cope with the problems of jet airplanes. Courts that have been able to cope with the problems of books can also cope with the problems of data-processing systems. Nevertheless, some new problems are bound to arise. One of the interesting questions is whether and, if so, when and to what extent, the use of information in a computer is a n infringement of a copyright. Suppose that the full text of a copyrighted book is recorded in machine-readable form. Then at what stage, if ever, does the owner or user of the computer infringe the copyright of the book? Certainly, if the computer is commanded at any time to reproduce the entire book, this would result in an infringement of the copyright. An exception might arise if the sole purpose of the printing is to check the accuracy of the recording. Suppose that only part of the book is reproduced.

336

REED C. LAWLOR

This may or may not be copyright infringement, depending upon whether or not the ordinary printing of that portion of the book would be copyright infringement under the same circumstances. I n this connection, it is to be borne in mind that certain uses of portions of books, for quotation or comparative analysis or argument-as in lawyers’ briefs-as distinguished from competitive use, are considered fair use. But now, suppose that, in the use of an information retrieval system in which the full text of books is recorded, the search system is operated in such a way that all it prints out are references to the pages and lines of the original book where the material may be found. This, it is submitted, would not involve an act of copyright infringement under the law as it now stands. Here the computer does not compete with the book; it helps a reader use the book more effectively. The manufacture of punched cards and magnetic tape records, thermoplastic tape records, and other machine-readable records which do not make use of visually legible printed format is analogous to the perforation of rolls of music and the manufacture of phonograph records. The latter acts are not considered infringements of copyrights, though under some circumstances they may involve trespass of some other kind of right. I n While Smith Music Co. v Apollo, 209 U.S. 1, 12; 52 L.E. 655; 28 S.C. 319, the United States Supreme Court held in 1908 that neither the act of perforating music on a roll nor the perforated roll itself constituted infringement of a musical copyright. The Court held, in effect, that a music roll is merely a mechanical device. The same rule applies to phonograph records, magnetic tapes, and the like when used in a normal way. I n 1909, in an effort to protect the interests of composers, a special law was enacted under which phonograph records could be produced by persons other than the owner of a copyright of the musical composition if, but only if, the owner of the musical composition first made phonograph records himself or permitted others to do it. Then the law went on to specify that if the newcomer made a phonograph record, he was required to pay certain royalties to the owner of the copyright on the musical composition. But that law does not apply to literary material. It applies only to musical compositions. Consequently, it would not apply, for example, to legal literature. It therefore appears that, as the law stands today, it might not be unlawful to record all of the copyrighted legal material in the United States in machine-readable form, so long as it is never reproduced in visually legible form. Whether such acts would constitute unfair competition would depend upon the law of the individual state in which the alleged wrongful act occurs. However, if one were to attempt such an ambitious project or even a


3 37

much smaller one, he should be cautioned that courts do sometimes change their minds; and a court might readily determine that, under modern conditions, even a punched card is legible. If, in fact, the literary work was to be recorded on magnetic tape or thermoplastic tape in a form which might be visually read by a trained user, such as by means of polarized light or fine iron dust, a court again, under modern circumstances, might hold that the recording is an infringement and, therefore, in effect overrule the precedent of White Smith Music Co. v. Apollo, which was handed down fifty-four years ago. Phonograph recordings per se have never been subject to copyright protection. Such a record submitted to the register of copyrights would be returned as not constituting copyright subject matter. But photographs and motion pictures are copyrightable. Recently a copyright was registered on a magnetic tape recording of Gian Carlo Menotti’s opera, “The Consul” [Iga]. The tape itself was submitted to the Copyright Office in lieu of a motion picture. Some attention has been given to the question as to whether computers think. An important related question arises as to whether the product of a computer can be copyrighted or patented. Datatron has programmed a computer to compose music. Librascope has programmed a computer to compose poetry. I t has been suggested that computers can make inventions [50, 401. Certainly a listing of a program can be copyrighted. It is usually just a printing of what was composed by a man. Mathematical tables representing the labor of a mathematician are the subject of copyright. What if these same tables were generated by a computer? There are some people who apparently feel that it is undignified to permit the musical compositions and poems produced by computers to be copyrighted. Undoubtedly there are others who think that inventions require the operation of the human intellect. These people would deny that a beautiful modern painting daubed on a canvas by a blind child is art. But is it the origin that determines whether a novel combination of paint strokes or musical notes or mechanical devices has merit? Or does the value depend solely on the result achieved? Under the copyright law, the term “author” includes the person who makes a composition for hire. There is no analogy in the patent law. I n the copyright law, the work of a composer can be copyrighted in the name of the person who employed him. But in the field of patent law, only the person who exercised the intellectual effort in making the invention can file a patent application. Though the subject is too long to discuss here, it is suggested that, in those cases where the owner of a computer employs someone to program the computer to generate a composition-whether it

3 38 REED C. LAWLOR be musical, literary, choreographic, or artistic-the composition would be copyrightable in the name of the owner of the program. I n the case of inventions first described by a computer, the person who programmed the computer to bring the invention into existence would be the inventor; or, if perchance the computer produced the invention accidentally or as a result of some random process, the person who first perceived and appreciated that the output of the computer was new and useful would be the inventor. This is somewhat analogous to what occurs under the patent law as it applies to newly discovered varieties of plants, where the person who first discovers that the new species exists is deemed to be the inventor, and as it applies to accidental discoveries that occur in the middle of research efforts directed to achieve wholly different results. Many of the problems of civil and criminal law take a new twist when applied to computer technology. Under certain conditions the personal property of a bank is not taxable by the county where the personal property exists; but real property is taxable. Real property includes fixtures and improvements. When this issue was considered in a recent case [?'I], the Court held that the computer is taxable as an improvement in real property, emphasizing that the building was constructed to house the computer and included expensive features especially adapting it for use with the computer as well as provisions for expansion, and further that it was intended that the computer remain permanently in place, except for removal for repair or obsolescence or for replacement by more efficient equipment. I n the law of evidence, certain types of original documents or business records can be introduced in evidence with very little proof about the manner in which they were made, other than that they were made in the ordinary course of business. A question arises as to whether a magnetic tape or other machine-readable record is an original document under the law of evidence, especially when the record was made by punching keys a t a time when no punched paper record was being made. Where original documents are not available, secondary evidence regarding their contents may usually be introduced upon proof that the original document is not available. Does this mean that a print-out made from a tape can be introduced in evidence instead of the tape itself? Or is it necessary to introduce the tape into evidence and put someone to the trouble of reading it out? The potentiality of computers for fraud and forgery is limited only by the imagination of evil men. When we get to the point where some computers write checks and other computers honor them in blind obedience to a program, we may have some very serious problems on our hands.


339

The Internal Revenue Service is instituting a program for recording the history of each taxpayer on 1/2-inch of tape and of updating these records every year, and purging the records every three years. Since, if adequately programmed, fraudulent concealment of income may be more easily detected than heretofore, the enforcement of the Internal Revenue Code may be simplified and made more certain, with resultant advantages in favor of honest taxpayers. Except for special problems such as the foregoing, it appears that computers are hardly in any different category than other types of machinery insofar as the law is concerned-whether it be the law of contracts, the law of torts, tax law, criminal law, or otherwise. It is to be anticipated that modern information technology, and computer technology especially, will solve more problems for lawyers and the courts than they will create.

10. Use of Computers in Court

Reference has already been made to the possible use of computers for indexing depositions and trial records, for information retrieval, and for the prediction of decisions. Computers can also be used in the administration of court proceedings not only in the docketing and scheduling of hearings and trials, but in the programming of steps to be taken in many kinds of proceedings. They may also be used to prove relations between facts. I n St. Louis, punch-card systems have been introduced for the control of various stages of probate proceedings [%I. Several years ago Judge Richard F. C. Hayden, while in the Attorney General’s office in San Diego, made an unusual use of computers [35].His case involved a charge of bribery of a public official. The question arose as to whether a positive correlatioh existed between the deposit of funds in certain bank accounts and the speed with which building permits were issued to those who made the deposits. I n this particular case, a large amount of data was involved. The statistical analysis was facilitated by means of a computer. And when opposing counsel challenged its reliability by asking for further analyses, those analyses merely added support to the charge of bribery! It is a general rule of trial practice that a lawyer should not ask a question unless he knows the answer. Interrogation of computers poses new problems for trial lawyers.

340

REED C. LAWLOR

11. New Horizons

Until March, 1960, we did not know when it would be possible to demonstrate how electronic data-processing systems can be actually used in finding statutes and cases of interest to lawyers. Nevertheless, such work was accomplished in the Health Law Center at the University of Pittsburgh at that time, and then eminently successful demonstrations were made at the American Bar Association Annual Convention in Washington, D.C., at the end of August, 1960. Then again successful demonstrations were made at the American Bar Association Annual Convention in St. Louis in August, 1961. Many lawyers are now aware of the possibilities of automatic legal research. Many wish it were here to serve them now. While automatic legal research is the major field of immediate interest to lawyers in their daily practice, there are other possible applications of information technology which could be of immediate value to lawyers and judges in their practice. Though we cannot anticipate what will be done next in this field, let us look into the crystal ball for a few minutes to try to see what the future may hold. (1) The application of peekaboo systems to the search for legal material on a practical and possibly commercially profitable scale is probably just around the corner. The systems developed at the National Bureau of Standards and the system undergoing development at Jonkers Business Machines, Inc., arc at the stage where they can be useful to lawyers. (2) The American Bar Foundation is considering issuance of a biweekly KWIC index of the titles of new laws enacted in all the states. ( 3 ) Today the bottleneck in the system involves the punching of cards as a preliminary step to storing information on tape or disc. Systems will soon come into use which will read books automatically and transcribe printed words into reproducible signals on magnetic tapes or other recording media. This would greatly expedite the automatic storage of large bodies of written information. Such records could then be analyzed by methods somewhat similar to those demonstrated at the 1960 and 1961 ABA Conventions for locating subject matter relevant to a lawyer’s question. (4) At the present time, methods are undergoing devcloprnent whereby printing of books and the like can he automated. The printing machines used for this purpose are driven by punched tape upon which the material to be printed has previously been recorded in code form. There is no technological reason that stands in the way of coding this material in such a form that the tapes can be used not


341

merely for the printing of books, but also for recording the same information on reproducible media where it will be available for retrieval purposes. By use of the same tape for printing books and for recording the text in retrievable form, great economies can be expected. (5) Systems are undergoing development for automatically abstracting documents. These systems are potentially applicable to the preparation of digests and abstracts of law decisions. (6) Special typewriters are available today which automatically punch paper tape during the typing process. Normally such tape is used for automatically retyping the same material when it is to be reproduced. Such tapes could be used for storing the material in retrievable form on magnetic tapes, magnetic discs, or the like. Such systems could find immediate use for automatically indexing depositions and trial records and also for automatically indexing new statutes and other verbal texts. (7) One of the most worthwhile and valuable projects that could be undertaken would be the development of a system for recording and retrieving new bills being submitted to Congress and to the various legislative bodies throughout the country in such a way that persons who are interested in proposed bills respecting various subjects can be informed about them at the time that they are first introduced. Such a project might include, for example, the recording of bills in machine-readable form at the time that they are originally typed in the office of a Congressman or at the time that they are first printed in the Government Printing Office, and then the transmission of those machine-readable records to a center where they could be stored and continuously analyzed to match the questions of persons who are interested in various subjects. With such a retrieval system the interested persons-whether they be Congressmen, businessmen, scientists, ordinary folks interested in the public welfare, or even lobbyists-could be notified of the introduction of bills of interest at the early stages of the consideration of such bills. By use of the same material, indexes of statutes could be automatically updated continuously for the benefit of all concerned. By applying electronic and other mechanical information storage aiid retrieval methods to law, it will be possible to improve the efficiency with which law-case searching and statute searching can be accomplished. Through this, and in other ways, legal processes can be improved and the ends of justice served more efficiently. Just how the application of information storage and retrieval techniques will be made available to members

342

REED C. LAWLOR

of the Bar throughout the country can only be guessed. If electronic processes are employed, due to the inherent nature of the problem and the costs involved, it may be necessary for the work to be sponsored by Bar Associations or law book publishers in various areas throughout the country and the information made available at various widely dispersed searching centers. When this work is first commenced, it will probably be by different organizations in different areas of the country in entirely different fields of law. While it may be that coding techniques may eventually be employed, it is not beyond the realm of possibility that large bodies of published decisions will be recorded on magnetic tape or some other informationstorage medium soon. It has been calculated that the total number of tapes required by present techniques to store all the words of all published United States Appellate decisions is of the same order of magnitude as the number of tapes used by a large life insurance company in the operation of its business. The task is not insurmountable. Only 20/20 hindsight will make it clear someday how simple the task really was. While the day may be far off when a machine will be able to think through a lawyer’s problems and present him in writing with the step-bystep solution, nevertheless, within the next decade, if not within just the next few years, it is very possible that electronic data-processing systems will go into regular use to supply lawyers with statutory material, case material, and textbook material which applies to the problems on which they are currently working for clients. The attainment of this dream will require close cooperation and imagination of scientists, lawyers, manufacturers, government, and business. To paraphrase the words written by Kelso [@]fifteen years ago: “The American Bar and American Science will do well to think seriously of mechanizing the drudgery of the practice of law, in order that the really irreplaceable human contributions of lawyers may be liberated for more effective use for the benefit of mankind.”

Just as the lawyer of today finds the use of the telephone, the dictaphone, the typewriter, and the adding machine indispensable to the modern law practice, the lawyer of tomorrow will also find the use of automatic information-processingsystems indispensable to his practice. John F. Horty’s reaction to the effect of computers and other phases of modern information technology on law has been summarized by him as follows [38] : “Though it is too early to determine just how this second revolution is going to occur in law, I could never be more convinced than now that, once we have opened up the box that lets the mathematicians and other scientists examine what the lawyer


343

does, the legal profession, legal rrsearch, the entire administration of justice will never be the same again.”

The law is here to stay. Information technology is here to stay. Thc twain have met. And they will beget a more efficient, more economical, more equitable, more logical, and more wise administration of justice for all. The application of modern information technology to the problems of law on a large scale will change the course of legal history. Bibliography NOTE:The abbreviation M.U.L.L. refers to Modern Uses of Logic in Law (quarterly newsletter of ABA Special Committee on Electronic Data Retrieval), American Bar Association, 1155 East 60 Street, Chicago 37, Illinois. 1. ACF Electronics, Data Processing Dept., Digitalized Logic and Its Applications. ACF Industries, Inc., Washington, D.C., 1955. 2. Allen, L. E., Symbolic logic: A razor-edged tool for drafting and interpreting legal documents. Yale Law J . 66,833 (1957). 3. Allen, L. E., Modern logic: A useful language for lawyers. Proc. 1st Natl. Law and Electronics Conf., Arrowhead Lake, California, October (1960), sponsored by Univ. of California a t Los Angeles. Matthew Bender, Albany, 1962. 4. Allen, L. E., Brooks, R. B. S., and James, P. A., Automatic retrieval of legal literature: Why and how. M.U.L.L. pp. 17-24 (1959). 5. Allen, L. E., Brooks, R. B. S., and James, P. A., Storage and retrieval of legal information; Possibilities of automation. M.U.L.L. pp. 6&84 (1960); see also Report to Walter E. Meyer Research Institute of Law, New Haven (March 1, 1961). 6. Allen, L. E., Uses of symbolic logic in law practice. Proc. A B A Electronic Data Retrieval Comm. Conf., St. Louis, Missouri (1961); see M.U.L.L. in press (1962). 7. Allen, L. E., W F F ’N PROOF series of educational games. ALL1 (Accelerated Learning of Logic Institute), 10822 Blucher Ave., Granada Hills, California. 8. Andrews, D. D., Application of random access techniques t o case law. Proc. ABA

9.

10. 11.

12.

13. 14.

Patent Trademark and Copgright Section Symposium on Information Retrieval, St. Louis, Missouri pp. 228-232 (1961). Ashby, W. R., Design for an intelligence-amplifier. In Automata Studies (C. E. Shannon and J. McCarthy, ed.), pp. 215-234. Princeton Univ. Press, Princeton, New Jersey, 1956. Bartholomew, P. C., The Supreme Court and modern objectivity. N . Y . State Bar J . 33, 157 (1961). Berkeley, E. C., Boolean Algebra (The Technique for Manipulating “and,” “or,” “not” and Conditions) and Applications to Insurance. Edmund C . Berkeley, New York, 1952; also see Record Am. Inst. Actuaries 96, 373 (1937); 97, 167 (1938). Biunno, V. P., History of electronic methods for legal research. Proc. A B A Data Retrieval Comm. Conf., Washington, D.C. p. 36 (1960), Bureau of National AiTairs, Washington, D.C., 1961; see also M.U.L.L. pp. 99-102 (1960). Boole, G., An Investigation of the Laws of Thought (1854), pp. 84 and 376. Dover Publications, New York, 1951. Brown, J. R., Thermo King Corp. v. White’s Trucking Service, Inc., 190 U.S.P.Q. 90, 95 (1961), Bureau of National Affairs, Washington, D.C., 1961.

344

REED

C. LAWLOR

15. Brown, J. R.,Electronic brains and the legal mind: Computing the data computer’s collision with law. Yale Law J . 71,239 (1961). 16. Brown, L., Manual of Preventive Law. Prentice-Hall, Englewood Cliffs, New Jersey, 1950. 17. Carroll, Lewis (C. L. Dodgson), Symbolic Logic, Part I: Elementary (1897). Berkeley Enterprises, New York, 1955. 18. Clark, R. L. (edited by Duke Univ. Faculty of Law), On Mr. Tammelo’s conception of juristic logic. J . Legal Educ. 8, 491-496 (1956). 18a. Colby, R., Letter dated April 28, 1961, re “The Consul,” Bull. Copyright SOC. U.S.A. 8,205-206 (196o-Sl), Copyright Society of U.S.A., New York, 1961. 19. Cournot, A. A., Exposition de la Thtbrie des Chances et des Probabilitb (1843), translation of Chapters XV and XVI. New Jersey Law Institute, Newark, New Jersey, 1954. (Copies can be obtained from Vincent P. Biunno, 605 Broad St., Newark 2, New Jersey.) 20. de Haan, C. S., Proc. 1st Intern. Patent Ofice Workshop on Information Retrieval, Washington, D.C. (1961),Patent Office Society, Washington, D.C., 1962. 21. de Laplace, Pierre Simon, Marquis, A Philosophical Essay on Probabilities (1819), Chapter XIII. Dover Publications, New York, 1951. 22. Dickerson, F. R.,The electronic searching of law, A m . Bar Assoc. J . 47 (9),902-908 (1961). 23. Fiordalisi, V., Progress and problems in the application of clectronic data processing systems to legal research. Proc. ABA Electronic Data Retrierial Comm. Conf., Washington, D.C. p. 22 (1960),Bureau of National Affairs, Washington, D.C., 1961; see also M.U.L.L.pp. 174-179 (1960). 24. Fisher, F. M., The mathematical analysis of Supreme Court decisions: The use and abuse of quantitative methods. Am. Polit. Sci.Rev. 52 (a),321 (1958). 25. Fipher, F. M.,On the existence and linearity of perfect predictors in “content analysis.” M.U.L.L. pp. 1-9 (1960). 26. Freed, R. N., A lawyer’s guide through the cornputcr mazc. I’ractical Lawyer 6, 1-5 (November 1960). 27. Freed, R.N., Some legal aspects of computer use in business and industry. J . I n d . Eng. pp. 289-291 (July-August 1961). 28. Freed, R.N., The importance of a systcrns approach to mechanized legal research. Proc. ABA Patent Trademark and Copyright Section Symposium on Information Retrieval, St. Louis, Missouri pp. 179-188 (1961). 29. Freed, R. N., Try suing a computer!-Legal tangles in E.D.P. Management Rev. pp. 4-11 (August 1961),Am. Management Assoc., New York. 30. Freed, R. N., Prepare now for machine-assisted legal research. Am. Bar Assoc. J . 4q (8), 764-767 (1961). 31. Freed, R.N., How computer specialists can help lawyers. J . Ind. Eng. 12,324-327 (September-October 1961). 32. Freed, R. N.,Machine data processing systcrns for the trial lawyer. Practical Lawyer 6,73-96 (April 1960). 33. Gelernter, H., Hansen, J. R., and Loveland, D. W., Empirical explorations of the geometry theorem machine. Proc. Western Joint Computer Conf., S u n Francisco, 17, 143 (1960). 34. Hayden, R. F. C., Electronics and the administration of justice. Proc. 1st Natl. Law and Electronics Conf., Arrowhead Lake, California, October (1960),sponsored by Univ. of California a t Los Angeles. Matthew Bender, Albany, 1962.


345

35. Hayden, R. F. C., How electronic computers work: A lawyer looks inside the new machines. Proc. ABA Electronic Data Retrieval Comm. Conf., S t . Louis, Missouri (1961); see M.U.L.L. in press (1962). 36. Hensley, D. R., Punched cards produce progress in Probate Court. Am. Bar Assoc. J . 48, (2) 138-139 (1962). 37. Horty, J. F., Experience with the application of electronic data processing systems in general law. Proc. ABA Electronic Data Retrieval Comm. Conf.,Washington, D.C. p. 3 (1960), Bureau of National Affairs, Washing&, D.C., 1961; see also M.U.L.L. pp. 158-168 (1960). 38. Horty, J. F., The keywords in combination approach to computer research in law with comments on costs. Proc. ABA Electronic Data Retrieval Comm. Conf., St. Loiiis, Missouri (1961), see M.U.L.L. pp. 54-64 (March, 1962). 39. Horty, J. F., Kehl, W. B., Bacon, C. R. T., and Mitchell, D. S., An information retrieval language for legal studies. Commiin. Assoc. Computing Machinery 4 (9), 380-389 (1961). 40. Jancin, J., The electronic inventor-A fantasy. J. Patent Ofice Soc. 43, 857-861 (Dec. 1961). 41. Jacobstein, J. M., The computer and legal implications. Quaere (Dec. 1960), Semiannual of the Univ. of Colorado School of Law. 42. Kalikow, M., Patent infringement determined by information retrieval, Proc. ABA Patent Trademark and Copyright Section Symposium o n Information Retrieval, St. Louis, Missouri pp. 18&200 (1961). 43. Kelso, L. O., Does the law need a technological revolution? Rocky Mountain Law Rez~.18, 378 (1946). 44. Kent, A., Experience with the application of electronic data processing systems in general law. M.U.L.L. pp. 179-185 (1960). 45. Kort, F., Predicting Supreme Court decisions mathematically: A quantitative analysis of the “right-to-counsel” cases. Am. Polit. Sci. Rev. 51, 1 (March 1957). 46. Kort, F., Reply to Fisher’s “Mathematical Analysis of Supreme Court Decisions.” Am. Polit. Sci. Rev. 52, 339 (June 1958). 47. Kort, F., The quantitative content analysis of judicial opinions. Polit. Research: Organization and Design 3, 11 (March 1960). 48. Kort, F., The quantitative content analysis of judicial decisions. Annaal of Political Science (Prof. Heinz Eulau, Stanford, California ed.) in press. The Free Press, Glencoe, Illinois, 1962. 49. Lawlor, R. C., Analysis of patent claims by mathematical logic. Proc. ABA Patent

Trademark and Copyright Section Symposium on Information Retrieval, St. Louis, Missouri pp. 201-228 (1961). (Proceedings are obtainable a t ABA, Chicago, Illinois.) 50. Lawlor, R. C . , Prediction of Supreme Court Decisions. Proc. 2nd Natl. Law and Electronics Conf., Arrowhead Lake, California (May 1962), sponsored by Univ. of California at Los Angeles and Systems Development Corp., Matthew Bender, Albany (In Press). 51. Ledley, R. S., and Lusted, L. B., The use of electronic computers to aid in medical diagnosis. Proc. I R E 47, 1970-1977 (November 1959). 52. Lewis, G. J., Electrical revolution in legal research. Illinois Bar J . 47, 680 (April 1959). 53. Loevinger, L., The element of predictability in judicial decision making. Proc. 1st Natl. Law and Electronics Conf., Arrowhead Lake, California, October (1960), sponsored by Univ. of California a t Los Angeles. Matthew Bender, Albany, 1962.

346

REED C. LAWLOR

54. Loevinger, L., Jurimetrics-Science and prediction in the field of law. Proc. A B A Electronic Data Retrieval Comm. Conf., St. Louis, Missouri (1961), see M.U.L.L. in press (1962). 55. Mathews, G. E., Computer dollars and sense in lawyer’s time records. Practical Lawyer 7, &22 (May 1961). 56. McKinnon, F. B., Leary, J. C., and Levinson, D., The American Bar Foundation project on the survey of American statutory law. Proc. S y s t e m of Information Retrieval Conf., CZeveZund, O@o, (April 1957), sponsored by Western Reserve Univ. Center for Documentation and Communication Research. 57. Mehl, L., Automation in the legal world-machine processing of legal information on the “law machine.” Proc. Symposium on the Mechanization of Thought Processes, Teddington, Middlesex, England (November 1958), sponsored by (British) National Physical Laboratory. 58. Melton, J. S., and Bensing, R. C., Searching legal literature electronically: Resalts of a test program. Minresota Law Rev. 45,229 (December 1960). 59. Molina, E. C., The Science of Chance Invades the Realm of the Law. New Jersey Law Institute, Newark, New Jersey, 1954. (See ref. 18.) 60. Morgan, R. T., The point of law approach to computer research in law. Proc. A B A Electronic Data Retrieval Comm. Conf., St. Louis, Missouri (1961), see M.U.L.L. pp. 44-48 (March, 1962). 61. Nagel, S., Weighting variables in judicial prediction. M.U.L.L. pp. 93-96 (1960). 62. Nagel, S., Using simple calculations to predict judicial decisions. Am. Behavioral Scientist 4‘24-28 (December 1960). 63. Pfeiffer, J. E., Symbolic logic. Sci. American, pp. 22-24 (December 1950). 64. Schubert, G. A., Quantitative Analysis of Judicial Behavior. The Free Press, Glencoe, Illinois, 1959; reviewed by F. Hort in M.U.L.L. pp. 143-145 (1960). 64a. Smith, B. M., Mens rea and murder by torture in California.Stanford Law Rev. 10, 672-693 (July 1958). 65. Suppes, P., Introduction to Logic. Van Nostrand, Princeton, New Jersey, 1957. 66. Tammelo, I. (edited by Duke Univ. Faculty of Law), Sketch for a symbolic juristic logic. J . Legal Educ. 8, 277 (1956); see also R. L. Clark’s criticism [18]. 67. U.S. Senate, 85th Congress, Mid-session.Hearings of subcommittee of the committee on government operations on 53126 (Science and Technology Act of 1958), Part I, pp. 250-251. 68. Waddell, W., Jr., Structure of Laws as Represented by Symbolic Methods. Ward Waddell, Jr., San Diego, California, 1961. 69. Warner, H. R., Toronto, A. F., Veasey, L. G., and Stephenson, R., A mathematical approach to medical diagnosis. J . A m . Med. Assoc. 177, 177-183 (1961). 70. Ziembinski, Z., Logic in law schools in Poland. M.U.L.L. pp. 98-99 (1960). 71. Bank of America National Trust and Saving, etc. us. County of Los Angebs, etc., Nos. 758,864 and 784,689, Superior Court of Los Angeles. (Decision not published,)

EXHIBIT“A”. EXAMPLES OF PREDICTIONS OF DECISIONS IN RIGHT-TO-COUNSEL CASESMADEON IBM iO90 .

W.

-

-.

F t GRUARY

lily62

JOHN 4AWYER

__

16-MPlN STREET HGPE T O h N U.S.A.

__ -.

-.

RE.

34

YOUR D O C K t T NO.

_ _

. _

O i A K MR.

- ___ ,

i

LAWYER

____

AN ANALYSIS HAS BEN M A D E O F THE R I G H T - T O C O U N S E L L S E - -THAT YOU PRESENTEI) E N T I T L t D ANONYMOUS NOS. 6 AND 7 V. B A K L K . 3 6 0 U . S . 287 (1959) T H E P N A L Y S I S WAS P E R F U K M E D ON A N I B M 7090 H I T H A S P E L I A L F O K T H A N PRUGKAM. E X C E P T FOR T H E M E l H O O D E V E L O P E D B Y D E V E L O P E D BYF R E D KDKT-!- THE M E T H O D S _ E M P L O Y E 0 -!EKEIN_WERf R t E U C. L A k L O R .

--

__

THE R E S U L T S S E T - F O R T H ~IELOW F O R E C A S T THE V O T E OF THE UNITEU S I A T - E S - S U P R E M E COURT. A S A h l l O L E r A N D CHA-VOTES O F THE I h 3 I VIUUAL JUSTICES..ASSUMING 1 H A T T h E Y A N D THk C O U R I K E C O L N I I C

__IN -

15

19

Y ~ U RC A S E

_ _

THE-FULLOMING

ARE-PRFGNT--- -

FACTS

--

T H E P E T I T I I I N E R L A C K E D A S S I S T A N C E - O F CUUNSE-L A T SOME P H A S E O F _TH$ P_RO_C_EEDING UTHkK T H A N A T

T I M E S O F T H E A K R A I G N M C N I , THE P R E P A R A T I O N F O R T R I A L , THE T R I A L , OR T H E S E N T E N C I N G . THE P t T I T I O N E R NEVER E X P L I C I r L Y H A I V t O R I G H T i TO COUNStL. -

Q LHE R-_FACTS- FQUNO -1 N -PK E V I OU

-.

- F I R S T TECHNIQUE

--

@ /

1-H Lb__F_I_c_L&!A t _A B S E N 1.

C A SE-S--LN-

- - -

..

c.-B”

30 0s .-

I

- -

-

__

Encn L ~ F THE

FOLLUWIN~ - P w - C R A N O C K R Y ~ n s t r sINCLCD~S O N L Y PRESENT I N _ Y O U s _ C $ S k . THE J U S T I C E S WHO v c r E D PRO ON T H E RESPECTIVE LASE?, A K E INDICATED.

F-A L T S - TH A. T - AKE -.

- . _.

. .

NUNt

.

.

.

.

.

.

.

.

.

-

-

- .........

C A L H O F T H t F O L L O U I N G -CONCI V C t O F C m S E L - - D E l h E t N A K R A I G h M E N T A N U T K I A L OK T H C H E A K I N G CN FHE P L t A O F G U I L T Y . T H E P k T I T I O J E K H A D 1 0 A S S I S T A N C t OF C O U i U S t L A T T h t T R I A L OK I H E H E A i t I N G ON T t l E P L E A O F G U I L T Y . T H E P t T I T I O N E R H A L : N!2 A b S I S T A N C E AT- T-tiE T I M E U F S t N T t NC I N G . T h t P E T I T I O I J E K S R t Q U t S T F U K A S S I G Q M E N T OF CUUNSEL W 4 b L)t\lIEL). T H t P t T I l I O ~ L KN t V t K t X P L I L I T L Y W A I V t D R I G H T

._ SUbJCCT

-

___

~

Ta cuu:istL. 39

, '

&

A COMPLEX I S S U t H A S P K t S C N T SUCH AS k l i t T H E K T H t T K I A L C U U K T I l A U J U K I S O I C T I O N LIF THE C A S t CR L b M P L I C A T C D CHARLL-S W t R C I N V U L V C D .

B1 350

.

......................

.

MK.

JOHN LAWYER FEBRUARY 1,1962 PAC;€ 2 . --. . . . .

.

.................

.

~____

......

..........

..

.~.

.

OTHER F A C T S FOUND I N P R E V I O U S C A S E S I N T H I S F I E L D A R t ABSENT._ _ _ ~ - .. - F I R S T TECHNIQUE

-

___

- _. . _.

T l i E R C A R E N O C R A N B E H R Y C A S E S € - I T H E R P R U OK C b N .

-

-

SECONU T E C H N I Q U E

BASEC UPON A S P E C I A L W E I G H T I N G FORMULA THAT H A S D E t N DEVELT ____ H t C U U K T A S A WHULC, I T IS P R E D I C A T E D T H A T Y O U R __F-O RCASE WILL B E CONSI[)EKED A - p k o - - C n s g ; ----

P E- D -U --

-

THIRD TELHNIPUE

-

B A S E D UPON S P E C I A L H E I G H l l N G F U R M U L A S T H A T H A V E BECN D t V E L IT- I S AN-TICIPArED THAT O P E D F O R - T H E - I N D I V I D U A L JUSTIC-E-S, t A C H O F T H E F O L L O h I N G J U S T I C E S NUW S I T T I N 6 O N T H E C O U R T CASE AS WILL-TFEAT YOUR C A S E A S A - P K O C A S E OK A S A - C O h INDICAlEG.

__ -

~

NO F O R M U L A S H A V E Y E T B t E N O E V E L O P E L I F U R P R E D I z r T h G THE V O T E I N ' T H I S K I N D OF C A S E .

-UF _.T__ hE FOLL-UUIJVG -JUSTJCJA HARLAN HH t N N A N nH I T 1 A K E R SlLWARl

iKA r T H t - v o i o F m

1 T- -1-SY-THt R E F O K E p PR t I) I C A T C O U R T O N T H I S C A S E WC)ULD t3

4 PRO V U T t i S 1 CON V O T E S - __ U h l " E T E ' { M I N L R V U T t - S ._-_ 4

-

f-OURTH T L C H N I C U t -

-s

u f%m---

-~

__

- _____________ __

B A S E 0 UPON T H E T t C H N I Q U E U E V C L O P t D B Y FRED KURT, PROFfSSOR O F P O L I T I C A L S C I E N C E A T THC U N I V t R S I T Y O F L O N N k C I I C U T , YOUR _ C A S E H A S !.CASE

__- - _ _

VAluE-OF_

..

__

__ - _ _ _

466.60

SIIYCL THE L A S t V A L U E I S t O U A L TO OR E X C t E O S 3 7 5 . C ~ K O K T S I E C H , \ I I O U t I N D I C A T E S T H A T THE S U P K t M E C C J U K l W U U L D T K C A T YUUK C A S E A S 4 -PROCASE. . . . . . . . . . . . . . . . . . .

---

""

B2'- .-

-

JOHN -LnHYER 1 9 1962 3 PAGE

MH.

- FtBRUARY

SUMMARY

__

...........

.............

...

-

1

CON VOTES -__U N C E R T A I N VOTES

- ~ _ _ _ ~ _ - _ - _ _ . . _ - I

4 __

._ .

.......

4

PREDICTION-

.

.

.

.

.

.

.

.

.

.

.

.

PRO

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

................

~ . .........

352

.......................

Author Index Numbers in parentheses are reference numbers and are included to assist in locating references when the authors’ names are not mentioned in the text. Numbers in italics refer to the page on which the reference is listed.

A Ablow, C. M., 185, 186 Alexander, S. N., 82 (1, l l ) , 152, 153 Allen, L. E., 300 (4, 5), 306, 309, 335 (5), 343 Andrews, D. D., 313 (S), 335 (8),343 Aroian, L. A., 50, 74 Arrow, K. J., 185 (2), 186 Ashby, W.R., 334, 343

B Bacon, C. R. T., 346 Baker, R. M. L., 7 (2), 15, 21 (2), 36, 49, 74

Brown, J. R., 308, 343 Brown, L., 324 (16), 344 Brown, R. R., 185, 187 Bruce, G. H., 271 Burns, A. J., 279 (8, 9), 281 (8, 9), 297 Burns, M. C., 285 (lo), 297

C Carroll, C. W., 185, 187 Carroll, Lewis, 309, 344 Clark, R. L., 305 (18), 344 Cocke, J., 78 (2), 152 Codd, E. F., 87, 105 (3), 152 Cohen, C. J., 28, 74 Colby, R., 337 (Ma), 344 Connelly, M. E., 287 ( l l ) , 297 Conte, S., 271 Cournot, A. A., 325, 344 Cox, F. B., 287 (22), 297

Bartholomew, P. C., 343 Bauer, W. F., 278 (I), 296 Baxter, D. C., 896 Beale, E. M. L., 186, 186 Bekey, G. A., 278 (24), 297 Benington, H. D., 84 (7), 152 D Bensing, R. C., 346 Berkeley, E. C., 305, 309 ( l l ) , 343 Birkel, G., Jr., 285 (4, 5), 296 Dahlquist, B., 22, 7 4 Birkhoff, G., 191, 196, 197 (l), 205 (l), Dames, R. J., 271 206 (I), 271 Dantzig, G. B., 186 (19), 187 Biunno, V . P., 343 de Boor, C. M., 261, 262, 271 Blackwell, D., 185 (g), 187 de Haan, C. S., 344 Blanyer, C. G., 297 DeLand, E . C., 185, 187 Boole, G., 305, 325, 343 de Laplace, Pierre Simon, 325, 344 Brenner, J. L., 30, 74 Dickerson, F. R., 344 Brigham, G., 185, 186 Diliberto, S . P., 30, 33, 74 Brooks, R. B. S., 300 (4, 5), 306 (4, 5), Dornheim, F. R., 186, 187 335 (5), 343 Douglas, J., Jr., 193, 205, 210, 271 Brouwer, D., 25, 30, 64, 74 Dreyfus, P., 78 (5), 152 353

AUTHOR INDEX

354

E Eckels, A., 29 (32), 75 Eckert, J. P., 82 (6), 152 Eckert, W. J., 64, 74 Ehricke, K., 74 Ehrlich, L., 211, 227, 232, 273 Ernst, A. A., 278 (34), 298 Everett, R. R., 84 (7), 152

F Fiacco, A. V., 185, 187 Fiordalisi, V., 344 Fisher, F. M., 325 (24, 25), 328, 335 (24), 344 Flanders, G. A., 229, 272 Forsythe, G. E., 191 (8), 195 (8), 263 (8), 271 Fort, T., 204 (9), 271 Frank, M., 185, 187 Frankel, S., 192 (lo), 271 Frankovich, J. M., 82 (8), 152 Freed, R. N., 335, 344 Freund, R., 30 (lo), 33 (lo), 74 Frisch, R., 185, 187

G Gantmakher, F., 204 (loa), 272 Garfinkel, B., 30, 74 Garrett, J. R., 24, 39 (14), 64, 74 Gass, S. I., 156 (12), 184, 187 Gelernter, H., 334, 344 Gill, S., 82, 152 Golub, G. H., 229 (ll), 272 Greenstein, J. L., 297 Griffith, R. E., 186, 187

H Habetler, G. J., 193, 194, 195, 196, 206 (24), 211 (24), 262 (24), 272 Hahn, W. R., Jr., 288 (38),298 Hamming, R. W., 22 (15), 74 Hansen, J. R., 334 (33), 3'44 Hanson, G., 62, 74 Hartsfield, E., 278 (14), 297 Hayden, R. F. C., 339, 344 Heid, J., 279 (27), 298

Heller, J., 209, 27.2 Henrici, P., 25, 40, 75 Hensley, D. R., 339 (36), 345 Herrick, C. E., 13, 14, 75 Herrick, S., 17, 75 Heraog, A. W., 287 (15), 297 Hildebrand, F., 13 (23), 20 (23), 23, 25, 75 Holt, J., 56 (30), 66 (30), 75 Hori, G., 31 (24), 75 Horty, J. F., 313 (37), 321 (38), 335 (37), 342, 345 Horwitz, R. D., 297 Householder, A. S., 195, 212 Hubbard, E. C., 28, 74 Hurney, P. A., Jr., 297 Hurwicz, L., 185 (2), 186

J Jackson, A., 279 (18), 297 Jacobstein, J. M., 345 James, P. A., 300 (4, 5), 306 (4, 5), 335 (5), 343 Jancin, J., 337 (40), 345

K Kahn, W., 96 (13), 153 Kalikow, M., 313 (42), 335 (42), 345 Kehl, W. B., 345 Kelley, J. E., Jr., 186, 187 Kelso, L. O., 310, 311, 342, 345 Kent, A., 312 (44), 345 King-Hele, D. G., 30, 75 Kolsky, H. G., 78 (2), 152 Kopal, Z., 39, 75 Kopp, R. E., 279 (8, 9), 281 (8, 9), 297 Kort, F., 325 (45, 46), 328, 330, 335 (45, 46, 47, 48), 345 Kosai, Y., 30, 75 Krein, M., 204 (loa), 272 Kryloff, N., 276 Kyner, W. T., 30 (lo), 33 (lo), 74

L Lapides, L., 279 (31), 298 Latta, G. E., 30 (4), 74 Lawlor, R. C., 308 (49), 331 (50), 333 (50), 335 (49, 50), 337 (50), 345

355

AUTHOR INDEX

Leary, J. C., 346 Ledley, R. S., 333 (51), 345 Lee, R. C., 287 (22), 297 Lees, M., 272 Leger, R. M., 297 Leiner, A. L., 82 (11, 12), 152 Levinson, D., 346 Lewis, G. J., 345 Loevinger, L., 325, 345 Lourie, N., 96 (13), 153 Loveland, D. W., 334 (33), 344 Lowry, E. S., 87 (4), 152 Lusted, L. B., 333 (51), 345

M McCarthy, J., 84 (19), 153 McDonough, E., 87 (4), 152 Mace, D., 28 (44), 76 McKinnon, F. B., 346 McLeod, J. H., 297 Manne, A. S., 185,187 Mathews, G. E., 304 (55), 346 Maxm-ell, M. E., 298 Mehl, L., 302, 346 Melton, J. S., 346 Mersel, J., 82 (14), 153 Miller, C. E., 186, 187 Milsum, J. H., 296 Mitchell, D. S., 345 Mitchell, H. F., 82 (6), 152 Molina, E. C., 325, 346 Morgan, R. T., 320, 346 Mori, H., 297 Morrison, D., 53 (29), 54 (28), 56 (30), 66 (30),75 Musen, P., 30, 75

N Nagel, S., 331, 335 (61, 62), 346 Neustadt, L. W., 278 (24), 297 Nigro, J. P., 278 (34), 298 Nothman, M. H., 297 Note, W. A., 82 (12), 153

0 O'Keefe, J. A., 29 (32), 'i5 Ostrowski, A. M., 224, 272 Ottoson, H., 279 (18), 297

P Palevsky, M., 297 Parter, S. V., 228, 272 Paskman, M., 279 (27), 298 Paul, R. J. A., 298 Payne, M., 7 (33), 36 (33), 75 Peaceman, D. W., 193, 211, 262, 271 Peet, W. J., 298 Perry, M. N., 85 (15), 153 Peterson, H. P., 82 (8), 152 Pfeiffer, J. E., 305, 346 Pines, S., 7 (33), 36, 75 Plugge, W. R., 85 (15), 153 Porter, R. E., 78 (16), 153 Pyne, I. B., 185, 187

R Rachford, H. H., 193, 205, 211, 219, 262, 271, 272

Rademacher, H., 25, 75 Reach, R., 96 (13), 153 Remes, E., 261, 272 Rice, J. R., 260, 261, 262, 271, 272 Robison, D. E., 50, 74 Rochester, N., 81, 153 Rosen, J. B., 185 (17), 187 Routh, D., 62, 74

S Scalzi, C. A., 87 (4),152 Schmid, H., 288 (30), 298 Schrimpf, H., 96 (13), 153 Schubert, G. A., 328, 3.46 Shapiro, I., 50 (36), 75 Shapiro, M., 186, 187 Shapiro, S., 279 (31), 298 Shortley, D., 229, 272 Shumate, M. S., 298 Skramstad, H. K., 278 (33, 34), 288 (35, 38), 298 Smart, W. M., 4 (37), 75 Smith, B. M., 308, 3.46 Smith, J. L., 82 (12), 153 Smith, N. M., 185 (9), 187 Smith, 0. K., 28, 31 (41), 63, 64, 75 Squires, R. K., 29 (32), 75 Stein, M. L., 279 (36), 298

AUTHOR INDEX

356

Stephenson, R., 333 (69), 346 Stewart, R. A., 186, 187 Stiefel, E. L., 261, 272 Struble, R., 30, 76 Suppes, P., 305 (65), 309 (65), $46 Susskind, A. K., 298 Swerling, P., 50 (43), 56, 75

W Wachspress, E. L., 193, 194, 195, 196, 204, 206 (24), 211, 232, 260, 262, 272 Waddell, W., Jr., 306, 346 Warner, H. R., 333 (69), 346 Wasow, W. R., 191 (8), 195 (8), 263 (8), 271

T Tammelo, I., 346 Teager, H. M., 84 (18, 19), 153 Thomas, L. M., 28, 76 Thrall, R. M., 205, 272 Titus, J., 76 Tornheim, L., 205, 272 Toronto, A. F., 333 (69), 346

U Urban, W. R., 288 (38), 298 Uzawa, H., 185 (a), 186

Weinberger, A., 82 (12), 153 Weiner, J. R., 82 (6), 162 Welsh, H. F., 82 (6), 152 West, G. P., 278 (l), 296, 298 Weyl, H., 202, 273 Wilson, A. N., 278 (42), 279 (42), 698 Witzgall, C., 186, 187 wolf, H., 7 (331, 36 (33), r5 Wolfe, P., 185, 186 (22), 187 Wortzman, D., 285 (43), 298 Wright, R. E., 298 Y Young, D., 211, 225 (as), 227, 229, 232, 240, 247, 273

V

Z Varga, R. S., 191, 192 (19), 196, 197 (I), 203, 204 (19), 205 (l), 206 (l), 226, 227, 228 (20, 21), 229 (ll), 230 (19), 240, 271 Veasey, L. G., 333 (69), 346

Ziembinski, Z., 309 (70), 346 Zoutendijk, G., 185, 187 Zraket, C. A., 84 (7), 152

Subject Index A Abstracting, automatic, 341 Accuracy tests for integration, 35-47 Adam-Moulton formulas, 22 Adjoint system of daerential equations, 39 Advance commitments, 115-116 Allocation of computer components, 105106; see also Tape, Disk, Core storage Analog-digital variables, 288-296; see also Arithmetic Analog-to-digital, see Converters Arithmetic, analog-digital, 283-288 hybrid, 285 Asymptotic rate of convergence, 195, 200, 212, 223, 225-227

B Batch processing, of computer problems, 83 Batch multiprogramming, 86, 87-104, 113 Biology, information growth in, 301, 303 BLOCKoperation, 132, 144, 145 Blocks, of core storage space, 112, 114 Boolean algebra, 305 Buffer service, 97-98 Burnout parameters, 49, 50-55, 64-66

among numerical integration methods, 38, 4 2 4 7 Concept profile, 322 Concurrency of computer operations, 78; see also Phase, Task c. Convergence, of simplex corrected gradient method, 170 of separable programming, 178 of cutting plane method, 181 of AD1 methods, see Asymptotic rate of c. Converters, analog-digital and digitalanalog, 277, 280, 283-285 Copyright, see Infringement Copyrighting of computer products, 337, 338 Core storage allocation, 112-115 modules, 90 Court decisions, prediction of, 324-333 retrieval of information, 311-323 Courts, see Law Cowell method, 6, 7, 14, 28, 42-47 Crank-Nicolson method, 230 Cutting-plane method, 174, 180-183, 186

D

Data editing, 49, 50, 71 Data exceptions, 128 C Decomposition procedure, 178-180, 186 Definitive orbit, 48, 49 Calculus of classes, 305 Delaunay theory, 30,31, 34, 35 of functions, 305 Density of atmosphere, 5 Determination of orbits, 48-69 Chebyshev polynomials, 229 Diagonally dominated matrices, 192, 202 property, 259 Differential corrections, 49, 51, 6649, 72 cyclic cemi-iterative method, 250 stagewise, 56-58 Columnar procedures, 174, 175-180 Combined analog-digital computers, 277, Differential gradient methods, 161-165 280 direct, 162-163 Combined analog-digital solution, examLagrangian, 163-165 ples, 281-283 Digital-to-analog, see Converters Comparison of analog and digital com- Diliberto theory, 30, 31-35 Dirichlet problem, 231-250 puters, 275-277 of analytic and numerical integration, 36 Disk allocation, 110-112 357

358

SUBJECT INDEX

Divergence of AD1 methods, 253 Double precision, 37 accumulation, 21, 23 Douglas-Rachford method, 193, 197, 206, 217-222, 256-257

parameters, 219, 223 Drag, 5, 49 Dumping, of computer programs, 102 Dump file, 111

E Eccentricity, 9 Eccentric anomaly, 10 Elliptic orbits, 9, 10 Encke method, 7-15, 4 2 4 7 with E or v as independent variable, 11-13

Ephemeris generation, 70 processor, 74 Equations of motion, 4-18 Error reduction matrix, 194-195, 197, 217 Evidence, use of computer records as, 338, 339

Execution control, 95-104 file, 111 phase, 9 1 region, 114

F Feasibility orbits, 2, 3 FIFO (fist-come-fist-serve), 121 Fraud by computers, 338 Frobenius, theorem of, 205 Function generation, 276, 278

G Gauss-Jackson method, 23, 27, 7 1 Gauss-Seidel method, 225, 229 Gerschgorin’s circle theorem, 204 Gradient methods, see differential, largestep, projected, simplex-corrected

Herrick’s method, 15, 28, 42-47, 4Y Hybrid arithmetic, 285 integrator, 290-293 multiplier, 293-295 systems, 278, 288-296 Hyperbolic orbits, 9, 10

I Ill-conditioning, 54 Improved orbit, 48, 49 Indexing, automatic, 304, 341 Inertial coordinate system, 4 Information growth, 299-302 retrieval, 310-323 storage, 310-323 Infringement of copyright by computer records, 335-337 Initiation of algorithms (in mathematical programming), 183-184 Injection parameters, see Burnout p. Input conversion, 70; see also Data editing Input-output requests, 95 Input-output service trees, 137, 138, 141150

Instability, 22 Insurance, application of Boolean algebra to, 305 Integrals of motion, 38 Integration (of ordinary differential equations), methods of, 18-28 Integrator, hybrid, 290-293 Interconnected analog and digital coinputers, 277-281 Internal Revenue Code, 305 Interprogram protection, 98-101, 132, 151 Interruption rules, 125-128 Inverse square law, 6 10, see Input-output Irreducible matrices, 192, 205, 227 Iteration parameters, 190, 193 good, 202, 203, 262 optimal, 198-199, 202, 231-250, 254258, 259-261 see also Douglas-Rachford, Peaceman-

H

Rachford, Wachspress

1

Hansen method, 30 Helmholtz equation, 200-202, 206, 222223, 225-227, 243

Jacobi matrix, 225

359

SUBJECT INDEX

Jacobi method, 225 k-line, 228 Jordan normal form, 248

K Kepler’s equation, 10-13

L Lagrange multiplier, 165, 173 Large-step gradient methods, 165-172 Law courts, use of computers in, 339; see also Court decisions Law, information growth in, 299-303 Law questions affecting computers, 335339 Lan-s, indexing and retrieval of, 320, 340 Least squares, 50-56, 72 weighted, 53 Legal questions, see Law Load, on time-shared computer components, 117 target, 118 Log (operating) file, 111

M Mean anomaly, 9, 17 Mean motion, 17 Mechanization in law practice, 302-305 Medicine, information growth in, 301, 303 hficroflnl images, automatic retrieval of, 324 Microprogramming, 125 Milne formulas, 22 Milne-Stormer formulas, 25 Minimax, 199, 211, 254-262 Missile intercept problem, 281-283 Mixed boundary conditions, 263-265 hlonotonicity principle, 199, 202, 203 Moulton, see Adams Multiplier, hybrid, 293-295 Multistep methods (of integration), 21-24, 25, 40 Multirevolution steps, 28

N Nondata exceptions, 126-128 Nonscheduled mode of computer operation, 106

Nonstandby programs, 112 Nonsystems program file, 111 Nonuniform mesh spacings, 263-265 Norm reduction, 195-196 Normal equations, 52-55 Normalized iterations, 250 Normalized weights, for computer programs, 118 NOT READY state, 93, 102 Notations of symbolic logic, 309 Notched cards, 323-324

0 Oblateness force, 4 Observation processing, 71 Observational errors, 53, 66 Operating staff of computer, communication with, 102-103 Optical observation, 48 Optimization, of computer programs, 104122 of iteration parameters, see Iteration parameters, optimal Orientation elements, 10 Ostrowski’s theorem, 224-225, 227 Overrelaxation, see Successive overrelaxation method

P Parabolic and near-parabolic orbits, 13-15 Parameters, see Douglas-Rachford, Iteration, Peaceman-Rachford, Wachspress Partial derivatives, computation of, 5 8 4 4 , 72 Patent claim, 308-309 Patent searching, 322, 323 Patenting of computer inventions, 337,338 Pattern recognizing machines, 321 Peaceman-Rachford method, 193, 200, 206, 210-217, 226,229,231-250,254256 parameters of, 202, 211-215, 231-250, 262 Peekaboo systems, 323, 340 Pending workload, of computers, 90-91 Periodic two surfaces, 33 Perturbation methods, 28-35, 42-47

SUBJECT INDEX

360

Petroleum industry, use of mathematical programming in, 157 Phase, see Execution, Preparation, Rewinddemount, Scheduling concurrency, 92 Point Jacobi matrix, 225 Post-execution control, 95 Precision orbits, 2, 3 Prediction, see Court decisions Predictor-corrector formulas, 22, 25 Pre-execution control, 95 Preliminary orbit, 48 Preparation phase, 91 Printing, automatic, 340 Priorities, for computer programs, 120, 125, 134 Processing service tree, 135-140 Program mixes, 109 Projected-gradient procedures, 170-172, 185-186 Projection operators, 266-267 Property “A” of matrices, 225 Protection, 128-130; see also Interprogram PPulsed analog techniques, 286-288 Punched cards, applications in law practice, 304, 323-324

Q Quadratic programming, see Simplex method Queue selection time, 121 Queuing of computer programs, 105-106, 120-122, 134-150 Queuing rules, 148-150

R Radar observation, 48 Ranking of judges, 328, 329 Rating, of computer programs, 118, 120 READREQUEST (pseudo-operation), 100 READYstate, 93, 102 Real-time processing, of computer problems, 85 multiprogramming, 86 Rectification, 7, 10 Reference conic, r. orbit, 7-10

Relaxation factor, 224 optimal, 225 Reliability, of court decision predictions, 325 Relocation, 101-102, 128-130 Remes algorithm, 261 Residuals, 54, 72 Restoration of computer programs, 102 Restricted operations, 100-101, 129-130 Rewind-demount phase, 91 Round-off errors, accumulation of, 21, 2528, 3 8 4 2 Run requests, 90 Runge-Kutta methods, 19-21, 24, 71

5 Scalogram, 328,329 Scheduling, of computer runs, 105-106 long-range, 115 short-range, 115, 116-120 algorithm, 116 Scheduling phase, 91 Search, mechanized, 311 SEEKREQUEST (pseudo-operation), 100 Seeking mechanism of storage disks, 88 Seeking time, 119 Segmentation of computer data and instructions, 124 Semimajor axis of orbits, 9 Semi-cymbolic programs, 101 addresses, 115 Separable programming, 177-178, 186 Service requests, 96-97 Service time, 108, 118 Shadow prices, 165 Simplex method for linear programming, 166-168, 172174 for quadratic programming, 174-175, 186 Simplex-corrected gradient method, 168170, 185 Simplicia1 methods, 172-175 Simulation problems, 275, 276, 280 Simulators, 275 Solar parallax, 3, 49 Space release, s. request (in computers), 103

SUBJECT INDEX

Space-scheduled computer operation, 106108 Space sharing by computer programs, 7980 Space-time scheduled computer operation, 106, 108-109 Spectral radius, 195, 202, 233, 251, 253, 257 Staging file, 108, 111 Standard deviation, of errors of observation, 53 Standby program file, 111, 115 Stare decisis, 331 STARToperation, 131, 140, 141 Stationary iterative methods, 195 Statistical estimate of errors, 67-69 Status preservation, 96 Statutes, see Laws Stieltjes matrices, 192, 196, 202, 204, 227 STOPoperation, 132, 142, 143 Successive overrelaxation method (SOR), 190, 224-230, 231-250 point, 224-227 block, 227-229 multiline, 227-229 Supervisory tables, 125 Suspension, of computer programs, 102 Symbolic logic, 305-310 Symmetric rounding, 27 Syntactic ambiguities, 306, 307 Systems file, 111

361

T Tape allocation, 109-110 Task concurrency, 92 Throughput rate, 83 Time sharing, by computer programs, 79, 117 Typewriters, automatic, 304, 341

U UNBLOCK operation, 132, 146, 147 Uniform loading of channels, 109 Unisolvent, 260

V Variance-covariance matrix, 56 Variation of parameters, 15-18 Varisolvent, 260 Virtual number of iterations, 233

W Wachspress parameters, 204,211,215-217, 231-250, 262 WAIT (pseudo-operation), 93, 95 Weakly cyclic matrices, 228 WRITE REQUEST (pseudo-operation), 100

Advances in Computers, Volume 3

Advances in Computers, Volume 3

Advances in Computers, Volume 49

Advances in Computers, Volume 55

Advances in Computers, Volume 56

Advances in Computers, Volume 19

Advances in Computers, Volume 12

Advances in Computers, Volume 43

Advances in Computers, Volume 28