Nonserial dynamic programming (Mathematics in Science and Engineering, Volume 91)

Nonserial Dynamic Programming

This page is intentionally left blank

Nonserial Dynamic Programming Umberto Bertele and Francesco Brioschi Istituto di Elettrotecnica ed Elettronica Politecnico di Milano Milan, Italy

@

1972

ACADEMIC PRESS

New York and London

C O W R I G H T 0 1972, BY ACADEMIC PRESS, INC. ALL RIGHTS RESERVED. NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT PERMISSION IN WRITING FROM THE PUBLISHER.

ACADEMIC PRESS, INC.

1 I 1 Fifth Avenue, New York, New York 10003

United Kingdom Edition published by ACADEMIC PRESS, INC. (LONDON) LTD.

24/28 Oval Road, London NW1

LIBRARY OF CONGRESS CATALOG CARDNUMBER: 78-187237 AMS (MOS) 1070 Subject Classifications: 90C99, 05C99,49C99 PRINTED IN THE UNITED STATES OF AMERICA

Contents Preface

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Acknowledgments

. . . . . . . . . . . . . . . . . . . . . . . . . . .

ix xi

Chapter 1 Nonserial Problems 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Serial Unconstrained Problem . . . . . . . . . . . . . . . . . . A Problem in Inventory Theory . . . . . . . . . . . . . . . . . . . The Nonserial Unconstrained Problem and Its Graph-Theoretical Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Problem in Pattern Recognition . . . . . . . . . . . . . . . . . A Problem in Traffic Control . . . . . . . . . . . . . . . . . . . . The Parametric Unconstrained Problem . . . . . . . . . . . . . . . The Constrained Problem . . . . . . . . . . . . . . . . . . . . . . Introduction to Nonserial Dynamic Programming and Plan of the Book .

1 2 2

Chapter 2 The Elimination of Variables One by One: Description of the Procedure 2.1 2.2 2.3 2.4

2.5 2.6

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . Serial Dynamic Programming . . . . . . . . . . . . . . . . . . . . Nonserial Dynamic Programming: The Description of the Solution of the Primary Problem . . . . . . . . . . . . . . . . . . . . . . . . . An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Secondary Optimization Problem . . . . . . . . . . . . . . . . The Elimination Process . . . . . . . . . . . . . . . . . . . . . . V

18 18 20 21

21 30

vi

Contents

Criterion Functions . . . . . . . . . . . . . . . . . . . . . . . . The Final Theorem and Other Dominance Relations among Elimination Orderings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 The Correspondence Theorem . . . . . . . . . . . . . . . . . . . . 2.10 The Parametric Unconstrained Problem . . . . . . . . . . . . . . .

2.7 2.8

35 39 46 50

Chapter 3 The Elimination of Variables One by One: Properties and Algorithms 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heuristic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . Optimal Path Algorithms . . . . . . . . . . . . . . . . . . . . . . Computational Implications of the Final Theorem . . . . . . . . . . . Further Dominance Relations among Elimination Orderings . . . . . . The Descendance Theorem . . . . . . . . . . . . . . . . . . . . . The Initial Theorem . . . . . . . . . . . . . . . . . . . . . . . . The Separating Set Theorem . . . . . . . . . . . . . . . . . . . . Connections between Structure and Dimension in a Graph . . . . . . Upper and Lower Bounds to the Dimension . . . . . . . . . . . . . A Branch and Bound Algorithm . . . . . . . . . . . . . . . . . .

.

54 55 58 61 65 68 76 78 80 85 96

Chapter 4 The Elimination of Variables in Blocks 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Description of the Procedure for the Solution of the Primary Problem An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . The Secondary Optimization Problem . . . . . . . . . . . . . . . . The Block Elimination Process . . . . . . . . . . . . . . . . . . . Some General Properties . . . . . . . . . . . . . . . . . . . . . . The Final Theorem . . . . . . . . . . . . . . . . . . . . . . . . The Descendance. Initial. and Separating Set Theorems . . . . . . . . Bounds and Algorithms: Some Hints . . . . . . . . . . . . . . . . The Correspondence Theorem . . . . . . . . . . . . . . . . . . . . The Parametric Unconstrained ProbIem . . . . . . . . . . . . . . . Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . .

100 101 103 108 111 115 121 126 133 135 137 139

Chapter 5 Multilevel Elimination Procedures 5.1 5.2 5.3 5.4 5.5 5.6

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multilevel Elimination Procedures for the Solution of the Primary Problem An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Secondary Optimization Problem . . . . . . . . . . . . . . . . . The Multilevel Elimination Process . . . . . . . . . . . . . . . . . The Final Theorem . . . . . . . . . . . . . . . . . . . . . . . .

140 141 144 156 160 164

vii

Contents

5.7 5.8 5.9

Some General Properties . . . . Heuristic Algorithms: Some Hints The Correspondence Theorem . .

..................

. . . . . . . . . . . . . . . . . . ..................

167 175 175

Chapter 6 Constrained Problems Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Penalty Function Approach . . . . . . . . . . . . . . . . . . . The Elimination of Variables One by One: Description of the Procedure An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . The Secondary Optimization Problem . . . . . . . . . . . . . . . . Univocal Constraints . . . . . . . . . . . . . . . . . . . . . . . . An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . Dynamic Systems and Other Applications : The Block Diagram Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.9 An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.10 A Discussion about Possible Improvements of the Optimization Procedures in Some Special Cases . . . . . . . . . . . . . . . . . . . . . . 6.11 An Allocation Problem . . . . . . . . . . . . . . . . . . . . . . .

6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8

183 184 185 187 193 194 195 197 209 213 214

Appendix A Review of Graph Theory . . . . . . . . . . . . . . . . 219 List of Symbols Glossary . . .

..........................

..........................

Appendix B Some Set-Theoretical Definitions . . . . . . . . . . .

219 220

222

Appendix C Combinatorial Aspects in the Solution of Linear Systems by Gaussian Elimination . . . . . . . . . . . . . . . . 224 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . Subject Index

..............................

229

233

This page is intentionally left blank

Up to the present dynamic programming has been regarded as a general type of approach to problem solving, essentially based on decomposition of the given problem into a sequence of smaller subproblems. It has permitted the solution of a remarkable number of problems (deterministic and stochastic, continuous and discrete, constrained and unconstrained, static and dynamic, and so on). In general, however, each individual situation has required derivation from the general philosophy of an ad hoc algorithm. This book, by restricting attention to the class of discrete deterministic optimization problems (constrained and unconstrained, parametric and nonparametric, serial and nonserial), aims at the following goals: (a) To give a general standard mathematical formulation for each class of problems. (b) To define, in a systematic way, three different classes of dynamic programming procedures, namely, elimination of variables one by one, elimination of variables in blocks, and multilevel elimination. The first two are a direct derivation of the principle of optimality, while the third can be regarded as a real generalization of dynamic programming. (c) To provide some simple criteria for measuring the computational effort (computing time and memory requirements) needed for each individual procedure. (d) To construct efficient algorithms, which exploit essentially graphtheoretical properties, for determining “optimal” or “good” elimination IX

X

Preface

procedures (secondary optimization problem). It is worthwhile noting that some of these algorithms are also appropriate for finding efficient (i.e., requiring minimal computing time) procedures for the solution, by means of Gaussian elimination, of a system of linear equations Mz = b with M symmetric, positive definite, and sparse. The approach of this book to dynamic programming, which is, as already noted, very general, also turns out to be effective from the didactic point of view, as the authors have verified while teaching Operations Research at the Politecnico di Milano.

Acknowledgments

It is a pleasure to acknowledge a number of persons who have influenced this work either directly or by means of their writings. In particular the initial researches (1965-1967) in the area of structural properties of dynamic programming at the Politecnico di Milano in the Laboratorio di Controlli Automatici were conducted by one of the authors together with A. De Maio, L. Divieti, G. Guardabassi, A. Locatelli S. Rinaldi, P. Rozzoni, and R. Schmid under the guidance of E. Biondi. The first paper in nonserial dynamic programming was written (1968) by one of the authors with S . Even at Harvard University. The bulk of the material presented comes from published or unpublished researches of the authors (1968-1971) at the Politecnico di Milano. However some interesting suggestions have come from a thesis of D. J. Rose, dealing with the elimination process, and from the work of R. Aris, G. Nemhauser, and D. Wilde, concerning the optimization of cyclic systems, which can be regarded as a first particular example of multilevel elimination procedures. Furthermore, some interesting theoretical results and applications, reported in the book, are due to R. E. Allsop, F. Harary and J. Krarup, J. A. Hillier, A. Korsak, A. Martelli, and U. Montanari. A. Beltrami, A. Colorni, and F. Forese provided the computer results of Sections 3.10-3.1 1 and also contributed many significant comments about the whole subject. A special debt of gratitude is due to R. Bellman who suggested the idea of this book. xi

xii

Acknowledgments

The typing of the manuscript was excellently accomplished by Mrs. Anna Maria Caputo and Mrs. Mariangela Rodolfi. Financial assistance was provided by the Consiglio Nazionale delle Ricerche either directly or through the Centro di Teoria dei Sistemi, Politecnico di Milano.

Nonserial Problems

1.1

Introduction

This chapter is devoted t o the general formulation of a sequence of optimization problems which, in the remainder of the volume, are examined and solved by a dynamic programming approach. The models considered, which are all deterministic and involve a finite number of discrete variables, are classified according to different criteria. In particular, serial and nonserial, unconstrained and constrained, and nonparametric and parametric models are discussed. The simplest model, namely, the serial unconstrained and nonparametric one, is introduced in Section 1.2. In Section 1.4 the seriality assumption is dropped. Parametric unconstrained problems are discussed in Section 1.7. Examples and applications of these models are given in Sections 1.3, 1.5, and 1.6. Finally, constrained problems are the concern of Section 1.8. In the last section of this chapter, the three different optimization procedures discussed in the book, all based on Bellman’s principle of optimality, are introduced. A plan of the whole work follows. 1

2

1. Nonserial Problems

1.2 The Serial Unconstrained Problem

As noted in the preceding section, the simplest optimization problem considered in this book is the serial unconstrained problem. It can be formulated as follows : minf(x) X

2

= min &(Xi), X icT

is a set of discrete variables, SzIbeing the definition set of the variable Xi(] Sq

I= T = {1,2, . . . , n - l};

and

xi = {Xi, xi+l}. The function f ( X ) is called the objective function and the functions f i ( X i ) are the components of the objective function and are specified, in general, by means of stored tables.

1.3

A Problem in Inventory Theory

Consider the following problem in inventory theory (see, for instance, [l]). A supply depot is required to meet a set of known demands for services or supplies at a preassigned set of times. If the demand is not satisfied, a cost is incurred. On the other hand, if the level of the depot is higher than the demand, another type of penalty exists, Finally, changing the level of services or supplies is costly. Letri ( i = 1, 2,. . . , n ) be thegiven demand at time i a n d x i (i = 1, 2,. . . n ) be the unknown level of services or supplies at time i. The level xo at time 0 is fixed. Each variable xi (i = 1,2,, . . ,n ) can assume a finite number ozt of values. Let fi‘(xi - ri) (i = 1,2, . . . , n ) be the cost due to the difference between the level of services or supplies and the demand. Let fi”(xifl - x i ) (i = 0,1, . . . , n - 1) be the cost inherent to adjusting the service level or stock level.

1.4

3

Nonserial Unconstrained Problem

Letting - r l ) +f,”(X, -

fl(X1, x2) =fir@,

&(xi, xi+l)=&’(xi

- ri)

xg)

+fl’(X2 - Xi),

+X’(X~+~ - xi),

i

= 2, . .

. , n - 2,

and fn-l(xn-19

Xn) = f L ( x n - 1

- rn-1) +fA(xn

- r,) + f L ( x n

-

Xn-d,

it is clear that the optimization problem is of the serial unconstrained type.

1.4

The Nonserial Unconstrained Problem and I t s Graph-Theoretical Representation

The statement of the nonserial unconstrained problem is minf(X) = min K

X

C fi(xi), ioT

is a set of discrete variables, Sz, being the definition set of the variable XAl

szj I

= %z,>;

T = {1,2, . . . , t } ;

and

Xic X .

The function f ( X ) us called the objective function, and the functions f i ( X i ) are the components of the objective function and are specified, in general, by means of stored tables. Some definitions which will be of use later are now given.

DEFINITION 1.4.1 Two variables x E X and y E X are said to interact if there exists a component f k ( X k ) such that both x and y belong to Xk.

DEFINITION 1.4.2 The set of the variables which interact with a variable x by Y 6).

E

X is denoted

4

1. Nonserial Problems

DEFINITION 1.4.3 Let Y c X . The set of the variables of X - Y, which interact with at least one variable of Y, is denoted by y ( Y ) and is given by

A graph-theoretical representation of a nonserial unconstrained problem is now introduced. The objective is to prepare a tool which will be useful in deriving an efficient solution of this problem via dynamic programming. In addition, this representation allows us to visualize the nonserial structure of the problem. DEFINITION 1.4.4 The interaction graph G = ( X , L ) of a nonserial unconstrained problem is an undirected graph, without self-loops and parallel edges, defined by the following.

(a) The vertex set X of the graph is the set of the variables of the problem. (b) Two vertices are adjacent if and only if the corresponding variables interact. The sets y ( x ) and y ( Y ) , previously defined, are hence obviously equal to the sets r(x)and r ( Y ) of the interaction graph (see Appendix A). It is interesting to note that the same interaction graph can originate from different problems.

EXAMPLE 1.4.1 Consider the nomerial problem 3

min C f i ( X i ) ,

x

i=l

The interaction graph of the problem is reported in Fig. 1.4.1. Note that the same interaction graph may also come, for instance, from the problem 4

min Cfi(Xi), x i=l

1.4

Nonserial Unconstrained Problem

5

Figure 1.4.1

where again but

EXAMPLE1.4.2 The interaction graph of a serial problem

is given in Fig. 1.4.2 and shows the origin of the adjective ‘‘serial.” In some cases it is easier to visualize the “structure” of a nonserial unconstrained problem by means of the complement G = ( X , 1)of the interaction graph G = ( X , L).

Figure 1.4.2

EXAMPLE1.4.3 The complement of the interaction graph of Example 1.4.1 is shown in Fig. 1.4.3. Finally it is worth noting that whenever the interaction graph G = ( X , L ) consists of k distinct components, the corresponding optimization problem can be divided into k independent subproblems.

6

1. Nonserial Problems

Figure 1.4.3

1.5 A Problem in Pattern Recognition An important problem in pattern recognition consists of automatically recognizing some preassigned elements of a given picture corrupted by noise. This can be performed by either local operations or a gIoba1 procedure. In particular, a global approach, which ends up in a nonserial unconstrained problem, taken from [2], will be described in this section. The aim is to find a picture as similar as possible to the one given, but with some regularity properties, in order to eliminate, if possible, the effects of distortion. This process is called optimal smoothing. It consists of three steps (for simplicity the picture is assumed to be rectangular). (a) The original picture is digitized into a certain number of points and a given property P (such as “dottedness,” ‘‘linearity,’’ “verticality,” “rectangularity”) is measured for each point. (b) The number of representative points of the picture is reduced. A set of points of a region Rij (i = 1,2,. . . ,r ; j = 1,2,. . . , c ) shrinks into a single point Q i j , and the most reliable (in some sense) value uij of property P is assigned to Q i j . (c) The smoothing algorithm is applied. A discrete variable x i j , which represents the value of property P in Qij for the smoothed picture, is defined. Its value is determined by minimizing an objective function, which takes into account both the degree of irregularity and the difference from the given picture. Lettingf, and fi be the costs resulting from the irregularities of the picture and g be the term which penalizes the difference between x i j and a i j , the objective function f can be written as t-1

c-1

i-1

j=1

f = 1 C fi(xij +g(aij

9

Xi&

3

xi+l,j)

+fi(xij

xi,j+l)

1.5

7

Problem in Pattern Recognition

Figure 1.5.1

.’ ..- ”

This minimization problem is a nonserial unconstrained problem. Its interaction graph has a special structure shown in Fig. 1.5.1. Optimal picture smoothing has, for instance, been applied to fingerprint classification (see [2] again). The original photograph of the fingerprint (Fig. 1.5.2) is digitized with 312 x 456 points and, in each one, the direction of the dermal ridges (property P) is computed [step (a)]. Then the photograph is divided into 247 “windows” of 24 x 24 points and the main direction aij is determined for each window [step (b)], thus obtaining a “sampling matrix” (Fig. 1.5.3). Finally [step (c)] the nonserial unconstrained

Figure 1.5.2 (From [2].)

8

1. Nonserial Problems

I

Figure 1.5.3 (From [21.)

0

Figure 1.5.4

problem is solved by dynamic programming. This provides the “smoothed matrix.” The smoothed matrix enables extraction of suitable paths (Fig. 1.5.4) from which some classificatory parameters for the fingerprint are derived. Another nonserial problem which arises in pattern recognition and is solved by dynamic programming is found in [3].

1.6

A Problem in Traffic Control

In this section a simplified traffic control problem is presented, and it is shown that it can be formulated as a nonserial unconstrained problem. The model was introduced in [4] for the coordination of 75 traffic signals in the central area of Glasgow, Scotland, and later developed by Allsop [5] and Korsak [6]. A network controlled by fixed-time signals is considered, and the following assumptions are made. (a) A cycle T and an integer o are given. All signals in the network have cycle times equal to T and their green-red ratio is known. One cycle is

1.7

Parametric Unconstrained Problem

9

selected and called the master cycle; the start of each one of the other cycles may be offset relative to the master cycle by any whole number of intervals T/o. (b) Whenever two or more signals exist or are taken into account, in some intersection, their relative offsets are preassigned. Hence each intersection A i (i = 1, 2,. . . , n) of the network is assigned an integer variable xi (0 5 x i 5 o - 1 ) which specifies the offset of its cycle (or cycles) relative to the master. (c) A delay function d i j for each one- or two-way link connecting intersection i with intersection j is defined. The function dij depends upon the average delay of a vehicle entering the link at one end, due to the fact that the signal at the other end is not always green, and upon the total number of vehicles per hour. It is assumed that the total delay per cycle d i j on each one- or two-way link of the network is a known function of the difference between the offsets of the signals at the ends of the link, and is independent of the other offsets in the network; that is, dij = dij(xi - xi). Clearly the functions dij are subject to hourly, daily, and seasonal variations. The problem of determining the values of the variables x i , which minimize the total delay, is a nonserial unconstrained problem, and the objective function can be written as

Whenever no link exists (or is considered) between intersections i and j , the delay dij is assumed to be equal to zero. The assumption that all signals have the same cycle may be easily removed. In fact, it is sufficient that each cycle time be a submultiple of T : The ranges of the xi’s are consequently reduced. Assumption (c) is not entirely realistic, but, nevertheless, the model has allowed appreciable reductions in delay.

1.7 The Parametric Unconstrained Problem

An unconstrained problem is in parametric f o r m when the objective function must not be minimized with respect to (w.r.t.) X , but only w.r.t. a subset X - P, f o r all possible assignments of the variables of P. The variables of the set P are, in this case, called parameters.

10

1. Nonserial Problems

Hence, the formulation of a parametric unconstrained problem is minf(X) x-P

C

= min fi(Xi), X-P ICT

is a set of discrete variables, Sxjbeing the definition set of the variable

%(I

I

sx, = @xJ;

T = {1727...,t};

xic x;

Pc

x;

\PI = p .

Again, the function f ( X ) is called the objective function and the functions h(Xi) are the components of the objective function and are specified, in general, by means of stored tables. The definitions of interacting variables and interaction graphs are the same as in Section 1.4. However, for sake of clarity, the vertices of the interaction graph corresponding to the parameters are distinguished from the graphical point of view. Two important classes of parametric problems are now introduced.

DEFINITION I .7.1 A parametric unconstrained problem, with interaction graph G is in quasi-standard form if

=

( X , L),

r ( X - P) = P. When a problem is not in quasi-standard form, a simpler minimization problem can be considered. In fact the set P may be replaced by the set r ( X - P) (r(X- P) 2 P) from the point of view of the optimization procedure.

EXAMPLE1.7.1 Consider the parametric unconstrained problem

the interaction graph of which is reported in Fig. 1.7.1. This problem is not in quasi-standard form. The parameter x4 and the component .f2 do not concern the optimization procedure.

1.7

Parametric Unconstrained Problem

Figure 1.7.1

DEFINITION 1.7.2 A parametric unconstrained problem, with interaction graph G = ( X , L), is in standard form if (a) it is in quasi-standard form; (b) the section graph of G w.r.t. X - P is connected.

Any problem in quasi-standard form, but not in standard form, may be divided into two or more distinct problems in standard form.

EXAMPLE 1.7.2 Consider the parametric unconstrained problem in quasi-standard form : minf,(x, x3 x,) 9

7

x-P

+

fi(X2

9

x4 Y

x,),

where X = {xl, x 2 , x3, x 4 , x,} and P = {x,}. The interaction graph is shown in Fig. I.7.2a. This problem can be partitioned into the two problems in standard form: min ($1

,ra 1

fi(X1,

x3 Y x5),

and

The interaction graphs of these problems are depicted in Fig. 1.7.2b,c.

EXAMPLE 1.7.3 The following parametric unconstrained problem,

12

1. Nonserial Problems

Figure 1.7.2

fb)

where X = { x , , x 2 , x , , x 4 ,x5, x,} and P = { x 5 , x,}, with the interaction graph shown in Fig. 1.7.3, is in standard form.

Figure 1.7.3

Note that a nonparametric problem is always in quasi-standard form and and is also in standard form when its interaction graph is connected.

1.8 The Constrained Problem

The aim of this section is to give the formulation of the constrained problem: First both the (a) nonparametric and (b) parametric forms of the problem are defined, and subsequently (c) some important applications are considered.

13

1.8 The Constrained Problem

a.

NONPARAMETRIC FORM.Letting

be a set of discrete variables, Xi(( s z j

Sz. being the

definition set of the variable

1 = %,L T = {1,2,..., t } ;

T ' = {1,2, . . . , t ' } ,

t'st;

and

Xi c A ',

ic T ;

the statement of a nonparametric constrained problem is minf(X)

=

1 fi(Xi),

min

X

x

(1.8.1)

id'#

subject to X 1 E Fi,

(1 X.2)

i E T.

The function f ( X ) is called the objective function; the functions Ji(Xi) are the components of the objective function; and the relations (1.8.2) are the constraints of the problem. Both components and constraints are specified, in general, by means of stored tables. Given Y = {yl, y,, . . . , y z >c X ( I 5 n ) and defining

s, = SYlx

Su2x

* * *

x

s,, ,

clearly

Fic Sxi,

i E T.

Note that, for i E T', Fiis not necessarily a proper subset of Sxi. Moreover, the set F of feasible assignments of X is given by

The unconstrained problem is a special case of this, specified by

For the constrained problem the definition of interaction between two variables (see Definition 1.4.1) splits into two distinct definitions.

14

1. Nonserial Problems

DEFINITION 1.8.1 Two variables x E X and y E X are said to interact, in a constrained problem, if there exists a constraint X k E Fk (k E T ) such that both x and y belong to Xk. DEFINITION 1.8.2 Two variables x E X and y E X are said to strongly interact (or interact w.r.t. the components), in a constrained problem, if there exists a component f k ( X k ) (k E T’) such that both x and y belong to X k . Clearly, two variables which strongly interact are also interacting, but not necessarily vice versa. Definitions 1.8.1 and 1.8.2 merge in Definition 1.4.1 when the problem is unconstrained. Similarly Definitions 1.4.2-1.4.4 extend to this case. DEFINITION 1.8.3 The set of variables which interact (strongly interact), in a constrained problem, with a variable x E A’, is denoted by y ( x ) (~’(x)). DEFINITION 1.8.4 Let Y c X . The set of the variables of X - Y which interact (strongly interact), in the constrained problem, with at least one variable of Y, is denoted by y ( Y ) (y‘(Y)) and is given by Y ( Y )=

u

x EY

Y(X)

- Y;

u

(Y’W)= x EY

Y’(X) - y).

DEFINITION 1.8.5 The interaction graph G = ( X , L ) [strong interaction graph G’ = ( X , L’)] oj’a constrained problem is an undirected graph, without self-loops and parallel edges, defined by the following. (a) The vertex set X of the graph is the set of the variables of the problem. (b) Two vertices are adjacent if and only if the corresponding variables interact (strongly interact).

Clearly Y’(X)

= y(x);

Y’(Y) = y ( Y ) ;

L’

= L;

15

1.8 The Constrained Problem

namely, the strong interaction graph is a subgraph of the interaction graph in a constrained problem.

Consider the constrained problem

The interaction graph of the constrained problem is depicted in Fig. 1.8.la, while the strong interaction graph is shown in Fig. 1.8.lb.

fbl

Figure 1.8.1

b. PARAMETRIC FORM.The statement of a parametric constrained problem is the same as that of a nonparametric one, with the exception of formula (1.8. l), which, letting P c X , becomes min

C A(Xi).

x-P ieT'

Definitions 1.8.1-1.8.5 extend to this case.

16

1. Nonserial Problems

c. APPLICATIONS. Obviously a very large number of optimization problems, which arise in many fields of engineering, operations research, and economics, can be formulated as constrained problems. Some of them, because of their special structure, have been conveniently approached by dynamic programming (see, for instance, [ l , 7-13]). Chapter 6 will examine some of these problems in detail.

1.9

Introduction to Nonserial Dynamic Programming and Plan of the Book

Dynamic programming is a decomposition technique, based on Bellman’s principle of optimality, which allows breaking a complex optimization problem into a sequence of easier subproblems. U p to now, dynamic programming has been applied to the solution of serial problems or some nonserial problems of special structure. The purpose of nonserial dynamic programming is to present a systematic framework of optimization techniques for nonserial problems. Some of these techniques are a direct generalization of dynamic programming; others are derived from it in a more sophisticated way. Nonserial dynamic programming not only has the obvious aim of finding the correct solution to the given optimization problem (primary optimization problem), but is also concerned with solving the primary problem efficiently, with regard to computing time and memory requirements of the computer (secondary optimization problem). Clearly, the efficiency of nonserial dynamic programming is strictly related to the “structure” of the specific problem: In general, the less “sparse” the interaction graph, the less efficient the nonserial dynamic programming method. The main part of this book (Chapters 2-5) is devoted to unconstrained nonserial problems. For their solution, three distinct procedures apply, namely, (a) elimination of variables one by one, (b) elimination of variables in blocks, (c) multilevel elimination procedures. The first procedure is a direct generalization of dynamic programming and is the easiest procedure from the point of view of the secondary optimiza-

tion problem. It allows minimizing meaningful indices of the computing time but cannot, in general, tackle efficiently the memory requirements.

1.9

Nonserial Dynamic Programming and Plan of Book

17

For this, procedure (b) is appropriate. However, only procedure (c) can take into account the two facets of the secondary optimization problem in the most efficient way. All procedures apply to nonparametric and parametric problems. Procedure (a) is a special case of procedure (b) which, in turn, is a special case of procedure (c). However, since the complexity of the secondary optimization problem greatly increases when passing from procedure (a) to procedure (c) (and also for didactic reasons), procedures (a), (b), and (c) will be presented in order. In particular, Chapters 2 and 3 deal with the description of procedure (a), its properties and algorithms. Chapter 4 is devoted to procedure (b), while procedure (c) is discussed in Chapter 5. All the primary procedures are extended in Chapter 6 to constrained problems, and some aspects of the relative secondary optimization problems are discussed. The interested reader will find the necessary graph- and set-theoretical definitions in Appendixes A and B. Finally, Appendix C deals with the combinatorial aspects of the Gaussian elimination process upon sparse positive-definite symmetric matrices. It is shown that finding efficient (i.e., requiring a minimal computing time) procedures for the solution, by means of Gaussian elimination, of a system of linear equations, is a graph-theoretical problem, very similar to the secondary optimization problem of nonserial dynamic programming.

Chapter 2

The Elimination of Variables One by One: Description of the Procedure

2.1

Introduction

This chapter is devoted to the solution of the nonserial unconstrained problem by means of a dynamic programming procedure consisting of the elimination of variables one by one. The primary procedure and the related secondary optimization problem are presented in the first part of the chapter. Then a graph-theoretical problem, namely, the problem of finding an optimal elimination ordering of the vertices in a graph, is considered. Finally, it is shown that the solution of the secondary optimization problem leads to the determination of an optimal elimination ordering on a suitable graph. This is done for both the nonparametric and the parametric problem.

2.2

Serial Dynamic Programming

In order to introduce nonserial dynamic programming, and also from the point of view of the nomenclature adopted, it is convenient to summarize 18

2.2

19

Serial Dynamic Programming

briefly the well-known dynamic programming procedure for the solution of the serial unconstrained problem : minf(x) = min X

X

C fi(Xi),

~ E T

is a set of discrete variables, oZ,being the number of assignments of variable xj;

T = {I, 2,. . . , n - 1 >;

and

Xi= { x i , xi+l>.

First, the variable x1 is considered. Since only the component &(XI) = &(xl, x,) is affected by x l , for minimizing f ( X ) w.r.t. x l , it is sufficient to compute minh(x1, x2) = hl(X2) (21 }

for all the assignments of x2, since x1 interacts only with x 2 , and to store the optimizing assignment xl*(xz).The stored table f l ( x l , x 2 ) is then replaced by the table h,(x,) which, therefore, behaves like a new component. The operation performed, which is called the elimination of the variable xl, thus leaves the following problem:

Next x2 is eliminated in this new problem. Since only x3 interacts with x 2 , an optimal value of x2 is computed for all the assignments of x 3 , and, as a result, the functions h,(x3) and x2*(xs)are stored. The procedure continues, eliminating successively the variables xs , x 4 , .. . , x,, thus finding an optimal value of f ( X ) . An optimal assignment for X is obtained by operating “backward,” namely, by determining SUCcessively x,*, x*,-l,. . . ,xl* from the stored tables x,*, x*,-l(xn),. . . ,x1*(x2). It is worth noting that the number of functional evaluations required for the solution of the serial problem via dynamic programming is

while, clearly, by employing exhaustive enumeration, the required number of functional evaluations is L’& az,.

20 2.3

2. Elimination of Variables O n e by O n e : Procedure

Nonserial Dynamic Programming: The Description of the Solution of the Primary Problem

Consider the nonserial unconstrained problem minf(X) X

=

min

x

C fi(Xi), a

d

1

is a set of discrete variables, crzj being the number of assignments of the variable xi; T = {1,2,. . . , t > ; and X i c X. Consider one ordering, out of the n! possible ones, of the variables of the set X . Let it be o = (y, ,y z , . . . ,y,). For this ordering the primary problem may be solved by dynamic programming, the selection of one ordering corresponding to a particular serialization of the nonserial problem. The procedure is divided into two parts.

a. FORWARD PART.The variable y , is considered first. Then

where

T, = { i : X i n {y,} f

0},

Hence the first step of the optimization procedure consists of computing, employing exhaustive enumeration,

and storing the optimizing assignment of y , as a function of y(y,), namely, YI*(Y(Yl)). The minimization off(Xj w.r.t. y , , for all possible assignments of y(y,), is called the elimination of the variable y , . The problem remaining after the elimination of y,,

2.4

21

An Example

is of the same form as the originalproblem, and the function h,(y(y,)) may be regarded as a component of the new objective function. Hence, the same procedure applies for the elimination of the variables y,, . . . , y , in this order. At each step j the new component hj and the optimizing assignment yj* are stored as functions of y b j I y1,. . . ,yjpl), namely, the set of variables interacting with yj in the current problem (that is, the problem obtained from the original problem by the elimination of y l , y z ,. . . , yj-l). Since the set y(y, I y l , . . . ,v,-~)is clearly empty, the elimination of y , yields the optimal value of ,f(X).

b. BACKWARDPART.This part consists in the successive determination of y,*, Y * , - ~., . . ,y l * , i.e., the optimizing assignment for X , from the stored tables Yl*(Y( Y l ) h YZ*(Y( Y2 I Yl))? . . * 7 Y,*. The forward and backward parts of the procedure can be combined into a more general and flexible procedure, consisting of eliminating the variables in the same order, but allowing that, at any step j , the optimizing assignments y,*, yz*, . . . , yj* be expressed as functions of the remaining variables . . . , y,. An Example

2.4

A procedure for the solution of the nonserial problem has been given in Section 2.3; an example of it will now be discussed in detail. This example also has the objective of introducing, in an easy way, the secondary optimization problem. Consider the nonserial problem

where x = {XI,x , , x 3 , xq, x5}, x1 = {x,,x,, x5}, x2 = {x,,x,}, x3 = {x,,x4,x5}, or,= r~ = 2, and the fi’s are given in the following tables. x1

0 0 0 0 1 1 1 1

x3

0 0 1 1 0 0 1 1

s

0 1 0 1 0 1 0 1

I

f3

fi .-

1 3 5 8 2 6 2 4

0 0 1 1

0 1 0 1

4 8 0 5

0 0 0 0

0 0 1 1

0 1 0 1

0 5 6

3 5 1 4 3 -

22

2. Elimination of Variables One by One: Procedure

It is now shown that an optimal assignment for X , out of all the 25 = 32 possible assignments, is found by eliminating xl,x2,x3,x4,x, in that order. When eliminating xl, only fl f, is considered, since x1 does not affect f,. Next, a table with x2,x3,and x5 as variables is constructed, since they are interacting with x1 throughf, and f, For every assignment of (x, ,x3,x,} a best value for x1 is chosen, called xl*,so that fl f 2 is minimal. This value of fi f,, as a function of x2,x3, and x,, is called hl(x,, x,, x,).

+

+

+

I

X2

Xa

X5

0 0 0 0 1

0 0

0

1

0

2

1 1 1

1

1

4

1

0 0 1

0 1 0

1

1

7 11 7 9

1 1 1

-h1 2 6

1

xl*

1

0 1 1

Next x2 is eliminated in the optimization problem min h,(x,

Y

x3 x,) 7

X-(qI

+

fdX2

9

x4 x5). 7

Let the new function of x3,x4,and x, be called h2 and the optimizing value of x, be called xz*.

~

h,

X2*

2 11

0 0

~

0 0

0 0

0

0

1 1

0

8

0

1

0

1

0 0

1 1

1 1

0

9 2 9 8

1

7

0 0 0 0 0

0 1

1

1

-

Next, x3 is eliminated in the optimization problem min

X - ( Z , ,z*1

hZ(x3, x49 x5)*

2.4

23

An Example

The new function of x4 and x5 is called h,, and the optimizing value of x, is called x3*. x4

x5

0

0 1 0 1

0 1 1

ha

x1*

2 9 8

0 1

-0 1

7

Next x, is eliminated in the optimization problem min h3(x4,x5). x-{z, ,z*,%I The new function of x5 is called h, and the optimizing value of x, is called x,*. x5

h4

x4*

0

2

0

1

7

1

---

Finally x, is eliminated; namely, h, is optimized w.r.t: x,. Thus min f = min h, and x5* = 0, x,* signment.

= 0 , x3* = 0, x2* =

Figure 2.4.1

=

0, xl*

2, =

1 is one optimizing as-

24

2. Elimination of Variables One by One: Procedure

The interaction graph of the original problem is shown in Fig. 2.4.1. The interaction graphs of the problems resulting from the successive elimination of the variables xl, x 2 , x3, x 4 , and x5 are shown in Fig. 2.4.2. The number of functional evaluations for this elimination ordering is 24

+ 24 + 23 + 22 + 2 = 46,

and the number of table look-ups is 2.24+2.24+

1.23+ 1.22+ 1.2=78.

The terms “functional evaluations” and “table look-ups” are self-explanatory; however, see the next section for a precise definition. If the elimination ordering is (x, , x, , xl, x 2 , xg), as the reader can verify for himself, the number of functional evaluations is 23

+ 23 + 23 + 22 + 2 = 30,

2.4

An Example

25

and the number of table look-ups is 1.23+1.23+2.23+2.22+1.2=42.

The interaction graphs of the problems resulting from the eliminatiop of the variables xg, x4, xl, x2, and x5, in this order, are shown in Fig. 2.4.3.

Figure 2.4.3

Thus the ordering (x,, x4, xl, x2, xs) does better than the ordering (xl, x, , x3, x,, x5), from the point of view of both the functional evaluations and the table look-ups required. It is also interesting to note that, for both orderings, the optimization procedure by successive eliminations is more convenient, at least w.r.t. the number of table look-ups, than exhaustive enumeration, which requires 25 = 32 functional evaluations and 3 25 = 96 table look-ups. It would seem that the advantages of the dynamic programming optimization procedure are modest indeed, but this is due to the fact that cr and n

.

26

2. Elimination of Variables O n e by O n e : Procedure

are small numbers. Consider, in fact, the same nonserial problem given above (hence the same n ) in which each variable can assume (T = 10 values. The number of functional evaluations and table look-ups for the previously considered procedures can be found in the following table. Procedures Exhaustive enumeration

Number of functional evaluations

105 = 100,ooo

+

+

+

+

Elimination 104 104 103 ordering = 21,110 (XI x* x3, x4, x5) 9

Number of table look-ups

3

+

. lo5 = 300,000

102

+ 10

z . 104 + 2 . 104 + I . 103 + 1 . 10' + 1 . 10 = 41,110

102

+ 10

1

1

Elimination 103 103 103 + ordering = 3110 (x3, XP, XI x * , x5)

. 103 + 1 . 103 + 2 . 103

+2

*

10'

+1

*

10

=

4210

1

The secondary optimization problem is to find an elimination ordering which is the best from the computing point of view. As already pointed out, this

implies the consideration of two different factors: computing time and memory requirements. The close connection between the computing time and the number of functional evaluations (and the number of table look-ups) needed is apparent. Moreover, the example shows that, at least when a and n are sufficiently large, the maximum number of interacting variables is also a reasonable index of the computing time [in fact this number is equal to 3 for the ordering (xl, x,, x 3 , x 4 , x5), while it is equal to 2 for the more efficient ordering (xI , x4, x1,x, ,x 5 ) ] . On the other hand, the example also shows that the optimization procedure that requires the least number of functional evaluations does not, in general, belong to the subclass of procedures in which the variables are eliminated one at a time. For instance, when the problem consists of a single table (as h, in the example), it is not efficient to go on eliminating variables one by one; instead, it is better to select directly a row which minimizes the function, i.e., to eliminate all the remaining variables simultaneously (see Chapter 4). As for the memory requirements, each elimination implies the storage of a table with the number of rows equal to the number of assignments of the interacting variables. Hence, for an optimization procedure in which variables are eliminated one by one, and when a,, = (T = const, an ordering

2.5

27

Secondary Optimization Problem

for which the maximum number of interacting variables is minimal, is such that the maximum “dimension” of the additional stored tables is minimal. However, as noted in Section 1.9, the optimization procedures consisting of eliminating variables one by one are not sufficient for fixing a bound, a priori, to the maximum dimension of the additional stored tables (see Chapter 4). 2.5

The Secondary Optimization Problem

The secondary optimization problem was introduced by means of an example in the previous section. Our present aim is to define this problem formally for the optimization procedure consisting of the elimination of variables one at a time, which is described in Section 2.3. The case cr%, = cr = const is now considered. This assumption will later be removed. Consider an elimination ordering w = ( y , ,yz,. . . ,y,) and examine, from the computational point of view, the elimination of the variable y,, i.e., the solution of the optimization problem

This solution implies two operations. (a) The function k,(y,, y(yl)) struction requires

I T,I

= C i e T , f i ( X i )is

constructed. This con-

table look-ups

and

I T, I

-1

additions

for each of the

1 S{VlIUY(V1)

I=

oIY(”l)l+l

points in which k,(y,, y(yl)) is defined. (b) The function k,(y, ,~ ( y , ) is ) minimized w.r.t. y , in connection with all I Slr(uJ I = crly(yl)l assignments of the variables of the set y(’yl), namely, min kl(Y17

rD1)) = hl(YD,)),

611

and the optimizing assignment yl*(y(yl)) is stored.

28

2. Elimination of Variables O n e by O n e : Procedure

The optimization procedure implies ol~(gJ(o- 1 )

comparisons.

From the point of view of the memory requirements, it is not convenient to perform operations (a) and (b) in this order, but it is better to adopt a more flexible procedure, which consists in storing only the current optimal value of y 1 as a function of y(yl). In this way a table with only oIy(yl)Iplaces must be stored. The same considerations apply for the elimination of the other variables. In order to specify the formulation of the secondary optimization problem, the following definitions are needed. Let o = ( y , , y z ,. . . , y,) be an elimination ordering and S, be the set of all possible n! orderings.

DEFINITION 2.5.1 The vector formed by the n nonnegative integers I y ( y j I y l , y,, . . . , y+*) 1, ordered from the largest to the smallest, is denoted by

40) = (dl(O),d & J ) ,. . .

Y

dll(w))

and called the vector dimension of the ordering w .

DEFINITION 2.5.2 The scalar

is called the number of functional evaluations of the ordering o.

DEFINITION 2.5.3 The degree of B(w), which is a polynomial in and called the cost of the ordering w . Note that C(o) = d l ( o )

0,

is denoted by C ( w )

+ 1.

DEFINITION 2.5.4 The scalar B(,9)

=

min B(w) weS,

is called the number of functional evaluations of the problem 9.

2.5

29

Secondary Optimization Problem

DEFINITION 2.5.5 The integer C(.P) = min C ( w ) w €SO

is called the cost of the problem 9 ' . Generally speaking, the secondary optimization problem consists of determining an ordering which minimizes the computing time. However, since a general formulation of this problem is exceedingly complex, it is reasonable to minimize some indices of the computing time. In particular, two versions of the secondary optimization problem are considered : (a) Find an ordering w such that B(o)= B ( . 9 ) . (b) Find an ordering cc) such that C(w) = C ( . P ) .

The index B ( . a ) , which takes into account only the interaction among variables and ignores the number of components of the current problem, is, in most cases, a very good index of the computing time. Moreover, when c is sufficiently large, since the maximum exponential factor is the most decisive in the expression of the number of functional evaluations, finding a minimal cost ordering is sufficient. As for the memory requirements the following two definitions are of interest.

DEFINITION 2.5.6 The integer dl(W) = max I YOtj I y , , y z , . . i

f

9

yj-1)

I

is denoted by D ( w ) and called the dimension of the ordering w .

DEFINITION 2.5.7 The integer D(.P)

=

min D ( w ) Lo €SW

is called the dimension of the problem .P In fact, the most meaningful parameter should be the maximal instantaneous storage occupation ;however it is believed that the integer D(w),which

2. Elimination of Variables One by One: Procedure

30

is equal to the maximum number of variables of the additional tables stored for the ordering w , is a good index of it. Finally, since C ( w ) = D ( w ) 1 and consequently C(,p) = D ( . p ) + 1, it is clear that a minimal cost ordering is also a minimal dimension ordering. The assumption aq = u = const is now removed. In this case Definition 2.5.2 is replaced by the following.

+

DEFINITION 2.5.8

and the secondary optimization problem is to find an ordering w such that B(w) = B ( ~ 9 ) . In the special case oZ, = 0a.l

(cxj = a positive integer),

(2.5.1)

B ( w) is a polynomial in u and Definition 2.5.3 is meaningful. Hence also version (b) of the secondary optimization problem is defined. If condition (2.5.1) is not met, it is convenient to select a value of u (which is not, in general, an integer) such that

az, N

@j

(ai= a positive integer);

to solve the secondary optimization problem as if

ax,= a o j and to adopt the resulting ordering for the original problem. Finally, it is important to note that the word “memory” is, in this book, synonymous with “fast memory.” If both fast and slow memory were used, the indices of the computing time would have to be appropriately modified.

2.6

The Elimination Process

Consider an undirected graph G = ( V , E), without self-loops and parallel edges, and the set S,, of all orderings l2 of the vertices of V. The aim of this section is to define the elimination process associated with an ordering l2 [14-171.

2.6

31

The Elimination Process

DEFINITION 2.6.1 Let G

=

( V, E ) be a graph and y

E

V. The graph obtained from G by

(a) deleting y and all the edges emanating from it and (b) connecting all the previously nonadjacent vertices in T(y); is called the ynamely, the graph ( V - (y), ( E - O(y)) u Z*(T(y))), elimination graph of G and is denoted by G., This operation is called the elimination of the vertex y. Given an ordering Q = (yl,y, ,. . . ,yN), the elimination process consists in the successive elimination, in the current graph, of the vertices y l , y,, . . . ,y N . This process forms a sequence of graphs Go = G , G',.

. . , Gj-l, G j , . . . , G N

so that Gj is the yj-elimination graph of Gj-l and GN is the null graph.

Theorem 2.6.1 (Invariance Theorem) Any graph obtained from G = ( V , E ) by eliminating all the vertices of the same subset Y c V, one by one, is independent of the order in which the vertices are eliminated.

PROOF. Let G' be the graph obtained from G by eliminating the vertices of the set Y in the order y, ,y, ,. . . ,yl. To prove the theorem, it is sufficient to show that two vertices of V - Y , a and b, which are not adjacent in G , are adjacent in G' iff there exists a path connecting a and b in G in which all the vertices, other than the terminals, belong to Y. Let the sequence of graphs resulting from the eliminations be G

=

GO,G1, G 2 , .. . , G1 = G'.

Assume that there exists a path connecting a and b through Y : a, yl', Y,', . . . ,Y;,

b.

This is a path in GO. Let G i be the last graph in which this path is preserved. In the transformation from Gi to Gi+l, one vertex y' is eliminated and an edge is put between the adjacent vertices on the path, on both sides of y ' . Thus, in Gi++'there is still a path from a to b. This property is preserved after each elimination. Therefore a and b are adjacent in Gz.

2. Elimination of Variables One by One: Procedure

32

Now assume that a and b are adjacent in GI but not in GO. Let Gi be the first graph in which a is adjacent to b. Thus the vertex y ” , eliminated in the transformation from Gi-l to Gi, is adjacent to both a and b. By repeated argument on the edges (a, y”) and (y”, b), it follows that there exists a path connecting a and b through Y. rn Theorem 2.6.1 allows us to state the following definition. DEFINITION 2.6.2

Let G = (V,E ) be a graph and Y c V. The graph obtained from G eliminating all the vertices of Y in some order is called the Y-elimination graph of G and denoted by G y ; this operation is called the elimination of the subset Y. The graph GY can be constructed directly from G as follows [18]. Theorem 2.6.2 Let G

=

(V, E ) and Y c V. The graph GY

=

(V’,E ‘ ) is given by

Y,= V - Y, E‘ = (E - (I( Y ) u O(Y ) ) )u Z*(T(Zl))u - * * u Z*(T(Zi)), where Z , ,Z2,. . . , Z l are the vertex sets of the connected components of the section graph of G w.r.t. Y.

PROOF. The proof follows directly from that of Theorem 2.6.1.

m

DEFINITION 2.6.3

The set of vertices adjacent to yj in the {y,, y 2 , . . . , yj-,}-elimination graph of G is denoted by

I

The set r(yj ( y , , . . . ,y j P l } )can be singled out directly in the original graph G [18]. Theorem 2.6.3 Let G = ( V , E ) and 9

=

( y l , y 2 , .. . , y N ) . Then

T(Yj I {Yl?.

..

3

Yj-1

=

w?,

2.6 The Elimination Process

33

where Yj is the vertex set of the connected component of the section graph of G w.r.t. {yl,, . . ,yj}, which contains y j .

PROOF.The proof follows directly from that of Theorem 2.6.1. M EXAMPLE2.6.1 Consider the graph of Fig. 2.6.la. The 0,-elimination graph of G is depicted in Fig. 2.6.lb and the 0,-elimination graph of G,,, namely,

Figure 2.6.1

34

2. Elimination of Variables O n e by O n e : Procedure

G{vl,Z)gJ, is given in Fig. 2.6.1~.Then, consider the elimination from G of u, and u1 in this order. The u,-elimination graph of G is shown in Fig. 2.6.ld, and the reader can check that the u,-elimination graph of Gv, equals the u,-elimination graph of G,, , according to the invariance theorem.

Note that G{vl,oa)can be constructed directly from G by fully connecting the set r ( { u l , u,}) = {us, u4, u,, u,} by Theorem 2.6.2. Furthermore, by Theorem 2.6.3, r(u,I {ul}) and r ( v 1 {v,}) can be singled out directly in G : Since the section graph of G w.r.t. { v l , u,} consists of a single component, we obtain

I

In some cases it may be helpful, given the complement G = ( V , B) of G = (V, E ) , to determine directly the complement of G,. This can be accomplished as follows. (a) The vertex set of cg is V - ( y } . (b) The edges ( u j , uk) are those, among the edges of G, for which at least one, between uj and v k , is adjacent to y in also the edges emanating from y are canceled.

c;

la)

@ @----a

Figure 2.6.2

35

2.7 Criterion Functions

EXAMPLE 2.6.2 The complement G of a graph G is given in Fig. 2.6.2a and the complement of the u,-elimination graph of G is given in Fig. 2.6.2b.

2.7

Criterion Functions

There are all together N ! elimination orderings of the vertices of a graph G = (V, E ) with [ V I = N . The problem of establishing a ranking among them is now discussed [14, 15, 17, 191. Let Q = (yl,y 2 , .. . ,ya) be an elimination ordering of the vertices of the set V .

DEFINITION 2.7.1 The vector formed by the N nonnegative integers I r(yjI {y, ,. . . ,yj-,}) I (that is, the degrees of the eliminated vertices at the time of their elimination), ordered from the largest to the smallest, is denoted by d(Q) = (dl(Q), d2(Q),. . . , d N ( Q ) )and called the vector dimension of the ordering Q. Let

Q, E S,.

Q17

DEFINITION 2.7.2 The orderings 52, and Q, are equivalent if d(Q1) = d(Q2).

DEFINITION 2.7.3 The ordering Q1 dominates the ordering 52, if

44) < d(Q2). DEFINITION 2.7.4 An ordering Q, which dominates or is equivalent to all the others, is a dominating ordering.

DEFINITION 2.7.5 An ordering Q, which is dominated by no ordering, is a noninferior ordering.

36

2. Elimination of Variables One by One: Procedure

EXAMPLE 2.7.1

Consider the graph of Fig. 2.7.1 and the orderings SZ, Qz = 03 vz ui), and Q 3 = (uq uz 3 vi ~ 3 ) . We have (049

9

3

= (u,

, vz , v 3 , v4),

9

Thus Q, and 9,are equivalent and both dominate Q3. Moreover it is easy to check that 52, and Q2 are dominating orderings.

...9 Figure 2.7.1

Figure 2.7.2

The existence of a dominating ordering has been demonstrated [17, 191 for the class of triangulated or chordal graphs. However there are also nontriangulated graphs which have the same property (see, for example, Fig. 2.7.2). Nevertheless, there do exist graphs that possess no ordering dominating all the others. EXAMPLE 2.7.2

Consider the graph of Fig. 2.7.3. The vector dimension of the orderings Q1 = ( ~ 1 ,~ 2 v, 3 , u4, u 5 , u6 , 21), and Q, = ( u 2 , u 3 , u 4 , ul , u 5 , us, v7) are, re-

Figure 2.7.3

2.7

Criterion Functions

37

spectively, 4 Q l ) = (594, 3, 3 , 2 , 1, 0 ) ;

d(Q,)

=

(4, 4,4, 3, 2, 1,O).

It is easy to verify that the two orderings Q, and Q, are both noninferior and that, consequently, no dominating ordering exists. For the graphs possessing no dominating ordering, a ranking may be obtained by an appropriate (single-valued) criterion function (to be minimized)f(d). In order for the ranking generated by the criterion function to be consistent with Definitions 2.7.2-2.7.5, it is required that the function f belong to a particular class defined as follows.

A criterion function f belongs to the class F if

and An example example of of aa criterion F is An criterion function function of of the the class class = =F is (2.7.1) with with

f,(k

+ 1) > A ( k ) ,

k = 0, 1,2,. . . ; j = 1,2,. . . , N .

Some special functions of the class .Fsatisfying (2.7.1), which are widely used in Chapters 2 and 3 and in Appendix C , are now defined. DEFINITION 2.7.7 The function f(d(Q)) such that fl(dl(Qn)) = dl(Q), fj(dj(Q)) = 0,

j

=

2,3,. . . , N ,

that is, the largest degree of the eliminated vertices for an ordering Q, is called the (scalar) dimension of the ordering and denoted by D(l2).

38

2. Elimination of Variables One by One: Procedure

DEFINITION 2.7.8

The integer max D(Q) QESQ

is denoted by D ( G ) and called the dimension of the graph G. DEFINITION 2.7.9

The function f(d(Q)) such that

fl(dl(Q>) = dl(Q) f,(dj(Q>)= 0,

+ 1, j = 2 , 3 ,..., N,

is called the cost of the ordering Q and denoted by C(Q). Clearly C(Q)

= D(Q)

+ 1.

DEFINITION 2.7.10

The integer max C(Q) RESQ

is denoted by C(G) and called the cost of the graph G. DEFINITION 2.7.1 1

The function f ( d ( Q ) ) , such that

fi(dj(Q)) = ~ where

IS

~ l ( ~ ) +j = ~ , 1,

2,.

.. ,N,

is a positive real number, is denoted by B(Q).

DEFINITION 2.7.12

The real number max B(Q) Q€SQ

is denoted by B(G).

:

2.8 Final Theorem and Other Dominance Relations

39

DEFINITION 2.7.13 The function f(d(Q)), such that

+ b d j ( 9 )+ c,

fj(dj(9))= a d j z ( 0 )

j

=

1, 2,.

. . ,N,

where a, b, and c are nonnegative real numbers, is denoted by A ( 0 ) . DEFINITION 2.7.14 The real number max A ( 0 ) RESQ

is denoted by A(G). 2.8

The Final Theorem and Other Dorn..iance Relations among Elimination Orderings

This section presents some initial results concerning dominance relations among elimination orderings. They will be first employed in Sections 2.9 and 2.10 for demonstrating that the secondary optimization problem can be solved as a graph elimination problem; their computational relevance will be treated in detail in Chapter 3.

Theorem 2.8.1 Let G = ( V , E ) be a graph and { Y, Z , Z} be a partition of V such that C is a separating set for G, and Y and Z are not adjacent. Then, letting 9' and 0"be two orderings such that (a) the vertices of Z are ordered last, and ( b ) the vertices of both Y and Z have the same relative position, we have d ( 0 ' ) = d(0"). PROOF.The theorem holds, since the degree of a vertex is not affected by the elimination of vertices belonging to a nonadjacent set. W

EXAMPLE2.8.1 Consider the graph of Fig. 2.8.1, and let Z: = { u 4 , u,}, Y and Z = {u3}. The two orderings

9' = (vl,

v2, u s , u4, us)

have the same vector dimension.

and

= { u l , u,},

0'' = (ul, u 3 , v 2 , uq, u5)

40

2. Elimination of Variables O n e by O n e : Procedure

Figure 2.8.1

Theorem 2.8.2 Let G = ( V , E ) be a graph, 2 be a subset of V fully connected in G, and Q be an elimination ordering. There exists at least one ordering Q', in which the vertices of 2 are ordered last, such that d(s2') 5 d(s2).

PROOF.If 2 = V, the theorem holds trivially. Otherwise the following two subpropositions must be proved. (a) It is possible to find at least one elimination ordering SZ", beginning with a vertex of A = V - Z,such that d(Q") 5 d(Q). (b) It is possible to find at least one elimination ordering Q', in which the vertices of Z are ordered last, such that d(Q') 5 d(Q). Proof of (a). Let Q = (yl,y 2 , .. . ,y N ) . Let ,F1) = 2 and A ( 1 )= A . If y1 E A(1),this proposition is satisfied. Otherwise (yl E P)) define 2'2'

= T(yl);

A(2)=

(V - {yl))- T(yl).

is fully connected, E2) 3 2")- {yl} and consequently A ( 2 ) A"). Also F2) is fully connected in Gyl. If, in GV1,y z E 27') and A'') f 0, the subsets F3)and A(3)are similarly defined. Since c

This way of proceeding continues, defining at each step j S j )

= m-11{Yl 9 Yz > ' . . , Yj-z}),

AG)= (Y- ( ~ 1 ,Y Z

3.

. ., ~ j - i > >- r C V j1I

( ~ 1 ,~2

9 .

.

. 7

~j-zl),

2.8

41

Final Theorem and O t h e r Dominance Relations

until either (1) for an integer h, yh E A(a), or (2) for an integer k, A(k)= 0.

Case (2) is now examined. Since = 0, the vertex yk--l is adjacent in %I1, Y2,...,Y&2} t o all other vertices. Hence, letting ym be a vertex of A(k-l)(which is, by definition, nonempty), it is clear that the ordering

dominates or is equivalent to Q. Thus, given the ordering Q, there exists another ordering, Q‘” = (yl’, y 2 ’ ,. . . ,yAv’),coincident with Q in case (l), such that d(Q”/) 5 d(Q), and, for a certain 1[1= h in case (1) and 1 = k - 1 in case (2)],

The vertex yl’ is not adjacent in G to the vertices yl‘, y2’,. . . , yl-, . Hence (see Theorem 2.8.1) the two orderings Q’” = (Yl’, Y2’, . ’

,

‘

>

I

Yl-1, Yi ,YZ,,

7 ’

.. Y YN’)

and Q“‘ = ( Y l ! , Yl’, Y 2 / ,. . . 2 Yl-1,

YI+19 ’

.

Y

YN‘)

are equivalent. This demonstrates proposition (a).

Proof of (b). An elimination ordering 9‘ can be found by repeated use ( N - I 2 I times) of the construction described in (a), taking into account the fact that, letting u E V - Z, if Z is fully connected in G, it is also fully connected in G,. rn The proof of Theorem 2.8.2 is constructive in nature: Figure 2.8.2 is of help in understanding the construction. Two examples, which illustrate it, are now given.

42

2. Elimination of Variables One by One: Procedure

EXAMPLE 2.8.2 Consider the graph G = (V, E) of Fig. 2.8.3, where V = {ul, u 2 , u 3 , v4, u 5 , v,}, the fully connected subset of vertices 2 = { u 4 , v5, u,}, and the ordering i2 = (u, , vl, u 2 , v5, v4, u3). We have d(Q) = (3,3,2,2, I, 0).

An ordering Q", beginning with a vertex of A = V - 2 = {ul, u z , us), which dominates or is equivalent to 0, is now determined using the construction of Theorem 2.8.2. Define Z(1)= z = {u4, u 5 , u,}; A(1)= n = {ul, u2, u3}.

2.8

Final Theorem and O t h e r Dominance Relations

43

Figure 2.8.3

Since u1 E A ( z )[case (I)], we have

In order to obtain an ordering f2'ending with the vertices of Zand dominating or equivalent to 52, the construction must begin with the graph Go, and the ordering ( u 6 , u 2 , u s , u 4 , v,). EXAMPLE 2.8.3

Reconsider the Fig. 2.8.3, Z = {u4, u 5 , us}, and the new ordering SZ, u 6 , u 2 , u 3 , v5). We have

=

(u4, vl,

44)= (493, 3 , 2 , I,()). An ordering Q;l,beginning with a vertex of A = V - 2 = ( u l , v 2 , u,}, which dominates or is equivalent to Lll, is now determined using the construction of Theorem 2.8.2. Define 2"1'

=

z = {u4, u s , v g ) ;

A(1)= A

= { u l , 02,us}.

2. Elimination of Variables One by One: Procedure

44

Since u4 E Z(l), define L”2’

= T(u,) = {u,

, u2, u 5 , u,};

A(Z)= (u3}. Since in G,,, u1 E .Fa) and A(2)# 0, define

Since, in

Then [case (2)] we have

and

Theorem 2.8.3 (Final Theorem) Let G = ( V , E ) be a graph, 2 a subset of V fully connected in G, and f a criterion function of the c l a s s F . Then there exists an optimal ordering in which the vertices of Z are ordered last.

PROOF.The proof derives directly from that of Theorem 2.8.2 when Q minimizes f. m The final theorem plays an important role in the optimization problem related to the elimination process. This fact will be apparent later. Another (weaker) result of a dominance relation between two elimination orderings is now given.

Theorem 2.8.4 Let G = (V, E ) be a graph; y’, y” E V , and I T ( y ’ ) I 5 1 F b ” ) I. Then, , . . YN-2) letting 52’ = (y’, y”, y l , y z ,. . . , Y N - ~ ) and Q” = (y”, Y’, ~ 1 YZ,.

2.8

Final Theorem and O t h e r Dominance Relations

45

be two elimination orderings, we have

PROOF. I f y ” and y’ are not adjacent in G, this theorem is a special case of Theorem 2.8.1. Otherwise, since

and, by the invariance theorem, the graph resulting from the elimination of y‘ and y” does not depend upon the elimination ordering, the proof

follows. I

EXAMPLE 2.8.4 Consider the graph of Fig. 2.8.4, and let

9’ = (ul, u z , v 3 , u, , u 5 , us>

and

9” = ( v z , u l , u 3 , v, , v 5 , v d .

Since = 3

we have d(Q‘) = (3,3,3,2, 1,O);

d(S2”) = (4,3, 3,2, 1 , O ) ;

and thus d ( 9 ’ ) < d(S”).

EXAMPLE 2.8.5 Theorem 2.8.4 substantially states that whenever it has been decided which two vertices must have the first two places in an elimination ordering, it is

2. Elimination of Variables One by One: Procedure

46

worthwhile beginning with the vertex of lower degree. This property does not extend, in general, to sets of three vertices. The graph of Fig. 2.7.3 is a counterexample. Consider the set {ul, u 2 , us). The vertex vI is the minimum degree vertex in the set. However, as the reader can check for himself, letting =

gr’=

(Dl9 V2

9

u3

7

u4

9

V5 9 v6

9

217);

O19 2)3, u 4 , u5, v6, u 7 ) ;

(%?

we have

d(SZ’) = (5, 4, 3, 3, 2, 1, 0); d(Q”)

=

(4,4, 4, 3, 2, 1, 0).

2.9 The Correspondence Theorem

This section shows that the solution of the secondary optimization problem, introduced in Section 2.5, can be obtained by the solution of an optimization problem for an elimination process in a suitable graph. The two cases, (a) (b)

a%,= u = const, crz, = 0”’ (aj = a positive integer),

are now treated in this order.

Theorem 2.9.1 (Correspondence Theorem) Consider the nonserial unconstrained problem minf(X) = min %/*(Xi) X x i€

with uz, = a = const.

Consider the graph G = (V, E ) with V= X

and

E = L,

namely, the interaction graph ( X , L ) of the problem 9. Then (a)

N p ) = B(G);

(b) C ( 9 ) = C(G).

2.9

47

The Correspondence Theorem

PROOF.Since, given 0 = 0 = (yl, y z , . . . , ynT),it is clear that y ( y l ) = I'(yl) and GyI equals the interaction graph of the problem obtained from 9by the elimination of yl, we have d ( 0 ) = d(0).

The proof follows taking into account Definitions 2.5.2-2.5.5 and 2.7.92.7.12. w Thus, given an optimal elimination ordering for the graph, the same ordering solves the secondary optimization problem.

EXAMPLE 2.9.1 Consider the nonserial unconstrained problem min[fl(xl X

7

x2>

+fax2

9

ff 2 ( x l

x5

7

9

x3 >

rsz,

=

x4) ff3(x3 x6) %

+f4(x4

y

x6 x?) Y

Xdl

with

rs = const.

Its interaction graph G = ( X , L ) is shown in Fig. 2.9.la. Consider the elimination ordering (x3, x, ,x, ,x 2 , x, , xl, x6). We find that Y(X3)

=

&3)

= {Xl

Y

x4

7

XI3

1.

Furthermore, the x3-elimination graph of G, given in Fig. 2.9.1b, equals the interaction graph of the problem obtained from the original problem after the elimination of x3. The same reasoning applies to the further

Figure 2.9.1

2. Elimination of Variables O n e by O n e : Procedure

48

eliminations. The reader can check that this ordering is a dominating one: d((x,, x 5 , x , , x 2 , x,, x l , x,)) = (3, 2, 2, 2,2, 1, 0), and that, consequently, B = g4

+ 4u3 + + gz

C = 4.

0;

Theorem 2.9.2 (Correspondence Theorem) Consider the nonserial unconstrained problem 9:

C fi(Xi)

minf(X> = min X

icT

X

with czj = ~ b j

(aj = a positive integer).

Consider the graph G = (V, E ) , with

v=

{x11, X I 2 , . . . ,$1,

x21,. . . , x52,. . . ,X N 1 ,

I

. . ,gi},w E

E = ((v,

W) u E

(Xi, Xi) E

{ x t , x:,.

L)

u I*({X11,X I * , . . . ,

X,Z,.

. . , x?},

{ x j l , x j 2 , .. , ,xyf}, i f j and

$I})

LJ* *

*

u Z*({XN1,xn*,.. . ,G n } ) ,

namely, the graph obtained from the interaction graph ( X , L ) of problem *!F’, substituting for each vertex xi a fully connected subset of vertices {xi1, x:,. . . ,x y } and fully connecting the subsets {xi1,x:,. . . , xzd} and { x j l , xi2,. . . ,q.}, i f xi and xi are adjacent. Then C ( 9 ) = C(G).

PROOF.Consider the set S, c S, of all elimination orderings such that, among the vertices of {x:, x?, . . . ,x p } , x t is eliminated first, and all the others, if any, follow immediately in the natural order. Let Q=

belong to ment Q E

(Yll, Y 1 2 , . . . ,p

1 ,

yzl, y2*,.

’

,y p z , . . . , Y1*l,Y,*, * . . Y 2 n ) 2

3,. There exists a one-to-one correspondence between an ele3, and an element ~=(Y1,Y2,...7Yn)

belonging to S,. It is easy to verify that

C(0)= C(Q)

49

2.9 The Correspondence Theorem

and that C ( 9 ) = C(G)

since by Theorems 2.8.1, 2.8.2, and 2.8.4, there exists an optimal ordering belonging to &. The proof of Theorem 2.9.2 is constructive in nature and allows, given the optimal elimination ordering for the graph, the determination of an ordering w which solves the secondary optimization problem. EXAMPLE 2.9.2

Consider the nonserial unconstrained problem

min[fi(xl? x2) X

+

f2@2

9

x3)

+

f3(x3 9

XJ

+fdxe

9

Xl)

+

fXX1

Y

%)I

where

x = {XI,

x2,

x3,x4};

OXl= 3 2 ;

OX%= UZs= ozp=

is considered (see Fig. 2.9.2b). The reader can verify that

n=

(X'1,

x41,XQ', xll, x12)

Figure 2.9.2

3.

2. Elimination of Variables One by One: Procedure

50

is an optimal ordering belonging to

3, and that

C ( Q ) = c = 4.

Then an ordering which solves the secondary optimization problem is cc)

= ( x 2 , x4,x 3 , x,),

2.10

and

C(w)

=

C ( 9 ) = 4.

The Parametric Unconstrained Problem

Consider the parametric unconstrained problem min f ( X ) = min x-P

C &(Xi),

X-P i e T

is a set of discrete variables, ozj being the number of assignments of the variable x j ;

T = {1,2, ..., t } ;

P

=

XicX;

{ x , ~ +,.~. . ,x,} c

X;

and

1PI

= p.

The optimization procedure for the solution of this problem consists of fixing one ordering w = (yl,y z ,. . . ,yn-J of the variables of the set X - P, eliminating (see Section 2.3) them one at a time according to such ordering, thus obtaining a function of P and, finally, operating “backward” in order to find the optimizing assignments y,*(P), y g * ( P ) ., . . ,Y Z - ~ ( P )Thus . the procedure in the parametric problem is very similar to that for the nonparametric problem, the only differences being that (a) only the variables of X - P are eliminated; (b) the variables interacting with a variable yi, at the time of its elimination, belong, in general, to both sets X - P and P . Point (b), however, implies no difference at all in the optimization procedure. On the contrary, it is a characteristic feature of dynamic programming, in general, that the interacting variables may be considered, temporarily, as parameters. The secondary 3ptimization problem has the same meaning as before and consists in determining an optimal ordering out of all the (n - p ) ! possibilities of the set S,. In particular, Definitions 2.5.1-2.5.7 extend to this case without modifications.

2.10

51

Parametric Unconstrained Problem

Also, in this case, for a problem in standard form, the solution of the secondary optimization problem can be obtained from the solution of an optimization problem for an elimination process in a suitable graph. For simplicity, only the special case cZj= (T = const is considered.

Theorem 2.10.1 (Correspondence Theorem)

Consider the parametric unconstrained problem in standard form 9: min f ( X ) = min

2fi(Xi)

X - P icT

x-P

with UZj = 0

= const.

Consider the graph G = (V,E ) with V= X

and

E= L

u Z*(P),

namely, the graph obtained from the interaction graph ( X , L ) of the problem

9, fully connecting the vertices of the subset P. Then ( a ) B ( 9 ) = B(G) -

. Cp d; j-1

(b) C ( 9 ) = C(G).

PROOF.Consider the set 3, c S, of all elimination orderings ending with the vertices x,-~+~,. . . ,x,, in this order. Let Q = CVl, Y Z

,. . . > Yn-p, x n - p + l ) . .

. ?

xn)

belong to &. There exists a one-to-one correspondence between an element

9 E 3, and an element W =

cY1,~z,***,~n-p)

belonging to S,. It is easy to verify that P

(a) B(o) = B(S2) -

C d; j-1

(b) C(W)= C(Q). Equality (b) is due to the fact that, since the problem9' is in standard form, yn-p is adjacent to all the vertices of P at the time of its elimination,

52

2. Elimination of Variables One by One: Procedure

+

and consequently C(Q) > _ p 1. Finally, since by Theorem 2.8.3 there exists an optimal ordering belonging to so,the proof follows. The proof of Theorem 2.10.1 is constructive in nature and allows, given the optimal elimination ordering for the graph, the determination of an ordering w , which solves the secondary optimization problem.

EXAMPLE 2.10.1 Consider the parametric unconstrained problem ,p in standard form :

The interaction graph ( X , L ) o f . 9 is depicted in Fig. 2.10.la and the graph

Figure 2.10.1

2.10

Parametric Unconstrained Problem

G

( V , E ) , such that

=

V == X

and

53

E

=

L

u {(xa,x6)1(,

is considered (see Fig. 2.IO.Ib). A dominating ordering Q E ,To is

and hence the ordering w , which solves the secondary optimization problem, is CJ = (xl, xs, x2, x4). Then C ( . F ) = C(G)

=

4;

B(,F') = O*

+ 3 0 ~= B(G) - o2 -

O.

Finally note that, if a dominating ordering Q' E S, - So is found, the construction given in the proof of Theorem 2.8.2 allows the determination of an equivalent ordering 9 E

so.

The Elimination of Variables One by One: Properties and Algorithms

3.1

Introduction

As demonstrated in Chapter 2, the secondary optimization problem, its various versions, for the nonserial unconstrained problem (both nonparametric and parametric form and for any number of assignments each variable) can be solved by finding an optimal elimination ordering a suitable graph.

in in of in

Finding an optimal elimination ordering in a graph is, therefore, the central problem of this chapter. In particular, a number of the properties reported refer to dominating orderings and, consequently, hold for any criterion function of the c l a s s y , while some others refer only to minimal dimension (and hence minimal cost) orderings. In Section 3.2 some heuristic algorithms for determining “good” elimination orderings are described. They are, in general, easy to implement and, in many cases, yield optimal or near-optimal orderings. 54

3.2

55

Heuristic Algorithms

The class of algorithms which yields an optimal solution is then considered. A conceptually simple but computationally heavy “optimal path” algorithm, based only on the invariance theorem, is first described (Section 3.3). Subsequently, some complex properties of optimal elimination orderings, which allow the construction of more efficient algorithms, are demonstrated (Sections 3.4-3.8). Section 3.9 shows some interesting relations between the structure of a graph and its dimensions. Section 3.10 is devoted to the determination of efficient upper and lower bounds, while a special branch and bound algorithm is described in Section 3.11. Some computational results are also given.

3.2

Heuristic Algorithms

It is evident that the problem of finding an optimal elimination ordering in a graph, that is, minf(W>), RESn

f E F,

is extremely complex. This section examines some easily implementable heuristic algorithms [14, 15, 19-21 3, which often determine, as already noted, an optimal or near-optimal ordering.

DEGREE ALGORITHM. At each step eliminate a minimum a. MINIMUM degree vertex in the current graph. EXAMPLE 3.2. I

Consider Fig. 3.2.la. A minimum degree ordering is

and its vector dimension is (see Fig. 3.2.1b-f which gives the graphs resulting from the various eliminations)

d(f2) = (3, 2, 2, 2, 1, 0). The reader can verify that Q is a dominating ordering. b. MINIMUM DEFICIENCY ALGORITHM. At each step eliminate a minimum deficiency vertex in the current graph.

56

3. Elimination of Variables One by One: Algorithms

(el

Id l

Figure 3.2.1

EXAMPLE 3.2.2 Consider Fig. 3.2.2a. A minimum deficiency ordering is

L? = ( V I ,

% 3 , 0 2 * v4, v 5 ,

v,,

v7, V g ) ,

and its vector dimension is (see Fig. 3.2.2b-i) d(Q) = (3, 3,2,2, 1, 1, 1 , 1 , O ) .

In this case f2 is again a dominating ordering. It is worthwhile noting that all minimum degree orderings are dominated by 9. DEGREEMINIMUM DEFICIENCY ALGORITHM. At each step c. MINIMUM eliminate, in the current graph, a minimum deficiency vertex in the class of minimum degree vertices. DEFICIENCY-MINIMUM DEGREE ALGORITHV. At each step d. MINIMUM eliminate, in the current graph, a minimum degree vertex in the class of minimum deficiency vertices.

57

3.2 Heuristic Algorithms

Algorithm a tries to minimize the entries of the vector dimension operating on a local basis. Algorithm b (which is usually better than algorithm a ) originates instead from the fact that the elimination of any zero-deficiency vertices in the current graph is an optimal move (see Theorems 2.8.1 and 2.8.3). Algorithms c and d combine, with a different priority, both ideas.

f el

q

I3

IFJ

P fhJ

fil

Figure 3.2.2

This list of meaningful heuristic algorithms is by no means complete. In particular, the properties demonstrated in the next sections will suggest some other more complex but powerful ones. Finally, an example in which algorithms a-d fail is reported.

3. Elimination of Variables One by One: Algorithms

58

Figure 3.2.3

EXAMPLE3.2.3 Consider the graph of Fig. 3.2.3. The two orderings, 1 '

=

v l , v 2 , v 3 , 0 4 , v 5 , v 6 r v 7 , v8)7

obtained via algorithms a-d (note that, in this case, all algorithms produce the same ordering), and

are compared in the following table.

d

3.3

D

B (a = lo)

Optimal Path Algorithms

The problem of determining an elimination ordering which minimizes a criterion function f E p,in a graph G = ( V , E ) , can lead to the well-known problem of finding an optimal path in a network.

3.3

Optimal Path Algorithms

59

The procedure, which is essentially based on the invariance theorem, comprises two steps. 1. Construct a network 9 as follows. (a)

A node w y is associated to each graph GY for all subsets Y c V . Hence the vertex set consists of 2"' nodes ( N = 1 V I).

(b) Two nodes w y g and wy.,are connected by an arc directed from w y ,to wyt, iff the graph GY,, is obtained eliminating a vertex z in Gyr (Y" = Y' u ( z } ) . (c) A cost I T(z I Y ' ) I is assigned to the arc (wy,,w y J , ) . 2. Determine an optimal (according to f) path from the node w0 (corresponding to the initial graph G) to the node wv (corresponding to the null graph). This can be done, for instance, using an algorithm due to Dijkstra [22, 231. From the computational point of view, this method performs better than the exhaustion method, which implies checking N ! orderings. However, since it requires constructing a network with 2N nodes and computing the cost for N 2N-1 arcs, it is not feasible for large-order graphs. Its feasibility can be augmented, if employed in connection with the final theorem and the other theorems that will be demonstrated in this chapter. Finally, a reduction in the complexity of the network can be obtained via rheorem 2.8.4 (see [24]).

EXAMPLE 3.3.1 Consider the graph G given in Fig. 3.3.1 and

The network G is depicted in Fig. 3.3.2, while Fig. 3.3.3 shows an optimal

Figure 3.3.1

60

3. Elimination of Variables One by One: Algorithms

Figure 3.3.2

path obtained by the Dijkstra algorithm. The corresponding minimal dimension ordering is SZ = ( v 2 , v l , v a , v 4 ) and D = 2. The reader should note that steps (1) and (2) of the method need not, in general, be executed in this order. On the contrary, in most cases it is possible to find the optimal path without tracing the entire network. Figure 3.3.4 shows the portion of the network which must actually be traced in this case.

3.4

Computational Implications of Final Theorem

61

Figure 3.3.3

3.4

Computational Implications of the Final Theorem

The final theorem, which has been introduced in Section 2.8 for the purpose of demonstrating the Correspondence Theorems 2.9.2 and 2.10. l , is also important for determining an optimal elimination ordering. Some of its computational consequences are now described. (a) Letting N* be the number of vertices of a maximal fully connected subset of V , the number of orderings to be checked for selecting an optimal

62

3. Elimination of Variables O n e by One: Algorithms

Figure 3.3.4

one is reduced to ( N - N * ) ! . Similarly, a network with only 2N-N' nodes must be constructed if we want t o employ the algorithm of Section 3.3. (b) Consider a graph G = ( V , E ) with the following property: Its complement partitions into distinct components. Let V , , V,, . . . , T/l be the vertex sets of such components and I V iI = N , ( x . f = N l i= N ) . Then the number of orderings to be checked is reduced from N ! to N , ! N,! . N l ! . This depends upon the fact that, if a vertex v E Viis eliminated first, all vertices in ( V - { u } - Vi) become fully connected and can then be ordered last in G, without affecting optimality.

-+

+

+ -

63

3.4 Computational Implications of Final Theorem

(c) An important special case arises when a subset Z c V , fully connected in G, is also a separating set for G. Then the problem of finding an optimal elimination ordering for the graph G becomes that of finding an optimal ordering in two graphs with a lower number of vertices. Let { Y, 2, Z} be a partition of V , such that Z is a separating set for G, and Y and Z are not adjacent. Then the dichatomic procedure for finding an optimal elimination ordering for G is as follows. Consider the graphs

G'

=

(Z u Y,Z(Z u Y ) ) ;

G"

=

(Z u Z , Z(Z u Z ) ) ;

namely, the section graphs of G w.r.t. Z , u Y and Z u Z. Successively, two optimal orderings, in which the vertices of Z are ordered last, are derived for these two graphs (eventually employing the construction described in the proof of Theorem 2.8.2). Let them be

Then an optimal ordering for G is

This is due to the fact that r ( y j I {h,. ..,

in G

=

r(yjI {yl,. . . ,yj-,})

in GI, j

=

1, 2,. . . , I

and

r ( z j 1 {zl,.. . , z ~ - ~ }in) G = r(zj1 {q,.. . , z ~ - ~ }in)G",

.j = I , 2,. . . , m.

Clearly this dichotomic procedure may be tried on the two graphs derived from the first, and so on.

EXAMPLE 3.4.1 Consider Fig. 3.4.la. Since the separating set Z = {u3, u 4 , u 5 } is fully connected in G, the problem of finding an optimal ordering for G may be split into the two problems of finding optimal elimination orderings in the two graphs G' and G" (see Fig. 3.4.1 b and c, respectively). Since the dominating orderings for G' and G" are

64

3. Elimination of Variables One by O n e : Algorithms

(bl

Figure 3.4.1

respectively, a dominating ordering for G is

'

= (vl? v 2 ,

v 6 ~v 7 , v 3 , v 4 , u6)-

(d) Consider a problem min X

1f i ( X i ) , ieT

which must be solved repeatedly in connection with the changes of one component. Let this component befl(X1). The set X1 is not supposed to change. It is easy to conceive of many cases in which this can happen. It is clear that each different fimay yield a totally different solution to the optimization problem; namely, all the x:s (not only the variables of X l ) may assume a different value. It is also clear that, if an elimination ordering is chosen, in which the variables belonging to X1 have the last I X'I places, a part of the forward dynamic programming procedure, namely, the part

3.5

65

Further Dominance Relations

consisting in the elimination of the variables of X - X1, must not be repeated each time. On the other hand, the existence of a component fl(X1) implies that the subset X’ is fully connected in the interaction graph of the problem. Hence there exists an optimal ordering, in which the variables of X1 are ordered last. This is also the best ordering from the point of view of the repetitive solution of the problem in connection with the changes in fl(X1). (e) Finally note that the final theorem is a basic tool for the proof of many properties and theorems which, in turn, form the basis of more efficient algorithms.

3.5

Further Dominance Relations among Elimination Orderings

This section presents some further results concerning dominance relations among elimination orderings.

Theorem 3.5.1 Let G = (V, E ) be a graph, v

E

V, and

D

an elimination ordering. r f

(a) the complement of the section graph of G w8.r.t. r ( v ) , denoted by G, is a forest of bushes, ( b ) corresponding to each bush of order k in G,there are k vertex disjoint paths in G, in which all the vertices except the terminals belong to V - (r(v) v { v } ) , connecting the root with the peaks, there exists at least one ordering Q r ,beginning with the vertex v, such that d(Q’) I d(SZ).

PROOF.Let A be the subset of r ( u ) formed by all those vertices which are roots of bushes in G. Let Z = T(v) - A . Since Z is fully connected in G, it is possible, by Theorem 2.8.2, to construct (from Q) an ordering Q”, ending with the vertices of 2, such that d(l.2”) I d(SZ). Let l.2” = (yl, y z , . . . , J J - ’ , y l , of D r rbelonging to { u > u A .

. . ,yAV)and y l be

the first vertex

66

3. Elimination of Variables One by One: Algorithms

(I) If y z = u, since v is not adjacent in G to y l , y z , . . . , y l P 1 , it is clear that the ordering SZ' = ( y z ,y ,,y ,,. . . ,y z - l , y Z + ] , . . , y X ) is such that d(SZ') = d(Q") 5 d(Q). On the other hand, (2) i f yz E A , since, by condition (b), in the graph G{yl,yz,,,,,yl-ll the degree of u is less than or equal to the degree of y l , and since, by condition (a), y, )...,yl-l,uz) then G{yl, yg)..., e } is isomorphic to a subgraph of there exists an ordering 9

cVl,Y2,.

* . ,.YZ-l,

u, w1,.

>

WN-d

which dominates or is equivalent to SZ". Then, letting Ql

= (v , y,, Y z 9 .

. . ,Yz-1,

. . ,W N - I ) ,

w1,.

we have d(Qr)5 d ( Q f r )5 d(SZ).

a

Theorem 3.5.2 (Initial Theorem) Let G = ( V , E ) be a graph, u E V, and f a criterion function of the class p. If conditions ( a ) and (b) of Theorem 3.5.1 are verified, then there exists an optimal ordering initiating with u. PROOF.The proof derives directly from that of Theorem 3.5.1 when

SZ minimizes f. An important special case arises when r ( u ) is fully connected in G . Then, clearly, conditions (a) and (b) of Theorem 3.5.1 are verified. Hence v can be eliminated first in an optimal dimension ordering (for f EF),

Theorem 3.5.3 Let G

=

(V, E ) be a graph, v E V, and Q an elimination ordering.

( a ) the complement of the section graph of G w.r.t. is a forest of bushes, and (b) G is I T ( u ) /-connected,

r(v),

denoted by

If G,

3.5

67

Further Dominance Relations

then there exists at least one ordering Q’, beginning with the vertex v, such that d(Q’) 5 d(D).

PROOF.In order to prove the theorem, it is sufficient to show that condition (b) of this theorem implies condition (b) of Theorem 3.5.1. Let r and p l , p 2 , . . . ,P k be vertices in r ( v ) which are, respectively, the root and peaks of any bush in G. From C construct a new graph G’ = (VI7 E’), given by V ’ = V u {w}

and

E‘=EuE,,

where w is a new vertex and

This new graph is also I r ( v ) ]-connected. Hence, by the Whitney-Menger theorem, there exist 1 F(u) I vertex-disjoint paths which connect w to r. Since the degree of w is exactly 1 T(u)1, each vertex of ( { u ) u T(u)- { r } ) belongs to a distinct path. In particular, there are k vertex-disjoint paths, connecting w to r through p l , p 2 , . . . ,p k , the vertices of which, with the exception of r, w , p l , p z , . . . ,P k , belong to v - (T(u) u {u}). Hence in G there are k vertex-disjoint paths connecting r to p l , p27 . . . ,p k . rn

Theorem 3.5.4 (Initial Theorem) Let G = ( V , E ) be a graph, v E V, and f a criterion function of the classF. If conditions (a) and (b) of Theorem 3.5.3 are v e r ~ e d ,then there exists an optimal ordering initiating with v.

PROOF.The proof derives directly from that of Theorem 3.5.3 when D minimizes f. EXAMPLE3.5.1 Consider Fig. 3.5.la. In minimizing any criterion function of the class In fact, it can be verified that

F,it is possible to eliminate u first.

(a) the complement of the section graph of G w.r.t {r,p l , p 2 } is a single bush and (b) either (1) the two vertex disjoint paths (r, q l , p l ) and (r, q 2 , p 2 ) connect the root r with the peaks p1 and p 2 , or ( 2 ) G is 3-connected.

68

3. Elimination of Variables One by One: Algorithms

Consider the graph G, (see Fig. 3.5.lb). The subset (p1,p2,r, q 2 ) is fully connected, and hence q1 can be eliminated next. Then, for instance, the ordering ( v , q l , p , ,p z , r, q2) is a dominating one.

( a1

(b) Figure 3.5.1

The properties illustrated up to now are valid for all criterion functions of the class F.The remainder of the chapter is will be devoted to the illustration of more powerful results which, however, hold (or at least are demonstrated) only for the scalar dimension D.

3.6

The Descendance Theorem

Given G = ( V , E ) and u E V , if the vertex u is erased with all its emanating edges, the resulting graph G' = ( V - ( u ) , E - O(u)) is such that D(G') 5 D ( G ) . This property may be preserved even if, after that the vertex u and the edges O(u) have been removed, some edges are added in F(u) [for the case in which T(u) is not fully connected]. The example which follows may be helpful for providing a firmer grasp of this idea. 3.6.1 EXAMPLE Consider the graph G of Fig. 3.6.la. We have D(G) = 2. The graphs G' (Fig. 3.6.lb) and G" (Fig. 3.6.lc), obtained by deleting u and O(u) and adding, respectively, one and two edges in Qu), have dimensions equal to

3.6

The Descendance Theorem

69

Figure 3.6.1

2, while the graph G”’ (Fig. 3.6.Id), obtained adding three edges in has dimension equal to 3.

r(u),

This idea is now defined precisely by means of the descendance theorem.

DEFINITION 3.6.1 Let G = ( V , E ) be a graph and u E V . A graph G* descendatit from G w.r.t. v if

v* = * v E*

=

(E

=

( V * , E * ) is called

-1 9

-

O(U))u E,

where?! , c (Z*(r(u)) -- Z(T(u))) is a set of new edges defined as follows. Let A = ( a l , a 2 , .. . , a,> be a subset of T(u)fully connected in G, and let Z = (zl, z 2 , . . . , z,} = T ( v ) - A . If at least one set between A and Z is empty, let I? = 8.Otherwise let I? be a set of new edges (ud,zj) such that, letting z i ={ Z E Z ( ( ~ ~ , Z ) E I ? ’ ) , i = l , 2 ,..., r, the following conditions hold for 1 5 h, k 5 r, and h -f k (a) 2, n Zk= $3; (b) at least one condition between

70

3. Elimination of Variables One by One: Algorithms

(1) a h is adjacent in G to all members of Zk; (2) ak is adjacent in G to all members of zh. The reader should note that there exist, in general, many different graphs descendant from a given graph G w.r.t. the same vertex u. For instance, both graphs G’ and G“ of Example 3.6.1 are descendant from G w.r.t u, while G”’ is not.

Theorem 3.6.1 (Descendance Theorem) Let G = (V, E ) be a graph, v from G w.r.t. v. Then

E

V, and G*

=

(V*, E*) a graph descendant

D(G*) 4 D(G). PROOF.

I. Recalling Definition 3.6.1, let H be the section graph of G w.r.t. V - ( A u {v}) and H* be the section graph of G* w.r.t. V* - A . Since, in the construction of G* from G, no edge joining vertices in I/ - ( A u { u } ) has been added or canceled, it follows that H = H * . 11. Consider a minimal dimension ordering for G ending with the fully connected subset A u { u } (see Theorem 2.8.3). Let it be bl,.. . ,y,, a1,. . . ,u,, u). It will be shown that the dimension of the ordering Oil,. . . ,yt , a,, . . . , a,) (not necessarily optimal) in G* is less than or equal to D(G). Since it is clear that, for i = 1, 2,. . . , r, { h , .. . a l , . . . , %,}) I in G* < I r ( a i I {YI,. . . ,~ t a,, , ..., I in G,

I r(ai 1

to prove the theorem, it is sufficient to show that, for i

=

1,2,.

. . , t.

111. Consider the elimination, both in G and G*, of the vertices y l , y , , . . . ,y i , all belonging t o the vertex set of the section graphs H = H * . Let H i and Hi*be the section graphs of G,, ,,...,}, and GTk,,..., w.r.t. ( V - {yl,. . . ,y i ) ) - ( A u { u } ) and ( V * - { y , ,. . . ,y i > ) - A , respectively. Since the creation of a new edge joining two vertices y , ,ym E {yi+l, . . . , y , } depends upon the existence of a path y n , zl, z2,.. . , z2,ym in which all the vertices, with the exception of the terminals, belong to {yl,

3.6

71

The Descendance Theorem

y 2 , . . . ,yj} (see the proof of the invariance theorem), and since H = H * ,

it follows that Hi= Hi*. IV. Clearly in the graph G r c v i

I { Y I ? .. .

3

Yi-1))

= (T(Yi

I {YL,. .

’ 9

u (Tbi I bl?.. .

7

Y i - l } ) n {Yi+l,. . . Y t } ) n ( A LJ{V>>)7 9

vi-1))

and in the graph G*

rotiI { Y ~ ~ . . . , Y ~ -(r ~> + ~ , . . . ~ Y ~ } ) u (T~i};

(]A‘].

Clearly the vertices of the set Q (other than yi) are connected by the same paths to yi in G, since H = H*. It is sufficient for meeting the requirements of condition (3.6.2) that the vertices of the set Q be adjacent to at least I A’ I vertices of the set A‘ u { u } in G. For convenience this condition is called condition (3.6.3). VI. Let I A’ 1 = 0 ; then, clearly, condition (3.6.3) holds. Let I A’ 1 = 1 ; A’ = { a } ; and Q = {q}. By Definition 3.6.1, if, in G*, the edge (a, q) E (i.e., it is one of the new edges), then q E r ( v ) , and condition (3.6.3) holds.

72

3. Elimination of Variables O n e by One: Algorithms

Let I A’ I 2 2. Then consider two vertices a’, a” E A’, and let q’ and q” be the vertices of Q adjacent to a’ and a’‘, respectively, in the given set of paths in G*. Consider the following cases separately. (a) q ’ = q” = q. By Definition 3.6.1 [condition (a)], at least one of the edges (q, a’) and (q, a ” ) belongs to E and, if one does not belong to E, q E r ( u ) . Summing up, q is adjacent in G to at least two vertices in the set (u, a’, a”). (b) q’ .tf q”. If at least one of the edges (q’,a’)and (q”, a ” ) belongs to E, the previous reasoning applies. Otherwise, by Definition 3.6.1 [conditions (b)], at least one of the edges (q’, a”) and (q”, a‘) belongs to E and, clearly, q’ E T ( v ) and g” E T(u).Hence in this case also, the set (q’, q” 1 is adjacent in G to at least two vertices in the set {u, a’, a ” } . The same reasoning applies for all vertex couples in A’. Condition (3.6.3) follows immediately, since two vertices belonging to A’ u { u ) cannot both be nonadjacent to Q in G. Loosely speaking, the descendance theorem states that it is possible to erase from a graph G a vertex v and all the emanating edges and to add a forest of bushes in r(u)without increasing the dimension, provided that conditions (a) and (b) of Definition 3.6.1 are satisfied. In particular, (1) a single bush can always be added; (2) a forest of bushes can be added if it fully connects the set

( a1

Figure 3.6.2

r(u).

73

3.6 The Descendance Theorem

161 Figure 3.6.3

EXAMPLE 3.6.2 Consider the graph G of Fig. 3.6.2a. The graph G* of Fig. 3.6.2b is descendant from G w.r.t. v. In this case a single bush has been added.

EXAMPLE 3.6.3 Consider the graph G of Fig. 3.6.3a. Its dimension is equal to 3. The graph G* (see Fig. 3.6.3b), obtained by erasing the vertex and the edges O(v) and adding a forest of bushes in r ( v ) , is descendant from G w.r.t. v. In fact, D(G*) = 3.

EXAMPLE 3.6.4 Consider the graph G of Fig. 3.6.4a. Its dimension is equal to 2. The graph G' (see Fig. 3.6.4b), obtained by erasing the vertex v and the edges

la)

fb)

Figure 3.6.4

74

3. Elimination of Variables O n e by O n e : Algorithms

T(u),is not descendant from G w.r.t. u, since condition (b) of Definition 3.6.1 is not met, In fact D(C') = 3. O(u) and adding a forest of bushes in

This fact could prove really surprising to a reader who reasons as follows: 1. The graph G" (see Fig. 3.6.5), obtained from G by splitting the vertex

u' and u" and partitioning the edges of the set O(u) between u' and u", must verify the condition u into the two nonadjacent vertices

D(G") 5 D(G).

2. The graph G' may be obtained from G" by repetitive use of the descendance theorem (that is, w.r.t. u' and u"). Hence D(G') 5 D(G").

3. Then D(C') 2 D(G). The fallacy of this syllogism lies in premise 1. In fact, it is not generally possible to split a vertex into two nonadjacent vertices and partition its emanating edges without increasing the dimension, even if this is counterintuitive.

Figure 3.6.5

DEFINITION 3.6.2 Let G = (V, E ) be a graph. A graph G* = ( V * , E*)is called descendant from G w.r.t. the set V - V* if there exists an ordering (ul', usr,. . . , u l ' ) of the vertices of V - V* and a corresponding sequence of graphs Go = G, G', G2,. . . , Gz= G*, such that Gi is a graph descendant from Gi-' w.r.t. Ui' (i = 1, 2 , . . . , Z).

3.6

75

The Descendance Theorem

Theorem 3.6.2 (Descendance Theorem) Let G = ( V , E ) be a graph, Y c V, and G* descendant from G w.r.t. Y. Then

=

( V c Y, E * ) be a graph

D(G*) 5 D(G).

PROOF.The proof is obvious.

rn

The descendance theorems have demonstrated that the dimension of a descendant graph G* does not exceed the dimension of the original graph G. The next question is: Which is the largest possible value of D(G) - D(G*)? The theorem which follows provides the answer. Theorem 3.6.3 Consider the two graphs G = (V, E ) and G' = ( V - { u ] , E - O(u)). Then D ( G ) - 1 5 D(G') 5 D(G).

PROOF.The second inequality is trivial. In order to prove the first one, let ( Y l ,Y z ,*

*

.

7

YN

)

be an optimal ordering for G'. Then it is easy to see that, in the graph G , the ordering bl,Y2,...,YN

9v>

has dimension less than or equal to D(G')

+ 1.

Hence a graph G* descendant from G w.r.t. a single vertex is such that

D(G) - 1 5 D(G*) 5 D(G), and a graph C* descendant from G w.r.t. k vertices is such that D(G) - k 5 D(G*) 5 D(G).

Finally, note that Theorem 3.6.3 gives an answer to another interesting question : What happens to the dimension of a nonserial unconstrained problem when one variable is added or dropped within the given com-

76

3. Elimination of Variables O n e by One: Algorithms

ponents. Again, the answer is that the dimension of the problem can increase or diminish at the most by one unit.

3.7 The Initial Theorem

Section 3.5 provided two initial theorems valid for all criterion functions of the class F.A more powerful initial theorem, based on the descendance theorem, is demonstrated for the scalar dimension D in this section.

Theorem 3.7.1. Let G

=

(Initial Theorem)

(V, E) be a graph, Y c V, and D(G) 2 k. If

( a ) Gy is a graph descendant from G w.r.t. Y ; (b) there exists one elimination ordering (yl,y z , . . . ,y L )of the vertices of the set Y such that

then there exists a minimal dimension ordering beginning with

PROOF.Consider the elimination ordering 9 = (yl, y 2 , .. . ,y z , zl, z2,.. . ,zm) for the graph G where (zl, z p , . . . ,2,) is an optimal elimination ordering for the graph G y .Then, by Theorem 3.6.2, we have

5 max(k, D(G)) = D(G) EXAMPLE 3.7.1 Consider the graph G of Fig. 3.7.1. It is clear that D(G) 2 3, since is a graph the vertices of G have minimum degree equal to 3. Since G{w,,c2} descendant from G w.r.t. {u,, vz}, and both I r(ul) 1 and I T ( u , I ( u , } ) I are equal to 3, a minimal dimension ordering beginning with the vertices u1 and uz (in that order) exists. When

1 Y I = 1, the initial

theorem specializes as follows.

77

3.7 The Initial Theorem

Figure 3.7.1

Theorem 3.7.2 (Initial Theorem)

Let G

=

( V , E ) be a graph, v

E

V, and D ( G ) 2 k .

If

( a ) the complement of the section graph of G w.r.t. r ( v ) is a forest of bushes, k, ( b ) I Qv) I I there exists a minimal dimension ordering beginning with v.

Figure 3.7.2

78

3. Elimination of Variables One by One: Algorithms

EXAMPLE 3.7.2 Consider the graph G of Fig. 3.7.2a. Since D(G) 2 3, the vertex vz can be eliminated first, and in Go, the vertex v5 can be eliminated next. Hence an optimal ordering is ( u 2 , u5 , v l, u3 , v4 , v6). Note that Theorem 3.7.2, which is a special case of Theorem 3.7.1, is a generalization of Theorem 3.5.4. (and hence of Theorem 3.5.2). In fact condition (a) is the same for both theorems, while condition (b) of Theorem 3.5.3 implies condition (b) of Theorem 3.7.2 (but not vice versa). The application of Initial Theorems 3.7.1 and 3.7.2 requires the knowledge of an appropriate lower bound to the dimension. The determination of good lower bounds is dealt with in detail in Section 3.10.

3.8

The Separating Set Theorem

Theorem 3.8.1 (Separating Set Theorem) Let G = ( V , E ) be a graph. Consider a partition of the vertex set V into the sets Y, Z , and Z such that 2 is a separating set of G, and Y and Z are not adjacent. If

( a ) GY and G, arc descendant froin G w.r.t. Y atid Z , respectively; ( b ) 2 is fully connected both in GY and Gz; then there exists a minimal dimension ordering in which the vertices of Z are ordered last. PROOF. Let (yl, yz ,. . . , Y Z, sl,s2,. . . ,sq) and (zl, z2 ,. . . , z, ,s1, s 2 , .. . , sp) be two minimal dimension orderings for Gz and G y , respectively (see Theorem 2.8.3). Then, by Theorem 3.6.2,

D(G) 2 (D(GZ)7 D ( G y ) ) =max[ max I I Y y i I { y l , . . . , y i - l ) I in G z , i C { l , 2, ..., 1 )

max

i e h , 2 , ..., 7%)

~ ~ ( Z ~ ~ { Z ~ , . . . , Z i~n -G~Y} ,) I~Z I - l ) ] .

Since

r(yi I {YI . . ,~ i - 1 ) ) ?.

in GZ = r ( y i 1 {y,, . . . ,yi-,}> in G, i = 1,2,...,1

79

3.8 The Separating Set Theorem

and

r(zj I {zl,.. . , ~

in

j - ~ } )

I

GY = T(zj {yl,. . . , y l , zl,.. . , z ~ - ~ } in ) G, j = 1,2, . . . , m

the ordering ( Y l , . . . ,yl, z1,.

. . ,,z,

s1,. . .

,s*)

is optimal for the graph G. rn The separating set theorem allows us to consider a subset Z , as if it were fully connected, thus considerably widening the application range of the dichotomic procedure illustrated in Section 3.4. It ought to be noted, however, that an efficient algorithm which exploits the dichotomic procedure requires, in its turn, a good procedure for generating separating sets with a low number of vertices and/or a low number of missing edges.

EXAMPLE 3.8.1 Consider the graph G of Fig. 3.8.1. Since Gi,l,,2) and G{VB,,lare graphs descendant from G w.r.t. ( u l , u 2 } and (u, , u,), respectively, and the separating set { u 3 , u4, v 5 } is fully connected both in G,,l,,, and G{V6,,,},then there exists an optimal dimension ordering ending with the vertices u 3 , uq, u5. This means that the subset { u 3 , u 4 , u g } behaves as if it were fully connected.

EXAMPLE 3.8.2 There exist graphs with no separating set satisfying the conditions of Theorem 3.8.1. The graph of Fig. 3.8.2 is an example. The reader can check this statement for himself.

Figure 3.8.1

Figure 3.8.2

80

3. Elimination of Variables One by One: Algorithms

Up to now, Theorems 3.5.2 and 3.5.4 have been considered as special initial theorems. However, since they require no previous knowledge of a lower bound, they can be regarded as particular separating set theorems. In effect, Theorem 3.5.2 turns out to be a special case of Theorem 3.8.1 when D is the criterion function.

3.9 Connections between Structure and Dimension in a Graph It is obvious that the dimension of a graph depends upon its “structure.” The nature of this connection is, in general, very complex. However, in some limiting cases, it is possible to characterize this relation in a simple way.

Theorem 3.9.1 The dimension of a graph G

=

PROOF.The proof is obvious.

( V , E ) is equal to zero iff it is empty.

rn

Theorem 3.9.2 The dimension of a nonempty graph G forest.

=

( V , E ) is equal to 1

ix

G is a

PROOF.If G is not empty and is a forest, there exists at least one vertex u of degree 1, and G, is still a forest. Conversely, if G is not a forest, there exists a subgraph G‘ of G which consists of a cycle. Then D ( G ) 2 D(G’) = 2. rn

Theorem 3.9.3

fi

The dimension of a graph G = ( V , E ) that is not a forest is equal to 2 no subgraph of G is isomorphic, within vertices of degree 2, to G(*).

PROOF.Consider I V I 2 4 (for I V I = 3, the theorem trivially holds). The condition is necessary. In fact, suppose the existence of a subgraph G’ of G isomorphic, within vertices of degree 2, to G(*)’,namely, the complete graph with 1 V 1 = 4 vertices. Next eliminate in G‘ (see Theorem 3.7.2) all vertices of degree 2, if any. The resulting graph is isomorphic to G4’.Hence

D(G) 2 D(G‘) = D(G‘4’)= 3.

3.9

Connections between Structure and Dimension

81

The condition is sufficient. The nonexistence of a subgraph of G isomorphic, within vertices of degree 2, to G4'implies, b y the Whitney-Menger theorem, that G is u-connected with u 5 2. There then exists a separating set Y(l Y I I 2) which satisfies Theorem 3.8.1. The graph G can then be split into two graphs Gt' and G"'. It is easy to see that neither C" nor G"' can contain a subgraph isomorphic, within vertices of degree 2. to G(". The proof follows by repetitive use of this argument. I The statement of this theorem can also be found in [25].

EXAMPLE 3.9.1 Consider the graph G of Fig. 3.9.la and the subgraph G' of Fig. 3.9.lb. Eliminating the two vertices of degree 2, u2 and u 3 , a graph isomorphic to G(4)is obtained. Then D ( G ) 2 3.

Figure 3.9.1

Theorems 3.9.1-3.9.3 can be formulated compactly as follows : For k = 0, 1, 2 the dimension of a graph G is less than or equal to k iff G does not contain a subgraph isomorphic. within vertices of degree 2, to G(?'"f". As the next example shows, this statement does not generalize to k > 2.

EXAMPLE3.9.2 Consider the graph G of Fig. 3.9.2. Since the graph is planar, by Kuratowski's theorem there exists no subgraph isomorphic to G5',within vertices of degree 2. However, repeated use of Theorem 3.7.2 can show that the dimension of G is not less than or equal to 3, but that

D(G) = 4.

82

3. Elimination of Variables One by One: Algorithms

Figure 3.9.2

Theorem 3.9.4 The dimension of a graph G G is complete.

=

I

(V, E ) is equal to N - 1 ( N = V 1) iff

PROOF.The proof is obvious. Theorem 3.9.5 The dimension of a noncomplete graph G the complement of G is a forest of bushes.

= (V, E )

is equal to N - 2 iff

PROOF.The condition is sufficient. In fact, for any elimination ordering

0 which initiates with a vertex of degree less than N - I , we have D(0)= N - 2. Hence D(G) = N - 2. The condition is necessary. In fact the existence of a 2-flower in the complement of G implies D(G) 5 N - 3.

EXAMPLE 3.9.3 Consider the graph of Fig. 3.9.3a and its complement, depicted in Fig. 3.9.3b, which is a forest of bushes. Consider the orderings

3.9 Connections between Structure and Dimension

83

Then

d(Q,) = (4, 3, 3, 2, 1,O); d(Q2)

=

D(52,)= 4; D(Q2) = 4;

(4, 3, 3, 2, I , 0 ) ;

B(Q,)

d(Q3) = (5,4, 3,2, 1,O);

=

5.

(aI

Figure 3.9.3

EXAMPLE 3.9.4 Consider the graph G of Fig. 3.9.4a and its complement depicted in Fig. 3.9.4b. Since C? is not a forest of bushes, but contains the 2-flower {u4, us> (or {u3, u,>>, we have

D(G) < 5 - 2

=

3.

a-.. ‘.a /

\

/

,,

\

’

@\ ,/‘ fdl Figure 3.9.4

(61

84

3. Elimination of Variables One by O n e : Algorithms

In fact choosing an ordering beginning with the vertices of the 2-flower { u 4 , u s } , namely, 0 = (u4, u s , u l , 0 2 , u s ) , we have

D(Q)= D(G) = 2. The complements of the graphs resulting from G by the successive elimination of u4 and u5 are depicted i n Fig. 3.9.4c,d. Theorems 3.9.4 and 3.9.5 can be formulated compactly as follows : Given a graph C and letting k be the cardinality of the maximum k-flower in the

I

I

I

I

Figure 3.9.5

3.10 Upper and Lower Bounds to the Dimension

complement of G, for k

=

85

0, 1, we have

D(G)=N-k-I.

Also in this case, the property does not extend to k 2 2. EXAMPLE 3.9.5 Consider, in fact, the graph G of Fig. 3.9.5a and its complement C of Fig. 3.9.5b, which contains no 3-flower. However, the ordering

is such that

D(Q)= 2. The complements of the graphs resulting from G by the successive elimination of u,, u 4 , and v5 are depicted in Fig. 3.9.5~-e, respectively.

3.10

Upper and Lower Bounds t o the Dimension

In the previous section, a class of graphs for which it is possible to determine immediately the dimension D, has been considered. However, since this class is very limited, it nearly always is necessary to use suitable algorithms which, in general, demand the knowledge of meaningful upper and lower bounds to D. This section is given over to the determination of such bounds. Some of them are fairly obvious and have already been introduced by means of examples; some others are less apparent. In part (a), the problem of determining upper bounds to D is examined; in part (b), lower bounds are considered successively; and finally in part (c), some computational results are reported. a. As anticipated in Section 3.9, the existence of a k-flower in C provides an upper bound to D. In fact, the following theorem holds.

Theorem 3.10.1 Consider a graph G = (V, E ) . r f there exists a k-power in the complement

G of G, then D(G) 5 N - k - 1.

3. Elimination of Variables One by One: Algorithms

86

PROOF. Let V‘ = {yl, y z , . . . ,y k > be a k-flower in { Y ~ +. .~. ,,y N } . Consider the ordering = b1, y 2 , . . . > yk

Then

3

ykfl,.

I roil)]I (N-

.

9

G

and V - V ‘ =

yS)-

1) - k ,

since the vertex yl is adjacent to at least k vertices in G. The proof follows since { y z , . . . ,y k } is a ( k - 1)-flower in the complement of GYl. W

,,

-0’ I

\

I

Figure 3.10.1

EXAMPLE 3.10.1 Consider the graph G of Fig. 3.10.la and its complement (Fig. 3.10.lb). Since there exists the 3-flower {us, u4, ug} in G and N = 6 , we have

D(G) 5 6 - 1 - 3 = 2. The ordering Q = (u,,

u4 9 V5

9

u1 Y u2 Y

4

beginning with the vertices of the 3-flower, has D(Q) = 2 and is, in this case, an optimal one. Another upper bound is obtained by considering an interesting relation between dimension and Mason’s index in a graph.

Theorem 3.10.2 Let G

=

( V , E ) be a graph and v its Mason’s index. Then DIY+l.

3.10

87

Upper and Lower Bounds to the Dimension

PROOF.If G is empty, the proof is trivial. Otherwise let Q

=

{ q l , q2,.. . ,

q v } be a Mason’s set. The section graph G‘ of G w.r.t. V - Q is, by definition, a tree or a forest. Letting (yl,y,, . . . ,yX-,) be a minimal dimension

elimination ordering in G‘ [D(G’) = 11, consider, for G, the ordering Q = 01,.. . , Y N - , ,

41,.. . q v ) . 3

Since a vertex y j (1 5 j 5 N - Y), at the time of its elimination, is adjacent . . . ,y N p v }and to at most v vertices of to at most one vertex of the set the set Q, it is clear that D k.

+

3.10

89

Upper and Lower Bounds t o the Dimension

PROOF.When no two nonadjacent vertices of degree k exist, the set of all vertices of degree k{s,, s2, . . . , s l } ( I = 1, 2, . . . ,k 1) is fully connected in G. Hence, by the final theorem, the minimal dimension ordering 9 = {yl,. . . ,y N P I ,sl,.. . ,sz} is such that I I‘(yl) I > k .

+

EXAMPLE 3.10.4 Consider the graph G of Fig. 3. 0.4. Only one vertex of degree 2 exists. Hence D(G) > 2. Another lower bound, which has been anticipated in Section 3.9, is the following.

Figure 3.10.4

Theorem 3.10.5 Let G = ( V , E ) be a graph and let G’ be a subgraph of G isomorphic, within vertices of degree 2, to Gk’.Then D ( C ) 2 k - 1.

PROOF.Consider k 2 3 (the cases k = 1, 2 are trivial). By Theorem 3.7.2, eliminate in G‘ all vertices of degree 2, if any. The resulting graph is isomorphic to G k ’ .Hence D ( G ) 2 D(C’) = D ( G k ’ ) = k - 1. W

EXAMPLE 3.10.5 Consider the graph G of Fig. 3.10.5. Since the subset { u 2 , v 4 , u 5 , u,} is fully connected in G, we have D(G) 2 3.

90

3. Elimination of Variables One by O n e : Algorithms

Figure 3.10.5

More powerful lower bounds can be obtained adopting methods which exploit the descendance theorem, A very efficient procedure is the one described by means of the flow chart of Fig. 3.10.6. It consists essentially in (1) assigning an initial lower bound Lo (obtained, for instance, via Theorems 3.10.3-3.10.5) ; (2) generating a sequence of descendant graphs, according to a certain rule, and using for each graph Theorem 3.10.3, for eventually improving the current lower bound; and finally (3) iterating step (2), starting from the new resulting lower bound, until no improvement can be obtained.

Within the procedure there may exist many a priori reasonable ways of

(i) selecting, in the current graph G”, a vertex w E W ; (ii) choosing a particular graph among the descendant ones w.r.t. w. An efficient way of operating such choices is to select a vertex z and a descendant graph G”* so that G* is as “similar” as possible to G”,. The two graphs turn out to be exactly the same whenever there exists a vertex w E W such that the complement of the section graph of G“ w.r.t. T(w)is a forest of bushes. Otherwise, the following criteria (see Fig. 3.10.6) may be used.

CRITERION a. Select a minimum deficiency-maximum degree vertex.

CRITERION ,8. Let G [ = (Vlr, ,El’) be the complement of the section graph of G” w.r.t. r(w). With reference to the nomenclature adopted in Definition 3.6.1, let a, be a maximum degree vertex and 2, = r(a,) in Glr. Then, if ( { a , } u 2,)

3.10

91

Upper and Lower Bounds to the Dimension

& V,', let a2 be a maximum degree vertex and 2, = r ( a J in the section graph Gi of G,' w.r.t. V,l - ( { a l ) u Zl). When (a,} u 2, = V:, the procedure ends. Then selecting the set of new edges

I?=

{ ( U ~ , Z ) ~ Z E Z ~ ,

i=1,2

,..., r } .

Definition 3.6.1 is satisfied. G=IV, E) L o I a n initial lower bound)

I Consider

the set

of

1

the complement ot the section graph at w r.t r w i s a forest of bushes, e x i s t s

G"

no Select d member w E w dCCOrdlng t o a given criterion a

descendant from

d

w . r .t . w,

( f i n a l lower bound)

Figure 3.10.6

92

3. Elimination of Variables One by One: Algorithms

TABLE 3.10.1" Graph

Lower bound L

Upper bound Ul

Upper bound

4 4 4 4 4 4 4 5 5 5 5 5 5 5 4 4 4 5 5 5

4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5

4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6

A B

C D E F G H Z

L M N 0

P

Q R S

T

U V a

N

= 30;

1) =

6 6 6

U2

min(Ci,, U,)- L

0 0 0 0 0

0 0 0 0 0

0 0 0 0 1 1 1 1 1 1

10%.

EXAMPLE3.10.6 Consider the graph G of Fig. 3.10.7a and determine a lower bound to D via the algorithm illustrated by the flow chart of Fig. 3.10.6. Then Lo = 4, since G contains the fully connected subset {v2, us, v6, v, ,v8} (see Theorem 3.10.5). Let c' = G and E = Lo. Then W = {vl} and the graph c'*, descendant from (? according to criterion /I,is depicted in Fig. 3.10.7b. Since I P* 1 = 7 > 1 = 5, let c' = c"*; i.e., the new current graph is that of Fig. 3.10.7b. Since W = 0, the new current lower bound becomes = 5 and the new set W is equal to {vz, v4}. Since the complement of the section graph 0f.c w.r.t. I'(v,) is empty, and, moreover, since I P*I = 6 = 2 1 , a new iteration then c'* = Gv,,, of the algorithm is made with the new initial lower bound Lo = 1 = 5. This iteration does not improve the current lower bound. Hence L = 5.

+

+

3.10

93

Upper and Lower Bounds to the Dimension

Figure 3.10.7 TABLE 3.10.2a

Graph

A B C D E F G H

Z L M N 0

P

Q R S

T

U V a

N

=

Lower bound L

13 14 12 13 14 12 12 12 13 13 11 11 12 12 12 12 12 11 11 12 30; 7 = 30%.

Upper bound Ul

Upper bound

13 14 13

13 14 13 14 16 14 14 14 15 16 14 14 15 15 15 15 15 15 15 16

15

15 14 14 15 15 15 14 14 15 15 15 15 15 15 15 16

U,

min(U,, U,)- L

0 0

1 1 1 2 2 2 2 2 3 3 3 3 3 3 3 4

4 4

94

3. Elimination of Variables One by One: Algorithms

TABLE 3.10.3a Lower bound L

Graph

19 17 17 17 16 16 17 17 17 17 17 17 17 17 17 16 16 17 16 17

A B

C D E F G

H Z

L M N 0

P

Q R S

T

U V a

N

= 30;

q

Upper bound Ul

Upper bound

min(U,, U,)- L

u 2

20

20 19 20 21 19 20

19 19 19 19 19 20

20

20 20 20 20 20 20 20 20 20 20 21 21 22

20

20 20 20 20 20 20 21 20 20 21

21 22

1 2 2 2 3 3 3 3 3 3 3 3 3 3 3 4 4 4 5 5

= 50%.

Note also that, since in this iteration the vertices successively selected satisfy Initial Theorem 3.7.2, the resulting ordering is an optimal one. This however, does not happen as a rule. c. Some computational results, taken from [ 2 6 ] , are now given. One hundred graphs G = ( V , E ) , with N = 30 vertices and different values of the filling index 17

=

I El

“ ( N - 1)]/2

*

100,

have been generated in a pseudorandom way. For each of these, the following computations have been made: (1) a lower bound L, by means of the algorithm described in part b of

this section;

3.10

95

Upper and Lower Bounds to the Dimension

TABLE 3.10.4”

Graph

A B

C D E F G H I L M N 0

P

Q R S

T

U Y

Lower bound L

Upper bound

Upper bound

u,

U,

21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 20 21 20 20

22 22 22 22 23 23 23 23 23 23 23 23 23 23 23 24 23 24 24 24

23 23 23 24 23 23 23 23 23 23 23 24 24 24 24 23 24 24 24 24

rnin(U,, U,) - L

1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 4 4

(2) an upper bound U , , via the minimum deficiency-minimum degree algorithm used in connection with the initial theorem ; (3) an upper bound U , , via the minimum degree-minimum deficiency algorithm, used in connection with the initial theorem. In Tables 3.10.1-3.10.5 for q = lo%, 30%, 50%) 70%) 90%, the graphs are ordered according to increasing values of min(U,, U,) - L. Clearly, it is possible to determine immediately an optimal ordering when (min(Ul, 17,) L ) = 0 and also to

(i) know the largest possible “error” of the heuristic algorithms; (ii) employ such bounds as tools in algorithms which determine D exactly. It is very interesting to note that the difference between the best upper bound and the lower bound are small indeed, especially for low or high values of q.

3. Elimination of Variables One by One: Algorithms

96 TABLE 3.10.F

Graph

A

B C D E F G

H I L

M N 0

P

Q R S

T U V

Lower bound L

Upper bound

Upper bound

u,

u2

25 25 25 25 25 25 25 25 26 26 25 25 25 25 25 25 25 25 25 25

25 25 25 25 25 25 25 25 26 26 26 26 26 26 26 26 26 26 26 26

25 25 25 25 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 21

min(U,, U,)- L

0

0 0

0 0

0 0

0 0

0 1 1 1 1 1 1 1 1 1 1

3.11 A Branch and Bound Algorithm

This section describes a branch and bound algorithm, taken from [26], the objective of which is to determine a minimal dimension ordering in a graph. It employs the upper and lower bounds determined in the previous section, the initial theorem in its simplified version (Theorem 3.7.2), and the final theorem. Its flow chart is shown in Figs. 3.11.1 and 3.11.2, while Table 3.11.1 shows the results for the same graphs considered in Table 3.10.1 (7 = 10%). This algorithm is efficient for sparse (e.g., r] 5 20%) and dense (e.g., q 2 80%) graphs, while it behaves poorly for the intermediate range of values of r]. However, it is important to note that, whenever D (which grows rapidly with q, as the reader can verify from Tables 3.10.1-3.10.5) exceeds a given value (which depends upon o), the memory requirements

3.11

97

Branch and Bound Algorithm

TABLE 3.11.1a

Lower bound L

Upper bound

A

4

4

4

B C D E F G H

4

4 4 4 4 4

4

Graph

4

N

4 4 4 4 5 5 5 5 5

0

5

Z

L M

P

5

Q

4 4

R S T

a

U

4 5 5

V

5

N

U

=

30; rj

=

5 5 5 5

4 4 4 4 4 5 5 5 5

5 5

5 5

5 5 5

5 5

4

5

5 5

6 6 6

5 6 5

10%.

I bound

use the h i t i d l theorem as many times as possible L e t G o = ( V n , E'I be the resulting graph i

0

Dimension D

Figure 3.11.1

98

3. Elimination of Variables One by One: Algorithms

t. 4

I

Use the initial theorem as many times as possible. Let G i b e the resu[tinq qraph.

( lvil

i 5 Li

+QA

no Construct in G' w ! = { w l Irrwl < L i j Z'={W(L'+.lb( T W J ( < U1-Z' w h e r e r' is d fully connected subset in G i

-xi

1

Figure 3.11.2

3.11

Branch and Bound Algorithm

99

practically forbid the use of the dynamic programming procedure based on the elimination of variables one by one for the solution of a nonserial problem. Moreover, the efficiency of the algorithms clearly decreases with increasing N . However, it is reasonable to think that the use of the separating set theorem, which allows splitting a single problem into problems with lower numbers of vertices, can greatly facilitate the determination of an optimal elimination ordering for large-order graphs.

The Elimination of Variables in Blocks

4.1

Introduction

A new dynamic programming procedure, the elimination of variables in blocks, for solving a nonserial unconstrained problem, is described in this chapter. This new procedure, which is a generalization of that given previously, is more complex, but also more convenient for the following two reasons. (a) It allows (possibly at the cost of additional computing time) solving the given optimization problem without exceeding a prescribed memory limitation. (b) If no memory limitation exists, it dominates, from the computingtime point of view, the procedure consisting in the elimination of variables one by one. The first part of the chapter is devoted to the formal description of the primary dynamic programming procedure and to the new secondary optimization problem. Then the block elimination process is defined, and its properties are derived: They generalize those given in Chapters 2 and 3 I00

4.2

101

Procedure for Solution of Primary Problem

for the elimination process. Little attention, however, is paid to algorithms, since no computational work has been carried out for this case. Finally, the correspondence between the secondary optimization problem of a nonserial unconstrained problem with memory limitation and an optimization problem for a block-elimination process in a suitable graph is demonstrated.

4.2 The Description of the Procedure for the Solution of the Primary Problem Again consider the nonserial unconstrained problem minf(X)

= min

x

X

Cfi(P), id'

oz. being the number of assignments of the

is a set of discrete variables, variable xj;

T = { 1 , 2 , . . . ,t } ;

and

Xi c X.

Consider one ordered partition of the variables of the set X . Let it be y = ( Y , , Y , , . . . , Ym),with rn 5 n , and let S, be the set of all ordered partitions of X. For this ordered partition, the problem may be solved by dynamic programming. The procedure is divided into two parts.

a. FORWARD PART.The block Y, is considered first. Then minf(X)

x

=

min min X-Y1

C fi(Xi)

Yl ~ G T

where

TI=

{i:

Xin Y, f @}.

Hence the first step of the optimization procedure consists in computing,

102

4. Elimination of Variables in Blocks

employing exhaustive enumeration,

and storing the optimizing assignment of Y, as a function of y( Y l ) ,namely, Yl * (Y( Yl)). The minimization off(X) w.r.t. Yl , for all possible assignments of y( Y,), is called the elimination of the block Y,. The problem remaining after the elimination of Y , ,

is of the same form as the originalproblem, and the function h,(y( Y,)) may be regarded as a component of the new objective function, Hence the same procedure applies to the elimination of the blocks Y,, Y 3 , .. . , Y,,, in order. At each step j the new component hj and the optimizing assignment Yj* are stored as functions of y(Yj I Y , , . . . , Y+,), namely, the set of variables interacting with at least one variable of Yj in the current problem (that is, the problem obtained from the original problem by the elimination of Y , , Y , , . . . , Yj-l). Since the set y ( Y , I Y,,. . . , Y,-,) is clearly empty, the elimination of Y , yields the optimal value of f ( X ) .

b. BACKWARD PART.This part of the procedure consists in the successive determination of Y,*, Y&, ,. . . , Yl*, i.e., the optimizing assignment for X , from the stored tables Y,*(y(Y,)), Y,*(y(Y, 1 Y J ) , . . . , Y,*. It is also clear that, in this case, the forward and backward parts of the procedure can be combined into one more general and flexible procedure, as described in Section 2.3. It is interesting to count the number of ordered partitions of the set X. Obviously this number is much larger than the number n! of orderings 0

E

s.,

Let a, be the number of all ordered partitions of a set of n elements (a, = I S, I). Then a, may be determined using the following difference equation :

In fact, the first block of the ordered partition can be of size k (1 I kI n) and can be chosen in ( 4 ) ways; after the first block is chosen, the remaining n - k variables can be partitioned in an-k ways. The integer a, can be computed, for instance, employing a method based on Pascal-like triangles.

4.3

An Example

103

EXAMPLE 4.2.1 Compute a4. The triangle is 1

3

1

2

4 6

75

13

16

20 26

The triangle is constructed starting from a, first two elements of the first row. Then

=

1 and a, = 1 , which are the

(1) the elements of the rows other than the first are obtained by adding the two upper elements, while ( 2 ) the ith element of the first row, which is is obtained by adding all the elements of the right-hand side of the current triangle.

4.3

An Example

An example of the optimization procedure described in Section 4.2 is now presented. Consider the nonserial problem 4

min f(X) = min

x

X

C fi(Xi), i=l

ci= rs = 2; and the f;s are given in the tables on page 104. An ordered partition is selected from among the 545,835 possibilities. Let it be XZ x3}, {x4 , x5 XI3 x7 XE}). v1 = 9

7

7

9

9

104

4. Elimination of Variables in Blocks

I

x1

x2

X4

X8

fi

0

0 0 0 0 1 1 1 1 0 0 0 0 1

0 0 1 1 0 0 1 1 0 0 1 1 0 0

0

3 7 2 14 9 12 12 15 6 5 3 0 1 2 11 17

0 0 0 0 0 0 0 1 1 1 1 1 1

1

1 1

1

1

1 1

1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

xE

x7

0

0

0

O

0 0 0 0 0 0 0 1 1 1 1 1 1 1 1

0 0 0 1 1 1 1 0 0

0

1

1 1

0

x4

x8

0 0 1 1 1 1

0 0 1 1 0 0 1 1 0 0 1 1

1 0 1 0 1 0 1 0 1 0 1 0 1

P15 16 5 15 11 4 2 2 3 1 21 18 6 7 6 3

0 0 0 0 0 0 0 0 1 1

1 1 1 1 1 1

0 0

0 0

0 1

0 0 1 1 1 1 0 0 0 0 1 1

1 1 0 0 1 1 0 0 1 1 0 0 1 1

0

0 1 0 1

xE

x8

1 1

1

0 1 0

1 0 1 0

1

x4

x5

0

0 0 0 0 1 1 1 1 0 0 0 0 1

0

0

0 1

1 0 1 0 1 0 1

1

0

1

1 1

1 1

0 1

0 0 0 0 0 0 0

1 1

1 1

1 1 1

1

1

0 0

1 1 0 0 1 1 0

4 6 8 7 14 5 9 15 9 3 5 21 4 3 20 19

0 1

0 1 0

I

f 4

8 10

0 17 10 0 2 4 19 2 6 3 1 3 5 10

4.3

105

An Example

The function hl(x4,x 8 ) and the optimizing assignments of xl, x2,and x3 are given in the table which follows.

0

0

5

1

1

1

0

1

5

1

1

0

1

0

10

0

0

0

1

1

7

1

0

0

The second and last step of the optimization procedure is

The optimizing assignment is x4* = 0; x5* = 0; x6* = I ;x,* = I ;xs*= 0. Operating backward, the optimal assignment for the remaining three variables is determined: xl* = 1; x2* = 1 ; x3* = 1.

Figure 4.3.1

The interaction graph of the original problem is given in Fig. 4.3.1, while Fig. 4.3.2 shows the interaction graph of the problem resulting after the elimination of the block {xl, xz,x3}. The number of functional evaluations required in this case is 25

+

25 =

64;

the maximum number of variables processed at a time is five; and the maximum number of variables in an additional stored table is two.

106

4. Elimination of Variables in Blocks

Figure 4.3.2

Next consider the ordered partition

(see in Fig. 4.3.3 the interaction graphs of the problems resulting from the successive eliminations of blocks {xl), {xz, x8), and {x5)) and the de-

Figure 4.3.3

4.3

An Example

107

generate ordered partition

corresponding to the exhaustive enumeration approach. With the aim of introducing the secondary optimization problem, it is worthwhile comparing, for such partitions, the number of functional evaluations for different values of u, the maximum number of variables processed at a time, and the maximum number of variables in an additional stored table. Number of functional evaluations Ordered partition u=2

v1

25

+ 25

=

u

u=5

64

55

+ 55 = 3,125

105

=

10

Maximum number of variables processed at a time

+ 105 = 200,000

Ya

Z4

+ 24 + 24 + 24 64

S4

28 = 256

5" = 390,625

=

Ys

+ 5* + S4

+ S4

= 2,500

Maximum number of variables in an additional stored table

+ lo4 + 104 + lo4 = 40,000

5

2

4

3

8

0

lo4

10" =

100,000,000

The number of functional evaluations may be considered, as in Chapters 2 and 3 for the procedure in which variables are eliminated one by one, to be a satisfactory measure of the computing time. The example shows that the maximum number of variables processed at a time is also a reasonable index of the computing time. On the contrary, for this procedure, the maximum number of variables in an additional stored table is not closely related to the maximum number of variables handled at one time. This allows us to give a more sophisticated formulation of the secondary optimization problem than that of Chapter 2: Find an ordered partition with respect to which the number of functional evaluations or the largest number of variables processed at a time is minimal, subject to the constraint that the largest number of variables in an additional stored table does not exceed a given integer p.

108

4. Elimination of Variables in Blocks

Clearly the problem is well posed if ,u is greater than or equal to the number of variables of the largest initial table. It must be noted, however, that this number cannot be deduced solely from the knowledge of the interaction graph. Consider, for instance, the interaction graph of Fig. 4.3.1 : It originates from a nonserial problem in which there are four tables with four variables, but it can also come from a problem for which I XiI = 2. Finally, if the components of the objective function are in analytic form, only the additional storage counts.

4.4 The Secondary Optimization Problem

The example developed in the preceding section has introduced a new secondary optimization problem for optimization procedures consisting of the elimination of variables in blocks. The aim of this section is to define the procedure formally, as was done in Section 2.5 for the elimination of variables one by one. The case crzf = cr = const is treated first. Consider an ordered partition y = ( Y , , Y 2 , .. . , Y,) and examine, from the computational point of view, the elimination of the block Y,; that is, the solution of the optimization problem min

C fi(Xi).

(4.4.1)

Y, D 1 = C. Then

+

C,

=

for p > D. rn

C,

Theorems 4.6.1 and 4.6.2 describe the shape of the function C,: It is monotone nonincreasing and reaches its lower bound for p 5 D. Since D 2 N - 1, the meaningful range of the integer ,LA, is 0 5 p 5 N - 1, as shown by Example 4.6.1. The function B, also has the same shape, with the difference that its lowest value is smaller than B, as the following theorem demonstrates.

Theorem 4.6.3 For any nonempty graph G and for p

2 D ( G ) , we have

B, < B. = PROOF.First consider the case p = D and an optimal ordering ( y , , y,, . . . , y X p l ,y N )such that the vertices yxpl and yAvare adjacent in G (see Theorem 2.8.3). Then the ordered partition

!P = ({Yl), (YZ),. . .

Y

(YS-21, (YS-19

ux))

is such that

B ( Y ) < B(Q) = B. The theorem also holds for ,u

>D

by Theorem 4.6.1. m

EXAMPLE 4.6.2 Again consider the graph of Fig. 4.6.1. An optimal elimination ordering is Q = (u,, u 5 , u s , u 2 . u,, u 3 , u4); thus ( c = 10) B(Q)

=B =4

. 103 + 2 .

102

+ 10.

4. Elimination of Variables i n Blocks

118

Instead, as noted in Example 4.6.1, for p 2 D ( G ) = 2, the optimal ordered partition Y = ( { u , ] , { u s , v g ) , {uz}, {vl, u3, v4}) yields B(Y)

=

B,

=4 .

103 < B.

It is now shown that to determine an optimal ordered partition, for any nonnegative integer p, only the subset $$ c S; of the regular partitions need be considered.

Theorem 4.6.4 Let G = ( V , E ) be a graph. For any nonnegative integer p and f o r all ordered partitions Y , such that

G

> 2,

B ( Y ) = BJG), belong to the subset L?$

of the regular partitions.

PROOF. Let Y = (Yl, Y , , . . . , Y,) E S$ be an optimal nonregular ordered partition, if such a partition exists. Let Yj be the first block such that the section graph of G{Y1,--, w.r.t. Yj is not connected. Let Y j , , . . . , Yj:, be the vertex sets of its connected components. Then substituting the elimination of Y j with the successive eliminations of the blocks Yjl, . . , , Yjpl,, we have (a) for c > 2 i=l

Theorem 4.6.5

Let G

=

( V , E ) be a graph. For any nonnegative integer p we have

min C(Y) = C,(G); YES&

119

4.6 Some General Properties

namely, there exists at least one optimal ordered partition in the class of the regular partitions.

PROOF.The proof is the same as that of Theorem 4.6.4, with the difference that the first inequality of point (a) is replaced by

EXAMPLE4.6.3 Consider the graph of Fig. 4.6.2 and u , = 2. Consider the nonregular ordered partition Y1 = ({ul ,u2}, {us u,}, {u,, u 6 , u, us}) E SY2 and de-

Figure 4.6.2

duct, using the construction of Theorems 4.6.4 and 4.6.5, the regular ordered partition u/, = ({ul}, { u d , (4,{u4}, {q,, u 7 , u , } ) . Then % 9

C(Ul,) = C(YJ = C,(C) = 4

Next, the relation between B,, and C, and the connectivity of a graph is discussed.

4. Elimination of Variables in Blocks

120

Theorem 4.6.6 Let G (a) B, (b) B,

=

(V, E ) be an a-connected graph. Then for c > 2

aN,for < crN, for

=

a!

> p;

a!

I p.

PROOF.

(a) The degenerate ordered partition !PI= ( V ) , for which

B(Y,) = O N , is optimal. In fact any other ordered partition U, = (Y , ,. . . , Y M ) ,since the graph G is a-connected with CY > p, is such that

Y, u F(Y1)=

v.

(b) Let Z be a separating set with I Z1 = a, and P and P be the two nonadjacent subsets in which V - Z is partitioned. The proof follows directly, since the ordered partition Y3= ( P , P LJ 2) is such that

+ TI^^^^ < crlvI,

B(Y/,) = eIpuzJ

c > 2,

D(Y/,)=IZI=a~p.

Theorem 4.6.7 Let G

=

(a) C , (b) C,

(V, E ) be an u-connected graph. Then

for

CY

>p;

< N , for

a!

I p.

= N,

PROOF.The proof derives directly from that of Theorem 4.6.6.

EXAMPLE 4.6.4 Consider the 2-connected graph of Fig. 4.6.3. A separating set 2 with I 21 = 2 is { u 4 , ug}. If p = 1, propositions (a) of Theorems 4.6.6 and 4.6.7 hold and B,(G) = c7 and C,(G) = 7. If p = 2, propositions (b) of Theorems 4.6.6 and 4.6.7 hold. In fact the partition !P = ( {vl, v 2 , v,}, {v4, u 5 , u 6 , u , } ) yields B ( Y ) = g5

+ 04;

C(Y) = 5 ;

D(!P) = 2.

4.7

The Final Theorem

121

Figure 4.6.3

4.7

The Final Theorem

Theorem 4.7.1 Let G = ( V , E ) be a graph and Z be any subset of V fully connected in G. Consider an ordered partition Y = ( Y , , Y z , .. . , Klf) E S $ . Then there exists at least one ordered partition Y ’ = (2, ,Z , ,. . . , Z,) E Syp, in which ‘z

zz,

such that

B( Y ’ ) 5 B ( Y )

and

C ( Y ’ ) 5 C(Y).

PROOF.The proof is divided into parts (a) and (b). (a) The possibilities are Yl u r ( Y , ) = V. In this case Y ‘ = ( V ) . Note that this condition is necessarily verified when G is a-connected with (I; > y. (2) Y , u F(Y,) 2 V. This can happen only when G is a-connected with 01 5 p. Then, using the construction which follows, it is possible to find an ordered partition ! P I = ( Yll, Yzl,. . . , YALl) E S,! such that Yll c V - Z, B ( P ) 5 B ( Y ) , and C(Y1) 5 C(Y>. (1)

Let D1)= ‘z and A“’ = A = V - Z[ A ( , )f 0;otherwise point (1) holds]. If Y, c A ( 1 ) then , Y1 = Y. Otherwise ( Y , n 2‘”f 0) define

E2’ = F(Yl)

and

A ( 2 )= ( V - Y , ) - T(Y,).

122

4. Elimination of Variables in Blocks

is fully connected, D2) 2 Zl(l) - Y , , and consequently c A ( , ) .Also L"2)is fully connected in Gyl and A ( , )f 0 [otherwise point (1) holds]. If Y, cf A(,' in GY1, the subsets Z3) and A(3)are similarly defined. This way of proceeding continues, defining at each step j

Since

r(YjPl I { Y,, . . . , Y j - 2 } ) ; A ( j )= (v - ( Y , u Y, u LJ Y j - l ) ) - D j ) ; D j )

=

'* *

until either

(i) for an integer h, Y, c A(*),or (ii) for an integer k, A ( k )= 0. Case (ii) is now examined. Since A ( k )= 0, the set Yk-1

u T(Yk-I I {YI,.. .

equals the vertex set of G { y l ,y~,-,'k-2}

Y

Yk-2))

and since

I T(A(k-1)I { Y l , .. . , Y,-,}) I 5 I Lw-"I I p, the partition

also belongs to S$ and is such that

B ( P )5 B ( Y )

and

C(!P')5 C(Y).

Thus, given the ordered partition Y, there exists another ordered partition Y" = (Yi', . . , , Y&),)E S$ , coincident with Y in case ( i ) , such that B(Y") 5 B ( Y )

and

C(Y") I C(Y),

and for a certain 1 [I equals h in case (i) and k - 2 in case (ii)]

Y;' c LI(2). Note that since c A(l-1' c

. . . c A(1) = A ,

then Y,l' c A = V - 2. It is clear that the set Yi' is not adjacent in G to the sets Yi',. . . , Yi!,

123

4.7 The Final Theorem

Hence the two ordered partitions

ly"

and

(Yi', Y;', . . . , Y&, Y&, . . . , Y&)

= !PI =

( Y l l , .. . , Yh,)

both belong to the set S$ and have the same B and C . (b) When point (2) holds, the same reasoning applies to GY:, in which Z is still fully connected, and to the new ordered partition ( Yzl,.. . , Y h l ) and so on, until point (1) is met. rn The proof of Theorem 4.7.1 is constructive in nature: Figures 4.7.1 and 4.7.2 are helpful in understanding the construction.

construction a l l , the

Figure 4.7.1

124

4. Elimination of Variables in Blocks

Q

Figure 4.7.2

EXAMPLE 4.7.1 Consider the graph G = (V,E ) of Fig. 4.7.3a and V= z= and Then B ( Y ) = fJ5

+d +

a3;

C(Y) = 5 ;

D(Y) = 3.

(1) This condition does not hold since {u3, u s } u {ul, u,, u s } is possible since G is 3-connected. ( 2 ) Define

Z(1)= 2 = {vl, A(1) = A

=

Let

v 2 , us);

V - 2 = { u 4 , u5, v,}.

c

V. This

4.7 The Final Theorem

125

Since Y, = { u 3 , u s } Q A(1),define

Since Y, = {u4} c A(,) (case i ) , we have

The graph G(‘4}, given in Fig. 4.7.3b, and the new ordered partition ( { u 3 , u s } , {ul, u 2 , u 5 } ) are now considered. Condition (1) is now met. Hence

Z2 = {ul, u 2 , u 3 , u 5 , us> and !Pf = ({u4}, {ul, u 2 , u s , v 5 , us}). Then

B ( Y ’ ) = (Y5

+

04;

C(!Pf)= 5 ;

D(Y’)= 3.

Figure 4.7.3

Theorem 4.7.2 (Final Theorem) Let G = ( V , E ) be a graph and 2 any subset of V fully connected in G. Then, for any nonnegative integer p, there exists an ordered partition Y f = ( Z , , . . . , 2,) E S g , in which c ZL,

such that B(!P’) = B,’(G).

PROOF.The proof derives directly from that of Theorem 4.7.1 when Y is optimal. w

126

4. Elimination of Variables i n Blocks

Theorem 4.7.3 (Final Theorem) Let G = (V, E ) be a graph and Zany subset of V fully connected in G. Then,for any nonnegative integer p, there exists a minimal-cost orderedpartition Y' = ( Z , , . . . , Z,) E S $ , in which

z c z,. PROOF.The proof follows directly from that of Theorem 4.7.1 when Y is optimal. rn Theorem 4.7.4 (Final Theorem) Let G = ( V , E ) be a graph and Zany subset of V fully connected in C. Then, for any nonnegative integer p and I Z 1 5 p, there exists a minimal-cost ordered partition Y" E S&, the last block of which is Z. PROOF.From the ordered partition Y' of Theorem 4.7.3, construct the new ordered partition Y ' ,= ( Z , , . . . , z L - l ,

z,

-

z,z>.

We then have C(Y") = C(!P') = CJG)

and

D(Y'")5 p. rn

The final theorems will be used to prove all the basic results of the remainder of this chapter. For additional computational implications the reader is referred to Section 3.4, since similar reasoning applies.

4.8

The Descendance, Initial, and Separating Set Theorems

The two previous sections have dealt with a number of properties regarding both criterion functions B, and C,. This section is devoted only to the criterion function C, and reports some theorems which are generalizations of the homonymous theorems of Sections 3.6-3.8.

Theorem 4.8.1 (Descendance Theorem) Let G = ( V , E ) be a graph, v E V, and G* = (V*, E * ) a graph descendant from G w.r.t. v. For any nonnegative integer p we have C,(G*) 5 CJG).

4.8

Descendance. Initial, Separating Set Theorems

127

I. Recalling Definition 3.6.1, let H be the section graph of G w.r.t. V - ( A u { u } ) and H* be the section graph of G* w.r.t. V* - A . Then (see step I of Theorem 3.6.1) H = H*. 11. Consider an optimal regular (see Theorem 4.6.5) ordered partition Y = ( Y , , Y,,. . . , Y,), such that Y, 3 ( A u { u } ) (see Theorem 4.7.3). It ‘will be shown that for the graph G* the ordered partition Y‘ = ( Y , , Y , , . . . , YM - { v } ) is such that

I

Since I Y, - {u> < I Y , I , and taking into account that the first M - 1 blocks are identical for the two partitions Y and Y’, to prove the theorem, it is sufficient to show that, for j = 1, 2 , . . . , M - 1,

111. Consider the elimination, both in G and in G*, of the blocks Y , , Y , , . . . , Y j , all belonging to the vertex set of the graphs N = H*. Let H j and Hj* be the section graphs of GY,u...Vyj and GS,u...uP, w.r.t. ( V - (Y, u u Y j ) )- ( A u ( u > ) and (V* - ( Y , u u Yi)) - A , respectively. It is now shown that H j = Hj*. In fact, the following points hold. (a) The vertex set W j of Hiequals the vertex set Wj* of Hi*. (b) The creation of a new edge joining two vertices y n , ym belonging to W j ( W j * )depends upon the existence in H ( H * ) of a path y n , wl,.. . , w z ,y , in which all the vertices, with the exception of the terminals, belong to Y, u ... u Yj (recall that the partition !P is regular). (c) H = H*.

IV. In the graph G,

and in the graph G"

128

4. Elimination of Variables in Blocks

Since

condition (4.8.1) becomes

I r ( Y j l {Y,,. . . , Yj-,}) n A I in G* 5 I r ( Y j I {Y, , . . . , Yj+,}) n ( A u

{ u } ) I in G, j = 1 , 2, . . . , M

-

1.

(4.8.2)

V. Consider the graph G*, and let

Since !P is regular, there exist I A’ I paths, connecting every vertex of A’ with some vertex of Y j , in which all the vertices, with the exception of the terminals, belong to Y, u Y, u u Y j P l .Let Q be the set of vertices adjacent to at least one member of A‘ in the given set of paths. It follows that

- -

(a) Q c ( Y , u -.. u Y j ) ; (b) I Q I 5 I A’ I. The vertices of the set Q - Y j are connected by the same paths to the vertices of Y j in G, since H = H * . Then it is sufficient, in order to meet the requirements of condition (4.8.2), that the vertices of the set Q be adjacent to at least 1 A‘ I vertices of the set A’ u { u } in G. This condition is condition (3.6.3) of Theorem 3.6.1. VI. Therefore step VI is exactly equal to step VI of Theorem 3.6.1.

Theorem 4.8.2 Let G = ( V , E ) be a graph and G * = ( V - Y, E * ) a graph descendant f r o m G w.r.t. Y. For any nonnegative integer ,LA we have C,(G*) 5 C,(G). PROOF.The proof is obvious. EXAMPLE 4.8.1 Consider the graph G of Fig. 4.8.la and the graph G* of Fig. 4.8.lb, which is descendant from G w.r.t. u. Consider p = 1, 2, 3.

4.8

129

Descendance, Initial. Separating Set Theorems

1. p = 3. Since D ( G ) = 3, C3(G) = C(G) minimal cost partition is yl

=

( { Y > ? { z l > ? {z2>9

(‘31,

=

{‘I>,

D(G)

+ 1 = 4, and one

{‘2>3

{’>)*

For the graph G*, the partition PI’

=

( { Y > ? {Z1>>

{z2>,

has C(lU,’) = 4

=

C(!P1) = C3(G)

Figure 4.8.1

2. p

=

2. The optimal partition y 2

=

,

{Zl, z 2 , z 3 ,

a,, a,, 0))

yields C(y/,) = C,(G) = 6.

For the graph G*, the partition Y2’

=

({Y},

{ZI,

z,,

a,, a,},

z3,

has C ( y 2 ’ )= 5

3. p

=

< C ( y 2 )= C,(G)

and

D(Ul,’) = 2.

1. Since no articulation point exists in G , one optimal partition is y3=

( ~ ~ l ~ z l ~ Z Z ~ z 3 ~ ~ ~ ~ a 2 , u ) )

and Cl(G)

=

C(UJ

=

7.

130

4. Elimination of Variables in Blocks

For the graph G*, the partition

u;’ = (Y, 21,

z2, z3,

a,, a,})

has C(Yi) = 6

< C(!P3)= C,(G)

and

D(U;’)

= 0.

EXAMPLE 4.8.2 Consider the graph G of Fig. 4.8.2a and the graph G* of Fig. 4.8.2b, which is descendant from G w.r.t. { D ~ u,}. , Consider p = 1, 2, 3. 1. p = 3. A minimal cost partition for G is Y I =

({%I>{%I,

{v5>,

{%I, {%I, {u3>,

(bJ

Figure 4.8.2

{u7>7

(4)

4.8

131

Descendance, Initial, Separating Set Theorems

Then C3(G*) = 3

< C3(G)= 4.

2. p = 2. An optimal partition for G is

while Yl* is again an optimal partition for C*. Then C2(G*)= 3 3. p have

=

< C2(G)= 5.

1. Since no articulation point exists either in G or in G*,we C,(G*)

=

7

< C1(C) = 8.

Theorem 4.8.3 (Initial Theorem) Let G CJG)

=

( V , E ) be a graph, Y c V, p be a nonnegative integer, and

2 k. If

( a ) GY is a graph descendant from G w.r.t. Y ; ( b ) there exists a regular ordered partition (Y, , Y, ,. . . , Y l ) of the set Y, such that, f o r j = 1,2,. . . , I,

I yjI+lr(yil

{ y 1 7 y ~ , . . . 7 y i - 1 } ) 1 ~ ~ 7

I r ( y j I { Y I , y2,.. .

9

Yi-l})

II p;

then there exists an ordered partition Y E S$, beginning with the blocks Y l , Y 2 , .. . , Y , , such that C(Y) = C,(G).

PROOF.Consider the ordered partition Y = (Yl, Y , , . . . , Y , , YL,,, . . . , Y,) for the graph G, where Y’ = ( Y l + l ,. . . , Y,) is an optimal ordered partition for the graph GY [i.e., C(Y’) = C:(G,) and D ( Y ’ ) 5 p]. Then Y E S,/’ and, by Descendance Theorem 4.8.2, we have C( Y ) = ma x (

max

j € { l ,2, ..., 1 )

(1

yjl+lr(yj\{yl,yz,...,y~-l})l),

C p d ) 5 max(k C,(G)) = C,(G)

Some interesting special cases of this theorem are developed in the examples which follow.

132

4. Elimination of Variables in Blocks

EXAMPLE 4.8.3

Consider the graph G of Fig. 4.8.3 and p = 4. If C,(G) 2 5, by Theorem 4.8.3 there exists an optimal ordered partition beginning with the block (v}.

Figure 4.8.3

EXAMPLE 4.8.4

Consider the graph G of Fig. 4.8.4 and p = 4. If C,(G) 2 6, by Theorem 4.8.3 there exists an optimal ordered partition beginning with the block h

9

v2>.

Figure 4.8.4

The separating set theorem, illustrated in Section 3.8, gives the conditions which a set 2 c V must satisfy, when vertices are eliminated one by one, so that the problem of finding a minimal dimension ordering can be split into two smaller subproblems. In this section it is shown that exactly the same conditions are required for a set Z(1 2 [ 5 p), when the elimination of blocks is considered.

4.9

133

Bounds and Algorithms: Some Hints

Theorem 4.8.4 (Separating Set Theorem) Let G

=

( V , E ) be a graph, and consider a nonnegative integer p. Consider

a partition of the vertex set V into sets Y, Z, and 2,such that 2 is a separating set of G (I Z I 5 p ) and Y and Z are not adjacent.

0(a) GY and GZ are descendant from G w.r.t. Y and Z , respectively; ( b ) ,Z is fully connected both in GY and GZ;

there exists an ordered partition !€' E S ; , the last block of which is Z, such that C(!€')= C,AG).

PROOF.Let ( Y , , . . . , Y l , 2) and (Z,,. . . , Z,, 2) be two optimal ordered partitions for GZ and Cy , respectively (see Theorem 4.7.4). Then, by Theorem 4.8.2

c,m 2 max(C,(Gz),

C,'(GY))

Since the set r ( Y j I { Y , , . . . , Yj-,}) in GZ is equal to the set r ( Y j I { Y , , . . . , Yj-,]) in G and the set r(Zj] Z,, . . . ,Z j - J in GY is equal to the set r(zjI { Y , , . . . , Yl,Z,, . . . ,Zj-,}) in G, the ordered partition !P= (YI,. .. , Yi,

z,,. . . ,z ,,Z)

is optimal for G, namely, !€' E S$ and C ( Y ) = C,A(G).

EXAMPLE 4.8.5 When p

2 3, Theorem 4.8.4 applies to Example 3.8.1. 4.9

Bounds and Algorithms: Some Hints

The properties of the block elimination process, derived in Sections 4.5-4.8, enable us to find some algorithms for determining an optimal (or good) ordered partition, as was done in Chapter 3 for the elimination

134

4. Elimination of Variables in Blocks

process. However, as pointed out in the introductory section of this chapter, no computational work has been undertaken. Hence this section gives only some hints about the possible extensions of the algorithms of Chapter 3. ALGORITHMS. Separating Set Theorem 4.8.4 suggests the I. HEURISTIC following heuristic algorithm. Consider the graph G and 3 nonnegative integer p. Select a separating set ,Z such that I Z I 5 p, if any. Then use the dichotomic procedure (see Sections 3.4 and 4.7) as if Z were fully connected, repeat the same construction on the two graphs obtained from the original one, and so on, until each graph generated in the procedure is a-connected with ct > p, The choice of .Z can be made according to different criteria; for example,

(a) choose a set ,Z with minimal cardinality (in this case this algorithm may be regarded, in some sense, as a generalization of the minimum degree algorithm of Section 5.2); (b) choose a set Z with minimal number of missing edges (in this case this algorithm may be regarded, in some sense, as a generalization of the minimum deficiency algorithm of Section 3.2). Note that these algorithms are considerably more difficult to implement than those of Section 3.2, since they require the determination of separating sets. PATHALGORITHMS. Taking into account Theorems 4.6.4 11. OPTIMAL and 4.6.5, it is possible to adopt the algorithm described in Section 3.3, modifying only points l(b) and l(c) as follows.

l(b) Two nodes w y t and wy.. are joined by an arc directed from wyt to wy,,, iff the graph G Y , ,, obtained by eliminating a block Z = Y" - Y' in GY,, is such that I I'(Z I Y ' ) I 5 p and the section graph of Gyt w.r.t. 2 is connected. 1(c) A cost I 2 I I T(ZI Y ' ) I is associated with the arc ( w y J ,wy,,).

+

Note that the number of arcs of the network 59 depends upon the nonnegative integer p considered.

111. UPPERAND LOWER BOUNDSTO THE p COST.For the block elimination process good upper bounds are obtained via the heuristic algorithms of point I. As for lower bounds, note that, by Theorem 4.6.1 and 4.6.2, C and, u fortiori, all its lower bounds are lower bounds to C,. In addition, specific lower bounds for each p can be determined by Descendance Theorems 4.8.1 and 4.8.2.

4.10

135

The Correspondence Theorem

FASTALGORITHMS. Finally, exact and, at the IV. EXACT,REASONABLY same time, reasonably fast algorithms can be constructed, as seen for the elimination process in Section 3.1 1, by using the invariance, final, initial, and separating set theorems, as well as the upper and lower bounds of point 111.

4.10 The Correspondence Theorem

The correspondence between the secondary optimization problem of a nonserial problem with memory limitation and an optimization problem for a block elimination process in a suitable graph is now shown. The two cases (a) uz, = u = const, (b) a3 = @I (a? = a positive integer), are treated in this order.

Theorem 4.10.1 [Correspondence Theorem for Case (a)] Consider the nonserial unconstrained problem

min f ( X ) = min X

with ax,

=

X

C fi(Xi)

icT

a = const. Consider the graph G V=X

and

=

( V , E ) with

E=L,

namely, the interaction graph ( X , L ) of the problem 9. Then, f o r any nonnegative integer p, (1) q L ( 9 ) = B,(G)? (2) C , ( 9 > = C,(G).

!P = ( Y , , Y 2 , .. . , Y M ) ,it is clear that y ( Y , ) = obtained from 9by the elimination of the block Y,, then PROOF. Since, given y

=

r(Y,) and Gyl equals the interaction graph of the problem Ny)=

W) ;

C ( y ) = C(U');

D ( y ) = WY).

The proof follows taking into account Definitions 4.4.5, 4.4.6, 4.5.8, and 4.5.9.

rn

136

4. Elimination of Variables in Blocks

Thus, given an optimal ordered partition for the graph, the same ordered partition solves the secondary optimization problem.

Theorem 4.10.2 [Correspondence Theorem for Case (b)] Consider the nonserial unconstrained problem 9 min f ( X ) = min X

X

with

az. =

OQI

C fi(Xi)

icT

(aj= a positive integer).

Consider the graph G = (V, E ) with

v = {Xll, XI2,. .. E = ( ( 0 , W ) I u E {xi’, x:,.

. . ,x;z,. . . , x n l , x,2,. . . , G S } ; . . ,e), w E {xi’, x j 2 , . . . , qj},

x 1 a * , x21, x 2 2 , .

7

and ( x i , x j ) E L}

i f j

u I*({X11,

x12,.

. . ,GI}) u

* * *

u Z*({xnl,x 2 , . . . , xg}),

namely, the graph obtained from the interaction graph ( X , L ) of the problem 9by substituting for each vertex xi a fully connected subset of vertices { x t , xi2,. . . , x f i } and fully connecting the subsets { x t , x:, . . . , g j } and {x,l, xi2,.. . ,qj} if x i and xj are adjacent. Then, for any nonnegative integer, (1) B / & ( P= )

qm,

(2) C/LP>= C,(G).

PROOF.Consider the set S, c Sp of all ordered partitions such that all the vertices of { x t , x?, . . . , x ; j } belong to a single block ( i = 1, 2,. . . , n). There exists a one-to-one correspondence between an element !P E S, and an element y E S,, obtained from !P by substituting for each set of vertices x t , xi2,.. . ,xri the variable x i . It is easy to check that (see Definitions 4.4.2, 4.4.4, 4.4.7, 4.4.8, and 4.5.54.5.7) N Y ) = B(Y/);

C(Y)

= C(W;

m y ) = D(W.

Hence there also exists a one-to-one correspondence between the sets 3,” and S,”. Finally, to prove the theorem, it must be shown that, for any nonnegative In integer p, there exists an optimal ordered partition belonging to 3”.,

137

4.11 Parametric Unconstrained Problem

fact consider an optimal ordered partition Y E Svp -

s$,

if any. Let

and let Y j be the first block such that, for some i, we have

Y j n {xt,x?, . . . , x?} # (ZI and

. . , x?> $ Y j .

(Xt,. , : x

Then deduct the ordered partition

Y'

=

( Y l , .. . , Yj-1, Yj Y3+1 .

-

u {x?, x?,. , xy}, * *

{x.1 2 7 32.2 2 9 . "

,x y } , . . . , Y,

-

{x?, . ,:x

. . ,x?}).

We have

B y ) < B(Y);

C(Y')

5 C(!P);

D(Yl) 5 D ( Y )

since

Yj u r ( Y j I {Y,, . . . , Yj-,}) = (Yj u {x?,,:x . . . , X T t } ) u r ( Y j u {x?, x:,. . . , x p } I { Y l , . . . , Yj-,}) is a subgraph of and G Iterating this reasoning, it is possible to find an optimal ordered partition belonging to The proof of Theorem 4.10.2 is constructive in nature and allows us, given the optimal ordered partition for the graph, to determine an ordered partition y which solves the secondary optimization problem.

4.11

The Parametric Unconstrained Problem

Consider the parametric unconstrained problem minf(x) X-P

= min

C &(Xi)

X - P if?'

138

4. Elimination of Variables in Blocks

is a set of discrete variables, a,, being the number of assignments of the variable xj; xi c x; T = (1,2,. . . , t > ; P = ( x n - p + l , .. . , x,} c x;

1 P 1 =p.

The optimization procedure for the solution of this problem consists in fixing one ordered partition y = ( Y , , Y, ,. . . , Y,) of the variables of the set X - P; eliminating the blocks Yj (j= 1,2,. . . , m) according to the procedure described in Section 4.2,thus obtaining a function of P; and, finally, operating backward in order to obtain the optimizing assignments Y,*(P), Y,*(P),. . . Y,*(P). The secondary optimization problem has the same meaning as before and consists in determining an optimal ordered partition y of the variables of the set X - P. In particular, Definitions 4.4.1-4.4.8 extend to this case. Also in this case, the solution of the secondary optimization problem, for a problem in standard form, can be obtained by the solution of an optimization problem for a block elimination process in a suitable graph. For simplicity, only the special case a,, = a = const is considered. 9

Theorem 4.11.1 (Correspondence Theorem) Consider the parametric unconstrained problem in standard form %p: minf(X)

=

x-P

min

C j;(Xi),

X-P icT

with

u, = a = const. Consider the graph G

=

(V, E), with

V= X

and

E

=L

u I*(p),

namely, the graph obtained from the interaction graph (X,L ) of the problem 9fully connecting the vertices of the subset P . Then,for any integer ,u 2 p ,

(0 B , ( P )

= B,(G),

(2) C , ( P ) = C,(G).

PROOF.Consider the set Syp c S,/ (p 2 p ) of all ordered partitions !P= ( Y l , Y , , . . . , Y M ) such that YM 2 P. There exists a one-to-one correspondence between an element !P E 3,” and an element y = ( Y l , Y 2 , .. . , Y, - P ) E S(. Since the problem 9

4.12

139

Concluding Remarks

is in standard form, we have

It remains to be shown that there exists an optimal ordered partition in In fact, by Final Theorems 4.7.2 and 4.7.3, there exists an optimal ordered partition Y’ = (Zl, Z z , .. . , Z,), with Z , =I P, and, if Z , = P, since the p r o b l e m 9 is in standard form, r ( Z L - , I { Z ,, Z , ,. . . ,Z,-z}) = P and the ordered partition Y” = ( Z ,, Z , ,. . . , ZLPlu P) E 3,” is better than Y‘.

s,”.

The proof of Theorem 4.11.1 is constructive in nature and allows us, given the optimal ordered partition for the graph, to determine an ordered partition y which solves the secondary optimization problem.

4.12

Concluding Remarks

The two dynamic programming procedures, (a) the elimination of variables one by one, and (b) the elimination of variables in blocks, are now compared. When no memory limitation exists, Theorem 4.6.3 shows that the number of functional evaluations required by an optimal ordered partition is always smaller than the one needed for procedure (a). However Theorem 4.6.2 demonstrates that both procedures (a) and (b) need a computing time of the same order of magnitude. Procedure (b) turns out to be the only feasible one when there is a “real” memory limitation (,u < D).Usually, however, the satisfaction of the memory requirements imply a cost, in terms of additional computing time, which increases with decreasing p (see Theorem 4.6.1). This additional computational burden, in turn, may often be reduced, using a more sophisticated technique, “the multilevel elimination procedure,” which will be described in the next chapter.

Mu1ti1eve1 El imi nation Procedures

5.1

Introduction

Classical dynamic programming (as specified in Chapter 4 for nonserial unconstrained problems) consists, essentially, in reducing the solution of the entire problem to the exhaustive solution of a sequence of subproblems. However, within each subproblem, there exists the possibility of processing exhaustively only a subset of the variables and, for each of their assignments, of using decomposition again, if possible, to eliminate other variables, and so on. This is the characterizing feature of the multilevel elimination procedure introduced in this chapter. In this new procedure, the power of the principle of optimality is enhanced in the sense that, for a given memory limitation, a greater degree of decomposition may be attained, and therefore the computing time is, in most cases, considerably reduced. This procedure includes the previous procedures as special cases and is, of course, more complex from both the primary and secondary points of view. This chapter is organized as follows. The first part (Sections 5.2-5.4) 140

5.2

141

Multilevel Elimination Procedures

describes the primary procedure and the secondary optimization problem. Then (Sections 5.5-5.8) the multilevel elimination process is defined and a limited number of its properties is given. In particular, Section 5.7 contains an example that stresses the advantages of this procedure. Finally (Section 5.9) the correspondence theorem is demonstrated and a further improvement in the primary procedure, when the number of assignments of the variables differ, is suggested. It is worthwhile noting that, from the outset, only parametric problems are considered in this chapter. This is so because of the nature of the multilevel elimination procedure, which is closed with respect to parametric problems since, in either case (namely, starting with either a parametric or a nonparametric problem), the lower level subproblems generally turn out to be parametric in form.

5.2

Multilevel Elimination Procedures for the Solution of the Primary Problem

Consider again the parametric unconstrained problem minf(X)

= min

x-P

where

9:

C fi(Xi),

X-P iel'

x = { x , , x z , . . . , x,}

is a set of discrete variables, azi being the number of assignments of the variable x j ; xi c T = {1,2,. . . ,t > ;

x;

P

= (X,a-p+l,.

. . , x,}

c

x; I P I = p .

The basic step of the procedure, applied to the original problem now described; it consists of two distinct substeps.

9, is

DECOMPOSITION. Select a nonempty set Yl c (X - P) a. HORIZONTAL and split the objective function as follows:

fW> = kl( Yl

9

Y(Y1))

+ El(X

-

where

and

TI= {i: X i n Y, f a } .

YlL

5. Multilevel Elimination Procedures

142

Then obtain, as indicated in substep b below, hl(Y(Y1)) = min k , ( L Y(Y1))

(5.2.1)

Yl

and the optimizing assignment Y,*(y( Y,)). The remaining problem (5.2.2)

is also a parametric unconstrained problem of the same form as the original problem. b. VERTICAL DECOMPOSITION. Problem (5.2.1) is now considered. Select a nonempty set W, c ( Y , u y(Y,)). Then rnin k,( Y, , y ( Y,)) = min Yl

min k,( Y, , y( Y,))

YlnW, Yl-W,

where k,’

=

C

fi(Xi);

i€Tl’

kit =

C

fi(Xi);

i e T -Tl’

and T,‘

=

( i : Xi n (Y,- W,) f @}

c

T,.

The solution of this optimization problem implies three operations. (1) For each assignment

P, of W, solve problem PI:

min k,‘ (Y, -

w,, y ( ~ -, w,) - w,, W,);

Yl -w1

thus obtaining the function h,‘(y( Y , - W,) - W,, PI)and the optimizing assignment of Y,- W,. Problem 9’ which ,,is of the same form’ as the original problem 9, can be solved, in turn, by a multilevel elimination procedure. Note that problem 9, has fewer variables than 9but must be solved I SWl I times.

5.2

Multilevel Elimination Procedures

143

(2) The function h:’(y(Y,) u W,) is constructed, by summing, for each assignment Fl of W,,the function h,’ and ki‘. (3) Finally the minimization of hi’ w.r.t. Y, n W , is performed, thus obtaining h,(y( Y,)) and the optimizing assignment Y,*(y( Y,)). From the point of view of the memory capacity needed, it is convenient to adopt a technique, which is a generalization of that described in Sections 2.5 and 4.4, consisting of storing only the current optimal value of Y, n W, (and the corresponding value of hi’) for each assignment of y( Y,). This point will be discussed in detail in the next section. Hence the basic step of the multilevel elimination procedure first described is completely specified by assigning the two nonempty sets Y, and W,. The problem (5.2.2), originating from the horizontal decomposition, equals that obtained f r o m 9 by the elimination of the block Y, (see Chapter 4), and is considered to be at the same level a s 9 .

Figure 5.2.1

On the other hand, the set of problemsg1, which derive from the vertical decomposition when Y, - W , f 0, is considered to be at a lower level. The existence of this new set of problems, introduced with the aim of further decomposing each subproblem of the block elimination procedure, is the characterizing feature of the multilevel elimination procedure.

144

5. Multilevel Elimination Procedures

The procedure continues by again applying the basic step to problems

9, and (5.2.2), and so on. The problem tree depicted in Fig. 5.2.1 shows the problems, at all levels, generated by this procedure. Formally, a multilevel elimination procedure cp can be defined recursively by assigning an ordered set of triplets for the problem 9:

v>),

9 = ((YI,w,, 9 4 , (YZ,w,,v”,. . . , (Ym,w,, where

(a) (Y,,Y2,. . . , Y,) is an ordered partition of X - P ; (b) Wjc(Y,U>)(YjIYi, Y2,..., Y,-i>),( j = l , Z . . . , m ) ; (c) 413 is a multilevel elimination procedure for the set of parametric unconstrained problems As already noted, whenever Y, - W, = @, no lower level problems ,p, are created, and q9 = q0. Moreover, since W, f @, the number of levels is finite and, consequently, the set of all multilevel elimination procedures, denoted by S,, is also finite. Finally, note that the procedure consisting of the elimination of variables in blocks is a special case of this procedure: In fact an ordered partition y = (Y,, Y2,. . . , YmJis exactly “equivalent” to the degenerate multilevel ordered partition

v = ((L Y, u Y(Y1L Yo), ( Y * ?yz u Y(Y2JYd.YO),.. . , (Yrn, y,n, v”). 5.3

An Example

The nonserial unconstrained problem, solved in Section 4.3 by eliminating variables in blocks, is now considered by means of the multilevel elimination procedure. This allows us to draw many interesting conclusions from a comparison of the two methods. For convenience the formulation of problem ,9 is‘repeated: 4

minf(X) X

and

( T ~= (T ==

= min

X

Cfi(X*), i=l

2. The components fiare given in the following tables.

5.3 An Example

145

0 0 0 0 1 1 I

0 0 1 1 0 0

0 1 0

1 1 3

1 0 I

14

1

0

12

1

1 0 0

15 6

1 1

1 1 1

1 0 0 1 1

1 0 1 0 1 0 1 0 1

x4

xs

x,

xs

f3

0 0 0 0 0 0 0 0 1 1 1

0

0

0

1s

0 0 0 1

0 1

1 0 1 0 1 0

16 5 15

0 0 0 0 0 0 0 0 1

I 1 1

1

I

1 1

1 1 1

0 0 0 0 I

1

1 1

0 0 0 0 1 1 1

1

1

1

0 0 1 1 0 0 1

1 0 0

1

0 1 0 1 0

1

1 0

1

I

'

2 9 12

5 3 0

0 0 0 0 0 0 0 0 1 1

1

0 0 0 0 1 1 1

1 0 0 0 0 1

0 0 1 1 0 0 1 1 0 0

1 1 0

2

1 1 1

I1

1

1 1

17

1

1

I 1

0 0

0

0

0 0 0 1

0

1

11 4 2 2

0 0 0 0 0

0

1

1

1 0 0

0 1 1 1 1

1 1

1 1

0 0 0 0

0 0 1

1

1

0

1

1 1

1 1 1

0

6 3

3

1 21 18 6

1

1

1

1

0 1 0 1 0 1 0 1 0 1 0 1 0

4 6 8 7 14 5 9 15 9 3 5 21

4 3 20 19

1 0

1

0 1

8 10 0

0 1 0 1 0 1 0 1 0 1 0 1 0 1

17 10 0 2 4 19 2 6 3 1 3 5 10 I

146

5. Mu Iti level Elimination Procedures

The interaction graph of problem tpis again reported in Fig. 5.3.1. Three different multilevel ordered partitions-y, , vb, and v,-are now considered.

Figure 5.3.1

The problem tree is illustrated in Fig. 5.3.2 and the interaction graphs of problems ,PI and .P2are reported in Fig. 5.3.3a,b.

Figure 5.3.2

Consider the first basic step specified by Y, = {xl, x 2 ,x3} and W , = {x4,x 8 } . The horizontal decomposition splits the objective function f into kl( 1‘

>

El(x

y ( ‘1))

= fi(xl

-

=f3(x4,

yl)

Y

x2

f

,

x4

3

x8)

x6 x7 > XS]

+

f2(x2

f f4(x4

2

x3

x4

9

x8)

9

x5 9 x6

3

x8).

7

Then the vertical decomposition is applied to the problem

;

5.3 An Example

147

Figure 5.3.3

we have

kl‘ = k ,

k;‘

and

= 0.

Moreover, since

Y, n W, = {x,, x2, x3) n {x4, x8>= 0 , then

Then h, is obtained directly by solving problem min h , 2 2 ,z3

,Fl; namely,

k,(x, > x2 x3, %, 2s) 9

1

for all assignments of x4 and x8 Let x4 = 0 and x8 = 0. Since

problem .p,is solved, in this case, by successively eliminating the blocks

148

5. Multilevel Elimination Procedures

{xl) and {x2,x3>,and no further level is created. Then the solution of the problem, minh(x1,

x2 Y

07 01,

(x,l

gives the following table.

0

3

0

1

1

1

The problem min {x2

yields and xz*(070) = 1 ;

x3*(0,0)

=

1;

and, by means of the backsolving operation,

X1*(O, 0) = 1. Thus the first row of the table hl(x4,x8) is constructed:

and the table hll(x2, 0,O) can be canceled. Successively let x4 = 0 and x8 = 1. Then the solution of the problem

5.3

An Example

149

gives the table

and the problem min hll(X2 0 , 1) 3

(22

.%}

+f,(x,, x 3 , 0 , 1)

yields

hl,(O, 1) = hl(0, 1)

=

5,

and

x,*(O, 1) = 1 ;

x,*(O, 1)

= 1;

1) = 0.

X,*(O,

By repeating the same calculations for the two remaining assignments of the variables x4 and x 8 , the table h, is completed:

0 0

0

5

1

1

5

1

1 1

1 1

0

10

0

0

1

7

1

0

1 0

0 0

After completing the first basic step, the problem remaining (at the first level) is

min

hl(x4

9

x8>

k4,z5,z6,z7,z81

+

f3(x4

9

x6

9

x7

9

x8)

+

fP(x4

9

x5

9

x6 > x8>*

The new basic step is specified by Y, = {x4,x 5 , x , , x,, x 8 } , that is, the set of all remaining variables; and W , = {x,}. Thus

+f 3 + A ;

k,

=

h,

h,

=

min h,'(x,)

E2 = 0;

k,'

= k,;

k: = 0.

Hence {Zd

=

min {Zd

min ~Z4,Zg,Z,

k,(x, ,x 5 , x6, x, ,x8). ,za}

150

5. Multilevel Elimination Procedures

The function h2'(x6) is obtained by solving, for all assignments of the problem PZy namely, ha

min 9%

327

$8

1

k2(~4 x5,

x6y

x7,x,>.

Let x, = 0. Since

problem q2is solved by successively eliminating the blocks { x , } and {x4,x, , x,}. In this case also, no further level is created. Then the solution of the problem

gives the table

0 0

0

0

0

8 0 1

1

1

1

2

0

1

1 1

The problem

yields

h,,(O) = h,'(O) = 10, and x4*(0) = 1;

x5*(0)= 0 ;

x,*(O)

=0;

x8*(0) = 1.

The table h21(~4,0, xa)is canceled and h2'(0)is stored as the current optimal value of h,. Next let x6 = 1. Then the solution of the problem

5.3

An Example

151

gives the table

0

0

0

0

0

1 0 I

4

1

5 3

0

1 1

1

The problem

yields

h2z(1) = h,’(I) = 7, and

x,*(l)

= 0;

x,*(l)

= 0;

x,*(l)

=

1;

x,*(l)

= 0.

Now h2’(l) is compared with the previous current optimal value of h z , namely, h,’(O), and the best value, namely, hz’(l) = 7, is retained as the new current optimal value of h, . Since G = 2, this value is also the final optimum. Then h, = minf(X) = 7, X

and an optimizing assignment is xl* = 1;

x,* = 1 ;

x3* = I ;

x** = 0 ;

x5* = 0;

X6* =

1;

x,* = 1;

x3* = 0.

This solution is obviously the same as that obtained by eliminating variables in blocks. From the computational point of view, the first basic step implies solving a2 times the second-level problem .P1, which requires c2

+ a2

functional evaluations (see Chapter 4), and no first-level optimization is made since Y, n W , = 0.

152

5. Multilevel Elimination Procedures

is computationally equivalent, disregarding the order in which the operations are performed, to the solution of the problems

and

Therefore, the global number of functional evaluations equals that needed for solving Q times the second-level problem ,q2, which requires a3

+

03

functional evaluations; i.e., the first-level optimization (w.r.t. x6) implies no additional functional evaluation in this case. The number of functional evaluations for the entire problem is then ayo2

+

Q2)

+ o(a3 +

03)

= 404,

which, for Q = 2, equals 64. The maximal dimension of an additional table, stored during the optimization procedure, is equal to 2.

Figure 5.3.4

5.3

153

An Example

The problem tree is again that of Fig. 5.3.2. The interaction graph of problem.!Pl is given in Fig. 5.3.4, while problem.9, is the same as that of the previous multilevel partition. The first basic step is specified by Y, = {xl,x z , xg} and Wl = {xz,x4}. Then k1

=fi

+fz;

Z1 = f a

+f4;

k1’

= k,;

ki’

= 0.

Hence

hl(x4, x8) = min h, (x, , x4,x8) = min hl’(xz,x4,x,) (22

}

= min (22)

{22

1

min k l ( x l , x2x3,x 4 ,x8). {21,23}

The function h:’(x,, x, ,x8) is obtained by solving, for all assignments of the variables of the set { x 2 , x,}, the parametric problem ,ql, namely, min kl(xl, %, x3,Xq,x8). (2,,z3}

Let x, = 0 and x4 = 0. The parametric problem P1is solved by successively eliminating the variables x1 and x3. Then the solution of the problem

gives the table

0

3

0

1

5

1

and the solution of the problem

gives the table

154

5. Multilevel Elimination Procedures

Summing hIl and h,,, one obtains x.9

0 1

Successively let xz = 1 and x4 = 0. We have

0

4

1

1

3

0

Then it is possible to begin the construction of table hl(x4,xe) performing the minimization w.r.t. x2:

I

hl

x4

x.9

0

0

5

0

1

5

1

0

I

x1*

xa*

x3*

1 1

1

1

1

0

5.3

155

An Example

The remaining computations are left to the reader: Clearly the resulting table must be the same as that of case a. The successive basic steps are the same as before. From the computational point of view, the solution of the problem min min kl(X1, x2, x3 > x4, xs), (22)

{z,,z,}

disregarding the order in which the operations are performed, is equivalent, from the point of view of the number of functional evaluations, to the solution of the problems minfl(xl7 x2 9 x4

9

X8)

{XI}

and min h(xz Iza ,za)

3

x3

9

x4 Y X8)

+ h,,(xz

9x4

9

x,).

Hence the number of functional evaluations for the entire problem is the same as for case a. The maximal dimension of an additional stored table again equals 2.

Figure 5.3.5

Figure 5.3.6

The problem tree is illustrated in Fig. 5.3.5, and the interaction graph of problem Pl is shown in Fig. 5.3.6. The reader can verify that the total

156

5. Multilevel Elimination Procedures

number of functional evaluations is again equal to

but that the maximal dimension of an additional stored table is only 1. In the table which follows, the number of functional evaluations, for different values of 6,of the multilevel ordered partition vcis compared with those of the ordered partitions y l , y z , and y 3 of Section 4.3. Maximal dimension Partitions

Pc W3

Wl

Wa

of an additional stored table

1 0 2 3

Number of functional evaluations

a=2

64 256 64 64

a=5

2,500 390,625

6,250 2,500

a = 10

~,OOo 100,000,000

200,000 40,000

Since, as is easily verified, y 3 and y1 are optimal (with respect to the number of functional evaluations) for p = 1 and p = 2, respectively, the table demonstrates the superiority of the multilevel elimination procedure. Note also that the number of functional evaluations of vCis equal to that of y z , which is the best ordered partition when no memory limitation exists.

5.4 The Secondary Optimization Problem The purpose of this section is to give a formal definition of the secondary optimization problem for multilevel elimination procedures. The case cz = 0 = const is discussed first. Consider the multilevel ordered partition ~1 E S, and examine, from the computational point of view, the first basic step specified by the sets Y, and W,. The minimization problem faced within the step is

5.4

157

Secondary Optimization Problem

Two cases must be distinguished. (a) Y, - W , = 0. In this case, as noted, no p r o b le m 9 , at the second level is created, and the number of functional evaluations clearly equals ulwluu(yl)las in the procedure of Chapter 4. Let {Yl*, Ylz,.. . , Yfl} be a partition of Y , - W, (b) Yl - W , f 0. such that Yli and Ylj ( i , j = 1,. . . , 6, and i # j ) are not adjacent in the section graph of the interaction graph of 9 w.r.t. ( Y , - W,). Then the solution of problem (5.4.1), by means of the procedure of Section 5.2 is equivalent, from the point of view of the number of functional evaluations (i.e., disregarding the order in which the operations are performed), to the solution of problems Q j :

= hlj(y(Yj) -

w,,wl),j = 1,2,. . . , 6,; (5.4.2)

and of the problem

where g,(P -

w1, W,) =f i ( P ) ,

i E T,’,

and ~

~

{ji : X = i n Y,i#pj],

j = 1,2 ,...,

(rl.

Hence the number of functional evaluations needed for solving problem (5.4.1) equals the number of functional evaluations for problems (5.4.2) plus, ifpl is not in quasi-standard form, the number of functional evaluations for problem (5.4.3). In fact, i f p l is in quasi-standard form, for some j

and, hence, the set of variables of problem (5.4.3) is a subset of that of problem Qlj (see the example of Section 5.3). The same considerations apply for all the other basic steps of the multilevel procedure. Then, given a multileveI ordered partition y E S, , the following indices of the computing time and the memory requirements are defined.

158

5. MuIti level El irnination Procedures

DEFINITION 5.4.1 The scalar m

B(F) =

C Pj5IW* UY(YjIY, >....Y,-J j=l

+ olWjl

where Pj is a Boolean variable, which equals 0 iff standard form, and B(p0) = 0

. B(@j, exists and is in quasi-

is called the number of functional evaluations of multilevel orderedpartition p. DEFINITION 5.4.2 The degree of B(pl), which is a polynomial in CT, is denoted by C(p) and called the cost of multilevel ordered partition ~ 1 .

where C(p0) = 0.

DEFINITION 5.4.3 The number of entries of the largest table stored during the multilevel optimization procedure is denoted by M(pl) and is given by M p ) = max(I Y(Yj I Yl,.

..

Y

yj-1)

I, WCp9);

3

M(p0) = 0.

DEFINITION 5.4.4 The degree of M(p), which is a monomial in CT, is denoted by D ( y ) and is called the dimension of muItileve1 ordered partition pl. Note that, in this case (c = Ta), ., we have D(pl) = my (I 3

r v jI

Yl,. . . , Yj-1)

where D(p0) = 0.

1, D ( V ) ) ,

5.4

Secondary Optimization Problem

159

The secondary optimization problem is now formally stated. Let

be the subset of S, formed by all those multilevel ordered partitions for which the maximal additional stored table has a number of entries not greater than a preassigned value.

DEFINITION 5.4.5 The scalar

is called the number of functiolzal evaluations of the problem 9with memory limitation p (for multilevel procedures). DEFINITION 5.4.6 The integer

is called the (multilevel) ,u cost of the problem 9. Definitions 5.4.5 and 5.4.6 specify the two versions of the secondary optimization problem for the multilevel elimination procedure when c~~~ = c ~ . When this assumption does not hold, Definitions 5.4.1 and 5.4.3 are replaced by the following. DEFINITION 5.4.7

B(po) = 0.

DEFINITION 5.4.8

and the first version (namely, the one specified by Definition 5.4.5) of the secondary optimization problem makes sense.

160

5. Multilevel Elimination Procedures

In the special case

cZ. = a o ~ t

(aj= a positive integer),

(5.4.4)

B ( q ) is again a polynomial and M(p) a monomial in 0. Hence Definitions 5.4.2 and 5.4.4 are meaningful, and both versions of the secondary optimization problem are defined. When Eq. (5.4.4) does not hold, see Section 2.5.

5.5 The Multilevel Elimination Process

Given an undirected graph G = ( V , E ) , without self-loops and parallel edges, a new more complex elimination process, the multilevel elimination process, which generalizes the block elimination process (see Section 4.5), is now defined. This process combines block eliminations and deletions of sets of vertices in a certain sequence which is specified by a multilevel ordered partition @. It consists of the ordered set of triplets @=

((Yl, Wl, @l), (YZ,

wz,@>“I,.. . , ( Y M , W M , @“I)

where (a) ( Y l , Y , , . . . , Y M )is an ordered partition of the vertex set V ; (b) for j = 1, 2,. . . , M , W j c (Y j u r ( Y j 1 {Yl,. . . , YjPl}j) and W j

f0; (c)

is a multilevel ordered partition for the graph

@j

Gj = (Vj, @)

with vj

= Yj

u r ( Y j ( {Yl,. . . , Yj-J)

-

w.. 3,

EJ’ = I ( V ) u Z*(I‘(Yj I { Y l , . . . , YjPl}) - Wj).

Hence this process originates, “at the first level,” the same sequence of graphs GYi, G ( Y I . Y Z } ., .. , G { y i ,yz,...,Y M }

of the block elimination procedure

but, in addition, “second-level” graphs Gj are generated, and so on.

5.5

161

Multilevel Elimination Process

The generation of such “lower level” graphs is the characterizing feature of the multilevel elimination process. The block elimination process is a special case of this one. In fact, when wj

=

Yj

u F(Y, I

{ Y l , . . . , Yj-l}),

j

=

1,2,.

.

I

,

M,

no second-level graphs are created. The set of all multilevel ordered partitions @ is denoted by So

EXAMPLE5.5.1

Figure 5.5.1

The graph G~l)l,l)2,Y3,1)6,,,l)8) is depicted in Fig. 5.5.1 b. The second-level graph C’ is shown in Fig. 5.5.2a, while Giv,) and G[l)3,t,til are given in Fig. 5.5.2b,c. Note that, since W2 = {u4, u 5 , u s, u,,,} = Y, u r ( Y 2I Y l ) ,the second-level graph G2 is not created. Finally Fig. 5.5.3 reports the graphs G13 (a) and Gr’”,,) (b). Some criterion functions, which generalize the ones pertinent to the block e!imination process, are now introduced in order to establish a ranking among the multilevel ordered partitions CD E So.

5. Multilevel Elimination Procedures

162

Figure 5.5.2

( aI

Figure 55.3

5.5

Multilevel Elimination Process

163

DEFINITION 5.5.1

where u is a positive real number and B(@O) = 1. DEFINITION 5.5.2 The integer

C ( @ )= max(1 W j I i

+ C(@j));

c(@) = 0,

is called the cost of the multilevel ordered partition @. DEFINITION 5.5.3 The integer

D(@) = max(T(Yj I { Yl ,. . . , I ' - ~ } ) , D(@j));

D(@O) = 0

i

is called the dimension of the multilevel ordered partition @. Let S$ = {@: D(@) 5 p } . DEFINITION 5.5.4 The scalar min B(@) @€S$

is denoted by B,(G)

=

B,.

DEFINITION 5.5.5 The integer CJG) =

cp

=

min C ( @ )

,,g

is called (multilevel) p cost of the graph G .

5. Multilevel Elimination Procedures

164

5.6

The Final Theorem

Theorem 5.6.1 Let G = ( V , E ) be a graph and Z any subset of V fully connected in G. Consider a multilevel ordered partition

belonging to S;. Then there exists in S; at least one multilevel ordered partition

6 = ((8, Rl,6% ( f 2 , Rz, @),. . . ,

a,R,, &)),

in which

zc FL,

PROOF.The proof is divided into parts (a) and (b). (a) Two possibilities can occur. 1. Y,u T(Y,) = V. In this case

6 = ( V , w,, dl), where @ is a suitable multilevel elimination procedure for the graph In fact, since is a subgraph of G', we have

el

B(6l)5 B(@)

and

el.

C ( @ ) 5 C(@l).

2. ( Y , u I'(Yl))& V. Then it is possible to find, using the construction which follows, a multilevel ordered partition

5.6

165

The Final Theorem

Let E l ) = ,Z and A ( , )= A = V - 2 ( A ( , )# 0;otherwise point 1 holds). If Yl c A(,),then = @. Otherwise ( Y , n ,L"'l' f 0) define Z'2'

=

r(Y 1 ) ;

A(,)= ( V - Y , ) - T(Y l ) .

Since P) is fully connected, D2)3 (2(l) - Y l ) and consequently A(2) c A ( 1 ) Also . ,F2) is fully connected in Gyl and A ( , )f 0(otherwise point 1 holds). If Y2 A(,)in GY1, the subsets F 3 )and are similarly defined. This way of proceeding continues, defining at each step J ,

+

Z(j)=

r(Yj+, I { Y , ,. . . , Yj-2}):

A(j)= (v - ( Y , u Y,

u-. u

Y j - J ) - Z(j),

until either

(i) for an integer h, Yh c A(h), or ( i i ) for an integer k , = 0. Case (ii) is now examined. Since A(k)= (3, the set yk-1

u r(yk-1 1

equals the vertex set of CfY1*y

(y1~. . . Yk-Z})

~ yk-al ~

9

and~ since ~

-

T(A'k-1) 1 { Y , , Y 2 , .. . , Y&3}) c

Z(k-I),

we have

T(A(k-1)I I { y i , y z , . . .Yk-3}>] ~

,.

both belong to the set S," and have the same B and C. (b) When proposition 2 holds, the same reasoning applies to Gyla, in which 2 is still fully connected, and to the new multilevel ordered partition ((Y25 Wza,@,"I,'

'

* 3

mu> W%,, @Fa)),

and so on, until condition 1 is met. rn Theorem 5.6.2

(Final Theorem)

Let G = (V, E ) be a graph and Z any subset of V fully connected in G . Then,f o r any nonnegative integer p, there exists a multilevel ordered partition

4 = (@l,

R , 6% (Fz, K , $9,. . . , ( F L , WL , W ) ,

belonging to S&', in which Zc

FL,

such that

B ( 6 ) = B,(G).

167

5.7 S o m e General Properties

PROOF.The proof derives directly from that of Theorem 5.6.1 when @ is optimal. H

Theorem 5.6.3 (Final Theorem) Let G = (V, E ) be a graph and .Z any subset of V fully connected in C. Then,for any nonnegative integer p, there exists a multilevel ordered partition

6 = ((L Fvl,

w,(92,

w2,

$9,. . . , (LWL, &”)),

belonging to S $ , in which Z C C

FA,

such that

C ( 6 ) = C,(G). PROOF.The proof derives directly from that of Theorem 5.6.1 when @ is optimal. H

5.7 Some General Properties

In this section some general properties of the multilevel elimination process with criterion functions B, and c, are presented.

Theorem 5.7.1 For any graph G and for any nonnegative integer p, we have

-

B,+l I 8,’

and

-

C,,, 5

c,.

PROOF.The theorem holds since

S g . 1

c

s;.

H

Theorem 5.7.2 For any graph G and f o r any nonnegative integer p, we have

8 , s B,

and

c, 5 C,.

168

5. Multilevel Elimination Procedures

PROOF. The theorem holds since the block elimination process is a special case of the multilevel elimination process. H Theorem 5.7.3

For any graph G (I)

min B(@) = min B(Y/); Y€SY

@E5*

( 2 ) min C(@) = min C(Y) = C. Q,

€S*

YE S P

PROOF.Only proposition (1) is demonstrated, the proof of proposition (2) being quite similar. The proof is divided into three parts. (a) When the optimal multilevel ordered partition has only one level, the theorem holds trivially. (b) Suppose that the optimal multilevel ordered partition has two levels. Let @ = (('1,

=

(('j17

w17

@'),

wjl7

@'),

. . (yl?f9 w;w, @")) (yjZ, wjZ7 @'),'. . (YjM,, WjMj, @'I), j = 1,2 ,... , M,

(y2>

wZ,

@,"),.

9

7

with, by Final Theorem 5.6.2, YjM,

= (WjI Yl,. . . , Yj-1)

-

Wj).

It is then possible to construct from @ a one-level ordered partition such that N @ a ) = B(@)

@a

so that case (a) applies [note that, in general, D(@,) 2 D(@)].Without loss of generality suppose @' f Do.It can be verified that, since D is optimal,

(YlM,

u Wl) =, T(Y1).

169

5.7 Some General Properties

Since ( Y 1 j U r ( Y l j \ { Y ~ ~ , . . . , Y ~ , ~ ~ ~ Wlj), ) > ) cj (= ~1 ~, 2u , . . . , ~ 1 - 1, we have, taking into account the fact that @ is optimal, B(@b) =

B(@).

By repetitive use of this construction, the one-level ordered partition @a is obtained. (c) When the optimal multilevel ordered partition @ has more than two levels, the proof follows, using repeatedly the construction described at point (b), starting from the lowest level.

c,,

Theorems 5.7.1-5.7.3 describe the shape of the functions B,, and for a given graph G, and compare them with the functions B,, and C,,. In particular Theorems 4.6.2 and 5.7.3 enable us to show that, for p 2 D ( G ) ,

- = c. c,

c,

The example which follows examines the functions C, and for a graph G which is the interaction graph of the smoothing problem described in Section 1.5.

EXAMPLE 5.7.1 Consider the graph G = (V, E ) of Fig. 5.7.1. Using Initial Theorems 3.7.1 and 3.7.2, it is easy to verify that D = 4 and, consequently, C = 5 (see also [IS]).

Figure 5.7.1

170

5. Multilevel Elimination Procedures

The following table gives the functions C , and Optimal ordered partition

c"

CP

0

16 16 12

YO

1 2

3

8

Fa

4

5

Y,

y o y 2

The multilevel ordered partitions

CP

8 6 5 5 5

c, for ,u 5 D. Optimal multilevel ordered partition @O

@I

@,

@z @a

01,and cB2 are now given.

5.7 Some General Properties

171

Figure 5.7.2

Q

(b)

@-@

9 Figure

5.7.3

172

5. Multilevel Elimination Procedures

The graphs G1, GI2, and

G17

are shown in Fig. 5.7.3a,b,c.

The graph G5 is shown in Fig. 5.7.4. Finally it is worthwhile introducing a special class of multilevel ordered partitions, the regular multilevel ordered partitions.

Figure 5.7.4

DEFINITION 5.7.1 A multilevel ordered partition di E SJ is regular if, for j = 1, 2,. . . , M , (a) the section graph of G { y l ~ y ~ ~ - yw.r.t. ~ - l l Yj is connected; (b) W j c r ( Y j - W j I { Y l , . . . , Yj-,}), if Yj - W j # 0, W j = Y j u T(Yj I { Yl ,. . . , Yj-l}), if Yj - Wj = 0; (c) Qij is regular. Let

&' c SJ

be the subset of regular multilevel ordered partitions.

Theorem 5.7.4 Let G = ( V , E ) be a graph. For any nonnegative integer p and f o r we have min B(@) = B,,(G) and min C ( @ )= C,u(G); @€8i

ad;

0

2 2,

173

5.7 Some General Properties

namely, there exists at least one optimal multilevel ordered partition in the class of regular partitions. The proof of this theorem, which is relatively easy but exceedingly lengthy and provides no additional insight into the subject, is left to the interested reader. Only two illustrative examples are given.

Figure 5.7.5

EXAMPLE 5.7.2 Considered the graph G of Fig. 5.7.5 and the nonregular multilevel ordered partition, which violates condition (a) of Definition 5.7.1 : @ = (((01, y2, 03, u4}, @l

= (({Ul),

{UI,

4,@%.. .) ({v4>, {U4? %I, @">,

{vz,

%)>@">,

< { u 5 ? u 6 , u7>,

'7),

@'I)

Then

B(@) = G Z . ~ ( 0 1 )+

. .. =

ay02

+ + $1 + ... = + 204 + ... . u2

G5

It is possible to derive from 0 the following regular multilevel ordered partition : @'

= (({Ul,

@a1

=

@bl

= (({u4},

uz),

(({ul}, { u l ( 04,

{%I?@a% @'),

9

U'J},

> {%>,@'bl),. ({.5 ({u59

7

'6

9

'7}7

u6> '7>,

u6

'

9

.)

u7},@o))

{%, u 6 > %'>, @'))

174

5. Multilevel Elimination Procedures

Then

+ 0.

B(@')= 0. * B(@,1) = o(oz

+ a3)+

*

.. + .. . = 204 + 2a3 + ...

B(@*l)+

+

O(O~

03)

*

Figure 5.7.6

EXAMPLE 5.7.3 Consider the graph G of Fig. 5.7.6 and the nonregular multilevel ordered partition, which violates condition (b) of Definition 5.7.1 :

Then B(@)= a2 I?(@')

+ -*

=

(Tya3

+ 04) + - -

= a6

+ +---. 05

It is possible to derive from @ the following regular multilevel ordered

5.9

175

The Correspondence Theorem

partition :

Then

B(@')= 0 .B(Qal)+

* * *

= 0($+

03

+ +- - = + 04)

*

05

04

-t 0 3 +

- .*.

Then, at least for d 2 2,

B ( 0 ' ) < B(@). Moreover,

D(@')5 D(@). 5.8

Heuristic Algorithms: Some Hints

The problem of finding an optimal multilevel ordered partition in a graph is exceedingly complex. The solution appears to be feasible only for graphs of a special structure. On the other hand, the study of heuristic algorithms, which, however, has not been considered in depth, looks promising. In particular, one simple heuristic algorithm follows : (a) Find, exploiting the results of Chapter 4, an optimal (or good) ordered partition ( Y l , Y 2 , .. . , YA,f)belonging to Syp. (b) Determine, for each subproblem j , a set W j according to suitable rules (for instance a vertex adjacent to all other vertices may be always included in Wj). (c) Iterate points (a) and (b) for all graphs Gj, and so on.

5.9 The Correspondence Theorem

The correspondence between the secondary optimization problem of a parametric unconstrained problem with memory limitation and an optimization problem for a multilevel elimination process in a suitable graph is now shown.

176

5. Multilevel Elimination Procedures

The two cases: (a) ox,= u = const; (b) oxj= ua, (aj = a positive integer); are treated in this order.

Theorem 5.9.1

[Correspondence Theorem for Case (a)]

Consider the parametric unconstrained problem in standard forin 9: minf(X) = min

C &(Xi)

X-P ~ E T

X-P

with

uzj = u

= const.

Consider the graph G = (V, E ) with

V=X

and

E

=L

u Z*(P),

namely, the graph obtained from the interaction graph ( X , L ) of the problem of the subset P .

9fully connecting the vertices Then, for any integer p 2 p ,

B,(p)

= B,L(G)

PROOF.Consider the set

c,(9)= c,(G),

and

sopc Sap of all multilevel

ordered partitions

such that

(1) at the first level Y,

3 P,

(2) at the second level, letting

mil {Yl, yz,. . . , Yj-1))

-

wj ,

j = l , 2 ,..., M - 1 j= My

iff Qjequals I'(R) in Gj, R being the vertex set of a connected component

5.9

The Correspondence Theorem

177

of the section graph of Gj w.r.t. Vj - Q j , YjLwj5

Qj;

otherwise Y j ~= j WjMj

= Qj.

such that, if the problem Pj is in quasi-standard form, then

I

y(zjmj[ Zjl,. . . , Zj,m,-l) in .Pj= y ( Z j Z,, . . . , Zj-l) - Uj in 9. There exists a one-to-one correspondence between the elements of Sop and S: defined as follows. (I)

First level

m

=

M;

z.= y. J

j = 1,2, . . . , M -

I’

1;

2, = Y, - P ;

u.= 3 w. 3 ,

j = l , 2 , . . . , M.

(2) Second level

( i ) If YjMj= Q j and, correspondingly,Pj is not in quasi-standard form,

m 3.= M . - 1 . 3

7

zjk = Yjk,

uj, = wj,,

k=1,2, ..., Mj-I; k = 1, 2,. . . , Mj - 1.

(ii) If YjAWj 3 Q j and, correspondingly,

9’ is in quasi-standard form,

m 3.= M3’:

zj, =

Yjk,

k

=

1,2,. . . , Mj - 1;

z j m j = Y j ~j Qj;

uj, = Wjk, etc.

k = l , 2 ,...,Mj.

178

5. Multilevel Elimination Procedures

Next it is shown that given two multilevel ordered partitions 9 E 3: and @ E sQp,corresponding according to the law stated above, we have (5.9.1) (5.9.2) (5.9.3) Equality (5.9.1) is now considered. Recalling Definitions 5.4.1 and 5.5.1 and taking into account that y(zj 1

zl,.. . , zj-l)= r(zjI {Z,,. . . ,Zjp1)),

j = 1,2,.

. . ,M ,

we have

Equality (5.9.1) is satisfied if, for j

=

1, 2,. . . , M ,

Take the following into account.

I. The graph Gj and the interaction graph Gj of 9 (j= 1,2,. . . ,M ) are such that Z(Zj - Wj) in Gj = Z(Zj - Wj) in @, and

O ( Z j - W j ) in Gj = O(Zj - Wj) in @.

9

11. Whenever the problem is not in quasi-standard form CS, = I), the “cost” of the elimination of the fully connected set Q j in the corresponding graph Gj is exactly equal to ar(zjl {z1*z2- zj-i j)-wd.

Then in both cases /lj = 0, 1, equality (5.9.4) is verified when a set of similar third-level equalities holds and so on, until, a t some level, two corresponding triplets (2,, W, , Go) and ( Z , , W, , po) are met. In this case, clwal

+ 0 = oIwal -

1,

The same reasoning applies to equality (5.9.2). The reader can verify equality (5.9.3).

179

5.9 The Correspondence Theorem

The proof follows taking into account that there exists an optimal multilevel ordered partition in SGp(by the final theorem) and an optimal multilevel ordered partition in SJ (see the procedure of Section 5.2). Since the correspondence law is rather complex, an example may be helpful for a complete understanding.

EXAMPLE 5.9.1 Consider a parametric problem 9:

the interaction graph (X,L ) of which is shown in Fig. 5.9.la. Consider the graph G = ( V , E ) derived from the interaction graph according to the cor-

Figure 5.9.1

180

5. Multilevel Elimination Procedures

respondence rule (see Fig. 5.9.lb) and the multilevel ordered partition belonging to 3,': @ =

(((xl, x2

@' = (({XI},

x3

3

9

x7

9

{x49

@'I>, ,

%},

xS

9

> x9},

{x.5,

@'I);

%}?

@,"I).

Using the correspondence rules, deduct the multilevel ordered partition belonging to S,2: p = (({xl

x2

9

9

x3, x4>? {.2

p1 = (({xl}, p2 = (((x6},

x.5>, {x69

x3>,

po)j({x4}, x9},

x 7 9

9

q'),

,

yo)?({x7>,

x6

9

x7},

p2));

{x5>,

Po>);

{x79 xS, x9},

9,")).

The graphs G,, G1, G2, and G2 are drawn in Figs. 5.9.2a,b and 5.9.3a,b. Note that the p r o b l e m q , is not in quasi-standard form (hence p1 = 1). Then B(@)= a2 * B(@') B ( ~= )

+ . 02

+

0

*

B(@2);

+ .

~ ( ~ 1 )0

B(W)

~ ( ~ 2 ) ; ~

(

~

= 302;

B(W) = 203;

= 1 )202;

~

Hence,

B(@)= B(p)

= 5a4;

Now consider case (b): OZj

= aaj,

aj = a positive integer.

Figure 5.9.2

(

~

= 2 )2a3.

5.9

181

The Correspondence Theorem

fa)

(61

Figure 5.9.3

It is possible to construct, starting from the interaction graph of problem 9, a graph G with the same rules adopted in Theorems 2.9.2 and 4.10.2, keeping in mind that , pis generally in parametric form. However, in this case, given a multilevel ordered partition E Swp,a corresponding multilevel ordered partition @ can be constructed but, not, as a rule, vice versa. Hence 8 , ( G ) 5 B p ( . P ) and c,(G) 5

C,( x5

7

+

x6)

f3(xS

9

x7

9

x8

9

x9)?

where

x=

ozj =

c

{xl, X Z , x3, x4, x5, x S , x 7 , x8, x9}; for j = 1, 2, 3, 7, 8, 9; azj = a2 for j

= 4,

5, 6;

and the memory limitation M ( v ) 5 cr. The interaction graph of problem 9 ' is shown in Fig. 5.9.4a and the graph G = ( V , E ) in Fig. 5.9.4b. An optimal multilevel ordered partition in S,l is @ =

( ( V ,iX4l7 X6l),

W) ;

= (({Xl? x2, x3,),

({x'/

9

x8

Y

x9},

{Xl, x2, x3, x42), @O>,

{x7

9

x8

2

x9

7

xG2},

({x,2, X5l, x52, x62>,b 4 2 , x;,

@'>,

x2, XG2>, @I);

182

5. Multilevel Elimination Procedures

Figure 5.9.4

then

B ( @ ) = 306

C ( @ )= 6 .

and

No corresponding multilevel ordered partition in 5F‘ can be derived, since are processed at different levels. the vertices x: and x42 (xgl and Moreover,

&(9) =07

+ us +

and

05

c,(.P) = 7.

This fact is very interesting. It suggests the necessity of splitting some ‘ in order to reduce the computational burden for variables of problem 9 its solution. In this case, for instance, the problem .PI,

minfi’(xl x2 X 9

+

is such that

f3’(x61,

9

x3

3

x41,

x62,

Xd2)

x7,

x8,

+

h’(X4’7

x4*9 X I

3

X6l3

x9>,

B;(.!P’)= 3 0 6 < &(.a),d 2 2; Ci(F’) = 6 < Cl(.a).

XS2)

Chapter 6 Constrained Problems

6.1

Introduction

The previous chapters have presented three increasingly complex different dynamic programming procedures for solving unconstrained optimization problems. On the other hand, most real optimization problems exhibit various kinds of constraints. In this chapter constrained problems of a general type are discussed, and two approaches for their solution are presented: (a) the penalty function approach in which the solution of a constrained problem leads t o the solution of an unconstrained one; and (b) the direct approach, which allows us to take the constraints directly into account and is, in general, computationally more convenient. The first approach, which is conceptually very simple, is described in Section 6.2, while the subsequent sections, which use References [7, 12, 13, 27-29] as a starting point, are devoted to the description of a systematic framework of the direct approach. As for the secondary optimization problem, it is clearly the same as before 183

184

6. Constrained Problems

when the penalty function is employed, while its definition becomes more difficult for the direct approach (see Section 6.5). In the last part of the chapter there are discussed a number of special, but important, cases in which some peculiarities of the constraints and/or components are used to improve the general procedure from the computational point of view. Finally it is worthwhile noting that, for the sake of brevity, only the procedure consisting of eliminating variables one by one is dealt with in this chapter: The extension of the block multilevel elimination procedures to the constrained case is, at least conceptually, immediate.

6.2

A Penalty Function Approach

Consider the constrained problem (see Section 1.8) minf(X)

=

min X

X

2 fi(Xi), iel"

subject to

XiE Fi,

i E T,

where

X=

h , X 2 , . . . , X n )

is a set of discrete variables, S , being the definition set of the variable

%(I sz, I = %& T = {I, 2,. . . , t } ;

T'

=

{I, 2 , . . . , t ' } ,

t' 5 t ;

and

Xic X ,

i E T.

The solution of a constrained problem can lead to the solution of an unconstrained one by a penalty function approach. Let M be a positive real number; for i E T', define

and, for i E T - T', fi'(Xi) =

{L

if X i €Fi, if X i $ Fi,

6.3

185

Elimination of Variables One by One: Procedure

and consider the unconstrained problem (6.2.1) Then, for M sufficiently large, (a) if the set F of feasible assignments of X is not empty, the solution of problem (6.2.1) equals the solution of the original constrained problem; (b) if F is empty, the minimal value of the objective function of problem (6.2,l) becomes “very large,” thus revealing infeasibility.

It is clear that the interaction graph of problem (6.2.1) equals the interaction graph of the original constrained problem. The computational features of this procedure will be discussed in Section 6.5.

6.3 The Elimination of Variables One by One: Description of the Procedure

In the previous section the solution of a constrained problem ha5 led to the solution of an unconstrained one. This section introduces another procedure. which allows us to consider the constraints directly and is, in general, more convenient from the computational point of view (computing time and storage space). Consider the constrained problem defined in the previous section and select an elimination ordering w = ( y , ,y z , . . . ,y,) of the variables of the problem. For this ordering, the problem may be solved by dynamic programming. The variable y1 is considered first. Let

TI = (i E T: X ic ({yl> u y ( y l ) ) } ;

T,’ = {i

E

T ‘ : X in { y l >f 0 ) ;

Since, taking into account the “structure” of the problem specified by the interaction graph, we have FX-{Yl) =

[fq(Yl)]x--IYl)

n

x,,

186

6. Constrained Problems

and

for y ( y l ) E

Fy(Y1);

then

Hence the first step of the optimization procedure consists in computing

and storing the optimizing assignment yl*(y(yl)) for all values of 70,) E Hl , where Hl = Kr(yl)includes Fy(yl). The problem remaining after the elimination of y , ,

subject to

X i €Fa,

ic T - T,;

Y(Y1) E HlY

is of the same form as the original problem. Hence the same procedure applies for the elimination of all the other variables. Clearly, when the set of the feasible assignments F is empty, at some step j the set K j turns out to be empty and the procedure stops. The backward part of the procedure is analogous to that described in Section 2.3. Finally it is interesting to note that the function h,(y(y,)) must be computed, according to the procedure illustrated, for ybl) E H , , which is a superset of Fy(’1).This fact is in conformity with the nature of dynamic programming, which considers at each step only those components and

6.4

I87

An Example

constraints concerning the eliminated variable. However, in some special cases, it may turn out to be useful to reduce the set over which h, must be computed, using some constraints of T - Tl.

An Example

6.4

An example of the optimization procedure described in Section 6.3 is now presented. Consider the constrained problem 5

min f ( X ) = min X

X

C fi(Xi), i=l

subject to

Xi€Fi,

i = l , 2 , . . . , 7,

where

x=

{XI, x2?x 3 , x4, x 5 , -%};

x2,

x 5 = {x5,

X6};

3 ( j = 1,2,, following tables : oj

7

=

x3};

x2=

CT =

x6 ==

0 1 2

1

0

1 1 2 2 2

1 2

0 1

2

XQ,x.5);

{xl, x.3, x*};

x4= (x8, x4, x6};

x7=

{XZ, x6};

. , , 6 ) ; and the f i s and the F i s are as given in the

I 0 0 0

iXZ,

X 1 = {XI? x2};

f, 0 0 0 0 0

0 0

0 2

1 7

1

1

8

2

1 1 1

0 0 0

1 2 0

1 4 6

1

8 4

1 1

2

2

2 2 2

2

1

0 1 1

2

2 2 2 2

2 7 5

3 9

1 2 2

10 I

188

6. Constrained Problems

I 0 0

1 2

0 0 2 0 0 0

0

1 1 1 1 1 2 2 2 2 2

4 I 5 3 2 5 4 9 11 8 13 6 1

0

0 1 2 1 2 0 1 2

1 1 0 0 0

2

0 2

2

0

0

0 0

1 1 2 2 1 1 2 2 0 0 1 1 1

0 0 1 1 1 1 2 2 2 2 2

I

F5

0 0 1 1 2

1

0 1 0 1 1 2 0 1 2

1

f4

15 2

~6 9 8

3 0

11 2 4 9 14 15 7

F6

f5 x5

xe

0 0 1 2

1 2 1 0

0

1

0 0 0 1 1 1 2 2 2

1 2 2 1 1 2 1 2 2

0 1 0 1 0 1 1 1 0 1

0 0 1 1 1 2 2

0

2 0 1 2 0 2

6.4

189

An Example

The interaction graph and the strong interaction graph of the problem are shown in Fig. 6.4.la,b. Consider the elimination ordering w = (xl, x 2 ,x3,x 4 ,x6,x6) of the variables of the set X and examine in detail the elimination of the variable xl.

Figure 6.4.1

Then Tl = (1, 2, 6 ) ;

Tl'

=

(1, 2);

The first step of the optimization procedure consists in computing min {z J E h ' p hzz,ZQ,XP)

kl(X13 xi?> x3) = h(x2 x3 x4) 7

Y

and storing the optimizing assignment for all values of

6 . Constrained Problems

190 (a)

Computation of Kl and k ,

0 0 1 1 2 2 2 2 0 0 0 1 2 0 0 1 1 1 2 2

0 0 0

0 0 0 0 0 1 1 1 1 1 2 2 2

2 2 2 2

0 0 0 0 1 1 1 1 2 2 2 2

1 1 2 2 1 1 2 2 1 1 2 2

2 2 1 1 1 1 2 2 1 1 2 2 2 2 2 1

2 2 2 2

0 1 0 1 0 1 0 1 0

1 0 1

0 1 0 1 0 1 0 1 0 1 1 1 1 0 1 1 0 1 0 1

11 11 14 14 1 1 4

4 9 9 5 5 14

9 9 5 11 11 15 15

6.4

(c)

191

An Example

Determination of hl(y(x,)) and x,*(y(xl)) f o r y ( x l ) E HI

XI

*

--

I 1 1

I

0 1 0 1 0 1

2

0

2

2 2 2

1

1 0 1

2

0

2

2

1

0 0 0

0 1

1 1 2

2 1

1

9 9 9 5 14 5 11 5 1 1 4 4

1 1 2

1 0 2

2 1 0 0

0 0

As usual, it is not generally convenient, from the point of view of the storage space required, to perform operations a, b, and c in this order, but it is preferable to adopt a more flexible procedure which will allow storing only the last table. The problem remaining after the elimination of x1 is

subject to

X i € Fi,

i € T - T,;

Y ( 4 E Hl*

The interaction and strong interaction graphs for this problem are given in Fig. 6.4.2a,b. Note that, while the interaction graph of the new problem is the x,-elimination graph of the original interaction graph, this is not true for the strong interaction graph. For the sake of brevity the operations regarding the elimination of the remaining variables are not reported here. The interested reader may verify the final result: min f ( X ) = 15, X EF

192

6. Constrained Problems

Figure 6.4.2

and XI* = 2;

x2* = 1;

x3* = 1 ;

x4* = 1 ;

x5* = 1 ;

X6* =

1.

Finally, as noted in the last part of Section 6.3, it is possible to reduce the set H , over which the function h, is computed. For instance, examining F('@a}, it is easy to check that the following assignments of y ( x , ) are not feasible :

0 0 1 1

1 1 2 2

2

1

1 0 1 0

2

1

1

0

Similarly, the list of forbidden assignments because of

0 1 2

1

1 1

0 0 0

is

Fq(53sZ4}

6.5

193

Secondary Optimization Problem

Hence, in this case, considering only F?2,'3}

and F,("33x41, the set HI reduces to

0 0

2

0

2

1

1

2

1 2

0

2

2

1

1

Of course this operation, which can produce a reduction of the computational complexity of the problem, implies, in turn, an additional computational effort. There seems to be no way of discussing the profitability of this operation in general.

6.5

The Secondary Optimization Problem

Sections 6.2 and 6.3 have presented two different approaches to the solution of a constrained problem: the penalty function and the direct approach, A comparison of these two approaches emphasizes the following differences. (a) The direct approach requires less storage space and fewer functional evaluations than the penalty function approach. (b) The direct approach, however, implies a new kind of operation: the determination of feasible assignments at each step. Hence, generally speaking, when I FiI E I SxiI , the simpler penalty function approach is preferable, while, when this condition is not verified, as happens in most cases, the direct approach seems more efficient. Clearly the secondary optimization problem for the procedure of Section 6.2 is exactly the same as previously described, since the constrained problem has led to an unconstrained one. As for the direct procedure, a precise definition of the secondary optimization problem becomes very complex. In fact, while the number of variables of each subproblem encountered during the optimization procedure can be determined as before by the interaction graph, the number of feasible assignments in each subprob-

194

6 . Constrained Problems

lem also depends upon the numerical values of the constraints. Thus, given an elimination ordering co, C(w) and D ( w ) have the same meaning as before, namely, the maximum number of variables in a subproblem and the maximum number of variables in a stored table; B(w) and M ( w ) are only upper bounds to the number of functional evaluations (and feasibility checks) and to the maximum number of entries of a stored table.

Hence an optimal ordering w , determined via the usual graph-theoretical procedure on the interaction graph of the problem, is not necessarily “optimal” for the constrained problem but is, in most cases, reasonably “good.” 6.6

Univocal Constraints

This section describes a variant of the direct elimination procedure (see Section 6.3), which applies when, at some step, a variable x can be univocally determined as a function of some (or all) interacting variables, by taking into account only some constraints. Loosely speaking, this means that the elimination of the variable x requires no optimization at all. Without loss of generality suppose that this phenomenon occurs at the first step. Let f1be a subset of Tl such that, defining

and

Kl is univocal w.r.t. y l . Then the usual optimization procedure simplifies as follows. (a) The optimizing assignment yl*(q(yl)) (b) The I Tl I constraints,

XiE Fi; reduce to the following

is deduced directly from

i E T,,

constraints:

k?,.

195

6.7 An Example

where f&

=p

HI,>

u ?(Yl>) =Wi-

{ Y l h Yl*(?(YJ))9

j E

Tl'

Then the problem resulting from the elimination of y , is of the same form as the original problem, so that this variant may be adopted a t any step, if possible. Note that in this case, the optimization procedure is not, in general, completely determined when the elimination ordering w is given. In fact, at each step j in which this variant is employed, the set Fj must also be specified. Finally note that the elimination of a variable according to this variant produces a minor filling up of the interaction graph, as the example of Section 6.7 shows. The secondary optimization problem of Section 6.5 loses some of its meaning, each class of problem requiring, in principle, an appropriate secondary optimization problem. An important class of optimization problems with univocal constraints is that of dynamic systems: Section 6.8 is given over to an examination of these systems. 6.7

An Example

Consider the optimization problem of Section 6.3, with the additional constraint x1 = x3,

namely,

0

0

1

1 2

2

196

6 . Constrained Problems

and the same elimination ordering o = ( x l , x2,.. . , x,). Note that the interaction graphs are again those of Fig. 6.4.la,b. The elimination of x1 can be performed, in this case, using the variant described in the preceding section. Then

and

Rl = F,. Hence, (a) the optimizing assignment is given by

I 0

0

1 2

2

1

and (b) the new constraints and (c) the new components are I

Hl,%

0 1

2

0 0 0 1 1 1

2 2 2

0 1

2 0 1

Xa

X3

0 0 0

0

1 2

1

2 2 2

H 1 . 6

h1,2

1 8

5 9 10

x3

x4

1 1 2 2

0 1 0 1

2 0 1

2

At this point, in order to reduce the storage space it is convenient to merge and H l , 2 , the two components hl,l and hl,2 and the two constraint sets

197

Dynamic Systems and Other Applications

6.8

obtaining HL ' hi' x2

x3

0

0 1

0

0 1 2

2

5 9 9 11

2

15

2

The interaction and strong interaction graphs of the new problem are shown in Fig. 6.7.la,b: They are subgraphs of those of Fig. 6.4.2a,b. The elimination of the remaining variables is then performed using the general procedure, thus obtaining minf(X) = 29, XEF

and X I * = 2;

x2* = 0 ;

x3* = 2;

x** = 1;

xs* = 0 ,.

xs* = 2.

Figure 6.7.Z

6.8

Dynamic Systems and Other Applications: The Block Diagram Representation

The aim of this section is to state a number of oft-cited problems dealing with dynamic programming, which are usually and usefully solved by adopting the variant of the optimization procedure described in Section

198

6. Constrained Problems

6.6. Moreover, in connection with each problem, the block diagram representation has often been employed [7, 12, 13, 27, 291 for visualizing the “structure” of each problem and attempting to introduce, in an embryonic stage, the secondary optimization problem.

a. Dynamic Systems The simplest problem regarding the optimization of dynamic systems can be formulated as

c Let

4 +f i . ( Z T > ,

(6.8.1)

t = 1, 2,. . . , T - 1.

(6.8.2)

T-1

min

(21. 22

,.... Z p ,

u1, u2 ,..., U T - J

>

t=1

subject to

z,+, = gt(Zt, u,),

The discrete variables z, and ut are, respectively, the “state” and the “control” (or “decision”) at time (or stage) t. Constraints (6.8.2), which determine the dependence of the state at a given stage upon the state and the control at the preceding stage, are called “transition” (or “transformation”) equations. At each transition a cost f t ( z t , ut) is associated. The term fT(zT)

Figure 6.8.1

6.8

199

Dynamic Systems and Other Applications

is the terminal cost [in some problems the variable z1 (initial state) or the variable zT (final state) or both are preassigned]. The interaction graph, strong interaction graph, and block diagram of this problem are shown in Fig. 6.8.la,b,c. This problem can be efficiently solved by eliminating the variables according to the ordering o = ( z T ,UT-1, Z T - 1 , . . . , u l , zl), and by using the variant of Section 6.7 for z T ,z T P 1 ,. .. , z2 and the general procedure for the remaining variables (see the example of Section 6.9). A more complex optimization problem occurs when the functions ft and g, depend not only upon the current values of the state and the control but also upon some states and/or controls of the previous stages (optimization of dynamic systems with time delays [30, 311). Furthermore, the extension to optimization problems for the nonscalar case (zt and vectors) is immediate. b. Other Applications. The optimization problem (6.8.1) having constraints (6.8.2) is frequently called, with reference to its block diagram representation, a serial (constrained) problem. Also considered in the literature [7, 12, 131 are some particular problems which arise in many fields of engineering (mainly chemical) and are called nonserial problems because of their block diagram representation. Four elementary “nonserial structures,” taken from Nemhauser [12], are now given: In connection with each of these, the interaction graph, the strong interaction graph, and the block diagram are provided. The determination of efficient elimination orderings, which exploit the variant of Section 6.6, is left to the reader. BRANCH”STRUCTURE (See Figs. 6.8.2a,b and 6.8.3.) 1. “DIVERGING

c fib$,4)

L-1

N-1

min X

+fN(Z,Y)

i=l

+ 1A r k r ,

U i r ) +fLr(zLr),

i=l

where

x = {zl,. . .,

ZA\TV,241,.

. . , U‘\T-l,

zl’,.

. . , ZL‘, U l l , . . . , U L } ,

200

P

6. Constrained Problems

6

P

6.8

Dynamic Systems and Other Applications

4

t

‘2

201

6. Constrained Problems

202

and 1 5M

< N - 1.

“CONVERGING BRANCH”STRUCTURE (See Figs. 6.8.4a,b and 6.8.5.)

2.

min X

subject to

where

and 1<M

+fdZ3 > %> +f4(z4),

where OIzi,ui53,

i = 1,2,3,

1 5 z4 5 5,

and thefi's are specified by the following tables.

fi

\ z1

f i

\

u1

0

1

2

3

5 9

6 3 5

8 6 6 3

3 7 4 2

2 11

4

22

0 1 2 3

u2

0

1

2

3

6 7 5 6

4

6 I 5 8

4 3 5 9

3 5 I

210

6. Constrained Problems

f3

\ z3

"3

I

0

1

2

3

0

0

1

2

1 2

5

6

I

3 8

4

2

4

2

3

5

3

6

7

The interaction graph, strong interaction graph, and block diagram of this problem are depicted in Fig. 6.9.la,b,c. Consider the elimination ordering w = (z4,ug, z, , u 2 ,z2,u l , zl).

Figure 6.9.1

6.9

211

An Example

(1) The variable z, is eliminated according to the variant of Section 6.6. Then

+

iz, = { { Z 3 > u3, 4 :z4 = z 3 u3, 0 Iz3, u3 5 3 , 1 Iz4 I 5 } ; &I = ( ( z 3 9 u3) 1 5 2 3 f u3 5 5, 0 z3, u3 5 3}; z,* = z3

+ u3;

and the new component, obtained from f4 by substitution, is given by

0 1 2

4

3

3

4 3 5

3 5 6

5

6

7

5

6 7

(2) The variable u3 is eliminated using the general procedure

Then

K2

=Hl,

and

The elimination of the variables z4 and u3 is often called, in the literature, elimination of block “3” (see Fig. 6.9. lc). The interaction graph, strong

212

6. Constrained Problems

@--@

Figure 6.9.2

interaction graph, and block diagram of the remaining problem are shown in Fig. 6.9.2a,b,c.

(3) Eliminating the variable z 3 , by means of the variant, we have z3* = z,

+

2.4,

and

0

5

1 2

9

3

9

I

9 7 9

I 9

9

6.10

21 3

Possible Improvements

(4) Eliminating the variable

u2 , we have ~

22

h4

u**

0

11 10

0

-1 2 3

1 0 0

12 15

Finally, eliminating the remaining variables, we have 4

2 fi

min

=

14,

i=l

and zl* = 2;

ul* = 0;

z2* = 2;

z3*= 2 ;

u3* = 0;

z4* = 2.

u2*

=

0;

6.10 A Discussion about Possible Improvements of the Optimization Procedures in Some Special Cases Dynamic programming consists essentially in splitting an optimization problem (either unconstrained or constrained) into a sequence (or collection in the multilevel case) of subproblems. Up to now the assumption has been made that each subproblem (with the exception of the variant of Section 6.6) is solved exhaustively. However, there do exist many problems in which the general procedure can be further improved by exploiting some peculiarities of the components and constraints. More specifically, given a subproblem min Y EK

k(Y, y ' ( ~ ) = ) h ( y( Y ) ) ,

y ( Y ) E KytP),

Y(y(Y,)

it may be possible to explore (a) only a subset A c P C Y and/or ) (b) in connection with each y ( Y ) E A , only a subset of K P ( y ( Y ) ) . Clearly this is advantageous in terms of both functional evaluations and storage space.

214

6. Constrained Problems

It is obvious that, in this case, the secondary optimization problem, as described in Section 6.5, becomes less meaningful, and, in principle, an ad hoc secondary optimization problem should be given in connection with each class of problems. The literature dealing with dynamic programming contains a number of problems of this kind [l, 9, 121. Section 6.1 1 reports, as an example, the well-known allocation problem.

6.11 An Allocation Problem In this section a simple allocation problem is described and two formulations are presented: The first is more straightforward and its solution is an example of the concepts developed in the preceding section, while the second, deduced by means of a suitable change of variables, allows an efficient solution through the use of the general procedure given in Section 6.3. A certain quantity q of an economic resource, which can be used in a number of different ways (activities), is given. Let xi(i = 1,2,. . . ,n) be the quantity of resource assigned (or allocated) to the ith activity. Assume that there exists a return (or utility) fi depending only upon x i , namely, fi=f i ( x i ) , and a utility (usually negative) fo inherent in the total resource x2 * . x~).The problem of maxiallocated, namely, fo = f o ( x l mizing the total return, subject to the availability constraints, is

+ +- +

f h+ XZ {zi, zs...., zn} max

n

+ + + Cfi(xi), * - -

Xn)

i=1

subject to

O<xi 4

maxfo(xl

= 3,

i=l

and an optimizing assignment is xl* = 2;

x2* = 0 ;

xg* = 1 ;

xq* = 1 .

Letting

the second formulation of the allocation problem is max {?/I, Ye,..., Itn)

f,(Yl)

+ %(Yi i=l

- Yi+d + f ( Y A

218

6. Constrained Problems

subject to O ~ y ~ ~ m Ci qni ,q

j = l , 2 ,..., n ;

0I Y j - Yj+l I qj,

j = l , 2 ) . . . ,n - I .

’ ,i

As the reader can verify by applying the general procedure of Section 6.3 for the elimination ordering co = (yl,y,, . . . ,yn), the computations to be performed are exactly the same as before. The change of variables, in fact, allows us, by reducing the complexity of the interaction graphs, to obtain, via the general procedure, the same computational advantages which are reachable only via a special procedure for the first formulation of the problem.

Appendix A

Review of Graph Theory

This appendix contains a brief and informal review of the graph-theoretical concepts necessary for a good understanding of this book. All notations and definitions refer to an undirected graph without self-loops and parallel edges, G = (V, E ) , where V is a finite set of I VI elements called vertices and E is a set of 1 E 1 vertex pairs called edges.

List of Symbols

r(y) The set of vertices adjacent to the vertex y. r ( Y ) The set of vertices of V - Y adjacent to at least one vertex of y [T(r) = UYEY Tb)- rl. I( Y) The set of edges of the section graph of G w.r.t. Y [clearly Z(y) = 01. O ( y ) The set of edges emanating from the vertex y. O ( Y ) The set of edges joining the vertices of Y with the vertices of v - Y[O(Y) = U y e Y Ob)-. Z(Y)I. Z*(Y) The set of edges of the complete graph with vertex set Y. 219

220

Appendix A

Glossary

such that Adjacent sets Two sets of vertices Y and Z , with Y n Z = 0, there exists at least one vertex of Y adjacent to a vertex of Z. Adjacent vertices Two vertices that are joined by an edge. Articulation point A separating set with one vertex.

+

Bush (of order k) A tree, with k 1 vertices, in which a vertex has degree k and k vertices have degree 1. The vertex of degree k is the root and the other vertices are the peaks of the bush. Complementary graph or complement of a graph A graph c = ( V , I?) obtained by deleting the edges of C = ( V , E ) from a complete graph having the same vertices [J!? = Z*(V) - El. Complete graph A graph in which every two distinct vertices are adjacent [ E = I * ( V ) ] .A complete graph with I V I = i vertices is denoted by G ( i ) . Connected component of a graph A connected nonnull subgraph of G which is not contained in any larger connected subgraph. Connected graph A graph in which every pair of distinct vertices are joined by at least one path. Connectivity The minimal cardinality of a separating set of G: A graph with connectivity h is said to be h-connected; a complete graph is defined as m-connected.

- -

, (vl,u l ) such that .vl, Cycle A sequence of edges (ul, v,), ( u z , u,), , uz are distinct vertices. The number of edges of a cycle is the length u2, of the cycle.

.

+

Deficiency of a vertex y The number of missing edges in the section graph of G w-r.t. T ( Y ) [I z*(KY>)- I ( r ( Y ) ) 11. Degree of a vertex y The number of vertices adjacent to y ( I r ( y ) I). Empty graph A graph consisting only of isolated vertices ( E = @). Flower A set of vertices Y 2 V such that Y is fully connected and each vertex of Y is adjacent to at least one vertex of V - Y. A flower with k vertices is a k-flower. Forest A graph which has no cycles so that each component is a tree. Fully connected set A set of vertices Y such that every two distinct vertices are adjacent [Z(Y)= Z*(Y)]. Fully connected sets Two sets of vertices Y and Z , with Y n 2 = 0, such that every vertex of Y is adjacent to all vertices of 2.

221

Review of Graph Theory

Homomorphism A homomorphism 5 of G = ( V , E ) into C’ = ( V ’ ,E l ) is a correspondence, generally many to one, of V into V’ such that corresponding to any edge ( u l , u2) E E, there exists an edge (~(q),r(uZ))E E‘, if T(UA f t ( u 2 ) . Isomorphic graphs A pair of graphs whose vertices and edges can be placed in one-to-one correspondence in such a way that adjacencies are preserved. Isomorphic graphs within vertices of degree 2 A pair of graphs such that the graphs, obtained from them by removing successively all the vertices of degree 2 and putting one edge between the adjacent vertices, are isomorphic. Mason’s index The minimal cardinality of a set of vertices Q (Mason’s set) such that the section graph of G w.r.t. V - Q is a tree or a forest. Null graph A graph with no vertex ( V = 0, E = 0). Path A sequence of edges (ul, u2), (u2, us), . , (u[-~,u l ) such that ul, u2, . , uz are distinct vertices. It can be also denoted by assigning the sequence of vertices u l , u 2 , * , u z . Planar graph A graph G which can be drawn in the plane in such a manner that all intersections of edges are vertices of G. Section graph w.r.t. a set Y The graph G‘, the vertex set of which is Y and the edges of which are all those edges in G which connect two vertices of Y [G‘ = ( Y , Z(Y))]. Separating set A set of vertices Y such that the section graph of G w.r.t. V - Y consists of two or more connected components. Subgraph A graph G’ = (V’, E ‘ ) is a subgraph of G = ( V , E ) if V’ c V and E’ c E. Supergraph A graph G‘ is a supergraph of G if G is a subgraph of G‘. Tree A connected graph which has no cycles. Triangulated or chordal graph A graph in which, for every cycle of length I > 3, there is an edge connecting two nonconsecutive vertices. Whitney-Menger (theorem of) A graph is h-connected iff every pair of vertices are joined by at least h vertex disjoint paths. +

--

-

More comprehensive expositions of these concepts are to be found, for instance, in references [32-351.

Appendix B Some Set-Theoret ical Definitions

This appendix gives four set-theoretical definitions which are used in Chapter 6. Let X = (xl, x2,. . . ,x,} be a set of discrete variables; Sz, be the definition set of the variable xj; Y = {yl, y,, . . . ,y z } c X ; Sy = SVlx S,, x

x SYL;

and

A c Sy

Consider a partition of the set Y into the sets W, 2, and U.

DEFINITION B. 1 The set AW = ( W : 3(W, Y - W ) E A ) ,

namely, the set of all assignments of W such that, for some Y - W , the assignment ( W , Y - W ) belongs to A , is called the projection o f A into W.

DEFINITION B.2 The set A w ( Z ) = ( W : 3 ( 2 , W, U ) E A } , 222

223

Some Set-Theoretical Definitions

namely, the set of all assignments of W such that, for the given Z and for some U , the assignment ( Z , W , U ) belongs to A , is called the Z-cut projection of A into W. Clearly AW =

(J AW(Z). ZEAZ

DEFINITION B.3 The set A is said to be univocal w.r.t. Z if, for 2 E A Z ,

IA

y z )I

=

1,

namely, if there exists only one Y - Z such that the assignment ( Z , Y - Z ) belongs t o A . Let Y c 2 c.'A

DEFINITION B.4 The set [A]Z = A x sz-y

is called the prolongation of A to 2.

Appendix

C

Combinatorial Aspects in the Solution of Linear Systems by Gaussian Elimination

This appendix deals with the combinatorial aspects of the Gaussian elimination process upon sparse positive definite symmetric matrices [ 14, 15, 17, 191. Consider the linear system 9: M Z = b,

(C. 1)

where M is an n x n symmetric positive definite matrix, the sparsity of which can be visualized as follows.

DEFINITION C. 1 The interaction graph G = ( X , F ) of a matrix M is an undirected graph, without self-loops and parallel edges, defined by the following. (a) The vertex set X is the set of the indices of the rows (and columns) of M . (b) Two vertices xi and xi are adjacent iff the matrix M has a nonzero (i,j ) entry (i f j ) . The linear system can be solved using the method of symmetric Gaussian 224

Combinatorial Aspects of Gaussian Elimination

225

elimination [19-211 (also known as the method of congruent transformations). Its first part (forward part) consists in factoring or decomposing M as M = LDLT. where L = (Zij) is a unit (lii = 1 ) lower triangular n x n real matrix and D is an n x n real diagonal matrix with positive entries. Since M is symmetric and positive definite, the decomposition exists and is unique. The factorization is obtained as follows. Let

where a1 is a scalar, rl is (n - 1) x 1, and @, is (n - 1 ) x (n - 1). Then

M,

=

Let M,

-

= MI

rlrlT

-

-a1

Also, M , is symmetric and positive definite. As before, we let

Then M can be written as follows:

" I

X

0

=

L,D2LZT.

226

Letting M ,

Appendix

=

RZ- (rz rzT)/az and

C

operating as before, we finally obtain

M = L,-1 Da-lLz-l

= LDLT.

Once this factorization has been performed, in order to solve the linear system of equations (C.l), we first consider

Ly"

= b.

D y'

= y"

LTy

= yl.

(C.2)

Then and finally

The solution of these systems (backsolving) is straightforward due to the special structure of L and D. Next examine the total number of operations required for the solution of the linear system (decomposition and backsolving). Let 6, be the number of nonzero entries of the first row of M , minus 1. Counting multiplications and divisions as multiplications and operations k 0 (k f 0) as additions, the first step of the decomposition procedure implies 6, multiplications to compute rJal and 6,(6, 1)/2 multiplications and additions to compute - (rl/al) rlT. Similarly, letting 6, be the number of nonzero entries of the first row of M i minus 1, the forward part of the solution (decomposition) requires

+

nl

+

.

2

"('i +

2

&l

3,

multiplications

and

Since one multiplication plus one addition are needed in Eqs. (C.2) and (C.4) for each nonzero off-diagonal entry of L and LT and n multiplications are needed for Eq. (C.3), the backsolving procedure requires

(2 C di ) + n

multiplications

ill

and

2

2 di

i-1

additions.

Combinatorial Aspects of Gaussian Elimination

227

Then note that system (C.1) can be written in the form ( P M P T ) P z= Pb,

(C.5)

where P is any permutation matrix. Since, if M is symmetric and positive definite, PMPT has the same property for any P, the method of symmetric Gaussian elimination applies to the solution of the linear system (C.5) for any P. It is clear that the number of operations required may depend on P . Hence a secondary optimization problem emerges: Which among the n! permutation matrices P minimizes a suitable index of the number of operations (for instance, a weighted sum of the number of multiplications and additions) and hence, approximately, the computing time needed for solving the system ((2.5) numerically? Since each matrix P corresponds to an ordering w = (yl, y z , . . . ,y,) of the indices xl, x 2 , . . . , x, of the rows (and columns) of M , an equivalent formulation of the secondary optimization problem is finding an ordering o such that the chosen index of the computing time is minimized. Letting S, be the set of all ordered partitions, M ( o ) = P M P T , and aj(w) be the number of nonzero entries of the first row of M j ( w ) minus 1, the secondary optimization problem can be formally stated by the following definitions. DEFINITION C.2

The vector formed by the n nonnegative integers dj(w), ordered from the largest to the smallest, is denoted by d ( o ) = (dl(o),d2(w),.. . , d,(w)) and called the vector dimension of the ordering o. DEFINITION C.3

j=1

DEFINITION C.4 A(57')

=

min A(@). w ESw

The scalar A(w) is the chosen index of the computing time and the coefficients a,b, and c are nonnegative scalars, depending upon the way in which additions and multiplications are weighted.

228

Appendix C

Finally it is shown that the secondary optimization problem can lead to a graph-theoretical elimination problem. Theorem C.l

(Correspondence Theorem)

Consider the linear system

M z = b, where M is an n x n symmetric positive definite matrix. Consider the graph G = ( V , E ) with V= X and E = F, namely, the interaction graph of the matrix M . Then

A ( 9 = A(G).

I]’(

= D = ( y , , y,, . . . ,y N ) ,it is clear that 6,(w) = yl) 1 and Ggl equals the interaction graph of the matrix M,(o), we have

PROOF.Since, given w

d ( o ) = d(s2).

The proof follows taking into account Definitions 2.7.13, 2.7.14, C.3, and c.4 Since the function A(D) belongs to the class F(as seen in Section 2.7), all the results of Sections 2.8 and 3.2-3.5 apply. A more comprehensive exposition of this subject can be found, for instance, in Reference [19].

References

[ l ] R. E. Bellman and S.E. Dreyfus, “Applied Dynamic Programming.” Princeton Univ. Press, Princeton, New Jersey, 1962. [2] A. Martelli and U. Montanari, Optimal smoothing in picture processing: An application to fingerprints. Proc. IFIP Cong., Lijubjana, Yugoslavia, Aug. 1971, North Holland Publ., Amsterdam. [3] U. Montanari, On the optimal detection of curves in noisy pictures, Comm. ACM 14, 335-345 (1971). [4] J. A. Hillier, Glasgow’s experiment in area traffic control, TrafJic Engrg. Control 7, 502-509 ; 569-571 (1965-1966). [5] R. E. Allsop, Selection of offsets to minimize delay to traffic in a network controlled by fixed-time signals, Transportation Sci. 2, 1-1 3 (1968). [6] A. Korsak, A proposed algorithm for globally optimal nonlinear-cost multidimensional flows in networks and some special applications. Presented at 5th Internat. Symp. on Traffic and Transportation, Berkeley, June 1971. [7] R. Aris, “Discrete Dynamic Programming.” Ginn (Blaisdell), Boston, Massachusetts, 1964. 229

23 0

References

[8] R. E. Bellman, “Dynamic Programming.” Princeton Univ. Press, Princeton, New Jersey, 1957. [9] R. E. Bellman, “Introduction to the Mathematical Theory of Control Processes,” Vols. I and 11. Academic Press, New York, 1967 and 1971. [lo] A. Kaufmann and R. Cruon, “Dynamic Programming: Sequential Scientific Management.” Academic Press, New York, 1967. [ l l ] R. E. Larson, “State Increment Dynamic Programming.” Amer. Elsevier, New York, 1968. [121 G. Nemhauser, “Introduction to Dynamic Programming.” Wiley, New York, 1966. [I31 D. Wilde and C. Beightler, “Foundations of Optimization.” PrenticeHall, Englewood Cliffs, New Jersey, 1967. [I41 U. Bertelb and F. Brioschi, Combinatorial aspects in dynamic programming and in the solution of linear systems, in “Differential Games and Related Topics” (H. W. Kuhn and G. P. Szego, Eds.). North-Holland, Publ., Amsterdam, 1971. [15] U. Bertelb and F. Brioschi, On the theory of the elimination process, J. Math. Anal. Appl. 35, 48-57 (1971). [16] F. Brioschi and S. Even, Minimizing the Number of Operations in Certain Discrete Variable Optimization Problems, Operations Res. 18, 66-8 1 (1970). [17] D. J. Rose, Triangulated graphs and the elimination process, J. Math. Anal. Appl. 32, 597-609 (1970). [18] A. Martelli and U. Montanari, Nonserial dynamic programming: On the optimal strategy of variable elimination for the rectangular lattice, J. Math. Anal. Appl. (to appear). [I91 D. J. Rose, “Symmetric Elimination on Sparse Positive Definite Systems and the Potential Flow Network Problem,” Ph. D. Dissertation, Harvard Univ., 1970. [20] W. R. Spillers and N. Hickerson, Optimal elimination for sparse symmetric systems as a graph problem, Quart. Appl. Math. 26, 425-432 (1968). [21] W. F. Tinney and J. W. Walker, Direct solution of sparse network equations by optimally ordered triangular factorization, Proc. ZEEE 55, 1801-1809 (1967). [22] E. W. Dijkstra, A note on two problems in connection with graphs, Numer. Math. 1, 269-271 (1959). [23] T. C. Hu, “Integer Programming and Network Flows,” AddisonWesley, Reading, Massachusetts, 1969.

References

231

[24] U. Bertele and F. Brioschi, A new algorithm for the solution of the secondary optimization problem in nonserial dynamic programming, J. Math. Anal. Appl. 27, 565-574 (1969). [25] F. Harary and J. Krarup, The relationship between outerplanar graphs and a class of discrete optimization problems. Presented at 7th Math. Programming Symp., The Hague, The Netherlands, September 14-1 8, 1970. [26] A. Beltrami, A. Colorni, and F. Forese, “Nonserial Dynamic Programming: An Algorithm for the Solution of the Secondary Optimization Problem.” Thesis of the Istituto di Elettrotecnica ed Elettronica, Politecnico di Milano (in Italian). [27] E. Biondi, F. Brioschi, L. Divieti, and P. Rozzoni, Multistage decision processes, in “Identificazione ed Ottimizzazione Problemi e Metodi.” Ed. C.N.R., Rome, 1967 (in Italian). [28] P. Bonzon, Necessary and sufficient conditions for dynamic programming of combinatorial type, J. Assoc. Comput. Mach. 17,675-682,1970. [29] F. Brioschi and A. Locatelli, Extremization of constrained multivariable function : Structural programming, IEEE Trans. Syst. Sci. Cybernetics SSC-3, 105-1 11 (1967). [30] G. Guardabassi and S. Rinaldi, Optimization of discrete systems with delayed control variables : A structural approach, Avtomat. i Telemeh., 7 , 54-59 (1968) (in Russian). [31] G. Guardabassi and S. Rinaldi, Optimization of discrete multivariable systems with time delays via structural programming. Presented at IFAC Symp. on Multivariable Control Systems, Dusseldorf, 1968. [32] C. Berge, “The Theory of Graphs and its Applications.” Wiley, New York, 1962. [33] R. G . Busacker and T. L. Saaty, “Finite Graphs and Networks: An Introduction with Applications.” McGraw-Hill, New York, 1965. [34] F. Harary, “Graph Theory.” Addison-Wesley, Reading, Massachusetts, 1969. [35] 0. Ore, “Theory of Graphs.’’ Amer. Math. SOC.,Providence, Rhode Island, 1962. [36] U. Bertelb and F. Brioschi, Contribution to nonserial dynamic proMath. Anal. Appl. 28, 313-325 (1969). gramming, .I. [37] U. Bertelb and F. Brioschi, A theorem in nonserial dynamic programming, J. Math. Anal. Appl. 29, 351-353 (1970). [38] U. Bertelb and F. Brioschi, Parametrization in nonserial dynamic programming, Rev. Francaise Informat. Recherche Opkationelle, v-2, 87-102 (1971).

232

References

[39] U. Bertelb and F. Brioschi, A note on a paper by Spillers and Hickerson, Quart. Appl. Math. 29, 311-313 (1971). [40] U. Bertelt: and F. Brioschi, On nonserial dynamic programming, J . Combinatorial Theory (to appear). [41] U. Bertelt: and F. Brioschi, New developments in the theory of nonserial dynamic programming. Presented at 7th Math. Programming Symp., 1970. The Hague, The Netherlands, September 14-18, 1970. [42] U. Berteli: and F. Brioschi, Nonserial dynamic programming: Multilevel procedures. Presented at 39th ORSA Meeting, Dallas, Texas, May 5-7, 1971.

Subject Index

A Additions, 27, 108 Algorithm branch and bound, 96 heuristic, 55, 134, 175 minimum deficiency, 55, 134 minimum deficiency-minimum degree, 56 minimum degree, 55, 134 minimum degree-minimum deficiency, 56 optimal-path, 58, 134 Allocation problem, 214

B Backward part, 21, 102, 186 Block diagram, 197 Block elimination graph, 111, 113 process, 111 of set, 111 Bounds, 85, 133, 134 Branch and bound algorithm, 96

Constrained problem, 12, 183, 184 Constraints, 13 Converging branch, 202 Correspondence theorem, 46, 48, 51, 135, 136, 138, 175, 176 cost of graph, 38 /&-costof graph, 115 p-cost of problem, 110 multilevel p-cost of graph, 163 multilevel p-cost of problem, 159 of multilevel ordered partition, 158, 163 of ordered partition, 109, 114 of ordering, 28, 38 of problem, 29 Criterion functions, 35, 37

D

C Comparison, 28, 109 Components, 2, 3, 10, 13 Computing time, 26, 29, 107, 109, 110, 157 23 3

Descendance theorem, 68, 70, 75, 126 Descendant graph, 69 Dichotomic procedure, 63, 79 Dimension of graph, 38, 80 of multilevel ordered partition, 158, 163 of ordered partition, 109, 114 of ordering, 29, 37 of problem, 29 vector dimension of ordering, 28, 35

234

Subject Index

Diverging branch, 199 Dominance relations, 39, 65 Dominating ordering, 35 Dynamic programming nonserial, 16, 20 serial, 18 Dynamic systems, 197, 198

E Elimination of block, 102 graph, 31 multilevel elimination procedures, 140, 141 multilevel elimination process, 160 ordering, 20, 28, 54 process, 30 of set, 32 of variable, 19, 20 of variables in blocks, I00 of variables one by one, 18, 54, 185 of vertex, 31 Equivalent orderings, 35

F Feedback loop, 202 Feedforward loop, 202 Filling index, 94 Final theorem, 39, 44, 61, 121, 125, 126, 164, 166, 167 Forward part, 20, 101 Functional evaluations, see also Number of functional evaluations

G Graph block elimination, 111, 113 cost of, 38 descendant, 69 dimension of, 38, 80 elimination, 31 interaction, 4, 14, 46, 48, 50, 51, 135, 136, 138, 176 ,&cost of, 1 1 5 multilevel p-cost of, 163 strong interaction, 14

H Heuristic algorithms, 55, 134, 175 Horizontal decomposition, 141

I Initial theorem, 66, 67, 76, 77, 126, 131 Interacting variables, 3, 4, 14 Interaction graph, 4, 14, 46, 48, 50, 51, 135, 136, 138, 176 Invariance theorem, 31, 112 Inventory theory, 2

M Memory, 26, 29, 107, 109, 110, 157 Minimum deficiency algorithm, 55, 134 Minimum deficiency-minimum degree algorithm, 56 Minimum degree algorithm, 55, 134 Minimum degree-minimum deficiency algorithm, 56 Multilevel elimination procedure, 140, 141 Multilevel elimination process, 160 Multilevel p, 159, 163, see Cost, Graph, Problem Multilevel ordered partition, 144, 160 cost of, 158, 163 dimension of, 158, 163 number of functional evaluations of, 156, 158 regular, 72

N Noninferior ordering, 35 Nonparametric form, 13 Nonserial dynamic programming, 16, 20 Nonserial unconstrained problem, 3, 20, 46,48, 101, 135, 136 Number of functional evaluations of multilevel ordered partition, 156, 158 of ordered partition, 109 of ordering, 28 of problem, 28, 107, 110, 159

Subject Index

235 0

Objective function, 2, 3, 10, 13 Optimal path algorithms, 58, 134 Ordered partition, 101 cost of, 109, 114 dimension of, 109, 114 multilevel, see Multilevel ordered partition number of functional evaluations of, 109 regular, 113 Ordering cost of, 28, 38 dimension of, 29, 37 dominating, 35 elimination, 20, 28, 54 equivalent, 35 noninferior, 35 number of functional evaluations of, 28 vector dimension of, 28, 35

P Parameters, 9, 50 Parametric form, 9, 15 Parametric unconstrained problem, 9, 10, 50, 51, 137, 138, 141, 176 Pascal-like triangle, 102 Pattern recognition, 6 Penalty function, 184 Primary optimization problem, 16, 101, 141 Problem constrained, 12, 183, 184 cost of, 29 dimension of, 29 p-cost of, 110 multilevel p-cost of, 159 nonserial unconstrained, 3, 20, 46, 48, 101, 135, 136 number of functional evaluations of, 28, 107, 110, 159 parametric unconstrained, 9, 10, 50, 51, 137, 138, 141, 176 primary optimization, 16, 101, 141 secondary optimization, 16, 26, 27, 107, 108, 110, 156, 159, 193 serial unconstrained, 2, 19

Process block elimination, 1 1 1 elimination, 30 multilevel elimination, 160

Q Quasi-standard form, 10, 157

R Regular multilevel ordered partition, 172 Regular ordered partition, 113

S Secondary optimization problem, 16, 26, 27, 107, 108, 110, 156, 159, 193 Separating set theorem, 78, 126, 133 Serial dynamic programming, 18 Serial unconstrained problem, 2, 19 Standard form, 11, 51, 138 Strong interaction graph, 14 Strongly interacting variables, 14

T Table look-ups, 24, 27, 108 Theorem correspondence, 46,48,51, 135, 136, 175, 176 descendance, 68, 70, 75, 126 final, 39, 44, 61, 121, 125, 126, 164, 167 initial, 66, 67, 76, 77, 126, 131 invariance, 31, 112 separating set, 78, 126, 133 Traffic control, 8

U Univocal constraints. 194 V

Vector dimension of an ordering, 28, 35 Vertical decomposition, 142

This page is intentionally left blank