Optimization of Discrete Time Systems: The Upper Boundary Approach

Lecture Notes in Control and Information Sciences Edited by A.V. Balakrishnan and M.Thoma

51 IIIII

III

Zbigniew Nahorski Hans E Ravn Ren6 V.V.Vidal

Optimization of Discrete Time Systems The Upper Boundary Approach IIII

Springer-Verlag Berlin Heidelberg New York Tokyo 1983

Series Editors A.V. Balakrishnan • M. Thoma Advisory Board L D. Davisson • A. G. J. MacFarlane • H. Kwakernaak J. L. Massey • Ya. Z. Tsypkin • A. J. Viterbi

Authors Dr. Zbigniew Nahorski Systems Research Inst. The Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw Poland Dr. Hans E Ravn Danish Energy Agency 11 Landemaerket DK-1119 Copenhagen K Denmark Dr. Ren6 V.V.Vidal The Institute of Mathematical Statistics and Operations Research The Technical University of Denmark DK-2800 Lyngby Denmark

ISBN 3-540-12258-3 Springer-Verlag Berlin Heidelberg NewYork ISBN 0-387-12258-3 Springer-Verlag NewYork Heidelberg Berlin This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machine or similar means, and storage in data banks. Under § 54 of the German Copyright Law where copies are made for other than private use, a fee is payable to 'Verwertungsgesellschaft Wort', Munich. © Springer-Verlag Berlin Heidelberg 1983 Printed in Germany Printing and binding: Beltz Offsetdruck, Hemsbach/Bergstr, 206113020-543210

PREFACE

The development of numerical methods for solving optimization problems has taken place with increasing speed along with the development of the electronic computer.

The underlying cause

for this to happen has been that a large number of technical and operational problems have shown themselves to be suitable for formulating and analyzing within the framework

of

optimi-

zation theory. However, the speed of development has been unequal for the different classes of methods.

Thus in the 1960s fundamental theo-

retical results were published in the area of control problems: the Maximum Principle and the Principle of Optimality.

There-

after numerical methods were developed and implemented, based on these principles and related theorems. In later years the main development within optimization theory has taken place in other areas,

leaving an impression that the

basic theoretical foundation for dealing with control problems had been successfully explored and mapped.

We do not share this

view.

In the present book we present a new approach to deal with the control problems

(specifically, the discrete time ones) which

indicates the possibility of exposing the fundamental theorems not only from control theory, but also from adjoining areas of mathematical programming.

But further it provides a convenient

base for formulating new and fundamental results,

in this book

exemplified by the theorem of the Nonlinear Maximum Principle. We call the approach the Upper Boundary Approach.

IV While thus the book sketches some promising perspectives and documents a number of new results, it does so, we admit, in a preliminary form. Rather than elaborating on presentation, we gave priority to the quick communication of the results.

The

reader will hopefully agree with us in the strategy chosen! The results presented here have their specific history in several years of research work at IMSOR, the Institute of Mathematical Statistics and Operations Research, The Technical University of Denmark.

Here Associate Professor, Lic. Techn. Ren~ Victor

Valqui Vidal in his lectures on optimization theory and applications pointed to some problems in the perception of basic properties and concepts in the theoretical foundation of the theory. They were basically the same in all areas of optimization theory, and not yet quite!

M.Sc. Hans Ravn, presently at the Danish

Energy Agency, at the time at IMSOR, was involved in the problems and worked with Vidal until the contours of the Upper Boundary Approach could be seen.

At this point Dr. Z. Nahorski,

of the Polish Academy of Sciences, joined the project and was main responsible that the ideas

(including some of his own} got

a firm shape at all.

We are very grateful that it was possible for the Danish Technical Research Council to support the later phase of the research economically.

Also we thank M.Sc. N.O. Olesen who con-

tributed during the later phases.

Last, but not least, we are

grateful to Miss Bente Wilkenschildt for her work with the preparation of this manuscript.

Copenhagen and Warsaw, July 1982 Z. Nahorski, H.F. Ravn, R.V.V. Vidal

CONTENTS

Chapter 1

INTRODUCTION

I

1.0

Introduction

3

i.i

Simple examples

4

1.2

The multistage optimization problem

12

1.3

Historical notes

14

1.4

The scope of the book

18

1.5

Literature

19

ONE-STAGE SYSTEMS

25

Literature

47

MULTISTAGE SYSTEMS

49

Literature

87

COMPUTER A L G O R I T H M

91

Chapter 2 2.1 Chapter 3 3.1 Chapter 4 4.1

Introduction

93

4.2

The general idea of the algorithm

93

4.3

The algorithm

4.4

Numerical examples

102

4.5

Complementary remarks

128

4.6

Literature

129

CONCLUSIONS AND FURTHER RESEARCH

133

Chapter 5

98

CHAPTER

I

INTRODUCTION

1.0

Introduction

Optimization

of discrete time systems is an activity which fre-

quently takes place as one of the central

steps in the design

process, when solving certain technical problems, stage problems. speaking, best

The purpose of this activity

called multi-

is, generally

to find a combination of parameter values, which will

(in some specified sense)

values, a mathematical

solve the problem.

model is constructed,

proposed solution to the problem. the multistage

the

of the discrete time

system.

Solution methods for the optimization is the subject of this book.

representing

The search for good parameter

values is taking place as an optimization model representing

To find these

of discrete time systems

We propose a new approach to this

area, the upper boundary approach,

which will allow us to derive

new and important results, while at the same time restating classical results within the same terminology. This first chapter serves as an introduction and to the book.

We shall therefore

examples of technical

and operational

to have some common properties

to the problem area

first give a number of problems, which can be seen

in their mathematical

representa-

tion. A subclass of these socalled multistage

optimization

will be identified.

and precision of this

After a discussion

subclass, we give a short outline of the historical

problems development

of solution methods for it, with emphasis on Dynamic Programming and the Maximum Principle.

1.1

Simple examples

Let us look to some problems which arise naturally in technical systems. I° Multistage Compression of a Gas A gas is to be isentropically compressed from the initial pressure Po to a final pressure PN" The compression proceeds in N stages.

In each stage the gas is first adiabatically compressed

and then isobatically cooled to its initial temperature. The energy consumption at the ith stage is given by

E i = mRT ¥/(y-I)[

(xi/xi_ I) (Y-I)/¥-I]

where m- the number of moles of gas compressed R- the universal gas constant T- the initial temperature of the gas ¥- the ratio of the specific heat of the gas at constant pressure to that at constant volume

(assumed to be

constant) xi-pressure of the gas at the end of the ith compression It is desired to determine the interstage pressure for which the total energy consumed in compression is minimal. variable in the ith stage u i is defined as

ui = xi+I/Xi then we can formulate our p r o b l e m as

N-I

min i=O

UOI---IUN_ I

u(Y-1 )/7 1

If the decision

subject

to

xi+1

= x.u. i i

Xo

= Po

XN

= PN

2° Transportation

Resources demand

i=0,1,...,N-I

Problem

are to be t r a n s p o r t e d

points

(sinks),

is only one type of r e s o u r c e to the total

from n depots

see Fig.

1.1.

and that

Demand

supply

points

• U

2

the total

J

•

uj 2 U..

i

UjN N

n

Fig.

•

1.1

A scheme

for t r a n s p o r t a t i o n

to N

that

demand.

Depots I

(sources)

It is s u p p o s e d

problem.

there

is equal

Let u~ - the q u a n t i t y i

of the resource

depot to the ith

sent from the jth

demand p o i n t

r~1 (u~)- the cost of this operation The p r o b l e m

is to determine

i=1,2,...,N

to minimize

the quantities

of u~l, j=1,2,...,n;

the total costs of transporting

the re-

sources

R

--

N ~

n ~

4 4 r~(u4J )

i=i j=1 subject to the c o n s t r a i n t s

u~>0 I -N

Z

U~ = W j, the supply of the resource

i=I En

jth depot, u~ = di, the demand

j=l

point,

j=1,2,...,n

at the

j=1,2,...,n for the resource

at the ith demand

i=1,2,...,N i=1,2,...,N

We can define

state v a r i a b l e s

which has been t r a n s p o r t e d mand points.

available

x~ as the total amount of resources l from the jth depot to the first i de-

Then we can write the following

equation

X~ = X j + U~ l i-I i xJ = 0 O j=1,2 .... ,n-1

XJ N = wJ

i=l,2,...,N

It s h o u l d

be n o t e d

there

n depots.

are

each d e m a n d nth d e p o t

point

there

This

mand by the

from

is p r e a s s i g n e d .

by all

rest

point

n = di _ ui

n-I E u3." i 9--I

to w r i t e

define

=

of n-1

the

2

above

. . ,x i

]

ri = [r11., r 2, .... rn] T w j = [w I, w 2, .... wn-1] T

N o w the p r o b l e m

c a n be

formulated

as

N

min Z u 1 , . . . , u N i=I subject

r i (u i)

to

xI

•

=

Xo

= 0

XN

= w

xi- I

U. > 0 , i --

÷

u.

di -

i = 1,2,...,N

the

although

the d e m a n d

supplies

the s u m of depots

problem

n-l~T

xi,-

variables

that

the

f r o m the

from

in the

supplies total

de-

d i i.e.

the v e c t o r s

I [x i,

state

fact

Therefore

2 n-l~T U i = [U 1, u i , . . . , u i J

xi

n-1

the

by subtracting

the

ith d e m a n d

convenient

Let us t h e n

are o n l y

arises

c a n be o b t a i n e d

of the r e s o u r c e

It is

that

n-1 Z u3"i -> 0 i--I

in v e c t o r

notation.

The last two constraints define for each i a region in the n-1 dimentional control space usually referred to as a set of admissible or feasible controls. n

~ j=l

U j. = d. 1 l

i=1,2,...,N The last five constraints define a region in the n-1 dimentional control space usually referred to as a set of admissible or feasible controls. 3 ° Catalyst replacement In a catalytic reactor the efficiency of the process gradually decreases as the catalyst gets older. Because of this, the best operating conditions change in time. The problem then is to find the best operating conditions

in some periods of time and the

best time for replacing the catalyst so as to obtain the maximum profit. Let us consider a system depicted in Fig. sists

1.2 which con-

of a tubular reactor and a distillation tower. In the

period i a material is feeded to the reactor with constant flow rate through the reactor F i. In the reactor a compound A cracks

I (I -C i )F i

i11

M.

1

O

FIJ TO1

Fi

O

Catalytic reactor Ti

I I I ei

I Fig.

1.2

Schematic flow sheet.

CiFi

to c o m p o u n d s

B and G. T h e n t h e y go to the d i s t i l l a t i o n

which the c o n v e r t e d m a t e r i a l CiF i and u n c o n v e r t e d recycled material (with rate Mi) have

(final product)

material

is r e c y c l e d w i t h rate

is added to the m a t e r i a l

=

(1-Ci)F i. The

f e e d e d to the p r o c e s

and inleted to the reactor.

Mi

tower f r o m

flows w i t h the rate

By m a t e r i a l

b a l a n c e we

C.F.

(1.1)

i i

(1.2)

F i = M i + (1-Ci)F i The c o n v e r s i o n

is a s s u m e d to be e x p r e s s e d

as

(1 .3)

C A = alTi-a2Fi-a3S i where T i is the exit t e m p e r a t u r e

in p e r i o d

tive flow rate t h r o u g h the catalyst,

Si =

S i represents

i and S i is the c u m u l a -

i.e.

i Z Fi j=l

(1.4)

the state of the s y s t e m w h i c h

age of catalyst, lance we have

is e q u i v a l e n t

al,a 2 and a 3 are g i v e n constants.

Qi = Fi Cp(Ti-To) where Qi is the heat input to the r e a c t o r reaction,

and T o is the t e m p e r a t u r e

(I .5)

+ hCiFi

in p e r i o d i, Cp is the

average heat c a p a c i t y of the r e a c t i n g mixture, reactor

to the

By e n e r g y ba-

h is the heat of

of the m i x t u r e

entering

the

(assumed constant).

The conversion, constraints

temperature,

and flow rates are subject to the

Cmi n _< C i r(u) which contradicts

the optimality of ~-

To facilitate things we shall in the sequel assume that ub(x) < ~ for all x £ X and

ui(xI£ X^ for

all x £ X

This is not a real limitation of the analysis. also assume that x £ X. An optimal u* and the corresponding 109.

We shall

solution we shall call

extended state ~*.

We shall define a support n°(.) function of an n-dimensional

at the point x 0 as a

argument,

defined on the set

X. It has the form

= i=lz N°I(xi) n

O{x)

It has the property that there exist a real number k, such that (i)

~O(x°)

+ k = ub(x O)

(ii)

N°(x)+ k > ub(x)

for all x £ X

Sl

We say t h a t n°(x) w h i c h ~°(x)

s u p p o r t s ub(x)

+ k = ub(x).

at the p o i n t s x for

A s u p p o r t at the p o i n t x we

d e n o t e n*(x) 109a.

If the c o n d i t i o n k > ub(x)

109b.

109(ii)

(ii)' n°(x)

+

shall call a s u p p o r t

n°(x)

a s t r o n g S u p p o r t at a p o i n t x O. T h e n it s u p p o r t s

ub(x)

o n l y at a single p o i n t x °.

If there exists a n e i g h b o u r h o o d f u n c t i o n n°(x)

satisfies

then w e shall call n°(x) If n°(x) by i09c.

is r e p l a c e d by

for all x £ X and x $ x ° , w e

of x O, Y c X

conditions

a local s u p p o r t at the p o i n t x °.

satisfies on Y conditions

(ii)' then w e shall call n°(x)

The d e f i n i t i o n

§109 w i t h

(ii) r e p l a c e d

a s t r o n g local support.

of a s u p p o r t g i v e n in §109 is a g e n e r a l i -

zation of a c l a s s i c a l Gould

such that a

of §109 on the set Y

support d e f i n i t i o n

(1969). The c l a s s i c a l

support defined

as given by

s u p p o r t is a s p e c i a l case of

in §109 s u b j e c t to

k--0 •

n*

1

(x i) = A i(x i) - A i(xi) + n ub(x)

w h e r e i i is any f u n c t i o n

such t h a t i09(i)

and

(ii) are

satisfied.

AlSo a support obtained after a redefinition r a l i z e d L a g r a n g i a n as g i v e n by Evans at al special case of a support d e f i n e d

of a g e n e -

(1971)

is a

in § 109 s u b j e c t to

k=0 * (X)

x± (0)

= I i(xi - Xl) + 1 ub(x)

= 0

w h e r e I , i = 1 , 2 , . . . , n are any f u n c t i o n s 1

§ 109

(i),

(ii), and the a b o v e conditions.

that s a t i s f y

32

109d.

All

theorems

are v a l i d

given

respectively,

ii0.

Theorem.

below

for s u p p o r t s

for local s u p p o r t s

If

or strong

or strong local

if the set X is r e s t r i c t e d

Q

has a f e a s i b l e

supports

supports,

to Y.

solution

then

a

~*(.)

exists. Proof.

Suppose

x is a f e a s i b l e

~*(x)

value.

Take

0

x=x

sup ub(x)

x ~ x

=

l

xCX

T h e n ~* (x) is a s u p p o r t

iii.

Theorem.

If

~x)

and ub(x)

~°(x°)_ 3x

Proof.

The function K°(x)

= O(x)

z(x)

and ub(x)

at x ° t h e n

the f u n c t i o n

- ub(x)

is d i f f e r e n t i a b l e

are d i f f e r e n t i a b l e

at x ° b e c a u s e at x 9 F r o m

both

the defi-

of a s u p p o r t z(xO) = k

then

are d i f f e r e n t i a b l e

3 u b ( x O) ~x

L e t us c o n s t r u c t

z(x)

nition

w i t h k = ub(x).

a n d z(x)

z (x) has a local m i n i m u m

>_ k

for all o x

in the point

and ~z(x°)~x

~°(x°) ~x

~ub(x°)= ~x

0

x 6X

33

112.

If ub(x)

is d i f f e r e n t i a b l e

Dub (x)

at x we call

the

s h a d o w prices. n

113.

L e t ~(x)

be a n y f u n c t i o n

We d e f i n e

the H a m i l t o n i a n

H(u,~)

ll3a.

The f u n c t i o n

H(u,~)

Hamiltonian

in c o n n e c t i o n

For g i v e n tical

=

as

Z ~ i (Xi) . i=l

- ~(f(u))

was p r e v i o u s l y

see G o u l d

see C h a p t e r

H(-,-)

= r(u)

Lagrangian,

a generalized

114.

of the f o r m K(x)

(1969).

Hamiltonian

called

generalized

We shall

call H(u,K)

the

with dynamic

programming

where

is d e f i n e d

in a s i m i l a r

manner,

3.

u and

distance

~,H(u,~) between

c a n be i n t e r p r e t e d

as the v e r -

the p o i n t

^

X =

in x and the p o i n t

on ~.

Lf(u)]

f(u)

[~

(u)]

115.

Theorem.

If u m a x i m i z e s

situated

Proof. U that

H(u,~)

over

U then x =

at {ub}.

is

(u)J

By c o n t r a d i c t i o n .

Suppose

~ maximizes

H(u,~)

over

is

H(~,~)

> H(u,~)

and

for all u 6 U

is n o t s i t u a t e d

at

{ub}.

Lf (~)J B u t f r o m the d e f i n i t i o n is a u* C U for w h i c h

of { ub} and a s s u m p t i o n

f(u*)

= f(~)

and

108 there

34

X* =

[r(u*)l

is s i t u a t e d

at

{ub}.

Hence

tf (u*)J r (u*)

> r (~) a n d c o n s e q u e n t l y N

H(u*,~)

which

116.

contradicts

Theorem. there

n°(x)

exists

Proof.

the supposition.

is a s u p p o r t

u ° which

and s a t i s f i e s

Then

> H(u,u)

at a p o i n t

is a s o l u t i o n

if

to: m a x H ( u , n O) o v e r U

f(u O) = x O.

Suppose

~°(x)

is a s u p p o r t

f r o m the d e f i n i t i o n

ub(x)

- ~°(x)

f r o m the d e f i n i t i o n

at the p o i n t

xO

of a s u p p o r t

ub(x°) - ~ ° ( x ° ) =

Now,

x ° if a n d o n l y

k

x6X

< k

w

of ub(x)

and a s s u m p t i o n

108 there

a r e u °, u C U t h a t

Inserting

f(u °) = x ° a n d

r(u O) = ub(x O)

f(u)

r(u)

= x

the above

and

formulae

= ub(x)

for a n y x C X

into the p r e v i o u s

ones we

have

r ( u °) - u ° ( f ( u ° ) )

= k

r(u)

< k

- ~°(f(u))

B u t for a n y ~ ~ W there a n d f(~) mulae

= f(u)

is a l w a y s

which means

is s a t i s f i e d

r(u)

u C W = {u:

a u E W t h a t r(~)

t h a t the s e c o n d

o

(f(u))

< k

~ r(u)

of a b o v e

also by u ~ W. T h u s we c o n c l u d e

- ~

(u)J 6 { u b } }

u £ U

for-

that

35

and from the d e f i n i t i o n of H a m i l t o n i a n

H(u°,= °) = k H(U,nO

which means N O W let u

o

) < k

uEU

that uO m a x i m i z e s maximizes

H(u°, ° )

H(u,~°).

H(u,n°). Then

> ~(u,~ °)

u£U

so there is a k that H(uO, n O) = k H ( U , K O)

< k

and from the d e f i n i t i o n

of H a m i l t o n i a n

r(u O) - n°(f(u°))

= k

r(u)

< k

- n°(f(u))

uEU

But from the t h e o r e m 115

I~ (uO)

t h e r e f o r e we h a v e

ub(x O) - n * ( x O) = k

and, d e f i n i n g

(u° )

for any u E U, f(u)

sup r(u)

is s i t u a t e d

at {ub}

= x, we have

- n O(f(u))

< k

of ub(x)

and X

all u £ U

f (u) =x

or from the d e f i n i t i o n ub(x)

- O(x)

< k

all x £ X

38

Rearranging

terms

we h a v e

n°(x O) + k = ub(x °)

n°(x)

+ k > ub(x)

from w h i c h we c o n c l u d e

ll6a.

x ° = f(u°).

Corollary.

Let u* be an o p t i m a l

. T h e n n*(x)

only

is a s u p p o r t

if u* is a s o l u t i o n

Proof.

the c o n d i t i o n

r e m 116 leads

Theorem. only

n°(x)

at a p o i n t

to the p r o b l e m x = f(u*) over

if and

U.

to the p r o b l e m ( A )

f(u*)

is a s t r o n g

then

= x. N o w a p p l i c a t i o n

it sa-

of Theo-

support

at a p o i n t

x ° if and

u ° to: m a x H ( u , n °) over U s a t i s f i e s

= x°

Proof.

T h e proof

Suppose

t h a t n°(x)

from T h e o r e m and

solution

to ub(x)

to l16a.

if any s o l u t i o n

f ( u °)

is a s u p p o r t

to: m a x H(u,n*)

As u* is a s o l u t i o n

tisfies

117.

that n°(x)

at a point

Q

all x £ X

conclusion

is a s t r o n g

116 there

satisfies

maximizes

is a s i m p l e

exists

support

Then

from T h e o r e m

which

contradicts

f(~)

at a p o i n t

u° which maximizes

f(u °) = x °. S u p p o s e

H ( u , ~ °) but

there

% x °. D e f i n e

i16 n°(x)

from T h e o r e m

exists f(~)

is a s u p p o r t

the a s s u m p t i o n

i16.

x °. T h e n H ( u , ~ °)

~ which

= ~ # x 0.

at the p o i n t

that n°(x)

is a strong

support. Suppose

n o w t h a t any

satisfies support

f(u °) = x °. T h e n

at x O. S u p p o s e

that there

o)

from T h e o r e m

~°(x)

116 t h e r e

o v e r U and

the assumption.

u O to: m a x H ( u , n O) o v e r 116 n°(x)

it is not a strong

is ~ ~ x ° that

then from Theorem H(u,

solution

is a s u p p o r t

exists

satisfies

support,

f(~)

U

is a i.e.

at x. But

~ which maximizes = ~, w h i c h

contradicts

37 ll7a.

Corollary. only

n*(x)

is a strong

if any s o l u t i o n

f(u*)

= x. M o r e o v e r

Proof.

The

first

u* to: m a x H(u,~*)

x if a n d

over U satisfies

any such u* is an o p t i m a l

sentence

c a s e of T h e o r e m from Theorem

118.

Deleted.

119.

Let K(x)

117.

The

of c o r o l l a r y second

solution

to

be a n y s u p p o r t

to ub(x).

of the s u p p o r t

Call

of s u p p o r t s

the c l a s s

functional

~(-,-)

define

is a c o n c l u s i o n

{ub}.

L e t the n u m b e r

for g i v e n

= r(u)

a saddle-point

of

be c o n s t a n t

on the p r o d u c t

~(u,~)

is just a s p e c i a l

sentence

115 and the d e f i n i t i o n

the d e f i n i t i o n

We

at a p o i n t

problem@.

the

120.

support

k from

for all supports.

k, C k. D e f i n e

the

Ux C k as

- K*(f(u))

of ~(.,-)

+ ~(x)=H(u,~*)+K(x)

as a p a i r

(u°,~v)EUXCk

w i t h the p r o p e r t y

~(u,~ °) < ~(u°,~ °) < ~(u°,~) u C U, ~ £ Ck, 121.

k E R.

Let S k c C k be the subset Then

for

Theorem.

supports

in a set C k .

(u,~ °) £ Ux S k we have:

~*(x)

is a strong

a n d u* is a s o l u t i o n saddle-point

Proof.

of strong

to < A J

of ~(-,.)

L e t ~*(x)

u* be a s o l u t i o n

support

restricted

be a s u p p o r t to ~ k_/

H(U,~*)

. Then

< H(u*,~*)

m

at the p o i n t x=f(u*)

if and o n l y

if

(u*,~*)

is a

to Ux Sk, k E R.

at the p o i n t

x = f(u*)

from Corollary

l16a

all u E U

and

38

A d d i n g to b o t h sides n*(x) we get the left i n e q u a l i t y the s a d d l e - p o i n t observing

definition.

that from the d e f i n i t i o n

n*(x) Then a d d i n g

of

The r i g h t i n e q u a l i t y w e get of a s u p p o r t

< ~ (x)

all

to b o t h sides H(u*,n*)

n E S

we get the right

in-

equality.

Let uS s u p p o s e n o w that ~(-,.).

(u°,n °) is a s a d d l e - p o i n t

T h e n from the r i g h t i n e q u a l i t y

of

in the s a d d l e - p o i n t

d e f i n i t i o n we have

H ( u ° , n *) + n°(x)

_< H ( u ° , n *) + n(x)

all 7tES k

or ~°(x) Taking

< n(x)

n = ~* rc° (x)

all ~ e S

we have

< n*

S u p p o s e n o w that ~°(x)

(R) is a strong

s u p p o r t to ub(x)

p o i n t x O # x. T h e n from the d e f i n i t i o n

at the

of strong s u p p o r t s

rc° (~) > n* (~) which contradicts

the e a r l i e r

inequality.

This proves that

n°(x) m u s t be a support to ub(x) at the p o i n t x, i.e. o n = n*. Now, from the left i n e q u a l i t y in the s a d d l e - p o i n t definition

H(u,~*)

+ ~*(x)

_< H ( u ° , ~ * )

H(u,n*)

< H ( u ° , ~ *)

+ n*(~)

all uEU

or all uEU

39

i.e.

uO maximizes

o

u is a n o p t i m a l u o = u*.

122.

H(u,n*)

o v e r U. T h e n

solution

T h e c a s e of l i n e a r

~*(-)

deserves

®

from Corollary

to the p r o b l e m

special

l17a

, i.e.

attention.

In

this case H(u,p)

= r(u)

- pTf(u)

w h e r e p is a n - d i m e n s i o n a l notes

122a.

a transpose

two points

X l , X 2 E X the l i n e s e g m e n t

A function

in X, t h a t

f: X~R,

to be c o n c a v e real

l,

0
uBi÷1 (xi+1) Comparing

Ci+ 1

expressions ~Bi+ 1 (xi+l)

xi+ I £ Xi+ I we conclude over

el+ 1

that at

66

i+i(i+i)

~*

+ ki+ 1 = ~Bi+l(

X*

i + l ) = UBi(X~)

X*

i+i (xi+1) + k i + I ~ ~ i + l [ x i + l )

~ UBi(x i)

+ ri{x*~i , - i,"*~ + ri(xi,u[)

Xi+ 1 6 Ci+ 1 , X i 6 X.1 which we can write UB i(x*) + r i(x*,u*) x.l 6 X.l

UB i(xi) + r i(x i,u*) From the above expressions we conclude that the function

P(xi)

= UBi(xi)

+ ri(xi,u*)

X$1

maximizes

- T~+I [Xi + fi(xi,ul) ]

over X i . Now, from the assumptions fi (xi'u*) , r i[xi,u* ) and UBi(xi) are differentiable at the point x*z " Moreover UBi+ I (Xi+l) is also continuously differentiable in a small open neighbourhood of xi+ 1. so that a continuously differentiable support ~*+1(Xi+l) exists. Then P [xi) is d i f f e r e n t i a b l e and ~P(X*) ~X i

~UB i (X*)

=

8x i

* 8r i (x*,ui)

+

X* ~Tt*+l(i+l) [

~x i

~Xi+ 1

[ I+

~fi [X *i 'u*) ] ~X i

But from Theorem 216

i [xl) ~x i

(xl) 8x i

which after insertion to the previous formula and after rearranging terms leads to

8xL{ri (xi ,ui ) - u~+l [fi (xi,u*)] }

=

~X.1

x i = x*l Now from the definition of Hamiltonian we get

(iii).

J =0

67

222a.

We s u m m a r i z e

n o w the a s s u m p t i o n s

we h a v e m a d e d u r i n g

(i)

no a d d i t i o n a l

(ii)

the n u m b e r s constant class

(iii)

proof

ki+ I

for all

Corollary.

Assume

i = 0,1,-.-,N-1 assume

that

that

222C.

that

of

x?

1

fi[xi'u[ 1 '

X

=

respect

to

xi ;

differentiable 1

in a

; then

uBi+ICx*

8X i

is a simple

the d e f i n i t i o n

in a small o p e n

with

in

* C i+i) qCx 'ui)

uBiCx

~Xi+ 1

consequence

~X.1

of T h e o r e m

of the H a m i l t o n i a n

§ 214,

and

216.

We g i v e an e x a m p l e cannot

and

is c o n t i n u o u s l y

~Xi+ 1

222(iii), Theorem

r i[x i,u~)

, are d i f f e r e n t i a b l e

The proof

and o n l y the

considered,

i = I ,2 - - - ,N

8UBi+ I x*

~X i

Proof.

has b e e n

differentiable

x*

small o p e n n e i g h b o u r h o o d

i i)

which

are:

of s u p p o r t s

~k+l(Xi+l)

supports

supports

of

UBi[xi)

~r. x*,u*

supports These

f r o m the d e f i n i t i o n

continuously

neighbourhood

222b.

222.

assumption,

of s t r o n g

~*~Xi)z

concerning

of T h e o r e m

be dropped.

showing

that the a s s u m p t i o n

Let us c o n s i d e r

r i x i , u i I = U 2i

Xo = 0

filxi'ui)

u,

= Xi + Ui

1

a 2-stage

6 [0,11

+

X1 = U0

~2 = ?

l

problem

i = 0,1

is

X 2 = 2X 1 + U I = 2U 0 + U I

x~ 6 Int X.

U o , U 1 6 [0,1]

1

with

68

We

have

the

xI E

the

sets

of

reachable

[0,1]

optimal

x2 6

states

[0,3]

control

o

u~--

~2 1

! I

~2

2

-

U~

=

x2 > 1

and

the

upper

X2 > 1

boundaries

2 x2

uB1(xI) : x21

UB2(~)

0<x2 I to zero.

it is This

gives

0 2a2 xt + b2 ul = 2(a 2 -1) Then we have

8x I

-2a 2 (x t + ul)

- b2

8x! = 2al xl + b l

and f r o m the e q u a t i o n

~H I

8~ I

8X 1

8X I

we obtain

b t = b 2 + 2x l ( a 2 - a t ) + 2a 2 u 1 A t last, the H a m i l t o n i a n

H0(U 0 , x 0 , a l , b l )

and we have

at stage

= r0(u0)

0

is

_ a l ( x 0 + u0 }2

b I (x 0 + % )

113

- 2 0 [ U 0 + 1)

-

- b1

2a l(x O + u O )

u 0 < -1

8H o

au °

-2a1 (X 0

--

+ UO)

- b1

-I < u 0
0

~x I ~x 2

NOW

as

x3 3X 2

then

the matrix

[x~,x~)

and the

The problem

may

3x 2

of

second

function

be

> 0

3x I 3X 2

derivatives takes

formulated

2 max ~. u 0 , u l , u 2 i=0

is n e g a t i v e d e f i n i t e

maximum

at t h i s

as a c o n t r o l

- [u£1 ~

point.

problem

at

119

subject

to

i = 0,1

Xi+ I = X i U i X0 = P0 X3 = P3 From e a r l i e r

calculations

the o p t i m a l

solution

to this p r o b l e m

is

seen to be t/3

i/3

113

t13

with the o p t i m a l

The

usual

Then the

values function

criterion

for

value

, see

~

R*(P3)

Happel

(1958)

, the g r e a t e r

p.

upper

260,

are

boundary,

form

uB( 31 >

x3

0