Stochastic Control by Functional Analysis Methods (Studies in mathematics and its applications)

STUDIES IN MATHEMATICS AND ITS APPLICATIONS VOLUME 11 Editors: J. L. LIONS, Paris G. PAPANICOLAOU,New York R. T. ROCKAF...

Author: Alain Bensoussan

16 downloads 341 Views 7MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

STUDIES IN MATHEMATICS AND ITS APPLICATIONS VOLUME 11

Editors: J. L. LIONS, Paris G. PAPANICOLAOU,New York R. T. ROCKAFELLAR, Seattle

N O R T H - H O L L A N D PUBLISHING COMPANY - AMSTERDAM

a

NEW YORK

a

OXFORD

STOCHASTIC CONTROL BY FUNCTIONAL ANALYSIS METHODS

ALAIN BENSOUSSAN Universite Paris Dauphine arid I N R I A

19x2

NORTH-HOLLAND PUBLISHING COMPANY - AMSTERDAM

NEW YORK

OXFORE

L

North-Holland Publishing Company, I982

All rights reserved. N o part of this publication may be reproduced, storedin a retrievalsystem,

or trammirted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without thepriorpermission of the copyright owner.

ISBN: 0444 86323 x

Publishers: N O R T H - H O L L A N D PUBLISHING COMPANY AMSTERDAM 0 NEW Y O R K 0 O X F O R D Sole distributors f o r the U.S.A. and Canada: ELSEVIER SCIENCE PUBLISHING COMPANY, INC. 5 2 VANDERBILT A V E N U E , NEW Y O R K , N.Y. 10017

Library of Congress Cataloging in Publiration Data

Bensoussan, A l a i n . S t o c h a s t i c c o n t r o l by f u n c t i o n a l a n a l y s i s methods. ( S t u d i e s i n mathematics and i t s a p p l i c a t i o n s ;

v . 11)

B ib li o g r a p h y : p .

. 1. C o n t r o l t h e o r y .

2. Stochastic analysis. I. T i t l e . 11. S e r i e s . QA402.3. B433 6 2 9 . 8 ' 3 12 81- 19900 ISBN 0-444-86329-X AACFC?

PRINTED IN T H E NETHERLANDS

V

INTRODUCTION

Our objective in this work is to give a presentation of some basic results of stochastic control. It is thus a text intended to advanced students and researchers willing to learn the theory. Stochastic control covers a broad area of disciplines and problems. It is also a field in full development, and some important aspects remain to be cleaned up. That is why in presenting stochastic control a choice is necessary. We have emphasized this choice in the title. The theory of partial differential equations, the semi-group theory, variational and quasi variational inequalities play a very important role in solving problems of stochastic control. We have tried to use them as much as possible, since they bring tools and results which are very important, specially for computational purposes, and which cannot be obtained in an other way, namely regularity results, and weak solution concepts. R. Rishel 1 1 1 , A. Friedman C 1 1 , N. Krylov [ l ] , The books by W. Fleming A . Bensoussan - J.L. Lions [ l ] , [ 2 ] show already the importance of the techniques of Functional Analysis. w. Fleming Rishel, Friedman besides covering many other topics, rely mostly on the classical theory of P.D.E. We try to emphasize here the importance of variational methods. Naturally, the present text has a lot in common with the books of J.L. Lions and the A. But here we have tried to simplify as much as possible the presentation, in particular leaving aside the most technical problems, which are treated there.

-

-

Also the books by J.L. Lions and the A. are devoted to variational and quasi variational inequalities. In the book of Krylov, one will find the study of the general Bellman equation, i.e., when the control enters into the drift as well as into the diffusion term. We do not treat this general case here, although it is certainly one of the nicest accomplishments of P.D.E. techniques in stochastic control. Quite fundamental results have been obtained for the general Bellman equation by P.L. Lions [ l ] , [ 2 ] , and more specialized ones by L.C. Evans A. Friedman [ l ] , H. Brezis - L.C. Evans C l ] . More recently R. Jensen - P.L. Lions [ l ] have introduced new important ideas of approximation. To report on that work would have gone beyond the objectives of the present text, and requested too much material.

There are many important other topics that we have not considered here. We have not reported on the developments of the so called "probabilistic approach" initiated by C. Striebel [ l ] , 1 2 1 and R . Rishel C 1 1 and developed extensively by M. Davis - P. Varayia C 1 3 , M. Davis C 1 1 and many other authors. A good report can be found in N. El Karoui [ l ] , (see also Lepeltier - Marchal [ l ] ) .

vi

INTRODUCTION

This approach is of course fundamental for very general processes, which are not Markov processes. It is certainly the most general one and very satisfactory from the probabilistic point of view. But for the applications, where the processes are mostly Markov, it seems less convenient than the analytic approach, especially for computational purposes. Also it requires technical developments which again would have gone beyond the scope of this text. The interested reader should consult besides the litterature, which has been briefly mentionned the recent book by 1.Gikhman - A. Skorokhod [11. Another very important area, which is in full development is the theory of non linear filtering and control under partial observation. Important results have been obtained recently by several authors in non linear filtering, T. Allinger - S.K. Mitter Cl], E. Pardoux [ l ] , and exploited for the control under partial observation by W. Fleming - E. Pardoux [ l ] , W. Fleming [ I ] . Stochastic P.D.E. play a very important role in this direction, and probably the field will progress fast (see E. Pardoux [2], M. Viot [ l ] , W. Fleming - M. Viot [ l ] ) . For the control of stochastic distributed parameter systems see A.V. Balakrishnan [ l ] , A. BensoussanC11, [ 2 ] , A. Bensoussan - M. Viot [ l ] , R. Curtain A.J. Pritchard [ I ] , S. Tzapestas C 1 3 . We consider in this work some stochastic control problems in discrete time, but mostly as an approximation to continuous time stochastic control. We refer to the books by D. Bertsekas [ I ] , D. Bertsekas S . Shreve [ I ] , E. Dynkin - A. Yushkevitch [ I ] for many more details. In a related direction, we have not discussed the numerical techniques which are used to solve stochastic control problems. We refer to J.P. Quadrat [ l ] , P.L. Lions - B. Mercier [ 1 1 and to the book of H.J. Kushner [ I ] . Let u s also mention the theory of large stochastic systems, with several players, the problems of identification, adaptive control, stochastic realization, stochastic stability etc., as interesting and important areas of research. In Chapter I we present the elements of Stochastic Calculus and Stochastic Differential Equations, in Chapter I1 the theory of partial differential equations, and in Chapter 111 the Martingale problem. This permits to deal with the various formulations of diffusion processes and to interpret the solution of elliptic and parabolic equations as functionals on the trajectory of the diffusion process (in a way similar to the well known method of characteristics for 1 s t order linear P.D.E.). This allows u s also to show the Markov semi-group property of diffusions. In Chapter IV we present the theory of Stochastic Control with complete information (when the control affects only the drift term). We study the Hamilton-Jacobi-Bellman equation, interpret its solution as a value function and solve the stochastic control problem in the stationary as well as non stationary case. We also present a semi group approach to stochastic control for general Markov processes. In Chapter V, we present the theory of filtering and prediction for linear stochastic differential equations, which leads to the Kalman

INTRODUCTION

Vii

filter. We show that the problem reduces to quadratic optimization problems, for which a decoupling argument yields the filter and the Riccati equation. In Chapter VI, we present the variational approach to stochastic control, in two situations, one with complete observation and one with uncomplete observation. We discuss also the separation principle. Chapter VII is devoted to optimal stopping problems which are solved by the theory of variational inequalities. We also develop a semi group approach in the case of general Markov processes. In Chapter VIII we present the theory of impulsive control and its solution by the method of quasi variational inequalities. Also a semi group formulation is given. We have tried to be self contained as such as possible, and have avoided too technical topics. Some basics on probability and Functional Analysis are the only requirements in order to read this book. Nevertheless, we recall the results that we need. We have restricted ourselves to stationary diffusions stopped at the exit of a domain, since one can explain the ideas in the simplest form in that context. But of course the methodology carries over to many other processes, like diffusions with reflexion, diffusions with jumps, random evolutions etc. see A. Bensoussan - J.L. Lions [Z], A. Bensoussan - P.L. Lions 111, A. Bensoussan - J.L. Menaldi I l l ) . When presenting the semi group approach we have kept a certain degree of generality, although we have not tried to describe all the examples which are covered in this approach (cf. M. Robin [ l ] for many ). examples like semi Markov processes, jump processes,

...

ix

ACKNOWLEDGMENTS

Many of the ideas presented here owe a lot to discussions and joint research with colleagues in France or abroad. I would like to thank particularly J. Frehse, A . Friedman, M. Goursat, J.L. Joly, J.L. Lions, P.L. Lions, J.L. Menaldi, U. Mosco, G. Papanicolaou, E. Pardoux, J.P. Quadrat, M. Robin, M. Viot with whom I had certainly the most fruitful exchanges and from whom I learnt a lot. The material covered here has been presented first as a course at the University of Illinois, during the fall 1980. I would like to thank particularly P. Kokotovic who originated the idea of a course and suggested the writing of a text. I would like to thank also the members of the Control Group, at Coordinated Science Laboratory, University of Illinois, in particular Prof. Cruz, Perkins for their help. Prof. B. Hajek at the University of Illinois read carefully the manuscript and made very useful comments. I would like to thank him very much for that.

I would like to thank very much Professors Lions, Papanicolaou and Rockafellar for accepting to publishing this book in their series with North Holland as well as the publisher for his agreement. Mrs Kurinckx realized a very nice typing of the manuscript. I would like to thank her very much for this excellent job.

1

CHAPTER I STOCHASTIC CALCULUS AND STOCHASTIC DIFFERENTIAL EQUATIONS

INTRODUCTION This chapter is devoted to the presentation of the stochastic dynamic systems which will be used throughout this work, namely those whose evolution is described by stochastic differential equations. This requires a stochastic calculus and the concept of stochastic integral, originated by K. Ito. The model looks like dy

=

g(y,t)dt

+

a(y,t)dw(t)

and g is called the drift term, 5 the diffusion term. This model generalizes the model of ordinary differential equations

and expresses the fact that the velocity is perturbed by a random term of mean 0. In the standard set up (strong solution) one assumes lipschitz properties of g, u with respect to the space variable. It is important for the applications to control to weaken the concept of solution in order to assume only measurability and boundedness of the drift term. This is achieved through Girsanov transformation. We have kept the presentation to what is essential within the scope of this text. But, aside basic preliminaries in Probability theory, we give complete proofs. We refer to the comments for indications on the natural extensions. Basic references for this chapter are the books by J. NEVEU C l l , I. GIKHMAN-ASKOROKHOD ~ 2 1 ,A. FRIEDMAN r i i , D. STROOCK - S.R.S. VARADHAN E l ] , E.B. DINKIN 1 1 3 .

2

CHAPTER I

R be a set. A 0-algebra

Let

R is a set of subsets

ff on

of

R

such that V

Ai

nAi i if

a

The elements of

, i

I , Ai

E

E

a,

(I countable)

then

, uAied i

Acff

,

@ A c a

are called e v e n t s .

The set

(@,a) is

a o-algebra.

It is contained in all o-algebras on R. It is called the trivial

a-algebra. A probability on i.e., a map

A

+

(Qg)is P(A)

from

a positive measure on 0

a

into

[O,l]

, with

total mass 1,

such that

P(Q) = 1

P(u

n When

P(A)

= 1,

An) = Z P(An) n

one says that A

if the An

are disjoint.

is aZmost certain

(Q is the certain

event). The triple If B of Q

c

(R,Q,P) is called a probability space.

a,and

is also a 0-algebra we say that i3

is a sub o-algebra

*

On

R

(the set of real numbers), the open intervals generate a o-algebra

on

R, which is called the Bore1 0-azgebra on R.

3

STOCHASTIC CALCULUS

On a product space X

1

x X2

, if X I ,X

x1 x X 2

respectively, the product o-algebra by the events of the form A'

x

Hence the Borel u-algebra on

Rn

' A

are u-algebras on

where

X

I

,

X

2

is the u-algebra generated

A'

x1 , A2

E

E

X2 .

is the a-algebra generated by open

cubes. The concept carries over to an infinite set of spaces X1, i

E

I

product u-algebra B X1 , is generated by the events of the form . I Ai E , Ai = X1 except for a finite number of i.

xi

A random variable is a measurable map

R

Borel u-algebra on If

fi

, i

I

E

,

u(fi)

I Note that if is a R.V.

E

a,ti

B

E

R

/3

.

,

i.e., if B

I

denotes the

It is called the o-algebra generated

fi are measurable.

by the family fi @

f-'(B)

The

TI 'A ,

are random variables, there is a smallest o-algebra for

which all maps u-algebra

,

R -tf

.

fk

.

and denoted by

u(fi,i

are random variables and

E

I).

fk(w)

It is also the product

-t

f(w)

br w, then

f

Also a positive random variable is the increasing limit of

a sequence of piecewise constant positive R.V. namely

We will need some results concerning extensions of probabilities,for which we refer to J. NEVEU [ I ] . Let

R be a set. We call algebra a set of subsets of R, satisfying

properties (l.l), ( 1 . 2 ) ,

( 1 . 3 ) except that in (1.1)

the set

I is not

countable, but only finite. We say that a class C5 {C,,

n

in C

t 1)

such that

n

nloreover, we have from (ii), to check that u(x)

From this, one easily deduces,by a limit argument as

(iii)

au - g.Du

=

f

6

+

0 ,

a.e.

which is a first order hyperbolic partial differential equation (P.D.E.). We have emphasized the point of view of computing a functional on the trajectory of the solution of an O.D.E. One can also have the reverse point of view, which consists in giving eqZCo7:t formulas for the solutions of P.D.E. In the litterature on P.D.E., this is known as the method of c h n r a c m r i s s k s . The O.D.E. ( 1 ) indexed by the parameter x, are called the characteristics of (iii). A s we will see, for elliptic and parabolic second order P.D.E., one can introduce characteristics curves, which are diffusion processes. In our presentation, except two difficJit regularity results on P.D.E. we have given full proofs of the results, but we have restricted ourselves to those which will be effectively used. The basic references are A . Friedman [21, O.A. Ladyzhenskaya and N . h . Ural'tseva [ I ; , O.A. Ladyzhenskaya - V . A . Solonnikov, N.J. Ural'ceva [ I ] , J.L. Lions [ I ] , S. Agmon - Douglis - Sirenberg [ I ; , C. Miranda [ I ] , C.B. Morrey [ I ] , G. Stampacchia [I], J.L. Lions -

52

CHAPTER I1

E. Magenes [ I ] .

For Sobolev spaces see for instance F. Trhves [ I ] .

1. FUNCTIONAL SPACES 1.1. &blgd-spaws Let

We will use constantly the SoboZev

8 be an open subset of R”.

spaces

, WzYp(S) ,

W”p(8)

W’%)

=

2

ff

5

p

E

LP(Cs)

< m,

integer. We recall that

1%

€

LPOS)}

1

,

W”pp)

For

p

=

W2”(0)

are Banach spaces for the norms

2 , we use the notation

HZ

= w192

= w292

They are HiZbert spaces. In the case when 8 follows (1.1)

(p,q

...

if n>p ,

€

is smooth, we have important i n c l u s i o n theorems, as

[I,=)) 2

W’’p(o) c L (8) with continuous injection

Clearly this result implies that if W2*p(c) c W’”((S)

n > p with continuous injection

53

PARTIAL DIFFERENTIAL EQUATIONS n > 2p

and if

(1.2) if p>n with

,

-1

a = 1

P

then

c

~ ~ ( (')6 )with

continuous injections

c

$65)

8 = 2 - 1

w"P(o)

.

Consequently

if n > p E 2

W2'Pf3)

(1.3)

Let

f

E

P

, and f

W2"f&)

# integer

20

for any

E

[ + , I 1 , and

c

I If/

I If1 For example taking

a

lwl,r
1,

0. Let

p
o

.

Under assumptions 12.171, 12.181 t h e r e exists f o r

one and o n l y one solution

Consider t h e f u n c t i o n a l on J(v) =

u

E

1

Ho(0)

f

E

L2@)

such that

H;)(O)

C '03ij i,j

a v dx axj 5

From P o i n c a r 6 ' s i n e q u a l i t y i t follows t h a t

+

a.

v2 dx - 2

f v dx

.

65

PARTIAL DIFFERENTIAL EQUATIONS

1

It i s a strictly convex continuous functionalonHo(B) admits a unique minimum.

.

Therefore it

Since clearly (2.19) is the Euler equation

0

for the minimum, it has also one and only one solution.

,?emark 2.2.

The result of Lemma 2.1 holds even when the matrix

not symmetric (wnich is here a restrictive assumption). Lax-Milgram Theorem (cf. for instance K. Yoshida [ I ] ,

a . . is li

This is the

J.L. Lions [ I ] ) .

The proof is different, since (2.8) cannot be interpreted as an Euler

D

equation. Remark 2.3. .The assumption

f

assume

E

f

E

L

2

(8)

is n o t really necessary.

(2.20)

W”~(R~)

a. lj

.

Consider also (2.21)

a

i

E

.

L~(R~)

We write (2.22)

We will set (2.23)

and

bi

b.(x)

E

3

H-l(O) with the same result.

Let us assume now that besides (2.17) the

Lffi(Rn)

.

=

aa. - ai(x) + C ax j j

We can

aij

verify

66

CHAPTER I1

We define next the bilinear form on

H1(0)

(2.24) +

Now if

v

E

1 Ho

1

(2.25)

Theorem 2 . 3 . f

E

,

Lp(0) , 2

fs.a

we deduce from Green’s formula (1.18) and (1.23) that a(u,v)

u

E

=

H1@)

+

,

Au

fsa. E

fi v

u v dx

E

p
O

some positive number

.

Indeed we make the change of unknown function

where yo

to be chosen and

xo

is fixed outside

6.

Clearly (2.26)

is equivalent to

Let us make explicit the operator w A(wz).

We have

(1) Because of (2.20) the symmetry assumption on

a.. is not restrictive 1J

67


= -

C

i,j

a

w - (a..(-

axi

L

Let u s s e t

We have f o r

hence

x

E

0

IJ

ao

ax

+

z

j I

OJ

aZ

-))

ax. J

t

68

CHAPTER I1

aij(x.-xoi) (x.-x )

c

+ yo

i,j

Ix-xol

2

J

Oj

-

C a.

ii

therefore

exp - ylx-xo/

outside 5, i t is possible to choose yo > 0 such that

Having fixed xo when

.

runs into 8 one has

x

, problem (2.26) is equivalent to the same problem with 2 aij changed into , ai changed into a.w2 , a. changed into aij w(aow + X) 2 y > 0 , and f changed into fw. Therefore we may assume Since

1 5 w 5 2

(2.27)

.

Let u s next prove that for

h

large enough, the problem

has one and only one solution. Let u s first show uniqueness.

f

=

0 , from Green's theorem, it follows that 0 0 a(u , u ) + A luo12

(1) We cannot take

y

=

o

arbitrary large.

Assume

69


A

and we can assume t h a t

hence

L e t u s prove e x i s t e n c e .

uo = 0 .

Define f o r

z

i s l a r g e enough s o t h a t

1

Ho

E

(2.32)

,

5

E

'0 a i j =

1

av dx a 1~a. ~ . 1

aZ Is ( f - Z a i %)v

i

V v r H Then

5

t o be t h e s o l u t i o n of

Ho

+ J

dx

( a + A ) < v dx 0

8

=

,

1 0 '

e x i s t s and i s d e f i n e d i n a unique way, by v i r t u e of Lemma 2 . 1 1 Ho i n t o i t s e l f S ( z ) = Consider

:.

We t h u s have d e f i n e d a map from

cl

z1,z2 and

Take

and i f

v =

tl-c2 ,

A

Let

, c2

=

S(z2).

We have

we deduce t h e e s t i m a t e

i s s u f f i c i e n t l y large it follows t h a t

which proves t h a t point.

S(zl)

=

uo

S

i s a c o n t r a c t i o n mapping i n

be t h e f i x e d p o i n t .

1

Ho, hence i t has a f i x e d

From ( 2 . 3 2 ) we have

70

CHAPTER I1

We next use Theorem 2 . 1 .

l-

(2.34)

Then

zo

i,j

There exists one and only one solution of

XTET

aij

1

+ Azo

=

f +

;

aU 0

bi

j

- a.

u0

1

satisfies

(2.35)

' 0 aij

-0 au

~i

Js zo v

+ X

JS

+ C

av dx K a~ dx

=

2i $i -0

aa.

v dx +

auO a u O )v dx (f+Zbjaxj- o

Js

J

y v c H 01 .

But from ( 2 . 3 3 )

uo

satisfies

auO av auO aaij v d x + A J u 0 8 axi d x + C /8 -ax j ax.i -U aU a u 0 )v dx . =Jo(f+;bjaxjc Jo

(2.36)

aij

By difference between (2.35) and ( 2 . 3 6 ) we obtain

Z JS aij

v

and since

Go

=

A

- uo

,

ax j

auO av ax j axi

dx + C JS

- -)-

(zo - u 0)v

+ A J8 Taking

a;'

(-

dx

=

0

.

(ea")'-' -0 j

-

0 aa..

ax j a..~i

vdx=

v dx +

we see that

is sufficiently large,

(1) The existence and uniqueness of consequence of Lax

-

Lo =

uo

Milgram theorem

uo

.

solution of ( 2 . 3 3 ) i s also a

71

PARTIAL DIFFERENTIAL EQUATIONS 1 uo E H2 n Ho

Hence

and

is solution of

uo

AUO + (ao+X)uo

=

f

a.e. in 0 ,

or

If

u

0

, (assuming n

H2

E

(by ( l . l ) ) ,

hence

g

E

L

0

>

axi

2 ) , we have

PAq 1

and

p,

=

pAq, > 2

uo E’W~’~~(;)

using Theorem 2.1 again we get

L q l , with

.

1 = r - 1 q1 2 n

p > 2).

(if

Therefore

By a bootstrap

argument, in a finite number of steps, we obtain uo

E

,

W2”(@)

hence

(2.30) is proved. The next step is to define the following sequence

which is well defined, by virtue of what we have done to study (2.30). We have (2.38)

The important fact to notice is that there is an improvement of regularity at each step. ul-uo

E

u2-u1

obtains for assert that

n

2

E

W2”(0)

n WA”(0)

hence

.

L q l with

Therefore

ul-u’

Indeed

E

-= -- n (if n > 2p) q1 P 2,q1 l’ql W n W, Again using a bootstrap argument, one

no , un - u’-’

.

E

Lm.

Using Lemma 2.2 below, we can then

12

CHAPTER I1

Hence at least for n

2

no, un - un0

is a Cauchy sequence in Lm.

Passing to the limit in (2.37) we conclude to the existence o f

u

solution

of (2.26).

Let us prove the uniqueness of the solution of (2.26). u

Let

f = 0

,

and

to be solution of A u + a u = O 0

then

and by the regularity argument used for (2.37), u

m

E

L

.

From Lemma 2.2

it follows that

hence

I lull

Lema 2 . 2 .

=

Lrn

0

.

0

Consider t h e equation

(f

E

which has one and onZy one s o l u t i o n f o r f

m

E

L , one has

u

E

m

L

Lp(8))

X large enough.

Then if

and

Before proving Lemma 2.2, we will recall an important Lattice property o f the space

1

H (8).We say that

73


Define

then u1

V

,

u2

u1

A

u2

E

1

H (0),

namely the following formula holds (2.41)

where

x

u zu particular

=

1

if

u1

u2

2

and

if

0

7

u 1 < u-

.

Setting in

then u = u

+

- u

-

.

+

, u

Remark that (2.42)

uiHA+u

This is clear when (2.43)

n u

u

B(G). Then one has to notice that

E

E H

1

implies This is clear since

1

6 H O .

, (un)+

in H

u n + 0 +

o

in

H'

1

.

14

CHAPTER I1

ahn)+

-=

ax

aU" axi X n

u LO

hence

Let us also note the relation (2.44)

a(v+,v-)

=

o

Y v

E

I

.

H (8)

W e can then give the

Proof of L e m a 2.2 Let

then

K

=

llLLl X+Y

(u-K)

+

E

I

Ho, hence from (2.39) a(u, (U-K) +) +

or

x (u,(U-K) +)

= (f

, (U-K) +)

PARTIAL DIFFERENTIAL EQUATIONS By a similar argument one obtains u t -K , hence (2.40).

Lemma 2 . 3 .

Assume in Zzenrem 2.3 t h a z

f

t

0. Then

u

0 2

0 (u

scltrtion

of ( 2 . 2 6 1 1 .

This is done by induction on the sequence Assume

un 2 0

,

then multiplying by

un

(un+I)-

defined by (2.37) we get

hence a((un+')-,(un+')-) which implies

n+l (u )

=

0

+

> 1 (un+l)-12

s

o

.

A similar argument holds for

uo, hence the desired result.

3 . PARABOLIC EQUATIONS

3.1. &nc;jmrm-~p~ces We will use

dz d t has to be explained. We note the inclusions The meaning of -

each space being dense in the following, with a continuous injection. Thus we can consider

z

also as an element of

L2(0,T;H-'(O)).

Its

derivative makes sense as a distribution and is defined as follows

76

CHAPTER I1

.

-1

the second integral being with values in H

aZ -6 8’ ( (0,T ) ; H - (0) ~ ). at

@

+

- JT

@ ’dt

z

E&(L~(O,T);H-’(@))

aZliE L2 (O,T;H-’@)) then we say that ; important property of

C

and that

z

E

W(0,T).

0 C ([0,Tl;L2(o))

with continuous injection.

Property (3.1) is true at least after a modification of

0

of measure If

the following integration by parts formula holds

JT

=

We write

on a set

z

.

z1,z2 E W(O,T),

An

is the following

W(0,T)

W(O,T)

(3.1)

Hence a priori

If

dz

< 1x z, 2 > dt + IT < z 1

, dt dz2 >

dt

.

Q = 8 X (0,T) and consider spaces of the form 2 $ 2 $ P ( ~ )=

(3* 3)

12

L~(Q) ,

6

ax; , ax;ax. , aLz

aZ

aZ

6

L~(Q)I

J

with the natural norm of Banach spaces.

...

Similarly we define bl””(Q) b Z y 1= b 2 ” ”

(3.4)

,

= {z E

1 5 p
O , p > 1 , p < m

1

Rn

/ z ( x ) I p exp - p p ( l + l ~ ~ dx ~ )< ~m /} ~

78

CHAPTER I1

We will use also the following embedding theorems (cf. Ladyzhenskaya Solonnikov- Ural’stseva If

z E

L ~ ’ l ’ p, p >

If

z E

h2””

Let C?

+ I

p > n + 2

, then z

We define

1

E

Co(q)

q

, then z , a z

be an open subset of

(3.9)

( 3 . 10)

,

2

111).

aij , ai ,ao

Rn

and m

E

L (Q)

E

C

o (Q) -

.

Q = 8x (0,T).

We assume

79


(3.11)

a(t;u,v)

au av Jsaij(x,t) ax. - dx ax. 1 1

Z,

=

+ C J

i

Theorem 3 .

+

~

i,J

8

* ax

a.(x,t) 1

v dx + J

8

a (x,t)u v dx

0

I. Assume 13.91 and

ii

(3.13)

.

L20)

E

Then t h e r e erists one and onZy one s o l a t i o n du - + a(t;u(t),v)

E

W(0,T)

of

f(t),v > a.e. t

in

(0,T) 1

u(T) =

Remark 3 . 1 .

=
y > O

, y arbitrary large

Indeed, make the change of unknown function y

exp - X(T-t)u

=

then problem (3.14) is equivalent to

hence we have the same problem, with a .

changed into

+

a .

I

'

a

changed into

f e-''(T-t)

* *

. 'Wn'

The space

V

being separable, there exists an

* * *

We look for an approximate solution as follows m

(3.16)

U,(t)

=

c

i= 1

eim(t)wi

where the scalar functions gim(t)

are solutions of the following

linear system of ordinary differential equations (3.17)

1

-

and

0

'5

We use Galerkin's method orthonormal basis w

f

du (2 ,w.) J

+ a(t;um,wj) = J

, j

=

I

,...,m

81


where

m

E

.

um u

Cwl,. .,w 1 and m

Multiplying (3.17) by

g

jm

as

m

+ m

.

and adding u p , we obtain

- Id td lUm(t) 1'

(3.18)

in L2(S)

-t

+ a(t;um(t),um(t))

=

.

However by (3.15) we can assume that

hence the estimate

which implies u

(3.20) Let

$

E

1

C (C0,TI)

is bounded in Lm(O,T;H) such that

@(O)

=

0

.

Set

We can extract a subsequence, still denoted u

(3.21)

Multiply (3.17) by

and letting (3.22)

m

+

u

in L 2 (0,T;V) weakly

u

and integrate.

Q(t)

tend to

and

+a:

$.(t) = $(t)wj J

such that

m ,

We obtain

, yields

1' (u,g')dt j

L 2 (0,T;V)

(U,Oj(T))

+ /' a(t;u,@.)dt = 3 J

.

.

82

CHAPTER I1 is arbitrary it follows from ( 3 . 2 2 ) that

Since j

V V € V

4

E

&(O,T))

hence V v

E

V

Taking

du E dt

,

2 L (0,T;V')

we obtain in the sense of distributions

d (u(t),v)

,

u

hence

L2(0,T)

E

implies (see 5 3.1),

.

W(0,T)

E

, which

Using next integration by parts in W(0,T)

and ( 3 . 2 3 ) yields u ( T )

Hence existence is proved. For uniqueness take

f =

0

,

u

=

0

.

Take v = u ( t )

in ( 3 . 1 4 ) .

We obtain

.

hence

u = 0

3.3.1.

Regularity with respect t o time

Theorem 3 . 2 .

We make t h e assumptions of Theorem 3 . 1 , and

(3.24)

a,. = a 1J ji

(3.25)

a t aij

(3.26)

f E L'(Q)

a

E

L-(Q)

, z

H~I

.

=

G.

83


Then we have

(3.27)

We define (3.28)

ao(t;u,v)

=

aU av dx Z J8 a.1 J.(x,t) ax.1 ax. 1

and

hence a(t;u,v)

=

ao(t,u,v) + (A1u,v)

.

We consider the Galerkin approximation (3.17) which is rewritten as f01lows (3.29)

+ -(dt,w.) J

ao(t;um,wj)

=

(f

-

A1um,wj) ,

j=l,

and we may assume that (3.30)

i +U

Multiplying (3.29) by du (3.31)

I t 1

in

-

HI:

g! (t) Jm

. and adding up in

+ ao (t;u, ,-u’) m

= (f

j, we obtain

- A u -ul) 1 m, m

.

But if we set

it follows from

(3.21) using the sdr’r%ri’d, a (t;u,v) = ao(t;v,u) 0

...,m

84

CHAPTER I1

hence

T 2 Jt 1- dt

1

d s + - a (t;u (t),u (t)) 2 0 m

+

Jtl

=

( f - A u -u’)ds 1 m, m

and from the assumptions we deduce

c

from which we easily deduce ( 3 . 2 7 ) .

3 . 3 . 2 . Regularity with respect t o space variables

Theorem -

3.3.

We make t h e asswrrptions o f Theorem 3 . 1 , and ( 3 . 2 4 ) , ( 3 . 2 6 )

and (3.32)

a.. 11

Then t h e solution u

E

w””(Q)

.

of (3.141 s a t i s f i e s

The idea is t o use Theorem 3 . 2 and the results on elliptic equations (cf. Theorem 2 . 1 ) . (3.34)

If we fix

Let us write ( 3 . 1 4 ) as

ao(t;u(t),v) t

= (f

+

du dt

A

1

U,V)

(outside a set of Lebesgue measure

( 3 . 3 4 ) as an elliptic equation, where

t

0), we can consider

is a parameter.

Since

85

PARTIAL DIFFERENTIAL EQUATIONS du 2 ;i-i - A1u E L ( Q ) can assert that f

and assumptions of Theorem 2.1 are satisfied, we

+

However there is a slight technical difficulty since this does not prove that t

-t

u(t)

is measurable from (0,T) into H2(8)

.

To overcome this difficulty, we use a time discretization procedure.

du -A u dt ln define a sequence u Set

g = f + -

(3.35)

L

E

2

(9) , We discretize O,k,...,Nk

=

T

by

a0(nk;un,v)

=

(gn,v)

ti v

E

HoI , un

E

1 Ho

where (3.36) Then un

E

2 H (8) and

(3.37) where

C

does not depend on n,k gk(t)

=

gn

for

. t

Define next E

[(n-l)k,nk[

remains in a bounded subset of It follows from (3.32) that uk 2 2 L (0,T;H 6)) hence, by extracting a subsequence (3.38) Let v

uk E

L2 (0,T;V)

-t

.

w

in L2(0,T;H2@))

weakly

We deduce from (3.35) that

.

and

86

CHAPTER I1

JT

(3.39)

0

a (t;uk(t),v(t))dt 0

=

JT

Cao(t;uk(t) ,v(t))

nt = integer part of

Therefore letting k

+

0

-

ao(ntk;uk(t),v(t)

(gk(t),v(t))dt

+

where

=

-.kt

But

in ( 3 . 3 9 ) we obtain

hence a.e. t

a0(t;w(t),v)

=

du (f(t) + d t - A1u,v)

.

This and ( 3 . 3 4 ) implies u(t)

=

w(t)

a.e.

which completes the proof of the desired result. R m a r k 3.5.

u

E

#*"(Q)

Under the assumptions of Theorem 3 . 3 we can write and

(3.40)

I

u(x,T)

=

i(x)

Idt +

87


3 . 3 . 3 . Other regularity results We now state without proofs (which are difficult) the analogue of the results stated in

§

2 . 1 for elliptic equations.

spaces of Hglder functions in x,t We denote for

0 < a < 1

,

We need to consider

.

by

+ sup X

t,t'

1 z (x,t)-2

(x,t )

I t-t' 1 a12

L

We then define

c1+a,a/2

and

and

- = (4)

{z(Z E C

0

-

(Q) , axi aZ

Ca,a/2(Q) - 1

CHAPTER I1

88

with

Let us assume a

(3.41)

ij

(x,t)

Ca , q ? j )

E

,

0 < a 4 p.

.

z E

Going back to (3.61), L! This

2’1yph4(QI).

then using the inclusion

~ 1 ’ 0 ’ 8 x( ~(0,T)).

2 , provided it is less than

*

2 L (O,T;H:oc(Rn)).

E

Using the inclusion result (3.7) it follows that

implies u

@

using Corollary 3.1, we see that

We can multiply the

After a finite number

13

of steps we obtain (3.62).

We now state without proofs (see Ladyzhenskaya

-

Solonnikov - Ural’tseva

[ l l ) a result on the Cauchy problem, when the operator A

not in divergence form as follows (3.65)

with the assumptions

a..

=

bi

Bore1 bounded.

13

(3.67)

a

ji

We consider the Cauchy problem

is writen

91

PARTIAL DIFFERENTIAL EQUATIONS with data (3.69) Then we have

Theorem 3 . 8 .

hie assume ( 3 . 6 6 1 ,

13.671,

13-69).

Then t h e r e exists m e

and o n l y one soZution o f 1 3 . 6 8 ) such t h a t

The solution u

of (3.68), (3.70) satisfies the following estimate

(Maximum principle)

which follows from what we have seen in Theorem 3.7.

Indeed for more

regular coefficients (3.71) will hold, and by a limit argument, it holds also under the assumptions of Theorem 3.8. COMMENTS ON CHAPTER I 1

I. By smooth boundary, we mean C2 boundary.

,

2. W l Y p

with

p

3. For Poincar6

=

n

,

W1”

c

Lq

V q c m

.

inequality to hold it is sufficient that

f

vanishes on

some part of the boundary, with positive capacity. 4 . Since

1 Ho c L2

with continuous injection and is dense then we have by

duality

-1

is dense in H with continuous injection. I Now since Ho is a Hilbert space, there is an isomorphism from

and

1 Ho

L2

into

H-l

.

Let us denote by

3

this isomorphism, then for

98

CHAPTER 11

u

E

Ho1 and v

E

1 Ho

we have

<Ju,v>

=

((u,v))

= J

aU av dx --

u v d x + I J

axi

ax;

hence

J

The fact that

is an isomorphism means that for given

L

in H-I,

the Dirichlet problem

-

nu + u

= L

, UEH;)

has one and only one solution. 5 . In the statement of problem ( 2 . 4 ) , it is sufficient t o assume

existence and uniqueness of the solution for

C"(6) .

f

in a dense subset of

Indeed from the Schadder estimate

and therefore one can extend the map linear continuous map from

6. In the case when A 0 estimation (2.16)

=

C"

-A , Theorem

f

+

u

by continuity, as a

c'+'.

in

2.2

and of Theorem 2 . 1 .

is an immediate consequence of Indeed, one has the a priori

estimate

7. In the proof of Theorem 2 . 3 , the idea to use the improvement of n+l n regularity for the sequence u -u was introduced by P.L. Lions [ l ] .

99


8. Property ( 2 . 4 0 ) clearly generalizes property (2.6) that we have seen

in the proof of Theorem 2.1.

This property is known as the Maximum

Principle.

9. If there is more regularity on the coefficients, a . . the data 1J

’

f

and the boundary, one can derive additional regularity properties on the solution. 10.The spaces

ip

,

...

are Frechet spaces.

101

CHAPTER I 1 1

MARTINGALE PROBLEM

INTRODUCTION The martingale approach to diffusion processes is due to D. Stroock S . R . S . Varadhan C11. The objective is to define diffusion process with minimal assumptions on the drift and the diffusion term. We have already seen in Chapter I, with the concept of weak solution that a measurable bounded drift was sufficient, but the diffusion term was still Lipschitz. S . V . have solved the problem assuming only continuity of the diffusion term. We will consider the same problem assuming a H6lder condition on the diffusion term. This will allow us to use as much as possible results on P.D.E., in obtaining a priori estimates.

1. PROPERTIES OF CONTINUOUS MARTINGALES

(R,O,P,3t) .

We consider Let

p(t)

be continuous

ztmartingale.

We say that it is square integra-

ble if we have

(1.1) We write

SUP

t>O p

E

is that for

M2 E p

E

E /v(t)12 < t

M2(Z ; P ) .

Mg,

We will assume

p ( 0 ) = 0. An important result

then

(1.2)

u(t)

-f

p(m)

and

u(t)

=

E [p(~)15~1

The space M2

.

,

in L '

as t

+ m

.

can be provided with a structure of Hilbert space by setting

102

CHAPTER 111

A second very important result follows from the Doob theorem (see C. Dellacherie - P.A. Meyer [ l ] ) .

If

-

p

Meyer decomposition E

M2,

then we can

write in a unique way

where

v(t)

is a continunus martingale, and

d a p t e d i ncr eas i ng prccess.

One says that

process associated with the martingale

(t) < p , u > (t)

is a continuous is the increasing

p(t).

It can be proved that

in the sense of convergence in

L'

One easily checks the following property

P[

(1.6)

Sup ()l(t)) > tc C O , TI

E l 5

N - + P[(T) 2

t

N1

I

The concept of square integrable martingale being too restrictive, one introduces the concept of Zocali,u square i n t e g r a b l e m a r t i n g a k . that

u

We say

is a locally square integrable martingale, if there exists an

increasing sequence of stopping times

- a.s.

We denote by

kM2

T~

+

+

T~

such that

m

the space of locally square integrable martingales.

The decomposition property ( . 4 ) extends to locally square integrable martingales.

In other words if

p

E

RM2,

there exists one and only one

KARTINGALE PROBLEM

increasing continuous adapted process

2 (tAT) -

!J

ti T

such that

< p , u > (t)

(tAT)

p(tA.r)

such that

103

is a 5 tAT martingale, E

M2

.

Example Consider the stochastic integral

v(t)

(1.8)

w

where

is

zt

=

it @(s).dw(s)

n dimensional standard Wiener process.

Assume that

E p

Then

E

RM2.

iT ($(t)(’dt

Indeed set un(t)

=

V T

c

T~ =

.

n and consider

p(tAn)

then

E 1pn(t)

I*

=

E

itAn

I$(s)l

2

ds

9

E

in /@(s)l2ds

.

We define (1.9)

9 , U >

(t) =

it/$(s)I2ds .

By Ito’s formula

2

lJ (t)

=

2

it l l ( s ) @ ( s ) . d w ( s ) + It

/ @ ( s ) 2ds

hence (1.9) defines the increasing process associated w th the martingale P(t)

*

104

CHAPTER 111

Remark 1.1.

One can justify the notation

(t)

as follows.

Let

1 ~ ~ ~ E1 -M2, 1 ~ then

1

PlU2 = 7 (lJ1+LI2)

(1.10)

=

2

< p , p > (t) 1

-

(t)

2

+

1 2

Pl(t)

- 71

2 U,(t)

martingale

where

Note that

is a difference of two increasing processes.

(t)

property and decomposition (1.10) uniquely defines

(t)

.

This

We also note that p1,p2 -t (t)

is a bilinear form on

M2

'

and

are orthogonai if pl(t)p 2 (t) is a martingale, This implies that they are also orthonormal 0 in the sense of the Hilbert space M2'

We say that i.e., if

Let now Assume

where

pl,p2

E

(t)

M2 =

.

p ( t ) = (pl(t), . . . , p

pk(t)

E

RM2 , and

(t))

be a vector continuous

Zt martingale.

10s

MARTINGALE PROBLEM

(1.13)

ajk(s)

is an adapted process and

/a.,(s) J

1

S C

.

We can define easily stochastic integrals with respect to integrands

4 be a step function. We define

Let

Then

and as for the case

a

=

I , we can extend

I(@)

to elements of

and denote it

Let

b(s)

be a matrix

processes and vector (1.16)

.EM2

JT

n+2 PI2

1

This completes the proof of (2.23).

Corollary 2 . 1 . such t h a t

Then if

Q

Let

0 be continuous and bounded on

LP(O,T;W~~~(Rn)),

E

Rn

x

[O,T1

,

, w i t h p > n+2.

LP(O,T;LToc(Rn))

Pxt is a s o l u t i o n o f t h e martingale probleir:, ue kave Ext @(x(TAT~),TAT~) = Q(x,t) +

(2.35)

where

E

Li

T~

is def ine d b y (2.21).

proos We can find a sequence Qn

an Since p > n+2 , Q~

E

C"(6

C0,TI) , such that

x

(Q) " ~

+

Q

in

1

-t

Q

in

c0(Q).

~

1

~

3

Q = 8 X [O,Tj

.

We can write Ito's formula for Q n and the process defined by (2.14)

But

117

MARTINGALE PROBLEM

0

Using Theorem 2.1, we obtain (2.35).

3. EXISTENCE A N D UNIQUENESS OF THE SOLUTION OF THE MARTINGALE PROBLEM

We assume here (3.1) a. ij

c

=

aij

b.

a,. 31

tjtj

2

B /t12

v 5

6

R” , B > o

Bore1 bounded.

Our objective is to prove the following

Theorem 3.1.

Under the assumptions ( 3 . 1 1 , ( 3 . 2 ) the s o k t i c n

the

0

martingale p r ~ b Z e mis unique. We will use several lemmas.

Lemma 3 . 1 .

It is sufficient to prcve uniqueness uhen bi

=

0

Considering the change of probability defined by (2.16), then

Fxt

implies the uniqueness of We assume that

Fxt

is

is defined -xt by formula ( 2 . 3 0 ) , then the uniqueness of P

solution of the martingale problem with unambiguously for

0. Since

.

PXt

.

b

=

Pxt

0

118

CHAPTER 111

A(t)

(3.3)

Let

=

-

C

a .(x,t)

iJ

i,j

4

E

,

C2+a(Rn)

a2 axiax j

consider the Cauchy problem

which has one and only one solution by Theorem 3.8, Chapter 11.

Lema 3 . 2 .

Let

pxt be a s o l u t i o n o f t h e martingale problem correspcnding

t o t h e operator

A(t)

given by ( 3 . 3 : , then we have

props Since

Property (3.5) is an immediate consequence of Ito's formula applied to the function u

and to the process

regularity of

Lemma 3 . 3 . (3.6)

x(s), which is possible by virtue of the

0

u.

~ e ~t O

YB

Borel subset of

P(x,t;T,B)

=

PXt(x(T)

R" E

B)

,

t 5 T

then t h e f u n c t i o n P(x,t;T,B) does not depend on t h e p a r t i c u l a r s c l u t i o n of t h e martingale probZem.

119

MARTINGALE PROBLEM

Proof Since on a metric space provided with its Bore1 o-algebra, a probability is uniquely defined by the values of

(see for instance J. Neveu [ I ] , V @

for

p. 60).

,

Co(Rn)

E

E@

@

continuous bounded

It is enough to check that

Ext @(x(T))

does not depend

on the particular solution Pxt. From Lemma 3 . 2 , the result is proved for @

E

C2"(Rn).

Since V I$ ll@kll 0 C

E

Co(Rn)

,3

I

0

and

Gk

+

' ll@l

C

a family

@k of smooth functions such that

I$ uniformly on compact sets,

we clearly have

If now

Ft is

another solution of the martingale problem, from the

equality

and from ( 3 . 7 ) it follows that EXt @(x(T))

=

Ext @ ( x ( T ) )

hence the desired result.

i e m a 3.4.

The function

i.e. it satisfies

P(x,t;T,B)

is

il

Markov transition function

CHAPTER 111

120

x

+

P(x,t;T,B)

is BoreZ

P(x,t;T,B)

7:s

a probability on Rn

(3.9)

B

(3.10)

P(r,T;T,B)

=

xB(x)

(3.11)

P(x,t;T,B)

=

J P(x,t;s,dy) P(y,s;T,B) R"

,

t S s 5 T

(3.9) and (3.10) are obvious from the representation formula (3.6). if

I$ E Co(Rn)

,

x

-Y

E

Xt

@(x(T))

is Bore1 for

approximation result (3.7) and since c2+a,1+a/2 function. Therefore (3.8) is verified if

B

Ext I$~(x(T))

=

uk(x,t)

is a

is a bounded ball, hence also for

compact and from Neveu C11, p. 61, for

B

Now

t < T , since we have the

B

borel.

It remains to prove (3.11), or which is sufficient

0

for any

E

.

O n C (R )

A s for Lemma 3.3, it is sufficient to check (3.12) for function which are @

E

c2+a( R ) .

Bu then considering the function u(x,t)

(3.4) and the function v(x,t,s)

i:

c

t

5

,

s

solution of

defined by

2

'

i,J

--

a. .(x,t) a v - 0 ax. ax. 1J

t 5 S

1 - 3

v(x,s;s)

=

u(x,s)

v(x,t;s)

=

u(x,t)

then we have

which is obvious by the uniqueness of the solution of ( 3 . 4 ) .

121

MARTINGALE PROBLEM

Lema 3.5. Is Pxt is avy solution of t h e martingale probLem, t h e n

(3.13)

Pxt [x(T)

E

Blms]

=

P(x(s),s;T,B)

a.s.

V t S s < T

It is sufficient to verify that for

I$

O n C (R )

E

and

cs,n

measurable

and bounded then we have

It is also enough to prove (3.14) with

@

C2'a.

E

In that case (3.14)

reads

But from Ito's formula and (3.4)

0

which easily implies (3.15).

Proof of Theorem 3 . 1 From formula (3.13), recalling that the functions P(x,t;T,B)

does not

depend on the particular solution of the martingale problem, we deduce easily that (3.16)

PxL [x(t,)

E

B ] , . . . , x(tn)

E

BnI =

Fxt [x(tl)EB1 ,..., x(t,)EBn!

V t l,...,t

2

t

,

,...,B

B1

Bore1 sets and where

PXt, pxt are two

solutions.n From Theorem 1n2, Chapter I, it follows that

Pxt

=

pxt . 0

122

CHAPTER 111

Remark 3 . 1 .

We have proved in addition that

x(s)

,m: , PXt

is a

0

Markov process.

We assume here that (3.17)

a

measurable bounded

(3.18)

bi measurable bounded.

ij

,

-1

aij = aji ; a

bounded

We recall that A(t)

Theorem 3 . 2 .

=

-

C

i,j

a. .(x,t) 'J

Under asswnptions 13.171,

a2 axiaxj

Ibi

a

13.181 there e x i s t s a s o l u t i o n of

t h e martingale problem.

Proof Assume first that (3.19)

I

a=-Do

*

2

with

Then according to Theorem 5.2, Chapter I, there exists a weak solution of the S.D.E.

at

Hence on some adequate 0,O ,5

n dimensional Wiener, and

y(t)

t

,

such that

there exists

, w(t)

123

MARTINGALE PROBLEM

F : R

Consider the map w

(3.22)

Ro

-+

y(.;w)

-f

Pxt the measure on 711

and

image of

t

?

F

by

.

Then PXt is a

solution of the martingale problem. Indeed let

...,xk

xl,

@

E

,

J%R")

5,

and

to be a measurable function of

; write

5,

(3.23)

=

t

Ss(Y(S1), ...,y (s,))

s

... s

s1 5

Sk

5 s

.

From Ito's formula we have

hence taking the image measure

and since

5,

generate 711;

,

it follows that

Pxt

is a solution of the

martingale problem. Assume now that we have (3.17), then consider a sequence an (3.26) and

sup lan(x,t)-a(x,t)l X,t

is Lipschitz in

a

(3.17).

x

+

o

as

n

-c

-

uniformly with respect to

such that

t, and satisfies

Such a sequence exists. There exists a solution Pn of the

martingale problem, corresponding to the pair

Therefore

(O,an) and

124

CHAPTER 111

This implies that the sequence Pn

remains in a relatively compact provided with the Ro,mt Let us extract a subsequence, still

subset of the set of probability measures on weak topology (see Parthasaratly [ I ] ) . denoted by

Pn

such that

(3.28)

Pn

-+

P weakly

...,

Consider a continuous function s(xl, xk) which is bounded; writing t 5 s1 5 s sk 5 s then we have, for 5 , = 6(x(sl), x(sk))

...

...,

0

E

B(Rn)

Using (3.26) and (3.28), we can let n

tend to

+m

in (3.29), and

obtain that

P

pair (0,a).

Use Girsanov transformation to obtain a solution of the

is a solution of the martingale problem relative to the

martingale problem relative to the pair

Remark 3.2.

7

(b,a).

We can get rid of the assumption

a

-1

exists and is bounded,

to obtain an existence result, provided that we replace (3.18) by the strongest assumption (3.30) Indeed replace a

SUP

1 x-y 1 s6 by

/bi(x,t)-b.(y,t:l/

p(6)

+

0 as

6

+

0

.

a+EI, then according to Theorem 3.2, there exists

a solution PE relative to the pair

and again

5

b,a+EI.

We have

MARTINGALE PROBLEM

Therefore PE

12s

remains in a relatively weakly compact subset of the set

of measures. Now we have, analogously to (3.29)

By virtue of (3.30) we can let

E +

0 in (3.31) and obtain the desired

0

result.

4. INTERPRETATION OF THE SOLUTION OF P.D.E.

Consider here

aij(x)

a

such that

i Borel bounded on Rn

,

Define (4.3)

Let also (4.4)

. a

Borel bounded

. a t 0 ,

According to Theorems 3.1 and 3.2, there exists one and only one solution of the martingale problem corresponding to the initial condition x

0. More precisely let Q o probability measure

Px

on

, m 0 , n to , there exists one and (Rono) such

that

only one

at

126

CHAPTER I11

(4.5)

PX[x(e)

= XI = 1

(4.6)

@(x(t))

-

@(x) +

Jt A$(x(s))ds &Rn)

.

is a smooth bounded domain of

Rn

,

(Px,?$)

Y @

E

is a martingale

Let

where 0

and consider the solution of

By Sobolev's inclusion theorems, we have (4.9)

u

E

C06)

.

Let

(4.10)

7 =

inf{x(t)

tro

the exit time of the process

Theorem 4 . 1 .

do} x(t)

from 8. We have

Asswne ( 4 . 1 1 , (4.2), 1 4 . 7 1 .

(4.11)

u(x) = EX

JT

f(x(t))(exp

Then we have

- Jt

ao(x(s))ds)dt

Proof 2

,

We can easily extend formula ( 2 . 3 5 ) to the function u(x)exp -

z,

Assume first p
T we have since X(T)

in (4.12).

, hence since

E

a,

is bounded

u

-

EX U(X(TAT) exp

J

TAT ao(x(s))ds

-t

0

.

Formula (4.12) shows that, by the monotone convergence theorem, (4.11) holds. Moreover

Thus formula (4.11) extends to f

E

Lp(8)

,

p >

4.

Let us now give the interpretation of parabolic equations. Let us assume (4.13)

a

I

ij

(x,t)

aij xai

(4.14)

a.

(4.15)

. a

E

=

E

C"'"'2(Rn

x

[O,T])

,

aaij

-E "k

aji

Sitj

2

R

Y 5

1512

L~(R"

x (0,~))

L-(R"

x

(0,~))

.

E

Rn , 8 > 0

Loo

128

CHAPTER I11

Let (4.16)

f

E

Lp(@

(0,T))

x

p >

$+

1

From Theorem 3.4, Chapter 11, there exists one and only one solution of

1

(4.17)

aU - at +

,

0

1 1 1 ~=

+ aou = f

A(t)u

u(x,T)

;(x)

2

where (4.19)

A(t)

-

=

C

a

~

axi

a

a

a.. - + C a 1J ax. i 2,. J i

According to Theorems 3.1 and 3.2, there exists one and only one solution Pxt of the martingale problem relatively to the operator A(t) Let (4.20)

401

S~ = inf{x(s)

S>t and note that since p >

$+

1

,

u

E

Assume 1 4 . 1 3 ) , 1 4 . 1 4 1 ,

Theorem 4 . 2 .

C

0 -

(Q)

,

Q = 8 xl0,TC.

(4.151, (4.161. Then we have

TAT (4.21)

Assume first 2

5

get

p < =.

f

E

u(x,t)

=

B(Q)

, ii

Ext[/

E

f(x(s),s)(exp

t

bo)

.

Then

u

E

- :1 ao(x(X),A)dA)ds

+

b 2 $ ’ * P ( ~ ), for any

Apply formula (2.35) extended asexplained in Theorem 4 . 1 , we

129

MARTINGALE PROBLEM

u(x,t)

=

TAT a ds Ext u(x(TAT~),TAT~) exp - it 0 TAT

+ Ext it

f(x(s),s)(exp

+

- it" aod))ds

from which we easily deduce (4.21).

U = 0 , we first extend (4.21) to functions f We observe the estimate

Taking first p >

+ 1.

Then assume

U

W z Y p n WAY'

(hence in

u

E

W2"

1 n Wo"(0)

COG))

solution of (4.17) with data

and consider a sequence

U +U

,

En, and letting

n

+ 03,

the desired result.

/Ext Jt

TAT^

f(x(s),s)ds

which improvesthe result of Theorem 2.1

Remark 4.2. We also have the estimate

Assume ( 4 . 1 ) , ( 4 . 2 ) and consider the forward Cauchy problem (5.1)

in

we obtain

0

Remark 4.1. We see that we have the es imate (4.22)

Lp(Q) ,

Un E B O ) . Applying ;L.21) with

and f

E

130

CHAPTER I11

with

4

(5.Ibis)

Borel bounded.

Since setting v(x,t) = u(x,T-t)

-

*

+ Av

at

=

0

then v

, v(x,T)

satisfies =

@(x)

we can assert that there exists one and only one solution of (5.1)

such

that

Since v(x,t) and noting

(5.3)

PX'O

=

= EXIt $(x(T))

Px, we can assert that

u(x,t)

=

EX $(x(t))

Y t

2

0

.

We write

which defines a family O(t) bounded functions on that

O(t)

of operators on

B , space of Borel

Rn, provided with the sup norm; it is easy to check

satisfies the following properties

131

MARTINGALE PROBLEM

(5.5) O(0)

O(t)@ Hence

I

=

t

0 if

@

z 0

.

i s a semi group of contractions on

@(t)

B , which preserves

positivity.

C

Denote by

the space of uniformly continuous bounded functions on

B.

which is a subspace of in x,t

for

(5.6)

,

t > 0

if :

O(t)

c

Then from 5 . 2 , we know

B.

@

E

+

c ,

that

u

Rn,

is continuous

Let u s check that

if we assume

In that case, we can consider the S.D.E. (5.8)

dy

=

Y(0)

+ a(y)dw

b(y)dt =

x

(R,Q,P,st ,w(t)) . Since b,a are Lipschitz, according to Theorem 4 . 1 of Chapter I, there is one and only one solution of (5.8). on a system

Denote the solution

yx(t)

.

Then clearly we have

and is an increasing function of

6. We have

132

CHAPTER 111

Therefore

IE O(Yx(t))-EO(y,~

(t))

~

5

E P( IY,(t)-YxI

(t)

5

p(6) +

C(t)

1) /x-x'i2

h2 for any

6 > 0. From this one easily deduces that

uniformly continuous in x, f o r

fixed, t t 0

t

.

@ ( t ) @ ( x ) is

Let us also check

that

(5.10)

in

O(t)b+b

C

as

t + O , ' d ~ c c .

Indeed

from which we obtain (5.10). The infinitesirnaL generator of (5.11)

~7 @

=

lim O(t)@-$ ti0 ~

O t)

is defined by

133

MARTINGALE PROBLEM

The domain of

B , such that (5.11) in the

is the set of functions

0

E

B.

E

C;"

sense of the convergence in EX @(x(t))

=

Assume

$(x)

@

, then from Ito's formula

.

- EX it A @(x(s))ds 0

But when (4.1), (5.7) are satisfied and

I$

E

c2+a b

then

A$

E

CEya

,

hence by the above reasoning we have

Therefore

Thus we have proved that

c ~ +D(c~J ~ ,

(5.13)

Let

b

and Q $

=

-

A@

8 be a smooth bounded domain of Rn. Let

where

T

Assuming

is the exit time from 8 q?

E

x(t)

E

B(6)

we define

.

S(6) , we may consider the non homogeneous Dirichlet

pr ob 1em (5.15)

of

@

1 2+Au=o

CHAPTER 111

134

Since u

-

=

v

satisfies

vIc

=

0

V(X,O)

=

0

then we see that there is one and o n l y one solution of (5.15) such that

Moreover we have

(5.17)

u(x,t) = EX @(x(tAT))

hence by definition of

If

@

E

H

1

@(t)

0 ) , we can give a meaning to (5.15) as follows u - $

(5.19)

E

1

Hob)

Problem (5.19) has one and only one solution such that

(5.20)

u

Uniqueness i s easy. I

E

2

1

L (0,T;H )

du

, ;ir E L

2

(0,T;H

-1

To prove existence we can consider an approximation

.

+ $ in H (s) , @n E B ( ~ ) Then we can define the solution u (5.15) with @ changed into $n. It is easy to check that we have

@n

of

135

MARTINGALE PROBLEM

'd dt (un(t),v)

a(u (t),v)

t

=

V v

0

E

1 Ho

.

Hence

I2 -

/un(t)-um(t) +

Jt

(un(t)-um(t),$n-($m)

-

a (un(s)-um(s) ,un(s)-um(s))ds

- It a

+

.

(u,(s)-~~(s),m~-~~)ds = 0

Using the fact that

($n- @m + 0 in H 1 as 1 2 is a Cauchy sequence in L (0,T;H ) and

n,m

u

2 C(0,T;L )

If

$

n satisfies (5.19), (5.20). E

H 1 (8)n Co(g) , then Qn

+

g?

+ m

,

we get that

.

The limit

in Co(g) ,

Since

we obtain (5.21)

u

E

.

CO( y > O , h = O .

Moreover t h e r e es4sts s i z cptirnal c m t r s l . We start with a Lemma.

Let

P:

0

be the probability defined by (1.13).

Then we have

It is enough to prove (3.5) assuming Y

where

=

E X JrAT $ ( x ( s ) ) d s

4 =

E

.

B(Rn) ,

'42

EX

$(x(s))ds)

0

We have XV(T)

15 1

STOCHASTIC CONTROL

XV(T)

=

exp CJT

0-l

- 71

gv(s)dw(s)

J

T

IU

-1

gv(s)

1

2

dsl

hence

We easily check that EX XV(T) 2

2

CT

independant of

v

.

Next $(x(s))d~)~]"~ [Ex (ITAT

using (4.22) of Chapter 11.

Hence (3.4)

2

C' T ( E x oJTAr $2(x(s))ds)1'2

2

c;\

IMLP since

p > n+2

.

, 3

l y o o f of Theorem 3 . 1 Since h 2

5

p .
n+2.

0

The function u

changed into

0

154

CHAPTER IV

Let us indicate now an other approach to study equation ( 2 . 8 ) ,

called the

method of policy iteration. 0

,...,un...

Let

u

W2"

n W;"

Knowing

, un

,

be a sequence of functions belonging to

p > n

,

defined as follows. We take

define vn(x)

uo

t o be a Bore1 function such that

+ Du"(x).g(x,v"(x)) Define next

un+l

Y x

.

as the solution of the linear equation AU~+'+

(4.15)

arbitrary.

a un+l 0

=

f(x,vn(x))

+ un+ 1 al(x,vn(x))

+

Theorem 3 . 2 .

We make t h e assumptions of Theorem 3.1 and a . 2 0, Lp , p > n , then un c u and in W2'p u e a k l y , where u i s t h e solution of 1 2 . 8 1 .

h

E

Clearly the sequence un al(x,vn(x))

,

g(x,v"(x))

is well defined. Moreover since are bounded,

(3.16)

Next Aun + a un = f(x,vn-l) + unal(x,vn-1)

+

0

+ Dun.g(x,vn-') t

f(x,vn) + unal(x,vn)

+

+

h

2

Dun.g(x,vn)

+

h

f(x,v"(x))

,

155

STOCHASTIC CONTROL

hence ~ ( u ~ + l - u+~ a ) (un+l-un) 0

u which implies u Hence

un

i

u

n -u 1,

n+ 1 +n

5

0

=

0

-

D(un+l -u n ).g(x,vn)

o

(un+l -un)al(x,vn) 5

n+l

-

(recall ( 3 . 3 ) ) .

pointwise and in W2"

weakly.

Therefore also, by compactness (3.17)

un

-t

u

in W'VP

Let us identify the limit.

We have for v

E

V

arbitrary

Aun + aOun

-

f(x,v)

+ aOun

-

f(x,vn)

~u~ + aOun

-

f(x,vn) + (un+l-un)a 1 (x,vn)

S Au"

=

strongly.

-

+ (Du n + l -Dun ).p(x,v")

unal(x,v)

-

unal(x,vn)

n+l)

n

+ (Dun+l-Dun).g(x,vn)

-+

Du".g(x,v)

-

+ (u

n+l

0 in

-

h

Uun.g(x,vn)

Dun+ 1 .g(x,vn)

-

= A(U~-U~+') + a0(u -u

-

-

-

h

=

.

Therefore Au + aou V V € V which means

-

f(x,v)

-

ua ( x , v ) 1

-

Du.g(x,v)

-

h

=

un+'al(x,vn) +

n -u )al(x,vn) +

Lp weakly

-

5

h 5 0

156

CHAPTER IV

(3.18)

AU + a u

-

H(x,u,Du)

S

h

Au + a u

-

H(x,u,Du)

-

h

0

a.e.

Also we have

0

2

Au + a u 0

=

+ m,

-

f(x,vn)

h + Aun + aOun

+

I

Au + a u 0

-

H(x,u,Du)

-

Dun.g(x,vn)

(DU"-D~) .g(x,v")

the right hand side tends to

which with (3.18)

ua (x,vn)

-

-

f(x,vn) - h =

-

h

Z

0 in L p

-

unal(x,vn)

weakly, hence

0

concludes to the desired result.

Let us give the analogue of the situation studied in section 2 and 3 .

4 . 1 . perabgljc-equetjgns

We consider here functions f(x,v,t)

: R" x ?/ x

g(x,v,t)

: R"

c(x,v,t)

: R" x ? / x C0,TI

x T

Bore1 and bounded and set

x

=

+ (un-u)al (x,vn)

4. EVOLUTION PROBLEMS

(4.1)

-

0

= ~(u-u") + a (u-u") 0

n

Du.g(x,vn)

~(u-u") + a (u-u") + (~u"-~u).g(x,v") + (un-u)a,(x,vn)

-

and as

-

[O,Tl

+

R

C0,Tl

+

Rn

+

R

0

157

STOCHASTIC CONTROL

(4.2)

H(x,t,X,p)

=

infCf(x,v,t)

-

Ac(x,v,t) + p.g(x,v,t)]

VEV

.

Next we assume (4.3)

CO,a,a/2

aij

E

h

Lp(Q)

aaij __

E

axk

3

Lrn

(4.4)

Let (4.5)

where 8

E

Q =8

x

, U

(0,T)

is a smooth bounded domain of

E

1

W2” n W0”

Rn.

Then we have

Theorem 4 . 1 .

We assume ( 4 . 1 ) , ( 4 . 3 1 , (4.4), (4.51. Then t h e r e e x i s t s

one and o n l y one s o h t i o n of (4.6)

b2y”p(Q)

u

E

-

au at

uIC

=

+

A(t)u

0

-

H(x,t,u,Du) = 0

, u(x,T)

=

u(x)

.

Similar to Theorem 3.4 of Chapter I1 and Theorem 2 . 1 . We can next give the interpretation o f the function u. (4.7)

f(x,v,t),g(x,v,t),c(x,v,t)

and measurable

We assume

are continuous in v

with respect to x,t,

V v

.

,

a.e.x.t

158

CHAPTER IV

(4.8)

Let

?J is a compact subset of

u

=

f(x,v,t)

which is Lebesgue measurable in a.e.

.

belong to I J J ~ ’ ~ ’ ~ (,Q )and define LU(X,V,t)

x,t.

OU(x,t)

Rd

x,t

-

u(x,t)c(x,v,t)

for any

It is a Caratheodory function.

v

+

and continuous in v

Hence there exists a function

which is Lebesgue measurable and such that

We can take

GU(x,t)

to be a Borel representative.

In the sequel, the

results will not depend on the choice of the Borel representative.

Let

Pxt

operator

be the solution of the martingale problem relatively to the A(t)

, with initial conditions (x,t).

Then if

x(s)

is the

canonical process as usual, we have

A control is an adapted process with values in V .

v(s)

(with respect to the family

We define the measure

:)

Pzt such that

(4.11)

and

Ptt

is the unique solution of the problem of controlled martingales

159

STOCHASTIC CONTROL

(4.12)

And (4.13)

where

We can state the

Theorem 4.2.

We make the a s s q t i o m of Theorem 4 . 1 and

7de

Then t h e s o l u t i o n of 14.61 is given explicitely by

h = 0.

(4.15)

u(x,t) = inf

J~~(V(.))

p > n+2.

.

V(.) ~ v ~ G P e G V e rzhere ,

(4.16)

exists an cptirnal control O ( s ) = Ou(x(s),s)

.

O(s)

defined by

CHAPTER IV

160

Similar to that of Theorem 3.1.

Theorem 4.3.

We make the assumptions on Theorem 4.2.

control v(.)

, the process SAT

u ( x ( s ~ ~ ~ ) , s ~ ? ~ ) -e xitp

for

t 5 s 2 T,

is a sub martingale

(PVXt

Then for any

t c(x(X),v(X))dX

A;)

-

u(x,t)

+

.

For v = 0 , it is a martingale.

Let

5,

be

measurable and bounded.

Using equation ( 4 . 6 ) we obtain

We have for

t 5 s 5 @ 5

T ,

161

STOCHASTIC CONTROL

This proves the first part of the theorem. Taking to

v

=

0 , we have equality

0

0, hence the second part of the theorem.

Remark 4.1. verifies

1.1~

It is easy to check that if =

0

,

u(x,T)

=

0

then ( 4 . 1 5 ) holds and

u

is Bore1 bounded and

and the properties of Theorem 4 . 3 ,

;(x)

3

is optimal.

5 . S E M I GROUP FORMULATION

5 * 1.

4-eroee r tr -2f - the -E9uatjon- - u

Let us go back t o equation ( 2 . 8 ) , and we will assume here (5.1)

For

a]

v

E

=

, .a

0

= c1

> 0 a constant

2 , a parameter we consider the operator Av = A

(5.3)

We note that

u

-

g(x,v).D

.

satisfies

(5.4)

1 Moreover let

w

A u + au s f

a.e.

in 8

.

satisfy ( 5 . 4 ) , then we have Aw + uw S f(x,v) + Dw.g(x,v) Au + au = inf[f(x,v) V

hence

fi v

+ Du.g(x,v)l

162

CHAPTER I V

A(w-u) +

U(W-U)

2

f(x,v) + DU g(x,v)

-

-

+ Du.g(x,v)l

inf[f(x,v

+

V

+ (Dw - Du).g(x,v)

< f(x,v) + Du.g(x,v) - inf[f(x

v ) + Du.g(x,v)l

+

V

+ ID(w-u) hence taking the inf in v

,

C

we obtain

(5.5)

Condition (5.5) imp1ies w - U S O .

(5.6)

Relation (5.6) is clear when a to prove the following result.

is large enough. Otherwise we have Let

h

E

0

.

Lp

given and

z

to be the

solution of the H.J.B. equation

I

then (5.8)

Indeed

h z

S

0

implies

z 2

can be obtained as the limit of the following iteration

163

STOCHASTIC CONTROL

(5.9)

zo

starting with

0 , and

=

zn Since

h

0

5

+

z

in

WzYp weakly

, one checks inductively that zn

5

0

,

hence

z

s 0

.

This

proves ( 5 . 8 ) , hence (5.6). We thus have proved the following ble make t h e assumptions o f Theorem 2.1,

Tkeorem 5 . 2 .

Then t h e soZution

u

and (5.1), ( 5 . 2 ) .

~f (2.8) i s t h e maximum element of ;he s e t o f

0

f u n c t i o n s s a t i s f y i n g (5.4). Remark 5 . 1 .

Assumption

(5.1) can be weakened, but it will be sufficient

0

for the remainder of the chapter. We note now

the solution of the martingale problem corresponding to

P:

the operator

Av

,

starting in x

at time

0

controlled martingale problem, with a control S,L.

Let

u

.

It corresponds to a

v(s)

=

s

, independant of

be a function satisfying ( 5 . 4 ) , then from Ito's formula

we have

= o hence u(x) Recalling that

ui,.

=

5

Ez[{tmfv(x(s))e-"sds

0

, we also have

+ E:u(x(tAT))e

-CltAT 1

.

164

CHAPTER IV

.

0

x (x(s~r))e-"~ds] + E:u(x(th~))e-'~l v o

which we have considered in

§

5.2 of Chapter I11 and noting that

(5.12)

+

Bo

u(x)

(5.10)

5

:E

[Jt f

Using the semi group

where

OV(t) : Bo

Bo

is the set of Borel bounded functions on

r,

we see that

(5.13)

u

on

u

6,

which vanish

satisfies the relation

s Jt OV(s)fv

w - " s d s + OV(t)u

This motivates the problem which will be studied in the next paragraph.

We make here the following assumptions.

Let

E

be a polish space

(1)

provided with the Borel a-algebra 8 . We note

B

the space of Borel bounded functions on

E.

uniformly continuous function on

E, C

the space of

We assume given a family

v c 1J, where (5.14)

'V

(5.15)

OV(t) : B

finite set

OV(t)OV(s) OV(t)Q

t

=

o

,

B

-f

Ov(0) = I

.

OV(t+s) if

Q

2

o

(1) This will be needed for the probabilistic interpretation.

OV(t)

,

165

STOCHASTIC CONTROL

We will also assume that (5.16)

aV(t)

(5.17)

t

-t

c

+

c

oV(t)G(x)

E

u

5

To find

u

maximum solution of

+ OV(t)ue-at

u e B

= fV(X)X

(4=

8

R ,

a > 0

1t 0v (s)Lve-"'ds

Lv(x)

-t

C

,

L (x) :L(x,v)

We consider the following problem. (5.19)

(0,m)

is continuous from

x fixed, Y I$

Y (5.18)

:

f(x,v)x

(x)

0

,

166

CHAPTER IV

t > 0

For

z(x,t)

is a regular function of

hence (5.18) is

x,t

satisfied. We will study ( 5 . 1 9 ) by a discretization procedure. Uh

h > 0

Let

define

by

(5.22)

uh

=

Min!Jh v

e-US QV(s)Lvds

+

QV(h)uhl

-"here mists one and onZg one s c l u t i o r cf (2.22).

Define for

z

E

C

T z h

.

B

E

z =

MinCJh e-"' v

since 7J'

'

OV(s)Lvds

is a finite set.

+

Note also that

which proves that

Th

T z h

z c B

such t h a t

z S

T z h

Th is increasing we have

2

T z S T z h h z 5

T2z h

E

C , when

0

proof

hence

C

is a contraction, hence has one and only one fixed

point , uh.

Since

t

OV(h)zl

Moreover

L e m o 5.2. L e t

uh

B

T and

,

o

S e m a 5.1.

z E

,

and by induction z S

Tnz h

+

u

h a s n - t m '

then

z

< uh '

167

STOCHASTIC CONTROL

Lemma 5 . 3 .

Ue have

(5.23)

Uh 5 UZh

.

procs We have f o r any

v

Uh 5

{

u

LZh e-"'

OV(s)Lvds

+ evah OV(h)uh

hence

which implies
- K h

which i m p l i e s ( 5 . 2 4 ) . Let u s s e t

q f +

then a s

u

m

that

q

uh s

Take

I h = 24

(5.25)

,

q

5

R 5 q

5

hence

q

,

j

imh e-"

integer.

R

/j/'

o

,

j

Let

u

Note t h a t

is

U.S.C.

Furthermore we s e e

OV(s)Lvds + e - a h 0v (rnh)uh

m = j 2q-R

and u

.

C u

R

with

e-"

5

,

q

QV(s)Lvds

+

ti rn

we g e t

e-a"2

9.

~ ~ ( j / 2 " . u, ~

integer

.

q

According t o Lemma 5 . 4 below,

-f

a.

integer

.

169

STOCHASTIC CONTROL

(5.26)

ti q.

j

Take next

= Ct

2

9"

I+

1

and let

R

tend to

+a

, we deduce from

( 5 . 2 6 ) using assumption ( 5 . 1 7 )

in which we may again let (5.19).

q

tend to

+m.

This proves that

,.

It is the maximum solution, since assuming u

u

satisfies

to satisfy

( 5 . 1 9 ) then clearly

which implies

-u

5

uh , hence the desired result.

G

Let us now state the result which has been used in the proof of Theorem 5 . 2 .

We refer to Dynkin 121. Let us briefly mention the main elements of the proof.

Let

Banach space.

be the space of bounded measures on We write

Then we have the following result

( E , 6 ) , which is a

170

CHAPTER IV

m

Define next an operator on

One checks that

by the formula

is a contraction semi group o n n , and that the

U(t)

following relation holds,

0

From (5.29) and (5.27) the desired result follows. Remark 5.2.

In example (5.20), we have

uh

Co =

E

T, and of course u

functions which vanish on

E

subspace of

.

Bo

C

of

L

Let u s define

R0

=

E

I

,

x(t;w)

is the canonical process,

Let u s assume for simplicity that

7 To

i

E

b'

=

...,m} .

{I,?,

, we associate a probability PTt

(5.30)

E2t @(x(s))

We will denote by values in V .

W

=

Oi(s-t)6(x)

Ro

, nt

fi s 2 t

such that

.

the class of step processes adapted to

V

E

... 5

7

More precisely, if To =

on

0

s

T1 5

not

with

W , then there exists a sequence 5

...

which is deterministic increasing and convergent to

+x

and

171

STOCHASTIC CONTROL

v

(5.31)

=

, v(t;o)

v(.)

=

vn(w)

t

€

IT,,?

n+ 1 )

T

where

is ?!on

v

measurable.

9"

We next define a family the pair

w,t

(w

E

07; , Oss A

,

.

u c cO"(E)

z c c'"((E).

Let u s fix xo

in

E , then there exists vO

(depending on xo) such that ThZ(XO) * j h e-as CJv~ ( s ) L Let x

vO

(xo)ds +

Ovo(h)a(xO)

arbitrary, we have Thz(x)

5

Jh

(x)ds +

Pvo(s)Lv

0 hence by difference

and from the assumptions (5.56), (5.57) it follows that

Ovo(h)z(x)

179

STOCHASTIC CONTROL

5

Lh e-asei's

Klx-xo

1 'ds

+

hence

and since

xo,x

are arbitrary, this implies

and iterating we see that

and letting

k

tend to

+a

,

it follows that

K

I I ~ h I 5l ~

Taking now h

(5.59)

=

1 and letting 24

IlU

Il&q

q

-f

m,

we deduce

K

which implies the desired result. Let u s now give an other regularity result. We assume (5.60)

I

I Z I

86 16 Ix-xgl

180

CHAPTER IV The rnuximwn so2ution of 15.191 is a l s o the maximwn s o l u t i o n of

L e m a 5.6. (5.61)

U E B ,

u

5

Jt e-”

tr v

+ e-Bt OV(t)u

Qv(s)(Lv+(G-a)u)ds

Y t

E l J ,

2 0

Proof We first show that (5.61) has a maximum element, which will be denoted by N

u.

Indeed, define for Bhz

=

z

E

B + e-Bh O v ( h ) z l

Min[Jh e-Bs aV(s)(Lv+(B-a)z)ds v o

It is well defined, by virtue if ( 5 . 6 0 ) . This is a contraction, since

’

I / o ~- ~~ ~~

z 5~ / jI z 1j-2 2 ’ = 1l2,-22/

Moreover when Let when

..r

uh

z

,

C

E

Ohz

E

.

C

-

be the fixed point, uh

z 2 0

.

6

C

Y

One checks as for Theorem 5.2 that

- -

Setting u

q

=

u

1/29

, we get

Y

u

.

J- u

-

uh 2 0

N

uh U

q

and

5

uZh

, since

Ohz 2 0

.

Then

9, 4

S

/j’*

0

e-Bs OV(s)(Lv+(C-a)u

L

and as for Theorem 5 . 2 , one checks that

,u.

4

)ds

t

is a solution of ( 5 . 6 1 ) , and

181

STOCHASTIC CONTROL

that it is the maximum solution, since any other solution satisfies v

5

0hv

,

-

v < uh '

hence

w

Let us show that u = u , where

u

is the maximum element of (5.19).

We will use Lemma 5.7 below to assert that & < I t e-"'

-

hence u

5

u.

w

i e m a 5.7.

,

Let

where (5.63)

E

B.

0

and the desired result is proved. O(t)

be a semi group on Let

w

w < Jt g

+ e-Bt OV(t)u

Jt e-Bs OV(s)(Lv+(B-a)u)ds

( 5 . 1 5 ) , and is.&Ol.

(5.62)

OV(t)u"

However, still using Lemma 5.7 we have u

hence u < u

+

QV(s)Lvds

@(s)g

5

ds +

@(t)w

B > 0 , one has

Then f o r any w

satisfying properties

be such t h a t

B

E

B

it e-as @(s)(g+(B-a)w)ds

+ edBt O(t)w

We set

we have H(0)

=

0

,

H(t)

5

0

In fact, we have the additional property

Y t

,Y

t t 0

182

(5.64)

CHAPTER IV

H(t) s H(s)

for

.

t 2 s

Indeed ( 5 . 6 4 ) amounts t o proving t h a t (5.65)

e

-us

O(s)w

O ( t ) w + Jst e-"O(A)g

5

,

di,

.

s 5 t

But f r o m ( 5 . 6 2 )

w s /

t-S

e

0

-a>

0 and

and i n t e g r a t i n g between

+ =

-a ( t - s )

O(t-s)w

we deduce

T

[ l-e-(B-a)Tjw =

+ /T ( 3 - i ) e

dA + e

O(A)g

JT (B-a)ewbt O ( t ) w d t + 0

H(t)dt

+

iT ( 3 - a ) e- ( 3 - a ) t ( i t e-" O ( s ) g d s ) d t iT (6-a)e - 6 t O ( t ) w d t + iT ( 0 - a ) e - ( p a ) t =

- e

-(B-a)T /T e - 2t 3 ( t ) g d t +

iT e-r't

H(t)dt

O(t)g d t

hence (5.66)

w =

/T .-3t 0

O(t)(g+(?-?)w)dt

+ e

-6T

O(T)w

+

- (6-3)T H ( T ) + J .T ( 3 - 5 ) e - (5-3) t H ( t ) d t

+ e

a

.

-

183

STOCHASTIC CONTROL

If B 2 u since H(t) < 0 , we clearly have ( 5 . 6 3 ) with 6 < a then using ( 5 . 6 4 ) we have

If

t = T.

hence H(T) +

e-(B-a)T

iT(@-a)e-(B-a)t

H(t)dt

H(T) + (@-a)H(T) JT e-(B-a)t

< e

therefore ( 5 . 6 3 ) holds in all cases for

t = T

.

5

dt

Since T

H(T)

=

5

is

C

arbitrary the desired result is proved.

Theorem 5 . 6 .

Let

z

We make t h e assumptions of Theorem 5.2,

15.561,

Then the m a x i m s o l u t i o n o f (5.19) belongs t o

(5.60).

E

C

5

and

5

1 5 . 5 7 1 and

C.

be the maximum element of

5

it e-Bs Ov(s)(Lv

+ (8-a)z)ds + e-Bt OV(t)

184

CHAPTER IV

Ch

where

=

Sh(z)

is defined by

ch

(5.67)

Min[Jh e-@'

=

v

OV(s)(Lv+(@-a)z)ds

o

+ e-ah QV(h)ch]

< h e C * Sh : B

Note that

B

-t

and

C

C

+

.

One easily checks the estimate

I Ish(zI)-sh(z2) 1 1

(5.68)

5

71

1z1-z21 j

from which one deduces

I IS(zl)-S(z2) 1 I

(5.69)

5

a-a 112 - 2 1 I , p 1 2'

when

We also note the relation, which follows from Lemma 5.7, (5.70)

u

5

sh (u)

.

Define now

,

un = S"(0)

Since

s

maps

coy6

u;

=

.

s;(o)

into itself, u n c

c0j6.

From (5.69) we have n+l

IIU

n

' I1

B-a n 1 (T)1Iu 1

and thus un+w We will show that

(5.71)

u = w .

in

C .

1

z 1 ,z2

E

C

185

STOCHASTIC CONTROL

which w i l l prove t h e d e s i r e d r e s u l t . We f i r s t remark t h a t from ( 5 . 6 8 ) belongs t o

C

,

has a fixed point i n

and

wh

s ~ ( o )+ wh

(5.72)

From ( 5 . 6 9 ) ,

denoted by

Sh

c

in

.

( 5 . 6 8 ) we have

(5.73) (5.74)

From ( 5 . 7 0 ) we c a n a s s e r t t h a t (5.75)

u s w h

We check by i n d u c t i o n on

n

that

un h

-

By i n d u c t i o n on

n

we check t h a t

(5.76)

un+un q

un 2h

hence

From ( 5 . 7 3 ) ,

as

q f m ,

=

~

( 5 . 7 4 ) , (5.76) follows t h a t

wq(x) 4- w(x) which w i t h ( 5 . 7 5 ) shows t h a t (5.77)

Gn

u < w .

v

x

un.

Hence n

.

B

,

which

186

CHAPTER IV

But a l s o

hence

+ e-pph OV(ph)wh

w < Jph edBS OV(s)(LV+(B-a)wh)ds h - 0 hence also

for

q 2 R.

Using a reasoning as in Theorem 5 . 2 , we obtain easily that w

5

+ e-Rt @"(t)w

J t e-Bs OV(s)(Lv+(p-a)w)ds

hence also, using Lemma 5.7 w which implies w

it

5

0

5 u

,

@"(s)Lvds

+ e-"

QV(t)w

and from (5.77) we see that ( 5 . 7 1 ) holds.

ci

completes the proof. Let us give an example where ( 5 . 5 7 ) is satisfied, with Consider the S.D.E, dy with (5.78)

This

=

g(y)dt

+

o(y)dw

y(0)

=

x

6

=

0

.

187

STOCHASTIC CONTROL

hence

which proves (5.77).

Remark 5 . 4 .

For other details (cf. M. Nisi0 [l], Bensoussan-Robin C11,

0

Bensoussan-Lions 121). COMMENTS ON CHAPTER I V

I . The method of improvement of regularity used in Theorem 2.1 is due to

P.L. Lions. 2 . Assumption (3.1) can be replaced by Lebesgue measurable in x

continuous in v

as mentionned in the evolution case

§

4.1.

, and In fact

we need a selection theorem. There are two types of such theorems that we may use. Consider Assume

F(x,v)

,

x

E

Rn

,

v

E

3'(compact subset of a metric space).

188

CHAPTER IV

F

1.s.c.

in x,v, F

bounded below.

Then there exists a Borel function C(x) F(x,C(x))

+V,

: Rn

such that

Y x

= inf F(x,v) V

(see for instance D. Berksekas, S . E . Shreve [ I ] ) . The other theorem uses more explicitely the Lebesgue measure on We assume that F F

Rn.

is a Caratheodory function, i.e.

, continuous in v, a.e. x.

V v

is Lebesgue measurable in x

Then there exists a Lebesgue measurable function C(x)

: Rn

+ V , such

that F(x,?(x))

=

inf F(x,v) a.e. V

We can take a Borel representation of We write

inf

G(x)

,

but it is not unique.

is a Lebesgue measurable

V

function such that if

?(x)

, which

for ess inf F(x,v)

5

then G(x)

F(x,v) 5

a.e.

Y v

ess infF(x,v)

a.e,

V

Note that

inf F(x,v)

when

F(x,v)

is Borel for any v

is not a

V

Borel function (cf. I. Ekeland - R. Temam [ I ] ) . 3 . The method of policy iteration was introduced by R. Bellman [ l ] ,

in the

general context of Dynamic Programming.

4 . For the study of degenerate Dynamic Programming equations (i.e., the matrix

-1

a

does not necessarily exist) we refer to P.L. Lions

-

J.L. Menaldi C11. 5. J.P. Quadrat has formulated a generalized martingale control problem,

which includes degeneracy (cf. J.P. Quadrat [ I ] ,

[Z]).

189

STOCHASTIC CONTROL

6. For numerical techniques t o solve the H.J.B. equation see J.P. Quadrat [ l l , P.L. Lions

- B.

Mercier [I].

7 . A s we have said in the general introduction, the most complete

treatment of the general Bellman equation is due t o P . L . Lions [ll,

[ZI. 8. The problem of semi group enveloppe was introduced by M. Nisi0 C21

9. Nisi0 has also introduced a problem of non linear semi group connected to stochastic control (cf. M. Nisi0

[ll).

10. In the context of Remark 5 . 3 . Under what conditions can we assert

that the solution u

of (5.55) coincides with that of ( 2 . 8 ) .

191

CHAPTER FILTERING AND PREDICTION

V

FOR

LINEAR S.D.E.

INTRODUCTION We present here the classical theory of linear filtering, due to R.E. Kalman [ l ] , R.E. Kalman - R.S. Bucy [ l l . Xe have chosen a presentation which can be easily carried over to infinite dimensional systems, for which we refer to A. Bensoussan [ I ] , R. Curtain - P.L. Falb [ l ] , R. Curtain - A . J . Pritchard [ l ] . For filtering of jump processes (cf. P. Bremaud [ l ] ) . For non linear filtering, cf. R. Bucy - P. Joseph [ I ] , and the recent developments in E. Pardoux [ I ] , T. Allinger S.K. Mitter [ I ] .

1. SETTING OF THE PROBLEM

We consider a usual system

(S2,a,P,5t,w(t)), and

solution of the linear S.D.E.

where (1.2)

F

E

L~(O,~$(R";R"))

G

E

L~(O,~;~(R";R"))

f(.)

E

L ~ ( o , ~ ; R ,~ )

Clearly the standard theory applies since

x(t)

to be the

192

CHAPTER V

g(x,t)

=

F(x)x

o(x,t)

=

G(t)

+ f(t)

.

5 is gaussian with mean x and covariance matrix

a

To the O.D.E.

corresponds a fundamental matrix

such that the solution of (1.4)

@(t,T)

can be expressed as x(t)

(1.5)

where

g

E

=

2

L (0,m;R").

@(t,O)x

+

Jt

The family

(1.6)

@(t,S)@(S,T)

(1.7)

@(t,t)

=

I

=

@(t,r)g(r)dT has the group property

@(t.T)

@(t,T)

d '

t,S,T

.

It is easy to check that the solution of (1.1) y(t) = @(t,O)c

(1.8)

+

Jt

can be expressed by

@(t,T)f(?)d?

+

It @(t,?)G(~)dw(r)

where the last integral is a stochastic integral. Formula (1.8) is a representation formula for the process

y(t).

It is also useful to

notice the following. Let (1.9)

h

E

Rn

and

-3 dt

=

F*(t)$

,

$(T)

=

h

FILTERING AND PREDICTION

193

then we have (1.10)

@(0).5 +

y(T).h

=

p(t)

O*(T,t)h

@(t).f(t)dt

+

iT @(t).G(t)dw(t)

.

Since

(1.11)

=

it is easy to deduce (1.8) from (1.10) and ( 1 . 1 1 )

It is clear from (1.8) or (1.10) that expectation y ( T ) (1.12)

i s a Gaussian variable with

y(T)

such that Q(T,O)x + {T @(T,t)f(t)dt

y(T)

=

-dy- -

F(t)y

;(t)

=

i.e. (1.13)

dt

,

y(0) = x

.

Let Y(t)

-

y(t)

then from (1.10) (1.14)

-Y(T).h

=

O(O).t

where (1.15)

Define next (1.16)

then

- b-'' dt

=

F* (t)+

+

iT @(t).G(t)dw(t)

194

CHAPTER V

hence from ( 1 . 1 4 ) , (1.18)

(1.17)

we deduce

E F(T).h

= Po @ ( O ) . $ ( O )

y(T).k

+

JT

G*(t)@(t).G*(t)$(t)dt

= II(T)h.k

where

II(T)

denotes the covariance operator of

y(T) (or

y(T)).

Hence we have the f o r m u l a (1.19)

n(T)h.k

(1.20)

n(T)

=

Po @ ( o ) . $ ( O )

=

+

O*(T,O) +

O(T,O)PO

J

T

G(t)G*(t)q(t).+(t)dt

iT O(T,t)G(t)G*(t)O*(T,t)dt

We will set for simplicity (1.21)

G(t)G*(t)

=

Q(t)

We can deduce from (1.20) that

.

II is solution of a differential

equation. We have (1.22)

TI(T)h.k

=

Po @*(T,O)h.O*(T,O)k

t

+ JT Q(t) @*(T,t)h.O*(T,t)k 0

The function s

1

O(s,t)h

E

H (t,T;Rn) , and

O*(s,t)h

E

H 1 (t,T;Rn)

+

hence

and

dt

.

.

195


(1.24)

d O*(s,t)h ds

=

O*(s,t)F*(s)h

We can approximate (1.22) with respect to

. T , using ( 1 . 2 4 ) .

We

obtain

dT

h.k

= Po

+

O*(T,O)F"(T)h.@*(T,O)k

+ Po O*(T,O)h.O*(T,O)F*(T)k +

Q(T)h.k

+

+ JT (Q(t)@*(T,t)F*(T)h.@*(T,t)k

.

+Q( t ) @*(T,t ) h O*(T, t) F*( T) k)d t

and from (1.20) we get

We thus have proved

L e m a 1.1.

The process

y

soZution of (1.1) i s

whose mathematical expectation y(t)

covariance matrix n(t) (1.25)

We next define a process

where (1.27)

G

Gaussian process

is solution of 11.131 and whose i s so2zrtion of the equation

z(t)

by setting

+

196

CHAPTER V

(1.28)

is a

n(t)

Rn

and

(1.29)

R

Y 0 is

e.n(t)

Zt continuous martingale with values in E

Rp , the increasing process of

it R(s)B.Bds

,

where

R

is symmetric invertible and R-1

E

Lm(O,m$(Tn;RP)) bounded.

From the representation theorem of continuous martingales, we have q(t) =

Lt R”2(s)db(s)

,

where

b

is a standard 3

Wiener process. We also assume q(t)

(1.30)

It i s clear that

Z(t) (1.31)

is independant from

z(t)

5

and

w(.)

.

is a Gaussian process, whose expectation

is given by

i ( t ) = {t H(s)y(s)ds

.

Set

-z(t)

=

z(t)

(1.32)

-z(t)

=

it H(s)y(s)ds

(1.33)

E Y*(s,)?(s,)

- Z(t)

then

=

+ q(t)

.

a(s1,s2)~(s2)

if

s 1 2 s2

t

197

F I L T E R I N G AND P R E D I C T I O N

= @(sl,s2)~(s2) +

Jssl @(sl,s)G(s)dw(s) 2

hence

Let h , k

E

R”.

We have from (1.34)

E y(sl).h y(s2).k

=

E Y(s2).0*(s1,s2)h

=

iI(s2)O*(s1,s2)h.k

y(s2).k

therefore

from which we deduce (1.33).

It is easy to deduce from (1.33) and (1.20)

that

From (1.34) and (1.32) it is easy, although tedious to deduce the covariance matrix of

z(t)

We consider that the process be observed.

and the correlation function.

y(t)

cannot be observed, whereas

z(t)

can

The filtering problem consists in estimating the value of

y(t), knowing the past observations. More precisely, we are interested in

We note the following

198

Lema 1.2.

CHAPTER V

Ye have

(1.36)

a(z(s),O<s

- H*(t)

(q(t) + R-’(t)zd{t))

X(T) = h

with the payoff

-

2 x.X(O)

This problem i s the following

.

209

FILTERING AND PREDICTION The optimal control 9 (3.5)

G(t)

is given by =

-

.

R-'(t)H(t)y,(t)

Define next

hence the pair

Y

-

yT,pT

is solution o f

(3.7)

-

- H*(t)R-'(t)(zd

-YT(0)

=

- Po PT(0)

Now from (3.7) and (2.25) we deduce

and from (2.26) we see that

,

H(t)y)

-pT(T)

= 0

.

2 10

CHAPTER V

Y,(T).h

(3.8)

=

(zd

-

H(.)Y(.),?$)

-

H(.)?(.))

therefore

-yT(T)

(3.9)

=

CT(zd

0

which with ( 2 . 2 7 ) completes the proof of ( 3 . 2 ) .

We write (3.10)

r(T) = r(T;zd) = iTzd + y(T)

-

iT H(.)?(.)

then from ( 3 . 2 ) we see that (3.11)

yT(T)

=

-

r(T)

P(T)h

.

Our next task will be to obtain evolution formulas for the pair r(.) P(.).

3.2.

&arsjye-formulas

We will describe a decoupling argument on system ( 3 . 1 ) .

L e m a 3.1.

Functions

(3.12)

yT(t)

,

pT t )

are r e Z a t e d by t h e r e l a t i c n

yT(t) = r(t) - p t)pT(t)

v

t

E

C0,TI.

proos be convenient to denote by y t) ~ ~ , ~ ( the t ) solution of T,h to emphasize the dependance in h Consider next system ( 3 . 1 ) (3.11, on the interval (0,t) instead of (0,T) with final condition pT(t) It wil

instead of (s)

h.

More precisely consider the functions

with

s 2 t.

Yt,PT(t)

(s)

,

Since Y ~ , ~ ( s ), P ~ , ~ ( s )satisfy this system and since the solution is unique, we can assert that

21 1


(3.13)

Pt 'PT (t) But by definition of

(s)

=

pT(s)

for

s 5 t

r,P we have

which with (3.13) proves ( 3 . 1 2 ) .

;emu 3 . 2 .

he fitnetion

P(t) c H'(O,T~(R~;R"))

alzd is the unique solzrtion o f t h e eqzration. o f R i c c a t i type

(3.4)

1

P(0)

P 0

=

?roo _" By Lemma 3.1 applied to system ( 2 . 2 5 ) we may assert that

(3.5)

aT(t)

=

.

-

P(t)BT(t)

=

1 DaT.aTdt + PoBT(0).QT(O)

Now from ( 2 . 2 4 ) we have P(T)h.h

T

+

CT QBT.BTdt

where we have set (3.6)

D(t)

=

H*(t)R-'(t)H(t)

From (3.5) we obtain (dropping index T (3.7)

P(T)h.h

=

on aT,BT)

T (PDP+Q)B.Bdt + PoB(0).B(O)

j 0

.

.

212

CHAPTER V

Next from the second equation (2.25) we get

- da = dt

(3.8)

(F*-Dp)a

B(T) = h

.

Define ( 3 9)

OP(t,S

9

F(t)

=

fundamental matrix associated to

-

Then the solution of ( 3 . 8) can be expressed as

which, going back to ( 3 . 7 ) yields

hence (3.10)

By analogy with (1.20), (1.25), formula ( 3 . 1 0 ) proves that differentiable in fact that i f c1

P

t

and that ( 3 . 4 ) holds.

is

Uniqueness follows from the

is a solution of ( 3 . 4 ) then defining

by ( 3 . 5 ) then the pair

in a unique way.

P(t)

6 by (3.8) and

a,@ is solution of (2.25) and thus defined

Applying (3.5) at time

T

shows that

P (T)h = P2(T)h

if

Pl,P2 are two solutions. Since h

uniqueness follows

is arbitrary, as well as

T, the

3

213


The f u n c t i o n

L e m a 3. 3.

r

E

1 H (0,T;R") and i s t h e unique s o l u t i o n o f

+ f(t) + P(t)H*(t)R-'(t)zd

dr = (F(t)-P(t)D(t))r(t)

(3.11)

r(0)

=

x

.

proos By (3.12) it is clear that

r

E

1 H (0,T;R").

Also

dt =

F~

-

Q~ + f + (FP+PF*

-

PDP+Q)~+

+ P(-F*~-D~+H * R-1 zd)

=

(F-PD)~+ (F-PD)P

P

*R-1

+ f + PH

zd

i.e. (3.11). Moreover r(0)

=

y(0) + P(O)p(O)

= x

which completes the proof of the desired result.

3.3.

Falr!ian-fjlter

Let us consider the observation process (1.26). z(t) where

b

is a

= Jt

H(s)y(s)ds

+

We have

it R''2(s)db(s)

Zt standard Wiener process with values in Rp. We

consider the S.D.E.

214

CHAPTER V

(3.12)

d9

((F(t)-P(t)D(t))?(t)

=

t

P(t)D(t)y(t)

+ f(t))dt

t

+ P( t)H*( t)R-’/’ (t)db (t) 9(0) = x

.

(3.13)

+

IT Op(T,t)P(t)H*(t)R-’/’(t)db(t)

.

Since dz = H y dt

R 1 ” db

t

we can rewrite (3.13) as (3.14)

?(T)

=

+

Op(T,O)x +

JT

{T

O,(T,t)P(t)H*(t)R-’(t)dz(t)

Op(T,t)f(t)dt

t

.

Now from (3.11) we also can assert (3.15)

r(T)

=

Op(T,O)x +

T Oh(T,t) (f(t)+P(t)H*(t)R-’

(t)zd(t))dt

and (3.16) Take x

r(T).h =

0

,

f

(3.17)

=

0

= (i;h,zd) + y(T).h

then

7

=

0

-

(i;h,H(.)F(.))

, hence

^*

r(T1.h

=

(ZTh,zd)

r (T) =

iT O p ( T ,

and from (3.15) t) P (t) H* (t) R-’zd (t)dt

.


which compared to (3.17) yields (3.18)

23

=

.

R-l(t)H(t)P(t)O~(T,t)h

It follows from formula (3.19) that

Let us show that (3.21) Indeed let @(T)

Op(T,O)x

Op(T,t)f(t)

+

=

y(T)

- iTOp(T,t)P(t)D(t)y(t)dt

to be the right hand side of (3.21).

Differentiating in T

, we get

hence -do- - (F - PD)e + f dt O(0) = x

therefore O(T) which proves (3.21).

=

Qp(T,O)x

+

JT Op(T,t)f(t)

Therefore from (3.21) and (3.20) we see that

Qp(T,O)x + which with (3.14) shows that

T 0

Qp(T,t)f(t)dt

=

y(T) - IT H(.)Y(.)

215

216

CHAPER

v

This formula compared to (2.1) shows that of

Y(T)

is the best estimate

y(T).

W e thus have proved the

Theorem 3 . 2 . Ve make the asswnptims of Thecrem 2 # i . Then zi7.e conditional ezpectation 9(T) of y(T) given z ( s ) , O<s/e a s s m e ( 2 . 2 1 , ( 2 . 3 ) , (2.4), (2.5), ( 2 . 8 / , / 2 . 9 / ,

(2.211, (2.22).

(2.27)

inf

Then 7;e have

J(v(.))

V(.)&

A s the proof of Theorem 1 . 1 ,

=

.

inf*J(v(.)) vn,

using Lemmas ( 2 . 1 ) ,

We start with differentiability Lemmas as in denote by

y

tion) still by

§

(2.29)

Let

u

E

;3

2 S;(O,T)

and

the corresponding state defined by (2.1), and (to save nota-

z

the corresponding observation, defined by (2.7).

1%

=

F y 1 + B(t;u(t))

dz

(2.30)

( 2 . 2 ) , (2.3).

1.3.

We write

where

(2.161,

_ _1

dt -

1'

Z ] ( O )

= 0

.

24 1

VARIATIONAL METHODS

Let

v

E

control

2 L (O,T), we will denote by

3 u+5v

We define

,

-

-dy- - F(t)y dt

(2.31)

Y(0) = 0

L e m a 2.4.

x,(t)

x I a defined by (2.12) with

The :unctionaZ

the state corresponding u

replaced by

co

the

u+&.

+ Bv(t;u(t))v(t)

. J(v(.))

is Gateauz di;lr"ereqtisble on

and r e have t h e formula

?TOO t^

I

We have (2.33)

Now

From assumption ( 2 . 2 2 ) , it i s easy to check, like in Lemma 1.5 that

0 , Y'

1

C'(C0,TI).

of ( 3 . 1 3 1 s a t i s f i e s

8 r e g u l a r bcunded donain of Rn.

Set

z = u$

where

@ E B(Rn).

Then

z

satisfies the equation

(3.54)

From Remark 2 . 2 ,

( 3 . 5 0 ) and ( 3 . 5 1 ) , ( 3 . 5 2 ) it follows that the right hand

side of ( 3 . 5 4 ) is hb'lder in x,t

with adequate constants, and

h@

Rn.

E

C2+'

on compact subsets of

an initial condition which is

We can consider

z-h$

0. We can then assert that

z

to obtain is

twice continuously differentiable in x with second derivatives hb'lder, on Q$ , and once continuously differentiable in t, with hb'lder derivative on

-

Q@

.

This f o l l o w s from the result of Ladyzenskaya et alii, mentionned in Chapter 11, Since

q?

§

3.3.3.

is arbitrary we have in particular the property ( 3 . 5 3 ) .

0

269

VARIATIONAL METHODS

With the assumptions of Theorem 3.2, the function L"(x,v,p,t) continuously differentiable in x,v,p

and holder in

There exists a Borel function

4

Here we use

0 -

5 E C (8)I hence Thus by (2.17) we have defined a map

explicitely the assumption that Lm@).

to imply that

5 from Lm into itself. Let us show that it is a contraction. x Indeed take z1,z2 E Lm and c l = Sh(zl) , q 2 = S A ( z 2 ) . Then from

S

(z) =

Lemma 2 . 3 we deduce that

which proves that

Sx

is a contraction. Since solution of ( 2 . 6 )

coincide with the fixed points of

Remark 2.2.

I3

S A , the proof is completed.

We say that problem ( 2 . 6 ) is a unilateral problem, since it

is defined by one sided conditions. The terminology comes from Mechanics where such problems are met frequently. Our motivation is completely different.

I7

3. VARIATIONAL INEQUALITIES

Assumptions ( 2 . 5 ) are two restrictive f o r the applicationsthat we have in mind.

We will relax them, but it will then not be possible to formulate

the problem like in ( 2 . 6 ) , since clearly regularity of the solution u requested in such a formulation.

is

288

LHAPTER V I I

We make the assumptions of Theorem 2.1, and the coercivity

Theorem 3.1. assumption

Then there exists one a d only one solution of the variational inequality

pi-ovided t h a t t h e set 1 K = { V E H Iv 0

some sufficiently

large constant. However since we do not want to rely on such an assumption, we will prove a result similar to Lemma 2 . 1 .

Lennna 3.1.

Without loss of generality, we may asswne

(3.6)

a o t y > O .

Proof Make the change of unknown function u = w z

.

Then we get the same problem with the following changes a

ij

a. 1

f

We recall that

1

+a

+

+

2

ijw

a.w2 1

wf

2 w S 2 ,

The result follows.

29 1

PROBLEMS OF OPTIMAL STOPPING

Let us now consider the problem a(u,v-u) + A(u,v-u)

(3.6)

B v X

For

1

,

Ho

v

5

$ ,

U E

I Ho

,u

5 $.

large enough, the assumptions of Theorem 3.1 are satisfied Therefore there exists one and o n l y one solution of

(namely (3.1)). (3.6).

E

(f,v-u)

2

We have the analogue of Lemma 2 . 3 .

Lema 3 . 2 .

Let

u

be t h e s o l u t i o n of (.3.6), and f changed i n t o

corresponding problem w i t h

f + @

be t h e s o l u t i o n o f t h e

,

@ E

Lm.

Then we have

Since the penalized problems are the same as in section 2, we still have the estimate

Hence E +

uE-UE

+ u-u

remains bounded in Lm.

0. Hence u"-';'

in Lm

However uE-EE

Y

-+

u-u

in L2

0

( 3 . 7 ) holds.

Lema 3.3.

kssume

f,$ bounded.

I If1

-uLet $, 5

Then t h e s o l u t i o n of 13.61 s a t i s f i e s

L

/ / ~ / /m < A + yI

(3.9)

+

L

-u

lwLa *

be the solution of (3.6) corresponding to hence

as

weak star, from which it follows that

5

1 IqlI

a

L

.

f

=

0. Note that

292

Let

CHAPTER V I I

K

=

I~I) 1

We s h a l l prove that

m.

L

(3.10)

-

u + K t O . Y

-

-

v = u+(u+K) as a test function in ( 3 . 6 ) , which is possible by the choice of K. We obtain Take

or

K(ao+X,(u+K)-)

L

hence

(L+K)-

=

0

t

0

, which proves (3.10).

Using next Lemma 3 . 2 , we obtain ( 3 . 9 ) .

0

We can then obtain the

Theorem 3 . 2 .

and

f E LP@) the V . I . (3.11)

We assume 11.1), i1.2), 1 1 . 1 0 ) ,

,p

>

f.

v

provided t h a t t h e s e t

*

@

E

H K

a(u

0

,v)

E

Lm

Then t h e r e e x i s t s one and o n l y one s o l u t i o n of

1

~ v ,2 Q , u

E

H; n L~

u 5

q

defined i n 13.31 i s n o t empty.

Consider first the equation

or

Let

a(u,v-u) t (f,v-u)

v

(3.12)

12.21.

=

(f,v)

,

293


AU

We know that

uo

0

+ aOuo

,

W2"@)

E

-

u = u-u u

when

=

,

f

u

0

lT

hence since

.

o

=

p >

5 , uo

E

Now set

COG).

0

is a solution of (3.11).

;;

Then

is a solution of the

following problem

1 0

Y V E H

u

HA

E

n L~

,

v 2 $-u

,

-u

5

0

0 i-u

which is the same problem for data

f

=

0, and

$ = I$-u

O

E

L

m

.

Moreover

we set

KO

=

{v

since it contains vo-uo

1 0

H 1v

E

5

where v

0

I

$-u 0

5

9

is not empty

,

v

0

E

1

Ho.

Therefore without loss of generality, we may assume z

E

Lm, define

5

=

f = 0. Let next

as the solution of

Sh(z)

(3.13)

For

A

large enough, we may apply Theorem 3.1, to ensure the existence

and uniqueness of

5.

Moreover, from Lemma 3.3, we see that

Hence we have defined a map contraction. Indeed if

S,

z1,z2

from Lm

La

and

5

E

Lm.

into itself. This map is a c1,c2 are the corresponding

sohtions of (3.13), it follows from Lemma 3.2 that

294

But clearly the fixed points of (3.11),

when

Remark 3 . 1 .

S A coincide with the solutions of

0

f=O. Hence the existence and uniqueness.

When we make the assumptions of Theorem 2.2, we have one and

only one solution of (2.6) and also-of ( 3 . 1 1 ) .

These solutions

Indeed let us check that the solution u

coincides.

of ( 2 . 6 ) is a

solution of ( 3 . 1 1 ) . Indeed let v

E

1

Ho

,

v

5

$, we have

J0 (Au+aou-f) (v-u)dx and by Green's formula we see that u

=

J

0

(Auta u-f) (v-$)dx

0

satisfies ( 3 . 1 1 ) .

2

0

This justifies

the introduction of V.I. as a weaker formulation to ( 2 . 6 ) , when

$

Lema 3 . 4 .

Let

$,T

L m J and

E

corresponding t o them.

Let u s consider

' U

(3.15)

Au'ta

with (3.16)

X

is

0

not regular.

large and

Then one has

and

Q

u

u,u t o be the s o l u t i o n o f 13.121

0

E

LE

uE

Lm.

to be solutions of

+ AuE +

1 (u'-$)~

Then we have

=

f

, ~

€

=

10 ~

29 5

PROBLEMS OF OPTIMAL STOPPING Set K = Max(/

l$-@l

1

w

= uE--uE-K

m,

L

We multiply (3.15) by

J8

(3.17)

lMl)

"

Y+X

w+

.

l w E Ho

,

and (3.15)"

by

-w+

and add up.

CA(uE-UE) + a (uE-uE) + A(uE-GE)lw+dx + 0

-E1 X

+ dx

where

x Indeed assume

zE 2 ",)I w

-

= ((U"$)+

C

(U"$)+,W+)

2

0

.

then "

uE-$-K

,.,

< $-$-K

0

S

,

hence ((UE-qJ)',w+) which proves that

X

2

0

.

=

0

Now from (3.17) we deduce

a(w,w+) + X(w,w+) + J

8

hence

w+

=

0

.

[(a0+A)K+qlw+

Therefore uE-zE 2 K

.

By a reverse argument we conclude that (3.16) holds. Therefore considering the solution of

We obtain

dx < 0

=

296

CHAPTER VII

(3.18)

we can assert tha (3.19) Consider next the iterative process

n+l)

+ (f,v-u

When

) I then the contraction argument mentionned in Theorem 3.2, . t , a guarantees that

un Defining similarly

+

u

in L~

.

zn. It follows from estimate (3.19) that

(3.21) Letting a .

2

y

2

n

-t m

we deduce that (3.14) holds true, at least provided that

0. A s this stage, it is not useful to make the change of

unknown function u

= wz

, which changes

$

into $/w.

Indeed we will

only obtain estimate (3.14) with twice the right hand side, which is not the estimate we want.

One proceeds as follows. Consider the V.I.

291


6 > 0 , will tend to 0.

where

Let also &:

be the solution of the same problem with

1 4

changed into

$.

6 > 0 , we have

Since

And it is enough to show that

u6

-+

u

6

as

+

0

,

in some sense. For

such a result, we may consider the change of functions u therefore it is sufficient to assume

a .

2

y > 0

.

= uz

, and

Consider next the

iterative process

2

J,(un,v-un) 6 6

+

(f,"-Ut++

then we have

from which it follows that

>

k = h+y

with as

6

-t

.

In particular it follows that

u6

0. From this and the V.I., one deduces

is then enough to obtain

u

6

+

u

in

1

us

Ho weakly and

completes the proof of the desired result. We can then state the following regularity result

is bounded in bounded in Lw

Lm

1

Ho

.

It

weak star, which

c

298

CHPATER VII

Theorem 3 . 3 .

We make t h e a s s u m p t i o n s o f Theorem 3 . 2 and

(3.22) Then t h e s o l u t i o n

u

of (3.11) belongs t o

Co(&

.

Define En = ll$n-j,l

I Lrn

,

En

-+

0

and

Clearly also

satisfy the assumptions of Theorem 2 . 2 .

The functions $n

Let

un

be the

It is also the solution of

solution of the V.I. corresponding to

@n. the unilateral problem ( 2 . 6 ) , hence in particular

u

E

Co(s).

But from

(3.14) we deduce that lIun-uI Hence

u

E

0 -

.

C (0)

I L"

5

llQn-vl

I L"

*

0

*

0

299


Let u s prove to end that section, the useful result that the solution uE of the penalized problem converges to the solution

in

.

Co(s)

u

of the V.I.

This result will be generalized in section 5 for general

semi groups, with some slight changes in the assumptions.

We will

however need it in Chapter VII, section 3 .

Theorem 3 . 4 .

Under t h e assumptions ~f Theorem 3 . 3 , t h e n t h e s o l u t i o n

of 1 2 . 3 ) converges towards t h e s o l u t i o n

u

of t h e 7 . 1 . (S.11) i n

Let u s first remark that it is sufficient to prove this result when is regular.

Indeed let

as in Theorem 2 . 3 , @n

@n

Lemma 3 . 4 , we have (noting u

I /Un-U/I

(3.23)

$

in

the V.I. corresponding to

Co.

.

y

From

on)

I lon-il I Lrn

5

L

+

uE

Co(6)

But the proof of Lemma 3 . 4 , in particular estimate ( 3 . 1 6 ) , together with an iterative procedure for the penalized problem, like in ( 3 . 2 0 ) shows that the same estimate is valid for the penalized problem, namely

From ( 3 . 2 3 ) , ( 3 . 2 4 ) it is clear that if we have

1

lu:-unl

1

+

0

in

then the desired result will follow. a We may of course assume . Lemma 3 . 1 ) (1) Now for

)t

large, replacing

Lemma 2 . 2 , that

uE

2

C

0

, for n fixed We may thus assume

$

regular

y > 0 , without loss of generality (cf

a .

by

aO+X

(cf. ( 2 . 1 0 ) ) , we know fron

remains bounded in W2'p.

Now consider the iterative

sequence

(L) We have however to consider a penalized problem with EIUJ

2

.

E

changed into

300

CHAPTER V I I

hence

< - kn

-

k

where

NOW

=

A x+y .

1 /uE>OlI

I-k

Therefore letting p

+ m,

~< c ~, from , Lemma ~ 2.2; similarly

1

lu E, 1

< c .

Hence we have

IIu~-u~'~I I

(3.25)

2

Lrn

C kn

.

We a l s o have

I lu-unt 1

(3.26)

L

< C

kn

which follows from (3.25) and continuity and convexity of the norm.

It

also follows directly from (2.16) and an iterative scheme. Now for any fixed

remains in a bounded set of

since uE'n depending on uE + u

in

n, we have by Lemma 2.2,

n).

COG) .

W2"

as

E

+

0

(a priori

From this and (3.25), (3.26) it follows that

0

30 1


4 . SOLUTION OF THE OPTIMAL STOPPING TIME PROBLEM

4.1.

Ihe-re9ular-case

We are going t o show the following

Theorem 4 . i .

We assume ( 1 . 1 1 , ( i . 2 ) , i l . i O / ,

(1.111, ( 2 . 2 ) , ( 2 . 5 1 .

the solution u

of 12.61 is given explicitely bg

(4.1)

u(x) = Inf Jx(6)

e

Then

.

Moreover there exists an optimal stopping time, characterized as follows. Define

and

then

6

If h

E

is an optimal stopping time.

Lp(&

,p >

5 , we know from Chapter 11, Theorem 4 . 1 ,

that

From this estimate follows that we can deduce the following Ito's the function u

integrated from formula

to

(4.4)

EX u(x(6A-c)exp

u(x)

=

+ EX

where

6

E

- J 6A.r

W2"

,

p >

ao(x(s))ds

4

+

JoAT (Au+aou) (x(s)) (exp-JS ao(x(X))dX)ds

is any stopping time.

302

CHAPTER V I I

Now using the relations ( 2 . 6 ) , it is easy to deduce from ( 4 . 4 ) that (4.5)

u(x)

5

V

JX(8)

8.

On the other hand we may assert that xc(x)

(Au+a u-f) = 0 p.p. 0

hence EX i8A?

xC(Au+aOu-f)

0

But for

s
0 such that

Let

u(x) < Y(X) - 6 and

Let

N,

i

such that

n t Nc:

110,- $ 1 1 Therefore for

s S

6

e 6A T

implies

0.

+a.

By virtue of assumption (5.76) we can assert that

We apply (5.86) with

u 5 J t e-"'

Therefore Indeed let

u

m

=

[t2'

+I],

O(s)L d s +

5

Jh e-"

O(s)L

ds +

-u

5

uh

tend

the maximum element.

O(h)u

hence

and from (5.82), it follows that

R

.

O(t)u

of (5.78). It i s -u isbeananelement other element, then

-u

and we let

-

, hence u

5

u.

to

339


Now if we do not assume (5.76), but (5.79) and (5.80), then we cannot let R L! tend m in (5.86) (with m = It2 ] + I ) . However since u 2 u we q'

deduce from (5.86)

But from (5.79), it follows that u 2 Jt e-"'

We let then

q

tend to

+m

@(s)L

u

q

Using then (5.80), we obtain

C.

E

ds +

O(t)uq

fi

q

.

as above and obtain the desired result.

The final statement of the theorem follows from the fact that both maximum elements of (5.78) and (5.27) can be approximated by the same sequence

h'

0

*

6 . I N T E R P R E T A T I O N AS A PROBLEM OF O P T I M A L S T O P P I N G

We assume is a semi compact ('1

E,

(6.1)

and the semi group

defined on

@(t)

(5.21), (5.23), (5.24). (6.2)

@(t)l

B

satisfies properties (5.19),

...

We replace (5.22) by =

1.

This assumption and (5.21) imply (5.22) Now in the case when assumption. (6.3)

E

is

ria5

csmpac;, we will need an additional

Let

i:

= {f

,3 K~ compact ~f(x)l < E , for x 4 KE}

continuous i ti

such that

E

(1) Locally Compact Hausdorff space, with denumerable base.

. Example

Rn,

340

The space

CHAPTER VII

is a closed subspace of

C.

Then we will assume that

We next define

for any Bore1 subset of

E.

We consider the canonical space

Ro

=

,

D([O,-);E)

is continuous to the right and

w(.)

has left limits.

no =

u(x(t),t

0)

2

According to the general Theorem of Markov processes, (cf. Dynkin [ 1 ] , [ 2 ] ) there exists a unique probability

Ft ''tl',?=

Px

on

completed ,

Ro, "mo , such that considering

c0 = n o

completed

then

o0,Go,

pX,

n-t

, x(t)

quasi continuous from t h e l e f t (I), PX(x(0)=x)

=

1

is a right continuous,

strong Markov process, and

.

(1) quasi continuous from the left means that tr A

of stopping times

-

...,T~ ..., + ~ ( w )

o

aaxa i j j

such t h a t :

(1.5)

1. C12 = 2

a

Let a l s o : (1.6)

Where # i s

a

2

0

a

bounded

an open bounded r e g u l a r domain o f R". k > 0,

c o ( s ) : R"+

+

R ~ c, o n t i n u o u s ,

c o ( 0 ) = 0, non d e c r e a s i n g CO(Sl +

s2)

5

co(5,)

+

C0(L2)

Let a l s o :

357

IMPULSIVE CONTROL

1 . 2 . The model

An i m p u l s i v e c o n t r o l i s d e s c r i b e d by a s e t a s f o l l o w s : (1.10)

w

= (5'

5

I

... < e n .. . .... cn ..

e2 , 52

0, c a l l e d t h e f i x e d c o s t ; co(S) i s c a l l e d t h e v a r i a b l e c o s t .

360

CHAPTER V I I I

2 . QUASI VARIATIONAL INEQUALITIES Orientation Dynamic Programming l e a d s t o a n a l y t i c problem which resembles t h e t y p e o f problems d i s c u s s e d i n c h a p t e r V I I . This i s n o t s u r p r i s i n g , s i n c e a s t o p p i n g t i m e problem i s a v e r y p a r t i c u l a r c a s e o f impulse contro1,namely when we

impose

e2

=

e3

=

... =

fm.

T h i s w i l l be r e f l e c t e d i n t h e a n a l y t i c as w e l l

a s i n t h e p r o b a b i l i s t i c t r e a t m e n t . We w i l l e n c o u n t e r q u a s i v a r i a t i o n a l i n e q u a l i t i e s ( Q . V . I . ) i n s t e a d of V . I .

We i n t r o d u c e a non l i n e a r o p e r a t o r a s f o l l o w s : (2.1)

MHx) = k + i n f [cp(x+S) + c o ( S ) l 620

x +

6 €8

which makes s e n s e f o r f u n c t i o n s (Pwhich a r e bounded below. We c a l l Q . V . I . t h e f o l l o w i n g problem :

where a s u s u a l :

(2.3)

+

a.

u v dx.

Remark 2 . 1 . The c o n d i t i o n u t 0 i m p l i e s t h a t Mu i s w e l l d e f i n e d .

I

Remark 2.2. Problem ( 2 . 2 ) i s a n i m p l i c i t V . 1 , s i n c e t h e o b s t a c l e depends on t h e s o l u t i o n . T h i s i s w h y , i t i s c a l l e d a Q . V . 1 , w i t h t h e V.I.

I t s h o u l d n o t be confused

36 1

IMPULSIVE CONTROL Vv 5 Mv,

u s Mu

I

which i s a t o t a l l y d i f f e r e n t problem.

Remark 2 . 3 The o p e r a t o r does n o t p r e s e r v e t h e r e g u l a r i t y o n d e r i v a t i v e s . I t does n o t map H 1 i n t o i t s e l f ' ] ) . However i t h a s a v e r y i m p o r t a n t p r o p e r t y which w i l l p l a y a fundamental r o l e i n t h e s e q u e l . I t i s monotone i n c r e a s i n g i n the following sense :

'Dl ( x )

:

+

1,l lu/

1 c.0 n +2.

Consider now t h e e l l i p t i c P . D . E . (3.32)

Au = f

Assuming p i s a s i n ( 3 . 3 1 ) , we can deduce from ( 3 . 3 1 ) , ( 3 . 3 2 ) and ( 3 . 5 ) , that : ~',T,T E u(y(8-T)) = E u(y(8'-T-Te))

+ E

'0 ,T

ef ( y ( t ) )

dt.

But n o t i n g t h e e s t i m a t e :

where t h e c o n s t a n t does n o t depend o n T , we deduce a b e t t e r e s t i m a t e t h a t ( 3 . 3 1 ) namely :

378

CHAPTER VIII

Proof of theorem 3 . l . We may c o n s i d e r t h e Q . V . I .

w i t h o b s t a c l e J1 = Mu. I t i s e a s y

(2.2) as a V . I .

t o check t h a t t h e assumptions o f theorem 4 . 2 ,

Qo,

W be a n i m p u l s i v e c o n t r o l . Consider

po,

p

t

c h a p t e r V I I a r e s a t i s f i e d . Let

, P;,

t h e n t h e p r o c e s s xn d e f i -

ned i n ( 1 . 1 4 ) s a t i s f i e s t h e e q u a t i o n :

We t a k e w i t h t h e n o t a t i o n of t h e b e g i n n i n g o f § 3.1

From ( l , l 7 ) ,

( I .18)

:

i t f o l l o w s t h a t assumption ( 3 . 5 ) i s s a t i s f i e d .

T h e r e f o r e from lemma 3 . 1 . ,

p r o p e r t y ( 3 . 8 ) a p p l i e d w i t h w = u we o b t a i n :

Let u s n e x t remark t h a t :

en

(3.38) Indeed i f 8""

(en,On+')

5

< T implies

r n , then x(s)

and x('J"),

x(e"-)

( 3 . 4 8 ) i s v e r i f i e d . I f 8""

x ( r n ) = x ( T ~ - )a @ ,

E

e n+ 1 - T n

d

for s

c &since

T.

If

en

= T~ T.

Ce n ,

1, x(s)

< T , therefore T 2

> T ~ t, h e n when

thus rn =

r n 2 T, which i s i m p o s s i b l e s i n c e 8"
I . We a p p l y t h e same method a s i n theorem n+ I 3.1, using the f a c t t h a t u i s a s o l u t i o n of a V.I. w i t h o b s t a c l e Mu". LetW

E

'Ikn+and l

take 0

2

j

2

n-l

We f i r s t have a s f o r ( 3 . 3 9 ) :

.

388

CHPATER V I I I

(3.66)

Operating a s f o r ( 3 . 4 5 ) , we o b t a i n :

(3.68)

We t a k e t h e mathematical e x p e c t a t i o n and add up t h e s e r e l a t i o n s , when j r u n s from 0 t o n-I. We o b t a i n :

n

389

IMPULSIVE CONTROL

n+ 1 =

Using t h e f a c t t h a t (3.69)

un(x)

5

a . s . We o b t a i n :

+m

J ~ ( w ) ,f o r

w

c

V+'. 1

Let u s prove t h e e x i s t e n c e o f a n o p t i m a l c o n t r o l . Definet"(x)

Bore1 2 0

function such t h a t : (3.70)

+ u n - l ( x + s"(x))

= k + co(;"(x))

Mun-l(x)

Vx

E

8, n

2 1

Consider xo a s dx

0

=

0

U ( X )dwo

x (0) = x T:

1

It

= inf

u n ( x o ( t ) ) = Mun-I(xo(t))}

then : (3.71)

-1

'n -1

5,

=

0

-gn

We have a g a i n x 0 (en) -1 c @ when p r o c e s s x j-1 and -8:,' follows : (3.72)

Then : (3.73)

if T:

Tn

5: f o r j

0 -1 ( x (en)) i f

;A = 1

O

(4.5)

f

E

B (Bore2 bounded on

Then t h e s o l u t i o n u o f Q . V . I .

8)

(2.21 s a t i s f i e s = 0, u 2 M u -a s

x6e

d s + $ ( t ) e-cYt u , V t 2 0.

of t h e s e t o f f u n c t i o n s s a t i s f y i n g ( 4 . 6 1 .

Proof u i s t h e s o l u t i o n of t h e V . I .

5.2, chapter V I I ,

w i t h o b s t a c l e $ = MU. T h e r e f o r e b y theorem

( r e l a t i o n s (5.11),

(5.12))

u s a t i s f i e s (4.6).

L e t us p r o v e t h a t i t i s t h e maximum element among t h e f u n c t i o n s s a t i s f y i n g %

( 4 . 6 ) . T h i s i s proved u s i n g t h e d e c r e a s i n g scheme. L e t u be an element o f ( 4 . 6 ) . We have

%

Assuming t h a t u

%

u

5

uo s i n c e :

%

un, t h e n M u 5 Mun, Therefore?; i s an element of t h e s e t

( 5 . 1 1 ) , (5.12) c h a p t e r V I I , w i t h $ = Mu". % n+l % then implies u < u T h e r e f o r e u S U.

.

But theorem 5 . 2 , c h a p t e r V I I ,

I

We c o n s i d e r now t h e a n a l o g u e of problem (5.27) c h a p t e r VII, w i t h an i m p l i c i t o b s t a c l e . Namely l e t $ ( t ) be a semi group s a t i s f y i n g ( 5 . 1 9 ) , c h a p t e r V I I . We a l s o c o n s i d e r :

..., ( 5 . 2 4 ) ,

393

IMPULSIVE CONTROL

L

(4.7)

E

B , t + $ ( t ) L i s measurable

from [ O , m l

into C

L t O

L e t now M be an o p e r a t o r such t h a t :

M :C

(4.8)

+

C i s L e p s c h i t z , concave, and monotone i n c r e a -

MYl

sing (i.e.

(4.9)

;

M(O)tk>O

5 M(D2 i f

Y1

5

(D2)

a > O

We c o n s i d e r t h e set of f u n c t i o n s : u

(4.10)

E

MU

C, u S

$(s)

e

Theorem 4.3.

We assume ( 5 . 1 9 ) ,

Lds + e

..., (5.24)

-at

$(t) u

chapter VII and 1 4 . 7 1 ,

(4.81,

( 4 . 9 ) . Then the s e t of s o l u t i o n of ( 4 . 1 0 ) i s not empty and has a rnadmwn

element, which i s a p o s i t i v e f u n c t i o n .

Let z

E

C and c o n s i d e r 5 t o be t h e maximum s o l u t i o n o f :

(4.11)

S i n c e Mz

E

C, 5 =

Tz i s w e l l d e f i n e d , a c c o r d i n g t o theorem 5 . 3 , c h a p t e r V I I .

I t w i l l be c o n v e n i e n t t o w r i t e :

(4.12) where u(9)

T

=

u o

M

: C -+ C i s t h e maximum element of t h e s e t ( 5 . 2 7 ) , c h a p t e r V I I .

O n e e a s i l y checks t h a t u i s monotone i n c r e a s i n g and concave. From t h e assumpt i o n ( 4 . 8 ) on M i t f o l l o w s t h a t : (4.13)

T i s i n c r e a s i n g and concave.

394

CHAPTER VIII

L e t next

1,

m

uo =

(4.14)

e -at $ ( t ) Ldt

and uo E C , uo 2 0. L e t 1 > p > 0, s u c h t h a t :

I

(4.15)

! ~ I l u ~ l5 k

Then one h a s : T(0) 2 u u

(4.16)

0

Indeed by ( 4 . 1 5 ) and assumption ( 4 . 9 ) . !J u0 9 M ( 0 )

and from ( 4 . 1 4 )

:

v

uo =

O

(1)

We can s t a t e t h e f o l l o w i n g :

Theorem 4 . 4 .

Assume (5.191,

. . ,,(5.24) chapter V I I and

14.S), (4.9), ( 4 . 1 9 ) .

Then problem 1 4 . 1 8 ) has one and o n l y one s o l u t i o n . Moreover h

-+

y,

-+

u in C, as

0, where C is the m d m u m element o f (4.10).

Let u s s e t oh($) t o be t h e s o l u t i o n o f ( 5 . 4 9 ) c h a p t e r V I I ( d i s c r e t i z e d V . 1 ) . Then uh : C (4.20)

+

C , and :

ah i s i n c r e a s i n g and concave.

T h i s w i l l f o l l o w from t h e f o l l o w i n g f a c c :

(4.21)

if i

5

S

E

C v e r i f i e s 5 2 $ and

hL + e

-Uh

$(h)

Stochastic Control by Functional Analysis Methods (Studies in mathematics and its applications)

Stochastic Control by Functional Analysis Methods (Studies in mathematics and its applications)

Stochastic Control by Functional Analysis Methods

Functional Analysis and its Applications

Functional analysis and its applications

Nonlinear functional analysis and its applications. Variational methods and optimization

Stochastic Analysis of Computer Storage (Mathematics and Its Applications)

Stochastic Analysis and Applications

Stochastic Control in Insurance (Probability and its Applications)

Functional Analysis and Applications

Boolean Valued Analysis (Mathematics and Its Applications)

Nonlinear Functional Analysis and its Applications: Nonlinear Functional Analysis and its Application. Fixed Point Theorems

Convexity Theory and its Applications in Functional Analysis

Topics in Stochastic Analysis and Nonparametric Estimation (The IMA Volumes in Mathematics and its Applications 145)

Classical and New Inequalities in Analysis (Mathematics and its Applications)

Nonparametric Methods in Change Point Problems (Mathematics and Its Applications)

Functional Analysis Methods in Numerical Analysis

Convex Functional Analysis and Applications

Functional analysis: Theory and applications

Functional Analysis: Theory and Applications

Discrete Mathematics and Its Applications

Superanalysis (Mathematics and Its Applications)

Discrete Mathematics and Its Applications

Discrete Mathematics and its Applications

Topics in Functional Analysis and Applications

Topics in Functional Analysis and Applications

Topics in functional analysis and applications

Topics in functional analysis and applications

Discrete mathematics and its applications

Superanalysis (Mathematics and Its Applications)

Topics in functional analysis and applications

Stochastic Control by Functional Analysis Methods (Studies in mathematics and its applications)