Stochastic Control by Functional Analysis Methods
STUDIES IN MATHEMATICS AND ITS APPLICATIONS VOLUME 11
Editors: J. ...
61 downloads
756 Views
11MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Stochastic Control by Functional Analysis Methods
STUDIES IN MATHEMATICS AND ITS APPLICATIONS VOLUME 11
Editors: J. L. LIONS, Paris G. PAPANICOLAOU,New York R. T. ROCKAFELLAR, Seattle
N O R T H - H O L L A N D PUBLISHING COMPANY - AMSTERDAM
a
NEW YORK
a
OXFORD
STOCHASTIC CONTROL BY FUNCTIONAL ANALYSIS METHODS
ALAIN BENSOUSSAN Universite Paris Dauphine arid I N R I A
19x2
NORTH-HOLLAND PUBLISHING COMPANY - AMSTERDAM
NEW YORK
OXFORE
L
North-Holland Publishing Company, I982
All rights reserved. N o part of this publication may be reproduced, storedin a retrievalsystem,
or trammirted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without thepriorpermission of the copyright owner.
ISBN: 0444 86323 x
Publishers: N O R T H - H O L L A N D PUBLISHING COMPANY AMSTERDAM 0 NEW Y O R K 0 O X F O R D Sole distributors f o r the U.S.A. and Canada: ELSEVIER SCIENCE PUBLISHING COMPANY, INC. 5 2 VANDERBILT A V E N U E , NEW Y O R K , N.Y. 10017
Library of Congress Cataloging in Publiration Data
Bensoussan, A l a i n . S t o c h a s t i c c o n t r o l by f u n c t i o n a l a n a l y s i s methods. ( S t u d i e s i n mathematics and i t s a p p l i c a t i o n s ;
v . 11) B ib li o g r a p h y : p . . 1. C o n t r o l t h e o r y .
2. Stochastic analysis. I. T i t l e . 11. S e r i e s . QA402.3. B433 6 2 9 . 8 ' 3 12 81- 19900 ISBN 0-444-86329-X AACFC?
PRINTED IN T H E NETHERLANDS
V
INTRODUCTION
Our objective in this work is to give a presentation of some basic results of stochastic control. It is thus a text intended to advanced students and researchers willing to learn the theory. Stochastic control covers a broad area of disciplines and problems. It is also a field in full development, and some important aspects remain to be cleaned up. That is why in presenting stochastic control a choice is necessary. We have emphasized this choice in the title. The theory of partial differential equations, the semi-group theory, variational and quasi variational inequalities play a very important role in solving problems of stochastic control. We have tried to use them as much as possible, since they bring tools and results which are very important, specially for computational purposes, and which cannot be obtained in an other way, namely regularity results, and weak solution concepts. The books by W. Fleming R. Rishel 1 1 1 , A. Friedman C 1 1 , N. Krylov [ l ] , A . Bensoussan - J.L. Lions [ l ] , [ 2 ] show already the importance of the techniques of Functional Analysis. w. Fleming Rishel, Friedman besides covering many other topics, rely mostly on the classical theory of P.D.E. We try to emphasize here the importance of variational methods. Naturally, the present text has a lot in common with the books of J.L. Lions and the A. But here we have tried to simplify as much as possible the presentation, in particular leaving aside the most technical problems, which are treated there.
-
-
Also the books by J.L. Lions and the A. are devoted to variational and quasi variational inequalities. In the book of Krylov, one will find the study of the general Bellman equation, i.e., when the control enters into the drift as well as into the diffusion term. We do not treat this general case here, although it is certainly one of the nicest accomplishments of P.D.E. techniques in stochastic control. Quite fundamental results have been obtained for the general Bellman equation by P.L. Lions [ l ] , [ 2 ] , and more specialized ones by L.C. Evans A. Friedman [ l ] , H. Brezis - L.C. Evans C l ] . More recently R. Jensen - P.L. Lions [ l ] have introduced new important ideas of approximation. To report on that work would have gone beyond the objectives of the present text, and requested too much material.
There are many important other topics that we have not considered here. We have not reported on the developments of the so called "probabilistic approach" initiated by C. Striebel [ l ] , 1 2 1 and R . Rishel C 1 1 and developed extensively by M. Davis - P. Varayia C 1 3 , M. Davis C 1 1 and many other authors. A good report can be found in N. El Karoui [ l ] , (see also Lepeltier - Marchal [ l ] ) .
vi
INTRODUCTION
This approach is of course fundamental for very general processes, which are not Markov processes. It is certainly the most general one and very satisfactory from the probabilistic point of view. But for the applications, where the processes are mostly Markov, it seems less convenient than the analytic approach, especially for computational purposes. Also it requires technical developments which again would have gone beyond the scope of this text. The interested reader should consult besides the litterature, which has been briefly mentionned the recent book by 1.Gikhman - A. Skorokhod [11. Another very important area, which is in full development is the theory of non linear filtering and control under partial observation. Important results have been obtained recently by several authors in non linear filtering, T. Allinger - S.K. Mitter Cl], E. Pardoux [ l ] , and exploited for the control under partial observation by W. Fleming - E. Pardoux [ l ] , W. Fleming [ I ] . Stochastic P.D.E. play a very important role in this direction, and probably the field will progress fast (see E. Pardoux [2], M. Viot [ l ] , W. Fleming - M. Viot [ l ] ) . For the control of stochastic distributed parameter systems see A.V. Balakrishnan [ l ] , A. BensoussanC11, [ 2 ] , A. Bensoussan - M. Viot [ l ] , R. Curtain A.J. Pritchard [ I ] , S. Tzapestas C 1 3 . We consider in this work some stochastic control problems in discrete time, but mostly as an approximation to continuous time stochastic control. We refer to the books by D. Bertsekas [ I ] , D. Bertsekas S . Shreve [ I ] , E. Dynkin - A. Yushkevitch [ I ] for many more details. In a related direction, we have not discussed the numerical techniques which are used to solve stochastic control problems. We refer to J.P. Quadrat [ l ] , P.L. Lions - B. Mercier [ 1 1 and to the book of H.J. Kushner [ I ] . Let u s also mention the theory of large stochastic systems, with several players, the problems of identification, adaptive control, stochastic realization, stochastic stability etc., as interesting and important areas of research. In Chapter I we present the elements of Stochastic Calculus and Stochastic Differential Equations, in Chapter I1 the theory of partial differential equations, and in Chapter 111 the Martingale problem. This permits to deal with the various formulations of diffusion processes and to interpret the solution of elliptic and parabolic equations as functionals on the trajectory of the diffusion process (in a way similar to the well known method of characteristics This allows u s also to show the Markov for 1 s t order linear P.D.E.). semi-group property of diffusions. In Chapter IV we present the theory of Stochastic Control with complete information (when the control affects only the drift term). We study the Hamilton-Jacobi-Bellman equation, interpret its solution as a value function and solve the stochastic control problem in the stationary as well as non stationary case. We also present a semi group approach to stochastic control for general Markov processes. In Chapter V, we present the theory of filtering and prediction for linear stochastic differential equations, which leads to the Kalman
INTRODUCTION
Vii
filter. We show that the problem reduces to quadratic optimization problems, for which a decoupling argument yields the filter and the Riccati equation. In Chapter VI, we present the variational approach to stochastic control, in two situations, one with complete observation and one with uncomplete observation. We discuss also the separation principle. Chapter VII is devoted to optimal stopping problems which are solved by the theory of variational inequalities. We also develop a semi group approach in the case of general Markov processes. In Chapter VIII we present the theory of impulsive control and its solution by the method of quasi variational inequalities. Also a semi group formulation is given. We have tried to be self contained as such as possible, and have avoided too technical topics. Some basics on probability and Functional Analysis are the only requirements in order to read this book. Nevertheless, we recall the results that we need. We have restricted ourselves to stationary diffusions stopped at the exit of a domain, since one can explain the ideas in the simplest form in that context. But of course the methodology carries over to many other processes, like diffusions with reflexion, diffusions with jumps, random evolutions etc. see A. Bensoussan - J.L. Lions [Z], A. Bensoussan - P.L. Lions 111, A. Bensoussan - J.L. Menaldi I l l ) . When presenting the semi group approach we have kept a certain degree of generality, although we have not tried to describe all the examples which are covered in this approach (cf. M. Robin [ l ] for many examples like semi Markov processes, jump processes, ).
...
This Page Intentionally Left Blank
ix
ACKNOWLEDGMENTS
Many of the ideas presented here owe a lot to discussions and joint research with colleagues in France or abroad. I would like to thank particularly J. Frehse, A . Friedman, M. Goursat, J.L. Joly, J.L. Lions, P.L. Lions, J.L. Menaldi, U. Mosco, G. Papanicolaou, E. Pardoux, J.P. Quadrat, M. Robin, M. Viot with whom I had certainly the most fruitful exchanges and from whom I learnt a lot. The material covered here has been presented first as a course at the University of Illinois, during the fall 1980. I would like to thank particularly P. Kokotovic who originated the idea of a course and suggested the writing of a text. I would like to thank also the members of the Control Group, at Coordinated Science Laboratory, University of Illinois, in particular Prof. Cruz, Perkins for their help. Prof. B. Hajek at the University of Illinois read carefully the manuscript and made very useful comments. I would like to thank him very much for that.
I would like to thank very much Professors Lions, Papanicolaou and Rockafellar for accepting to publishing this book in their series with North Holland as well as the publisher for his agreement. Mrs Kurinckx realized a very nice typing of the manuscript. I would like to thank her very much for this excellent job.
This Page Intentionally Left Blank
xi
CONTENTS
CHAPTER I .
STOCHASTIC CALCULUS AND STOCHASTIC DIFFERENTIAL EQUATIONS
1
INTRODUCTION
1
1. PRELIMINARIES
2
1.1. 1.2. 1.3. 1.4.
Random v a r i a b l e s Conditional expectation Expectation S t o c h a s t i c processes Martingales
-
2. STOCHASTIC INTEGRALS 2.1. 2.2. 2.3. 2.4. 3.
Wiener process Construction o f the stochastic integral S t o c h a s t i c process d e f i n e d by a s t o c h a s t i c i n t e g r a l Extension o f t h e s t o c h a s t i c i n t e g r a l
ITO'S FORMULA
4. STOCHASTIC DIFFERENTIAL EQUATIONS 4.1. 4.2.
S e t t i n g o f t h e problem L i p s c h i t z case
5. GIRSANOV TRANSFORMATION
5.1. Fundamental lemma 5.2. G i r s a n o v ' s Theorem 5.3. A p p l i c a t i o n t o t h e concept o f weak s o l u t i o n o f a s t o c h a s t i c d i f f e r e n t i a1 e q u a t i o n CHAPTER 11.
PARTIAL DIFFERENTIAL EQUATIONS
2 5 7 9
10 10 12 16 18 20 32 32 33 37 37 42 46 51
INTRODUCTION
51
1. FUNCTIONAL SPACES
52
1.1. Sobolev spaces 1.2. Concept o f t r a c e 1.3. G r e e n ' s f o r m u l a
52 54
57
xii
CONTENTS
2. THE DIRICHLET PROBLEM FOR ELLIPTIC EQUATIONS 2.1. The b a s i c e x i s t e n c e and uniqueness r e s u l t s 2.2. V a r i a t i o n a l t e c h n i q u e s
59 59 64 75
3. PARABOLIC EQUATIONS 3.1. F u n c t i o n a l spaces 3.2. V a r i a t i o n a l f o r m u l a t i o n 3.3. R e g u l a r i t y
75 78 82
3.3.1. R e g u l a r i t y w i t h r e s p e c t t o t i m e 3.3.2. R e g u l a r i t y w i t h r e s p e c t t o space v a r i a b l e s 3.3.3. O t h e r r e g u l a r i t y r e s u l t s 3.4. The Cauchy problem
82 84 87 94
CHAPTER 111.
101
MARTINGALE PROBLEM
INTRODUCTION
101
1. PROPERTIES OF CONTINUOUS MARTINGALES
101
1.1. Square i n t e g r a b l e c o n t i n u o u s m a r t i n g a l e s 1.2. S t o c h a s t i c i n t e g r a l s 1.3. A r e p r e s e n t a t i o n theorem f o r c o n t i n u o u s m a r t i n g a l e s
101 104 105
2. DEFINITION OF THE MARTINGALE PROBLEM
107
2.1. S e t t i n g o f t h e problem 2.2. P r o p e r t i e s o f t h e m a r t i n g a l e 2.3. Some a p r i o r i e s t i m a t e s
107 108 112
problem
3 . EXISTENCE AND UNIQUENESS OF THE SOLUTION OF THE MARTINGALE PROBLEM 3.1. Uniqueness 3.2. E x i s t e n c e 4 . INTERPRETATION OF THE SOLUTION OF P.D.E. 4.1. 4.2.
E l l i p t i c equations Parabolic equations
5. SEMI GROUPS 5.1. Semi group o f d i f f u s i o n s 5.2. Stopped d i f f u s i o n s
117 117 122 125 125 127 129 129 133
CONTENTS
CHAPTER I V .
STOCHASTIC CONTROL WITH COMPLETE INFORMATION
Xiii
139
INTRODUCTION
139
1. SETTING OF THE PROBLEM
139
1.1. N o t a t i o n . Assumptions 1.2. C o n t r o l l e d m a r t i n g a l e 2. THE EQUATION OF DYNAMIC PROGRAMMING Orientation 2.1. N o t a t i o n 2.2. Study o f t h e H.J.B.
equation
139 140 142 142 142 143
3. SOLUTION OF THE STOCHASTIC CONTROL PROBLEM
150
4. EVOLUTION PROBLEMS
156
4.1. P a r a b o l i c e q u a t i o n s 4.2. S t o c h a s t i c c o n t r o l problem 4.3. M a r t i n g a l e p r o p e r t y 5. SEMI GROUP FORMULATION 5.1. 5.2. 5.3. 5.4. CHAPTER V .
156 158 160 16 1
A property o f t h e equation u The problem o f semi group envelope I n t e r p r e t a t i o n o f t h e d i s c r e t i z e d problem A regularity result
161 164 170 178
FILTERING AND PREDICTION FOR LINEAR S.D.E.
191
INTRODUCTION
191
1. SETTING OF THE PROBLEM
191
1.1. S t a t e e q u a t i o n 1.2. The o b s e r v a t i o n process 1.3. Statement o f t h e problem 2. CHARACTERIZATION OF THE BEST ESTIMATE 2.1. Main r e s u l t 2.2. Q u a d r a t i c o p t i m i z a t i o n problems
199 199 203
KALMAN FILTER
208
A g e n e r a l system Recursive formulas Kalman f i l t e r The i n n o v a t i o n process
208 210 213 216
3. RECURSIVITY 3.1. 3.2. 3.3. 3.4.
-
191 195 197
4. PREDICTION
219
CONTENTS
XiV
CHAPTER V I .
VARIATIONAL METHODS I N STOCHASTIC CONTROL
221
INTRODUCTION
221
1. MODEL WITH ADDITIVE N O I S E
22 1
1.1. S e t t i n g o f t h e model 1.2. A d e n s i t y r e s u l t 1.3. Necessary c o n d i t i o n s
221 224 227
2. THE CASE WITH INCOMPLETE OBSERVATION 2.1. 2.2. 2.3. 2.4.
Statement o f t h e problem Prel i m inary r e s u l t s Necessary c o n d i t i o n s The q u a d r a t i c case. S e p a r a t i o n p r i n c i p l e
3. SEPARATION PRINCIPLE Orientation 3.1. The Kalman f i l t e r 3.2. A dynamic programming e q u a t i o n 3.3. S o l u t i o n o f t h e s t o c h a s t i c c o n t r o l problem w i t h incomplete information CHAPTER V I I .
PROBLEFIS OF OPTIMAL STOPPING
234 234 237 240 243 253 253 254 255 26 9 279
INTRODUCTION
279
1. SETTING OF THE PROBLEM
279
1.1. Assumptions. 1.2. Remarks
Notation
2. UNILATERAL PROBLEMS 2.1. P e n a l i z e d problem 2.2. L i m i t problem i n t h e r e g u l a r case 3. VARIATIONAL INEQUALITIES
279 281 281 281 282 287
Orientation 3.1. L i m i t o f t h e p e n a l i z e d problem 3.2. Weakening t h e c o e r c i t i v i t y assumption
287 288 290
4. SOLUTION OF THE OPTIVAL STOPPING TIME PROBLEM
301
4.1. The r e g u l a r case 4.2. The non r e g u l a r case 4.3. I n t e r p r e t a t i o n o f t h e p e n a l i z e d problem 5. S E M I GROUP APPROACH 5.1. S o l u t i o n o f t h e V . I . as a maximum element 5.2. The case o f g e n e r a l semi group
30 1 303 305 307 307 315
CONTENTS 5.3. D i s c r e t i z a t i o n 5.4. Case w i t h o u t c o n t i n u i t y 6. INTERPRETATION AS A PROBLEM OF OPTIMAL STOPPING 6.1. Markov process r e l a t e d t o t h e semi group 6.2. Problem o f o p t i m a l s t o p p i n g 6.3. I n t e r p r e t a t i o n o f t h e d i s c r e t i z e d problem CHAPTER V I I I .
IMPULSIVE CONTROL
xv 326 336 339 339 34 1 345 355
INTRODUCTION
355
1. SETTING OF THE PROBLEM
356
1.1. Assumptions. 1.2. The model 1.3. Some remarks
Notation
356 357 359
2. QUASI VARIATIONAL INEQUALITIES
360
Orientation 2.1. S e t t i n g o f t h e problem 2.2. S o l u t i o n o f t h e Q . V . I .
360 360 36 1
3. SOLUTION OF THE IMPULSIVE CONTROL PROBLEM 3.1. 3.2.
The main r e s u l t I n t e r p r e t a t i o n o f t h e d e c r e a s i n g scheme
4. SEMI GROUP APPROACH 4.1. S o l u t i o n o f t h e Q . V . I . as a maximum element 4.2. Case o f g e n e r a l semi groups 4.3. D i s c r e t i z a t i o n REFERENCES
368 368 387 39 1 391 392 395 399
This Page Intentionally Left Blank
1
CHAPTER I
STOCHASTIC CALCULUS AND STOCHASTIC DIFFERENTIAL EQUATIONS
INTRODUCTION This chapter is devoted to the presentation of the stochastic dynamic systems which will be used throughout this work, namely those whose evolution is described by stochastic differential equations. This requires a stochastic calculus and the concept of stochastic integral, originated by K. Ito. The model looks like dy
=
g(y,t)dt
+
a(y,t)dw(t)
and g is called the drift term, 5 the diffusion term. This model generalizes the model of ordinary differential equations
and expresses the fact that the velocity is perturbed by a random term of mean 0. In the standard set up (strong solution) one assumes lipschitz properties of g, u with respect to the space variable. It is important for the applications to control to weaken the concept of solution in order to assume only measurability and boundedness of the drift term. This is achieved through Girsanov transformation. We have kept the presentation to what is essential within the scope of this text. But, aside basic preliminaries in Probability theory, we give complete proofs. We refer to the comments for indications on the natural extensions. Basic references for this chapter are the books by J. NEVEU C l l , I. GIKHMAN-ASKOROKHOD ~ 2 1 ,A. FRIEDMAN r i i , D. STROOCK - S.R.S. VARADHAN E l ] , E.B. DINKIN 1 1 3 .
2
CHAPTER I
R be a set. A 0-algebra
Let
R is a set of subsets
ff on
of
R
such that V
Ai
nAi i if
a
The elements of
, i
I , Ai
E
E
a,
(I countable)
then
, uAied i
Acff
,
@ A c a
are called e v e n t s .
The set
(@,a) is
a o-algebra.
It is contained in all o-algebras on R. It is called the trivial
a-algebra. A probability on i.e., a map
A
+
(Qg)is P(A)
from
a positive measure on 0
a
into
[O,l]
, with
total mass 1,
such that
P(Q) = 1
P(u
n When
P(A)
= 1,
An) = Z P(An) n
one says that A
if the An
are disjoint.
is aZmost certain
(Q is the certain
event). The triple If B of Q
c
(R,Q,P) is called a probability space.
a,and
is also a 0-algebra we say that i3
is a sub o-algebra
*
On
R
(the set of real numbers), the open intervals generate a o-algebra
on
R, which is called the Bore1 0-azgebra on R.
3
STOCHASTIC CALCULUS
On a product space X
1
x X2
, if X I ,X
x1 x X 2
respectively, the product o-algebra by the events of the form A'
x
Hence the Borel u-algebra on
Rn
' A
are u-algebras on
where
X
I
,
X
2
is the u-algebra generated
A'
x1 , A2
E
E
X2 .
is the a-algebra generated by open
cubes. The concept carries over to an infinite set of spaces X1, i
E
I
product u-algebra B X1 , is generated by the events of the form . I , Ai = X1 except for a finite number of i. Ai E
xi
A random variable is a measurable map
R
Borel u-algebra on If
fi
, i
I
E
E
a,ti
B
E
,
R
/3
i.e., if B
I
denotes the
.
It is called the o-algebra generated
fi are measurable.
by the family fi @
f-'(B)
The
TI 'A ,
are random variables, there is a smallest o-algebra for
which all maps u-algebra
,
R -tf
.
,
and denoted by
u(fi,i
E
I).
It is also the product
.
u(fi)
I Note that if is a R.V.
fk
are random variables and
fk(w)
-t
f(w)
br w, then
f
Also a positive random variable is the increasing limit of
a sequence of piecewise constant positive R.V. namely
We will need some results concerning extensions of probabilities,for which we refer to J. NEVEU [ I ] . Let
R be a set. We call algebra a set of subsets of R, satisfying
properties (l.l), ( 1 . 2 ) ,
( 1 . 3 ) except that in (1.1)
the set
I is not
countable, but only finite. We say that a class C5 {C,,
n
in C
t 1)
such that
n
nloreover, we have from (ii), to check that u(x)
From this, one easily deduces,by a limit argument as
(iii)
au - g.Du
=
f
6
+
0 ,
a.e.
which is a first order hyperbolic partial differential equation (P.D.E.). We have emphasized the point of view of computing a functional on the trajectory of the solution of an O.D.E. One can also have the reverse point of view, which consists in giving eqZCo7:t formulas for the solutions of P.D.E. In the litterature on P.D.E., this is known as the method of c h n r a c m r i s s k s . The O.D.E. ( 1 ) indexed by the parameter x, are called the characteristics of (iii). A s we will see, for elliptic and parabolic second order P.D.E., one can introduce characteristics curves, which are diffusion processes. In our presentation, except two difficJit regularity results on P.D.E. we have given full proofs of the results, but we have restricted ourselves to those which will be effectively used. The basic references are A . Friedman [21, O.A. Ladyzhenskaya and N . h . Ural'tseva [ I ; , O.A. Ladyzhenskaya - V . A . Solonnikov, N.J. Ural'ceva [ I ] , J.L. Lions [ I ] , S. Agmon - Douglis - Sirenberg [ I ; , C. Miranda [ I ] , C.B. Morrey [ I ] , G. Stampacchia [I], J.L. Lions -
52
CHAPTER I1
E. Magenes [ I ] .
For Sobolev spaces see for instance F. Trhves [ I ] .
1. FUNCTIONAL SPACES 1.1. &blgd-spaws
Let
We will use constantly the SoboZev
8 be an open subset of R”.
spaces
, WzYp(S) ,
W”p(8)
W’%)
=
2
ff
5
p
E
LP(Cs)
< m,
integer. We recall that
1%
€
LPOS)}
1
,
W”pp)
For
p
=
W2”(0)
are Banach spaces for the norms
2 , we use the notation
HZ
= w192
= w292
They are HiZbert spaces. In the case when 8 follows (1.1)
(p,q
...
if n>p ,
€
is smooth, we have important i n c l u s i o n theorems, as
[I,=)) 2
W’’p(o) c L (8) with continuous injection
Clearly this result implies that if W2*p(c) c W’”((S)
n > p with continuous injection
53
PARTIAL DIFFERENTIAL EQUATIONS n > 2p
and if
(1.2) if p>n with
,
-1
a = 1
P
then
c
~ ~ ( (')6 )with
continuous injections
c
$65)
8 = 2 - 1
w"P(o)
.
Consequently
if n > p E 2
W2'Pf3)
(1.3)
Let
f
E
P
, and f
W2"f&)
# integer
20
for any
E
[ + , I 1 , and
c
I If/
I If1 For example taking
a
lwl,r
1,
0. Let
p
o
.
Under assumptions 12.171, 12.181 t h e r e exists f o r
one and o n l y one solution
Consider t h e f u n c t i o n a l on
J(v) =
u
E
1
Ho(0)
f
E
L2@)
such that
H;)(O)
C '03ij i,j
a v dx axj 5
From P o i n c a r 6 ' s i n e q u a l i t y i t follows t h a t
+
a.
v2 dx - 2
f v dx
.
65
PARTIAL DIFFERENTIAL EQUATIONS
1
It i s a strictly convex continuous functionalonHo(B) admits a unique minimum.
.
Therefore it
Since clearly (2.19) is the Euler equation
0
for the minimum, it has also one and only one solution.
,?emark 2.2.
The result of Lemma 2.1 holds even when the matrix
not symmetric (wnich is here a restrictive assumption). Lax-Milgram Theorem (cf. for instance K. Yoshida [ I ] ,
a . . is li
This is the
J.L. Lions [ I ] ) .
The proof is different, since (2.8) cannot be interpreted as an Euler
D
equation. Remark 2.3. .The assumption
f
assume
E
f
E
L
2
(8)
is n o t really necessary.
(2.20)
W”~(R~)
a. lj
.
Consider also (2.21)
a
i
E
.
L~(R~)
We write (2.22)
We will set (2.23)
and
bi
b.(x)
E
3
H-l(O) with the same result.
Let us assume now that besides (2.17) the
Lffi(Rn)
.
=
aa. - ai(x) + C ax j j
We can
aij
verify
66
CHAPTER I1
We define next the bilinear form on
H1(0)
(2.24) +
Now if
v
E
1 Ho
1
(2.25)
Theorem 2 . 3 . f
E
,
Lp(0) , 2
fs.a
we deduce from Green’s formula (1.18) and (1.23) that a(u,v)
u
E
=
H1@)
+
,
Au
fsa. E
u v dx
fi v
E
p
O
some positive number
.
Indeed we make the change of unknown function
where yo
to be chosen and
xo
is fixed outside
6.
Clearly (2.26)
is equivalent to
Let us make explicit the operator w A(wz).
We have
(1) Because of (2.20) the symmetry assumption on
a.. is not restrictive 1J
67
PARTIAL DIFFERENTIAL EQUATIONS
= -
C
i,j
a
w - (a..(-
axi
L
Let u s s e t
We have f o r
hence
x
E
0
IJ
ao
ax
+
z
j
I
OJ
aZ
-))
ax. J
t
68
CHAPTER I1
aij(x.-xoi) (x.-x ) J
c
+ yo
i,j
Ix-xol
2
Oj
-
C a.
ii
therefore
exp - ylx-xo/
outside 5, i t is possible to choose yo > 0 such that
Having fixed xo when
.
runs into 8 one has
x
, problem (2.26) is equivalent to the same problem with 2 aij changed into , ai changed into a.w2 , a. changed into aij w(aow + X) 2 y > 0 , and f changed into fw. Therefore we may assume Since
1 5 w 5 2
(2.27)
.
Let u s next prove that for
h
large enough, the problem
has one and only one solution. Let u s first show uniqueness.
f
=
0 , from Green's theorem, it follows that 0 0 a(u , u ) + A luo12
(1) We cannot take
y
=
o
arbitrary large.
Assume
69
PARTIAL DIFFERENTIAL EQUATIONS
A
and we can assume t h a t
hence
L e t u s prove e x i s t e n c e .
uo = 0 .
Define f o r
z
i s l a r g e enough s o t h a t
1
Ho
E
(2.32)
,
5
1
E
'0 a i j =
t o be t h e s o l u t i o n of
Ho
av dx a 1~a. ~ . 1
aZ Is ( f - Z a i %)v
+ J
dx
( a + A ) < v dx 0
8
=
,
i V v r H
Then
5
1 0 '
e x i s t s and i s d e f i n e d i n a unique way, by v i r t u e of Lemma 2 . 1 1 Ho i n t o i t s e l f S ( z ) = Consider
:.
We t h u s have d e f i n e d a map from
cl
z1,z2 and
Take
and i f
v =
tl-c2 ,
A
Let
, c2
=
S(z2).
We have
we deduce t h e e s t i m a t e
i s s u f f i c i e n t l y large it follows t h a t
which proves t h a t point.
S(zl)
=
uo
S
i s a c o n t r a c t i o n mapping i n
be t h e f i x e d p o i n t .
1
Ho, hence i t has a f i x e d
From ( 2 . 3 2 ) we have
70
CHAPTER I1
We next use Theorem 2 . 1 .
l-
(2.34)
Then
zo
i,j
There exists one and only one solution of
XTET
aij
1
+ Azo
=
f +
;
aU 0
bi
j
- a.
u0
1
satisfies -0 au
(2.35)
' 0 aij
~i
Js zo v
+ X
2i $i -0
JS
+ C
av dx K a~ dx
=
aa.
v dx +
auO a u O )v dx (f+Zbjaxj- o
Js
J
y v c H 01 .
But from ( 2 . 3 3 )
satisfies
uo
auO av auO aaij v d x + A J u 0 8 axi d x + C /8 -ax j ax.i -U aU a u 0 )v dx . =Jo(f+;bjaxjc Jo
(2.36)
aij
vdx=
By difference between (2.35) and ( 2 . 3 6 ) we obtain
Z JS aij
v
and since
Go
=
A
- uo
,
ax j
dx
=
(ea")'-' -0
auO av ax j axi
dx + C JS
- -)-
(zo - u 0)v
+ A J8 Taking
a;'
(-
j
0
0 aa..
-
ax j a..~i
v dx +
.
we see that
is sufficiently large,
(1) The existence and uniqueness of consequence of Lax
-
Lo =
uo
Milgram theorem
uo
.
solution of ( 2 . 3 3 ) i s also a
71
PARTIAL DIFFERENTIAL EQUATIONS 1 uo E H2 n Ho
Hence
and
is solution of
uo
AUO + (ao+X)uo
=
f
a.e. in 0 ,
or
If
u
0
0
, (assuming n
H2
E
>
PAq 1 (by ( l . l ) ) ,
hence
g
E
L
and
p,
=
pAq, > 2
uo E’W~’~~(;)
using Theorem 2.1 again we get
L q l , with
axi
2 ) , we have
.
1 = r - 1 q1 2 n
p > 2).
(if
Therefore
By a bootstrap
argument, in a finite number of steps, we obtain uo
E
,
W2”(@)
hence
(2.30) is proved. The next step is to define the following sequence
which is well defined, by virtue of what we have done to study (2.30). We have (2.38)
The important fact to notice is that there is an improvement of regularity at each step. ul-uo
E
u2-u1
obtains for assert that
n
2
E
W2”(0)
n WA”(0)
hence
.
L q l with
Therefore
ul-u’
Indeed
E
-= -- n (if n > 2p) q1 P 2,q1 l’ql W n W, Again using a bootstrap argument, one
no , un - u’-’
.
E
Lm.
Using Lemma 2.2 below, we can then
12
CHAPTER I1
Hence at least for n
2
no, un - un0
is a Cauchy sequence in Lm.
Passing to the limit in (2.37) we conclude to the existence o f
u
solution
of (2.26).
Let us prove the uniqueness of the solution of (2.26). u
Let
f = 0
,
and
to be solution of A u + a u = O 0
then
and by the regularity argument used for (2.37), u
m
E
L
.
From Lemma 2.2
it follows that
hence
I lull
=
0
.
0
Lrn
Lema 2 . 2 .
Consider t h e equation
(f
E
which has one and onZy one s o l u t i o n f o r f
m
E
L , one has
u
m
E
L
Lp(8))
X large enough.
Then if
and
Before proving Lemma 2.2, we will recall an important Lattice property o f the space
1
H (8).We say that
73
PARTIAL DIFFERENTIAL EQUATIONS
Define
then u1
V
,
u2
u1
A
u2
E
1
H (0),
namely the following formula holds (2.41)
where
x
7
=
1
if
u1
u2
2
and
if
0
u 1 < u-
.
Setting in
u zu particular
then u = u
+
- u
-
.
+
, u
Remark that (2.42)
uiHA+u
This is clear when (2.43)
n u
u
B(G). Then one has to notice that
E
1 E H
implies This is clear since
1
6 H O .
, (un)+
in H
u n + 0 +
o
in
H'
1
.
14
CHAPTER I1
ahn)+
-=
ax
aU" axi X n
u LO
hence
Let us also note the relation (2.44)
a(v+,v-)
=
o
Y v
E
I
.
H (8)
W e can then give the
Proof of L e m a 2.2 Let
K
=
llLLl X+Y
then
(u-K)
+
E
I
Ho, hence from (2.39) a(u, (U-K) +) +
or
x (u,(U-K) +)
= (f
, (U-K) +)
PARTIAL DIFFERENTIAL EQUATIONS By a similar argument one obtains u t -K , hence (2.40).
Lemma 2 . 3 .
Assume in Zzenrem 2.3 t h a z
f
t
0. Then
u
0 2
0 (u
scltrtion
of ( 2 . 2 6 1 1 .
This is done by induction on the sequence Assume
un 2 0
,
then multiplying by
un
(un+I)-
defined by (2.37) we get
hence a((un+')-,(un+')-) which implies
n+l (u )
=
0
+
> 1 (un+l)-12
s
o
.
A similar argument holds for
uo, hence the desired result.
3 . PARABOLIC EQUATIONS
3.1. &nc;jmrm-~p~ces
We will use
dz The meaning of d t has to be explained. We note the inclusions
each space being dense in the following, with a continuous injection. Thus we can consider
z
also as an element of
L2(0,T;H-'(O)).
Its
derivative makes sense as a distribution and is defined as follows
76
CHAPTER I1
.
-1
the second integral being with values in H
aZ -6 8’ ( (0,T ) ; H - (0) ~ ). at
@
+
- JT
@ ’dt
z
E&(L~(O,T);H-’(@))
aZliE L2 (O,T;H-’@)) then we say that ; important property of
C
and that
z
E
W(0,T).
0 C ([0,Tl;L2(o))
with continuous injection.
Property (3.1) is true at least after a modification of
0
of measure If
on a set
z
.
z1,z2 E W(O,T),
the following integration by parts formula holds
dz
JT
=
We write
An
is the following
W(0,T)
W(O,T)
(3.1)
Hence a priori
If
< 1x z, 2 > dt + IT < z 1
, dt dz2 >
dt
.
Q = 8 X (0,T) and consider spaces of the form 2 $ 2 $ P ( ~ )=
(3* 3)
12
L~(Q) ,
6
ax; , ax;ax. , aLz
aZ
aZ
6
L~(Q)I
J
with the natural norm of Banach spaces.
...
Similarly we define bl””(Q) b Z y 1= b 2 ” ”
(3.4)
,
= {z E
1 5 p
O , p > 1 , p < m
/ z ( x ) I p exp - p p ( l + l ~ ~ dx ~ )< ~m /} ~
78
CHAPTER I1
We will use also the following embedding theorems (cf. Ladyzhenskaya Solonnikov- Ural’stseva If
z E
L ~ ’ l ’ p, p >
If
z E
h2””
Let C?
+ I
p > n + 2
, then z
We define
1
E
Co(q)
q
, then z , a z
be an open subset of
(3.9)
( 3 . 10)
,
2
111).
aij , ai ,ao
Rn
and m
E
L (Q)
E
C
o (Q) -
.
Q = 8x (0,T).
We assume
79
PARTIAL DIFFERENTIAL EQUATIONS
(3.11)
a(t;u,v)
au av Jsaij(x,t) ax. - dx ax. 1 1
Z,
=
+ C J
i
Theorem 3 .
+
~
i,J
8
* ax
a.(x,t) 1
v dx + J
8
a (x,t)u v dx
0
I. Assume 13.91 and
ii
(3.13)
.
L20)
E
Then t h e r e erists one and onZy one s o l a t i o n du - + a(t;u(t),v)
=
a.e. t
in
(0,T) 1
V V E V = H 0 u(T) =
Remark 3 . 1 .
u
Formulation (3.14) is the evolution analogue of a(u,v)
(f,v)
=
, u
E
Here we will not need to split the form
1
Ho
,
V v
a(t;u,v)
E
1 H0 '
in two parts as we
have done in the elliptic case (cf. Lemma 2.1 and Theorem 2.1). we will
not need to assume the symmetry of
gemark 3.2.
a. ij
Problem (3.14) is backward in time.
Also
*
We have considered a
backward evolution problem instead of a forward evolution problem for convenience, since we will consider them more often. the change
t = T - s
,
s
Note that making
[O,T], one can formulate (3.14) as a
E
forward evolution problem.
Remark 3.3. (3.15)
Since
I
A(t)
-;if u
E
d(V;V')
+ A(t)u
'd u(T)
E
=
=
6
W(0,T)
.
,
f(t)
we can write (3.14) as follows
.
80
CHAPTER I1
The differential equality has to be viewed as an equality between
.
2
elements of
L (0,T;V')
3ernark 3 . 4 .
We have not assumed a. 2 0.
In fact we can without loss of
generality assume
(3.15)
a o > y > O
, y arbitrary large
Indeed, make the change of unknown function y
exp - X(T-t)u
=
then problem (3.14) is equivalent to
hence we have the same problem, with a .
changed into
+
a .
I
'
a
changed into
f e-''(T-t)
* *
. 'Wn'
The space
V
being separable, there exists an
* * *
We look for an approximate solution as follows m
(3.16)
U,(t)
=
c
eim(t)wi
i= 1 where the scalar functions gim(t)
are solutions of the following
linear system of ordinary differential equations (3.17)
1
-
and
0
'5
We use Galerkin's method orthonormal basis w
f
du (2 ,w.) J
+ a(t;um,wj) = J
, j
=
I
,...,m
81
PARTIAL DIFFERENTIAL EQUATIONS
where
m
E
.
um u
Cwl,. .,w 1 and m
Multiplying (3.17) by
g
jm
as
m
+ m
.
and adding u p , we obtain
- Id td lUm(t) 1'
(3.18)
in L2(S)
-t
+ a(t;um(t),um(t))
=
.
However by (3.15) we can assume that
hence the estimate
which implies (3.20) Let
$
is bounded in Lm(O,T;H)
u
E
1
C (C0,TI)
such that
@(O)
=
0
.
Set
We can extract a subsequence, still denoted (3.21)
u
Multiply (3.17) by
and letting (3.22)
m
+
u
in L 2 (0,T;V) weakly
u
and integrate.
Q(t)
tend to
and
+a:
$.(t) = $(t)wj J
such that
m ,
We obtain
, yields
1' (u,g')dt j
L 2 (0,T;V)
(U,Oj(T))
+ /' a(t;u,@.)dt = 3 J
.
.
82
CHAPTER I1
is arbitrary it follows from ( 3 . 2 2 ) that
Since j
V V € V
4
E
&(O,T))
hence V v
E
V
Taking
du E dt
,
2 L (0,T;V')
we obtain in the sense of distributions
d (u(t),v)
,
hence
u
L2(0,T)
E
implies (see 5 3.1),
.
W(0,T)
E
, which
Using next integration by parts in W(0,T)
and ( 3 . 2 3 ) yields u ( T )
Hence existence is proved. For uniqueness take
f =
0
,
u
=
0
.
Take v = u ( t )
in ( 3 . 1 4 ) .
We obtain
.
hence
u = 0
3.3.1.
Regularity with respect t o time
Theorem 3 . 2 .
We make t h e assumptions of Theorem 3 . 1 , and
(3.24)
a,. = a 1J ji
(3.25)
a t aij
(3.26)
f E L'(Q)
a
E
L-(Q)
, z
H~I
.
=
G.
83
PARTIAL DIFFERENTIAL EQUATIONS
Then we have
(3.27)
We define (3.28)
ao(t;u,v)
=
aU av dx Z J8 a.1 J.(x,t) ax.1 ax. 1
and
hence a(t;u,v)
=
ao(t,u,v) + (A1u,v)
.
We consider the Galerkin approximation (3.17) which is rewritten as f01lows (3.29)
+ -(dt,w.) J
ao(t;um,wj)
=
(f
-
A1um,wj) ,
j=l,
and we may assume that (3.30)
i +U
Multiplying (3.29) by
in
-
HI:
g! (t) Jm
. and adding up in
j, we obtain
du (3.31)
I t 1
+ ao (t;u, ,-u’) m
= (f
- A u -ul) 1 m, m
.
But if we set
it follows from
(3.21) using the sdr’r%ri’d, a (t;u,v) = ao(t;v,u) 0
...,m
84
CHAPTER I1
hence
T 2 Jt 1- dt
1
d s + - a (t;u (t),u (t)) 2 0 m
+
Jtl
=
( f - A u -u’)ds 1 m, m
and from the assumptions we deduce
c
from which we easily deduce ( 3 . 2 7 ) .
3 . 3 . 2 . Regularity with respect t o space variables
Theorem -
3.3.
We make t h e asswrrptions o f Theorem 3 . 1 , and ( 3 . 2 4 ) , ( 3 . 2 6 )
and (3.32)
a.. 11
Then t h e solution u
E
w””(Q)
.
of (3.141 s a t i s f i e s
The idea is t o use Theorem 3 . 2 and the results on elliptic equations (cf. Theorem 2 . 1 ) . (3.34)
If we fix
Let us write ( 3 . 1 4 ) as
ao(t;u(t),v) t
= (f
+
du dt
A
1
U,V)
(outside a set of Lebesgue measure
( 3 . 3 4 ) as an elliptic equation, where
t
0), we can consider
is a parameter.
Since
85
PARTIAL DIFFERENTIAL EQUATIONS du 2 ;i-i - A1u E L ( Q ) can assert that f
and assumptions of Theorem 2.1 are satisfied, we
+
However there is a slight technical difficulty since this does not prove that t
-t
u(t)
is measurable from (0,T) into H2(8)
.
To overcome this difficulty, we use a time discretization procedure.
du -A u dt ln define a sequence u Set
g = f + -
(3.35)
L
E
2
(9) , We discretize O,k,...,Nk
=
T
by
a0(nk;un,v)
=
(gn,v)
ti v
E
HoI , un
E
1 Ho
where (3.36) Then un
E
2 H (8) and
(3.37) where
C
does not depend on n,k gk(t)
=
gn
for
. t
Define next
E
[(n-l)k,nk[
remains in a bounded subset of It follows from (3.32) that uk 2 2 L (0,T;H 6)) hence, by extracting a subsequence (3.38) Let v
uk
E
L2 (0,T;V)
-t
.
w
in L2(0,T;H2@))
weakly
We deduce from (3.35) that
.
and
86
CHAPTER I1
JT
(3.39)
0
a (t;uk(t),v(t))dt 0
=
JT
Cao(t;uk(t) ,v(t))
nt = integer part of
Therefore letting k
+
0
-
ao(ntk;uk(t),v(t)
(gk(t),v(t))dt
+
where
=
-.kt
But
in ( 3 . 3 9 ) we obtain
hence a.e. t
a0(t;w(t),v)
=
du (f(t) + d t - A1u,v)
.
This and ( 3 . 3 4 ) implies u(t)
=
w(t)
a.e.
which completes the proof of the desired result. R m a r k 3.5.
u
E
#*"(Q)
Under the assumptions of Theorem 3 . 3 we can write and
(3.40)
I
u(x,T)
=
i(x)
Idt +
87
PARTIAL DIFFERENTIAL EQUATIONS
3 . 3 . 3 . Other regularity results We now state without proofs (which are difficult) the analogue of the results stated in
§
2 . 1 for elliptic equations.
spaces of Hglder functions in x,t We denote for
0 < a < 1
,
We need to consider
.
by
+ sup X
1 z (x,t)-2
(x,t )
L
I t-t' 1 a12
t,t'
We then define
c1+a,a/2
and
and
- = (4)
{z(Z E C
0
-
aZ
(Q) , axi
Ca,a/2(Q) - 1
CHAPTER I1
88
with
Let us assume a
(3.41)
ij
(x,t)
Ca , q ? j )
E
,
0 < a 1
,
p
o
Bore1 bounded.
Our objective is to prove the following
Theorem 3.1.
Under the assumptions ( 3 . 1 1 , ( 3 . 2 ) the s o k t i c n
the
0
martingale p r ~ b Z e mis unique. We will use several lemmas.
Lemma 3 . 1 .
It is sufficient to prcve uniqueness uhen bi
=
0
Considering the change of probability defined by (2.16), then solution of the martingale problem with unambiguously for
Fxt
0. Since
Fxt
is
is defined -xt by formula ( 2 . 3 0 ) , then the uniqueness of P
implies the uniqueness of We assume that
b
.
PXt
.
=
Pxt
0
118
CHAPTER 111
A(t)
(3.3)
=
-
C
a .(x,t)
iJ
i,j Let
4
E
,
C2+a(Rn)
a2 axiax j
consider the Cauchy problem
which has one and only one solution by Theorem 3.8, Chapter 11.
Lema 3 . 2 .
Let
pxt be a s o l u t i o n o f t h e martingale problem correspcnding
t o t h e operator
A(t)
given by ( 3 . 3 : , then we have
props Since
Property (3.5) is an immediate consequence of Ito's formula applied to the function u
and to the process
regularity of
Lemma 3 . 3 . (3.6)
x(s), which is possible by virtue of the
0
u.
~ e ~t O
YB
Borel subset of
P(x,t;T,B)
=
PXt(x(T)
R"
E
B)
,
t 5 T
then t h e f u n c t i o n P(x,t;T,B) does not depend on t h e p a r t i c u l a r s c l u t i o n of t h e martingale probZem.
119
MARTINGALE PROBLEM
Proof Since on a metric space provided with its Bore1 o-algebra, a probability is uniquely defined by the values of
(see for instance J. Neveu [ I ] , V @
for
p. 60).
,
Co(Rn)
E
E@
@
continuous bounded
It is enough to check that
Ext @(x(T))
does not depend
on the particular solution Pxt. From Lemma 3 . 2 , the result is proved for @
E
C2"(Rn).
Since V I$ ll@kll 0 C
E
Co(Rn)
' ll@l
I
0
,3
a family
@k of smooth functions such that
and
C
Gk
+
I$ uniformly on compact sets,
we clearly have
If now
Ft is
another solution of the martingale problem, from the
equality
and from ( 3 . 7 ) it follows that EXt @(x(T))
=
Ext @ ( x ( T ) )
hence the desired result.
i e m a 3.4.
The function
i.e. it satisfies
P(x,t;T,B)
is
il
Markov transition function
CHAPTER 111
120
x
+
P(x,t;T,B)
is BoreZ
P(x,t;T,B)
7:s
a probability on Rn
(3.9)
B
(3.10)
P(r,T;T,B)
=
xB(x)
(3.11)
P(x,t;T,B)
=
J P(x,t;s,dy) P(y,s;T,B) R"
,
t S s 5 T
(3.9) and (3.10) are obvious from the representation formula (3.6). if
I$ E Co(Rn)
,
x
-Y
E
Xt
@(x(T))
is Bore1 for
approximation result (3.7) and since c2+a,1+a/2 function. Therefore (3.8) is verified if
B
Ext I$~(x(T))
=
uk(x,t)
is a
is a bounded ball, hence also for
compact and from Neveu C11, p. 61, for
B
Now
t < T , since we have the
B
borel.
It remains to prove (3.11), or which is sufficient
0
for any
E
.
O n C (R )
A s for Lemma 3.3, it is sufficient to check (3.12) for function which are @
E
c2+a( R ) .
Bu then considering the function u(x,t)
(3.4) and the function v(x,t,s)
i:
t
5
,
s
solution of
defined by
2
--
c '
i,J
a. .(x,t) a v - 0 ax. ax. 1J
t 5 S
1 - 3
v(x,s;s)
=
u(x,s)
v(x,t;s)
=
u(x,t)
then we have
which is obvious by the uniqueness of the solution of ( 3 . 4 ) .
121
MARTINGALE PROBLEM
Lema 3.5. Is Pxt is avy solution of t h e martingale probLem, t h e n
(3.13)
Pxt [x(T)
E
Blms]
=
P(x(s),s;T,B)
a.s.
V t S s < T
It is sufficient to verify that for
I$
O n C (R )
E
and
cs,n
measurable
and bounded then we have
It is also enough to prove (3.14) with
@
C2'a.
E
In that case (3.14)
reads
But from Ito's formula and (3.4)
0
which easily implies (3.15).
Proof of Theorem 3 . 1 From formula (3.13), recalling that the functions P(x,t;T,B)
does not
depend on the particular solution of the martingale problem, we deduce easily that (3.16)
PxL [x(t,)
E
B ] , . . . , x(tn)
E
BnI =
Fxt [x(tl)EB1 ,..., x(t,)EBn!
V t l,...,t
2
t
,
,...,B
B1
Bore1 sets and where
PXt, pxt are two
solutions.n From Theorem 1n2, Chapter I, it follows that
Pxt
=
pxt . 0
122
CHAPTER 111
Remark 3 . 1 .
We have proved in addition that
x(s)
,m: , PXt
is a
0
Markov process.
We assume here that (3.17)
a
measurable bounded
(3.18)
bi measurable bounded.
ij
,
-1
aij = aji ; a
bounded
We recall that A(t)
Theorem 3 . 2 .
=
-
C
i,j
a. .(x,t) 'J
Under asswnptions 13.171,
a2 axiaxj
Ibi
a
13.181 there e x i s t s a s o l u t i o n of
t h e martingale problem.
Proof Assume first that (3.19)
I
a=-Do
*
2
with
Then according to Theorem 5.2, Chapter I, there exists a weak solution of the S.D.E.
at
Hence on some adequate 0,O ,5
n dimensional Wiener, and
y(t)
t
,
such that
there exists
, w(t)
123
MARTINGALE PROBLEM
F : R
Consider the map w
(3.22)
Ro
-+
y(.;w)
-f
Pxt the measure on 711
and
image of
t
?
F
by
.
Then PXt is a
solution of the martingale problem. Indeed let
...,xk
xl,
@
E
,
J%R")
5,
and
to be a measurable function of
; write
5,
(3.23)
=
t
Ss(Y(S1), ...,y (s,))
s
... s
s1 5
Sk
5 s
.
From Ito's formula we have
hence taking the image measure
and since
5,
generate 711;
,
it follows that
Pxt
is a solution of the
martingale problem. Assume now that we have (3.17), then consider a sequence an (3.26) and
sup lan(x,t)-a(x,t)l X,t
is Lipschitz in
a
(3.17).
x
+
o
as
n
-c
such that
-
uniformly with respect to
t, and satisfies
Such a sequence exists. There exists a solution Pn of the
martingale problem, corresponding to the pair
Therefore
(O,an) and
124
CHAPTER 111
This implies that the sequence Pn
remains in a relatively compact provided with the Ro,mt Let us extract a subsequence, still
subset of the set of probability measures on weak topology (see Parthasaratly [ I ] ) . denoted by
Pn
such that
(3.28)
Pn
-+
P weakly
...,
Consider a continuous function s(xl, xk) which is bounded; writing t 5 s1 5 s sk 5 s then we have, for 5 , = 6(x(sl), x(sk))
...
...,
0
E
B(Rn)
Using (3.26) and (3.28), we can let n
tend to
+m
in (3.29), and
obtain that
P
pair (0,a).
Use Girsanov transformation to obtain a solution of the
is a solution of the martingale problem relative to the
martingale problem relative to the pair
Remark 3.2.
7
(b,a).
We can get rid of the assumption
a
-1
exists and is bounded,
to obtain an existence result, provided that we replace (3.18) by the strongest assumption (3.30) Indeed replace a
SUP
1 x-y 1 s6 by
/bi(x,t)-b.(y,t:l/
p(6)
+
0 as
6
+
0
.
a+EI, then according to Theorem 3.2, there exists
a solution PE relative to the pair
and again
5
b,a+EI.
We have
MARTINGALE PROBLEM
Therefore PE
12s
remains in a relatively weakly compact subset of the set
of measures. Now we have, analogously to (3.29)
By virtue of (3.30) we can let
E +
0 in (3.31) and obtain the desired
0
result.
4. INTERPRETATION OF THE SOLUTION OF P.D.E.
Consider here
aij(x)
a
such that
i Borel bounded on Rn
,
Define (4.3)
Let also (4.4)
. a
Borel bounded
. a t 0 ,
According to Theorems 3.1 and 3.2, there exists one and only one solution of the martingale problem corresponding to the initial condition x
0. More precisely let Q o probability measure
Px
on
, m 0 , n to , there exists one and (Rono) such
that
only one
at
126
CHAPTER I11
(4.5)
PX[x(e)
= XI = 1
(4.6)
@(x(t))
-
@(x) +
Jt A$(x(s))ds &Rn)
.
is a smooth bounded domain of
Rn
,
(Px,?$)
Y @
E
is a martingale
Let
where 0
and consider the solution of
By Sobolev's inclusion theorems, we have (4.9)
u
E
C06)
.
Let
(4.10)
7 =
inf{x(t)
do}
tro the exit time of the process
Theorem 4 . 1 .
x(t)
from 8. We have
Asswne ( 4 . 1 1 , (4.2), 1 4 . 7 1 .
(4.11)
u(x) = EX
JT
f(x(t))(exp
Then we have
- Jt
ao(x(s))ds)dt
Proof 2
,
We can easily extend formula ( 2 . 3 5 ) to the function u(x)exp -
z,
Assume first p
T we have since X(T)
in (4.12).
, hence since
E
a,
is bounded
u
-
EX U(X(TAT) exp
J
TAT ao(x(s))ds
-t
0
.
Formula (4.12) shows that, by the monotone convergence theorem, (4.11) holds. Moreover
Thus formula (4.11) extends to f
E
Lp(8)
,
p >
4.
Let us now give the interpretation of parabolic equations. Let us assume aaij
(4.13)
(x,t)
a
E
C"'"'2(Rn
x
[O,T])
ij
I
aij xai
(4.14)
a.
(4.15)
. a
E
,
-E "k
=
aji
Sitj
2
R
Y 5
1512
L~(R"
x (0,~))
L-(R"
x
(0,~))
.
E
Rn , 8 > 0
Loo
128
CHAPTER I11
Let (4.16)
f
E
Lp(@
(0,T))
x
p >
$+
1
From Theorem 3.4, Chapter 11, there exists one and only one solution of
1
(4.17)
aU - at +
,
0
1 1 1 ~=
+ aou = f
A(t)u
u(x,T)
;(x)
2
where (4.19)
A(t)
-
=
C
a ~
a
axi
a
a.. - + C a 1J ax. i 2,. J i
According to Theorems 3.1 and 3.2, there exists one and only one solution Pxt of the martingale problem relatively to the operator A(t) Let (4.20)
401
S~ = inf{x(s)
S>t and note that since p >
$+
1
,
u
E
Assume 1 4 . 1 3 ) , 1 4 . 1 4 1 ,
Theorem 4 . 2 .
C
0 -
(Q)
,
Q = 8 xl0,TC.
(4.151, (4.161. Then we have
TAT (4.21)
Assume first 2
5
get
p < =.
f
E
u(x,t)
=
B(Q)
, ii
Ext[/
E
f(x(s),s)(exp
t
bo)
.
Then
u
E
- :1 ao(x(X),A)dA)ds
+
b 2 $ ’ * P ( ~ ), for any
Apply formula (2.35) extended asexplained in Theorem 4 . 1 , we
129
MARTINGALE PROBLEM
u(x,t)
=
TAT a ds Ext u(x(TAT~),TAT~) exp - it 0
+
TAT
+ Ext it
f(x(s),s)(exp
- it" aod))ds
from which we easily deduce (4.21).
U = 0 , we first extend (4.21) to functions f We observe the estimate
Taking first p >
+ 1.
Then assume
U
W z Y p n WAY'
(hence in
u
E
W2"
1 n Wo"(0)
COG))
solution of (4.17) with data
and consider a sequence
U +U
,
En, and letting
n
+ 03,
the desired result.
Remark 4.1.
We see that we have the es imate /Ext Jt
f(x(s),s)ds
which improvesthe result of Theorem 2.1
Remark 4.2. We also have the estimate
Assume ( 4 . 1 ) , ( 4 . 2 ) and consider the forward Cauchy problem (5.1)
in
we obtain
0
TAT^ (4.22)
Lp(Q) ,
Un E B O ) . Applying ;L.21) with
and f
E
130
CHAPTER I11
with
4
(5.Ibis)
Borel bounded.
Since setting v(x,t) = u(x,T-t)
-
*
+ Av
at
=
0
then v
, v(x,T)
satisfies
=
@(x)
we can assert that there exists one and only one solution of (5.1)
such
that
Since v(x,t) and noting
(5.3)
PX'O
=
= EXIt $(x(T))
Px, we can assert that
u(x,t)
=
EX $(x(t))
Y t
2
0
.
We write
which defines a family O(t) bounded functions on that
O(t)
of operators on
B , space of Borel
Rn, provided with the sup norm; it is easy to check
satisfies the following properties
131
MARTINGALE PROBLEM
(5.5) O(0)
O(t)@ Hence
I
=
t
0 if
@
z 0
.
i s a semi group of contractions on
@(t)
B , which preserves
positivity.
C
Denote by
the space of uniformly continuous bounded functions on
B.
which is a subspace of in x,t
for
(5.6)
,
t > 0
if :
O(t)
c
Then from 5 . 2 , we know
B.
@
E
+
c ,
that
u
Rn,
is continuous
Let u s check that
if we assume
In that case, we can consider the S.D.E. (5.8)
dy
=
Y(0)
+ a(y)dw
b(y)dt
=
x
(R,Q,P,st ,w(t)) . Since b,a are Lipschitz, according to Theorem 4 . 1 of Chapter I, there is one and only one solution of (5.8). on a system
Denote the solution
yx(t)
.
Then clearly we have
and is an increasing function of
6. We have
132
CHAPTER 111
Therefore
IE O(Yx(t))-EO(y,~
(t))
~
5
E P( IY,(t)-YxI
(t)
5
p(6) +
C(t)
1) /x-x'i2
h2 for any
6 > 0. From this one easily deduces that
uniformly continuous in x, f o r
fixed, t t 0
t
.
@ ( t ) @ ( x ) is
Let us also check
that
(5.10)
in
O(t)b+b
C
as
t + O , ' d ~ c c .
Indeed
from which we obtain (5.10). The infinitesirnaL generator of (5.11)
~7 @
=
lim O(t)@-$ ti0 ~
O t)
is defined by
133
MARTINGALE PROBLEM
The domain of
B , such that (5.11) in the
is the set of functions
0
E
B.
E
C;"
sense of the convergence in EX @(x(t))
=
Assume
$(x)
@
, then from Ito's formula
.
- EX it A @(x(s))ds 0
But when (4.1), (5.7) are satisfied and
I$
E
c2+a b
then
A$
E
CEya
,
hence by the above reasoning we have
Therefore
Thus we have proved that
c ~ +D(c~J ~ ,
(5.13)
Let
b
and Q $
=
-
A@
8 be a smooth bounded domain of Rn. Let
where
T
Assuming
is the exit time from 8
q?
E
x(t)
E
B(6)
we define
.
S(6) , we may consider the non homogeneous Dirichlet
pr ob 1em (5.15)
of
@
1 2+Au=o
CHAPTER 111
134
Since u
-
=
v
satisfies
vIc
=
0
V(X,O)
=
0
then we see that there is one and o n l y one solution of (5.15) such that
Moreover we have
(5.17)
u(x,t) = EX @(x(tAT))
hence by definition of
If
@
E
H
1
@(t)
0 ) , we can give a meaning to (5.15) as follows
(5.19)
1
u - $
E
Hob)
Problem (5.19) has one and only one solution such that
(5.20)
u
Uniqueness i s easy. I
E
2
1
L (0,T;H )
du
, ;ir E L
2
(0,T;H
-1
To prove existence we can consider an approximation
.
+ $ in H (s) , @n E B ( ~ ) Then we can define the solution u (5.15) with @ changed into $n. It is easy to check that we have
@n
of
135
MARTINGALE PROBLEM
'd dt (un(t),v)
a(u (t),v)
t
=
V v
0
E
1 Ho
.
Hence
I2 -
/un(t)-um(t) +
Jt
(un(t)-um(t),$n-($m)
-
a (un(s)-um(s) ,un(s)-um(s))ds
- It a
+
.
(u,(s)-~~(s),m~-~~)ds = 0
Using the fact that
($n- @m + 0 in H 1 as 1 2 is a Cauchy sequence in L (0,T;H ) and
n,m
u
2 C(0,T;L )
If
$
n satisfies (5.19), (5.20).
E
H 1 (8)n Co(g) , then Qn
+
g?
+ m
,
we get that
.
The limit
in Co(g) ,
Since
we obtain (5.21)
u
E
.
CO( y > O , h = O .
Moreover t h e r e es4sts s i z cptirnal c m t r s l . We start with a Lemma.
Let
P:
0
be the probability defined by (1.13).
Then we have
It is enough to prove (3.5) assuming Y
where
=
E X JrAT $ ( x ( s ) ) d s
4 =
E
.
B(Rn) ,
'42
EX
$(x(s))ds)
0
We have
XV(T)
15 1
STOCHASTIC CONTROL
XV(T)
=
exp CJT
0-l
- 71
gv(s)dw(s)
J
T
IU
-1
gv(s)
1
2
dsl
hence
We easily check that EX XV(T) 2
2
CT
independant of
v
.
Next $(x(s))d~)~]"~ [Ex (ITAT
using (4.22) of Chapter 11.
Hence (3.4)
2
C' T ( E x oJTAr $2(x(s))ds)1'2
2
c;\
IMLP since
p > n+2
.
, 3
l y o o f of Theorem 3 . 1 Since h 2
5
p .
n+2.
0
The function u
changed into
0
154
CHAPTER IV
Let us indicate now an other approach to study equation ( 2 . 8 ) ,
called the
method of policy iteration. 0
,...,un...
Let
u
W2"
n W;"
Knowing
, un
,
be a sequence of functions belonging to
p > n
,
defined as follows. We take
define vn(x)
uo
t o be a Bore1 function such that
+ Du"(x).g(x,v"(x)) Define next
un+l
Y x
.
as the solution of the linear equation AU~+'+
(4.15)
arbitrary.
a un+l
=
f(x,vn(x))
+ un+ 1 al(x,vn(x))
+
0
Theorem 3 . 2 .
We make t h e assumptions of Theorem 3.1 and a . 2 0, Lp , p > n , then un c u and in W2'p u e a k l y , where u i s t h e solution of 1 2 . 8 1 .
h
E
Clearly the sequence un al(x,vn(x))
,
g(x,v"(x))
is well defined. Moreover since are bounded,
(3.16)
Next Aun + a un = f(x,vn-l) + unal(x,vn-1)
+
0
+ Dun.g(x,vn-') t
f(x,vn) + unal(x,vn)
+
+
h
2
Dun.g(x,vn)
+
h
f(x,v"(x))
,
155
STOCHASTIC CONTROL
hence ~ ( u ~ + l - u+~ a ) (un+l-un) 0
u which implies u Hence
un
i
u
n -u 1,
n+ 1 +n
5
0
=
0
-
D(un+l -u n ).g(x,vn)
o
(un+l -un)al(x,vn) 5
n+l
-
(recall ( 3 . 3 ) ) .
pointwise and in W2"
weakly.
Therefore also, by compactness (3.17)
un
-t
u
in W'VP
Let us identify the limit.
We have for v
E
V
arbitrary
Aun + aOun
-
f(x,v)
+ aOun
-
f(x,vn)
~u~ + aOun
-
f(x,vn) + (un+l-un)a 1 (x,vn)
S Au"
=
strongly.
-
+ (Du n + l -Dun ).p(x,v")
unal(x,v)
-
unal(x,vn)
n+l)
n
+ (Dun+l-Dun).g(x,vn)
-+
Du".g(x,v)
-
+ (u
n+l
0 in
-
h
Uun.g(x,vn)
Dun+ 1 .g(x,vn)
-
= A(U~-U~+') + a0(u -u
-
-
-
h
=
.
Therefore Au + aou V V € V which means
-
f(x,v)
-
ua ( x , v ) 1
-
Du.g(x,v)
-
h
=
un+'al(x,vn) +
n -u )al(x,vn) +
Lp weakly
-
5
h 5 0
156
CHAPTER IV
(3.18)
AU + a u
-
H(x,u,Du)
S
h
Au + a u
-
H(x,u,Du)
-
h
0
a.e.
Also we have
0
2
Au + a u 0
=
+ m,
-
f(x,vn)
h + Aun + aOun
+
I
Au + a u 0
-
H(x,u,Du)
-
Dun.g(x,vn)
(DU"-D~) .g(x,v")
the right hand side tends to
which with (3.18)
ua (x,vn)
-
-
f(x,vn) - h =
-
h
Z
0 in L p
-
unal(x,vn)
weakly, hence
0
concludes to the desired result.
Let us give the analogue of the situation studied in section 2 and 3 .
4 . 1 . perabgljc-equetjgns
We consider here functions f(x,v,t)
: R" x ?/ x
g(x,v,t)
: R"
c(x,v,t)
: R" x ? / x C0,TI
x T
Bore1 and bounded and set
x
=
+ (un-u)al (x,vn)
4. EVOLUTION PROBLEMS
(4.1)
-
0
= ~(u-u") + a (u-u") 0
n
Du.g(x,vn)
~(u-u") + a (u-u") + (~u"-~u).g(x,v") + (un-u)a,(x,vn)
-
and as
-
[O,Tl
+
R
C0,Tl
+
Rn
+
R
0
157
STOCHASTIC CONTROL
(4.2)
H(x,t,X,p)
=
infCf(x,v,t)
-
Ac(x,v,t) + p.g(x,v,t)]
.
VEV
Next we assume aij
(4.3)
aaij __
CO,a,a/2
E
E
axk
3
Lrn
(4.4)
Let (4.5)
where 8
h
E
Q =8
Lp(Q)
x
, U
(0,T)
is a smooth bounded domain of
E
1
W2” n W0”
Rn.
Then we have
Theorem 4 . 1 .
We assume ( 4 . 1 ) , ( 4 . 3 1 , (4.4), (4.51. Then t h e r e e x i s t s
one and o n l y one s o h t i o n of (4.6)
b2y”p(Q)
u
E
-
au at
uIC
=
+
A(t)u
0
-
H(x,t,u,Du) = 0
, u(x,T)
=
u(x)
.
Similar to Theorem 3.4 of Chapter I1 and Theorem 2 . 1 . We can next give the interpretation o f the function u. (4.7)
f(x,v,t),g(x,v,t),c(x,v,t)
and measurable
We assume
are continuous in v
with respect to x,t,
V v
.
,
a.e.x.t
158
CHAPTER IV
(4.8)
Let
?J is a compact subset of
u
=
f(x,v,t)
which is Lebesgue measurable in a.e.
.
belong to I J J ~ ’ ~ ’ ~ (,Q )and define LU(X,V,t)
x,t.
OU(x,t)
Rd
x,t
-
u(x,t)c(x,v,t)
for any
It is a Caratheodory function.
v
+
and continuous in v
Hence there exists a function
which is Lebesgue measurable and such that
We can take
GU(x,t)
to be a Borel representative.
In the sequel, the
results will not depend on the choice of the Borel representative.
Let
Pxt
operator
be the solution of the martingale problem relatively to the A(t)
, with initial conditions (x,t).
Then if
x(s)
is the
canonical process as usual, we have
A control is an adapted process with values in V .
v(s)
(with respect to the family
We define the measure
:)
Pzt such that
(4.11)
and
Ptt
is the unique solution of the problem of controlled martingales
159
STOCHASTIC CONTROL
(4.12)
And
(4.13)
where
We can state the
Theorem 4.2.
We make the a s s q t i o m of Theorem 4 . 1 and
7de
Then t h e s o l u t i o n of 14.61 is given explicitely by
h = 0.
(4.15)
u(x,t) = inf
J~~(V(.))
p > n+2.
.
V(.) ~ v ~ G P e G V e rzhere ,
(4.16)
exists an cptirnal control O ( s ) = Ou(x(s),s)
.
O(s)
defined by
CHAPTER IV
160
Similar to that of Theorem 3.1.
Theorem 4.3.
We make the assumptions on Theorem 4.2.
control v(.)
, the process SAT
u ( x ( s ~ ~ ~ ) , s ~ ? ~ ) -e xitp
for
t 5 s 2 T,
is a sub martingale
(PVXt
Then for any
t c(x(X),v(X))dX
A;)
-
u(x,t)
+
.
For v = 0 , it is a martingale.
Let
5,
be
measurable and bounded.
Using equation ( 4 . 6 ) we obtain
We have for
t 5 s 5 @ 5
T ,
161
STOCHASTIC CONTROL
This proves the first part of the theorem. Taking to
v
=
0 , we have equality
0
0, hence the second part of the theorem.
Remark 4.1. verifies
1.1~
It is easy to check that if =
0
,
u(x,T)
=
0
then ( 4 . 1 5 ) holds and
u
is Bore1 bounded and
and the properties of Theorem 4 . 3 ,
;(x)
3
is optimal.
5 . S E M I GROUP FORMULATION
5 * 1.
4-eroee r tr -2f - the -E9uatjon- - u
Let us go back t o equation ( 2 . 8 ) , and we will assume here (5.1)
For
a]
v
E
=
, .a
0
= c1
> 0 a constant
2 , a parameter we consider the operator
(5.3)
We note that
Av = A
u
-
g(x,v).D
.
satisfies
(5.4)
1 Moreover let
w
A u + au s f
a.e.
in 8
.
satisfy ( 5 . 4 ) , then we have Aw + uw S f(x,v) + Dw.g(x,v) Au + au = inf[f(x,v) V
hence
fi v
+ Du.g(x,v)l
162
CHAPTER I V
A(w-u) +
U(W-U)
2
f(x,v) + DU g(x,v)
-
-
+ Du.g(x,v)l
inf[f(x,v
+
V
+ (Dw - Du).g(x,v)
< f(x,v) + Du.g(x,v) - inf[f(x
v ) + Du.g(x,v)l
+
V
+ ID(w-u) hence taking the inf in v
,
C
we obtain
(5.5)
Condition (5.5) imp1ies w - U S O .
(5.6)
Relation (5.6) is clear when a to prove the following result.
is large enough. Otherwise we have Let
h
E
0
.
Lp
given and
z
to be the
solution of the H.J.B. equation
I
then (5.8)
Indeed
h z
S
0
implies
z 2
can be obtained as the limit of the following iteration
163
STOCHASTIC CONTROL
(5.9)
zo
starting with
0 , and
=
zn Since
h
0
5
+
z
in
WzYp weakly
, one checks inductively that zn
5
0
,
hence
z
s 0
.
This
proves ( 5 . 8 ) , hence (5.6). We thus have proved the following ble make t h e assumptions o f Theorem 2.1,
Tkeorem 5 . 2 .
Then t h e soZution
u
and (5.1), ( 5 . 2 ) .
~f (2.8) i s t h e maximum element of ;he s e t o f
0
f u n c t i o n s s a t i s f y i n g (5.4). Remark 5 . 1 .
Assumption
(5.1) can be weakened, but it will be sufficient
0
for the remainder of the chapter. We note now
the solution of the martingale problem corresponding to
P:
the operator
Av
,
starting in x
at time
0
controlled martingale problem, with a control S,L.
Let
u
.
It corresponds to a
v(s)
=
s
, independant of
be a function satisfying ( 5 . 4 ) , then from Ito's formula
we have
= o hence u(x) Recalling that
ui,.
=
5
Ez[{tmfv(x(s))e-"sds
0
, we also have
+ E:u(x(tAT))e
-CltAT 1
.
164
CHAPTER IV
.
0
x (x(s~r))e-"~ds] + E:u(x(th~))e-'~l v o
which we have considered in
§
5.2 of Chapter I11 and noting that
(5.12)
+
Bo
u(x)
(5.10)
5
:E
[Jt f
Using the semi group
where
OV(t) : Bo
Bo
is the set of Borel bounded functions on
r,
we see that
(5.13)
u
on
u
6,
which vanish
satisfies the relation
s Jt OV(s)fv
w - " s d s + OV(t)u
This motivates the problem which will be studied in the next paragraph.
We make here the following assumptions.
Let
E
be a polish space
(1)
provided with the Borel a-algebra 8 . We note
B
the space of Borel bounded functions on
E.
uniformly continuous function on
E, C
the space of
We assume given a family
v c 1J, where (5.14)
'V
(5.15)
OV(t) : B
finite set
OV(t)OV(s) OV(t)Q
t
=
o
,
B
-f
Ov(0) = I
.
OV(t+s) if
Q
2
o
(1) This will be needed for the probabilistic interpretation.
OV(t)
,
165
STOCHASTIC CONTROL
We will also assume that (5.16)
aV(t)
(5.17)
t
-t
c
+
c
oV(t)G(x)
E
u
5
To find
u
maximum solution of
+ OV(t)ue-at
u e B
= fV(X)X
(4=
8
R ,
a > 0
1t 0v (s)Lve-"'ds
Lv(x)
-t
C
,
L (x) :L(x,v)
We consider the following problem.
(5.19)
(0,m)
is continuous from
x fixed, Y I$
Y (5.18)
:
f(x,v)x
(x)
0
,
166
CHAPTER IV
t > 0
For
z(x,t)
is a regular function of
hence (5.18) is
x,t
satisfied. We will study ( 5 . 1 9 ) by a discretization procedure. Uh
h > 0
Let
define
by
(5.22)
uh
=
Min!Jh v
e-US QV(s)Lvds
+
QV(h)uhl
-"here mists one and onZg one s c l u t i o r cf (2.22).
Define for
z
E
z =
MinCJh e-"' v
T z h C
.
B
E
C
since 7J'
OV(s)Lvds
+
OV(h)zl
'
is a finite set.
Note also that
T z h
Th
E
C , when
is a contraction, hence has one and only one fixed
point , uh. L e m o 5.2. L e t
0 z c B
such t h a t
z S
T z h
proof Th is increasing we have
2
T z S T z h h hence
t
Moreover
which proves that
Since
uh
B
T and
,
o
S e m a 5.1.
z E
,
z 5
T2z h
and by induction z S
Tnz h
+
u
h a s n - t m '
then
z
< uh '
167
STOCHASTIC CONTROL
Lemma 5 . 3 .
Ue have
(5.23)
Uh 5 UZh
.
procs We have f o r any
v
Uh 5
{
u
LZh e-"'
OV(s)Lvds
+ evah OV(h)uh
hence
which implies
- K h
which i m p l i e s ( 5 . 2 4 ) .
Let u s s e t
q f +
then a s
u
m
q
.
C u
u
Note t h a t
is
U.S.C.
Furthermore we s e e
that uh s
Take
I h = 24
,
imh e-"
OV(s)Lvds + e - a h 0v (rnh)uh
m = j 2q-R
and
R
with
5
R (5.25)
u
q
5
R 5 q
5
hence
q
,
j
integer.
/j/'
e-"
o
,
j
Let
,
q
QV(s)Lvds
+
ti rn
we g e t
9. e-a"2
~ ~ ( j / 2 " . u, ~
integer
.
q
According t o Lemma 5 . 4 below,
-f
a.
integer
.
169
STOCHASTIC CONTROL
(5.26)
ti q.
j
Take next
= Ct
2
9"
I+
1
and let
R
tend to
+a
, we deduce from
( 5 . 2 6 ) using assumption ( 5 . 1 7 )
in which we may again let (5.19).
q
tend to
+m.
This proves that
,.
It is the maximum solution, since assuming u
u
satisfies
to satisfy
( 5 . 1 9 ) then clearly
which implies
-u
5
uh , hence the desired result.
G
Let us now state the result which has been used in the proof of Theorem 5 . 2 .
We refer to Dynkin 121. Let us briefly mention the main elements of the proof.
Let
Banach space.
be the space of bounded measures on We write
Then we have the following result
( E , 6 ) , which is a
170
CHAPTER IV
m
Define next an operator on
One checks that
by the formula
is a contraction semi group o n n , and that the
U(t)
following relation holds,
0
From (5.29) and (5.27) the desired result follows. Remark 5.2.
In example (5.20), we have
uh
Co =
E
T, and of course u
functions which vanish on
E
subspace of
C
of
.
Bo
L
Let u s define
R0
=
E
I
,
x(t;w)
is the canonical process,
Let u s assume for simplicity that
7 To
i
E
b'
=
...,m} .
{I,?,
, we associate a probability PTt
(5.30)
E2t @(x(s))
We will denote by values in V .
W
=
Oi(s-t)6(x)
Ro
, nt
fi s 2 t
such that
.
the class of step processes adapted to
V
E
... 5
7
More precisely, if To =
on
0
s
T1 5
not
with
W , then there exists a sequence 5
...
which is deterministic increasing and convergent to
+x
and
171
STOCHASTIC CONTROL
v
(5.31)
=
, v(t;o)
v(.)
=
vn(w)
t
€
IT,,?
n+ 1 )
T
where
is ?!on
v
measurable.
9"
We next define a family the pair
w,t
(w
E
07; , Oss A
,
.
u c cO"(E)
z c c'"((E).
Let u s fix xo
in
E , then there exists vO
(depending on xo) such that ThZ(XO) * j h e-as CJv~ ( s ) L
(xo)ds +
Ovo(h)a(xO)
vO Let x
arbitrary, we have Thz(x)
5
Jh
(x)ds +
Pvo(s)Lv
0 hence by difference
and from the assumptions (5.56), (5.57) it follows that
Ovo(h)z(x)
179
STOCHASTIC CONTROL
5
Lh e-asei's
Klx-xo
1 'ds
+
hence
and since
xo,x
are arbitrary, this implies
and iterating we see that
and letting
k
tend to
+a
,
it follows that
K
I I ~ h I 5l ~
Taking now h
=
1 and letting -
q
-f
m,
we deduce
24
(5.59)
IlU
Il&q
K
which implies the desired result. Let u s now give an other regularity result. We assume (5.60)
I
I Z I
86 16 Ix-xgl
180
CHAPTER IV The rnuximwn so2ution of 15.191 is a l s o the maximwn s o l u t i o n of
L e m a 5.6. (5.61)
U E B ,
u
5
Jt e-”
tr v
+ e-Bt OV(t)u
Qv(s)(Lv+(G-a)u)ds
Y t
E l J ,
2 0
Proof We first show that (5.61) has a maximum element, which will be denoted by N
u.
Indeed, define for Bhz
=
z
E
B + e-Bh O v ( h ) z l
Min[Jh e-Bs aV(s)(Lv+(B-a)z)ds v o
It is well defined, by virtue if ( 5 . 6 0 ) . This is a contraction, since
’
I / o ~- ~~ ~~
z 5~ / jI z 1j-2 2 ’
= 1l2,-22/
Moreover when
z
,
C
E
Ohz
when
uh
.
C
-
..r
Let
E
be the fixed point, uh
z 2 0
6
.
C
and
Y
One checks as for Theorem 5.2 that
- -
Setting u
q
=
u
1/29
, we get
Y
u
.
J- u
, since
Ohz 2 0
N
uh U
q
-
uh 2 0
5
uZh
.
Then
9, S 4
/j’*
e-Bs OV(s)(Lv+(C-a)u
0
4
)ds
t
L
and as for Theorem 5 . 2 , one checks that
,u.
is a solution of ( 5 . 6 1 ) , and
181
STOCHASTIC CONTROL
that it is the maximum solution, since any other solution satisfies v
5
0hv
,
-
v < uh '
hence
w
Let us show that u = u , where
u
is the maximum element of (5.19).
We will use Lemma 5.7 below to assert that & < I t e-"'
-
hence u
5
u.
w
i e m a 5.7.
,
Let
where (5.63)
E
B.
0
and the desired result is proved. O(t)
be a semi group on Let
w
w < Jt g
+ e-Bt OV(t)u
Jt e-Bs OV(s)(Lv+(B-a)u)ds
( 5 . 1 5 ) , and is.&Ol.
(5.62)
OV(t)u"
However, still using Lemma 5.7 we have u
hence u < u
+
QV(s)Lvds
@(s)g
5
ds +
@(t)w
B > 0 , one has
Then f o r any w
satisfying properties
be such t h a t
B
E
B
it e-as @(s)(g+(B-a)w)ds
+ edBt O(t)w
We set
we have H(0)
=
0
,
H(t)
5
0
In fact, we have the additional property
Y t
,Y
t t 0
182
(5.64)
CHAPTER IV
H(t) s H(s)
for
.
t 2 s
Indeed ( 5 . 6 4 ) amounts t o proving t h a t
(5.65)
e
-us
O(s)w
O ( t ) w + Jst e-"O(A)g
5
,
di,
.
s 5 t
But f r o m ( 5 . 6 2 )
w s /
t-S
e
-a>
dA + e
O(A)g
-a ( t - s )
O(t-s)w
0
0 and
and i n t e g r a t i n g between
we deduce
T
[ l-e-(B-a)Tjw =
JT (B-a)ewbt O ( t ) w d t + 0
+ /T ( 3 - i ) e
+
=
H(t)dt
+
iT ( 3 - a ) e- ( 3 - a ) t ( i t e-" O ( s ) g d s ) d t iT (6-a)e - 6 t O ( t ) w d t + iT ( 0 - a ) e - ( p a ) t =
- e
-(B-a)T /T e - 2t 3 ( t ) g d t +
iT e-r't
H(t)dt
O(t)g d t
hence (5.66)
w =
/T .-3t
O(t)(g+(?-?)w)dt
+ e
-6T
O(T)w
+
0
- (6-3)T H ( T ) + J .T ( 3 - 5 ) e - (5-3) t H ( t ) d t
+ e
a
.
-
183
STOCHASTIC CONTROL
If B 2 u since H(t) < 0 , we clearly have ( 5 . 6 3 ) with 6 < a then using ( 5 . 6 4 ) we have
If
t = T.
hence H(T) +
e-(B-a)T
iT(@-a)e-(B-a)t
H(t)dt
H(T) + (@-a)H(T) JT e-(B-a)t
< e
therefore ( 5 . 6 3 ) holds in all cases for
t = T
.
5
dt
Since T
H(T)
=
5
is
C
arbitrary the desired result is proved.
Theorem 5 . 6 .
Let
z
We make t h e assumptions of Theorem 5.2,
15.561,
Then the m a x i m s o l u t i o n o f (5.19) belongs t o
(5.60).
E
C
5
and
5
1 5 . 5 7 1 and
C.
be the maximum element of
5
it e-Bs Ov(s)(Lv
+ (8-a)z)ds + e-Bt OV(t)
184
CHAPTER IV
Ch
where
=
Sh(z)
is defined by
ch
(5.67)
Min[Jh e-@'
=
v
OV(s)(Lv+(@-a)z)ds
+ e-ah QV(h)ch]
o
< h e C * Sh : B
Note that
B
-t
and
C
C
+
.
One easily checks the estimate
I Ish(zI)-sh(z2) 1 1
(5.68)
5
71
1z1-z21 j
from which one deduces
I IS(zl)-S(z2) 1 I
(5.69)
5
a-a 112 - 2 1 I , p 1 2'
when
We also note the relation, which follows from Lemma 5.7, (5.70)
u
5
sh (u)
.
Define now
,
un = S"(0)
Since
s
maps
coy6
u;
=
.
s;(o)
into itself, u n c
c0j6.
From (5.69) we have n+l IIU
n
' I1
B-a n 1 (T)1Iu 1
and thus un+w We will show that
(5.71)
u = w .
in
C .
1
z 1 ,z2
E
C
185
STOCHASTIC CONTROL
which w i l l prove t h e d e s i r e d r e s u l t .
We f i r s t remark t h a t from ( 5 . 6 8 ) belongs t o
C
,
has a fixed point i n
and
wh
s ~ ( o )+ wh
(5.72)
From ( 5 . 6 9 ) ,
denoted by
Sh
c
in
.
( 5 . 6 8 ) we have
(5.73)
(5.74)
From ( 5 . 7 0 ) we c a n a s s e r t t h a t (5.75)
u s w
h
We check by i n d u c t i o n on
n
that
un h
-
By i n d u c t i o n on
n
we check t h a t
(5.76)
un+un q
un 2h
hence
From ( 5 . 7 3 ) ,
as
q f m ,
=
~
( 5 . 7 4 ) , (5.76) follows t h a t
wq(x) 4- w(x) which w i t h ( 5 . 7 5 ) shows t h a t
(5.77)
Gn
u < w .
v
x
un.
Hence
n
.
B
,
which
186
CHAPTER IV
But a l s o
hence
+ e-pph OV(ph)wh
w < Jph edBS OV(s)(LV+(B-a)wh)ds h - 0 hence also
for
q 2 R.
Using a reasoning as in Theorem 5 . 2 , we obtain easily that w
5
+ e-Rt @"(t)w
J t e-Bs OV(s)(Lv+(p-a)w)ds
hence also, using Lemma 5.7 w
it
5
@"(s)Lvds
+ e-"
QV(t)w
0
which implies w
5 u
,
and from (5.77) we see that ( 5 . 7 1 ) holds.
ci
completes the proof. Let us give an example where ( 5 . 5 7 ) is satisfied, with Consider the S.D.E, dy with (5.78)
This
=
g(y)dt
+
o(y)dw
y(0)
=
x
6
=
0
.
187
STOCHASTIC CONTROL
hence
which proves (5.77).
Remark 5 . 4 .
For other details (cf. M. Nisi0 [l], Bensoussan-Robin C11,
0
Bensoussan-Lions 121). COMMENTS ON CHAPTER I V
I . The method of improvement of regularity used in Theorem 2.1 is due to
P.L. Lions. 2 . Assumption (3.1) can be replaced by Lebesgue measurable in x
continuous in v
as mentionned in the evolution case
§
4.1.
, and In fact
we need a selection theorem. There are two types of such theorems that we may use. Consider Assume
F(x,v)
,
x
E
Rn
,
v
E
3'(compact subset of a metric space).
188
CHAPTER IV
F
1.s.c.
in x,v, F
bounded below.
Then there exists a Borel function C(x) F(x,C(x))
= inf F(x,v)
+V,
: Rn
such that
Y x
V
(see for instance D. Berksekas, S . E . Shreve [ I ] ) . The other theorem uses more explicitely the Lebesgue measure on We assume that F F
Rn.
is a Caratheodory function, i.e.
, continuous in v, a.e. x.
V v
is Lebesgue measurable in x
Then there exists a Lebesgue measurable function C(x)
: Rn
+ V , such
that F(x,?(x))
=
inf F(x,v) a.e. V
We can take a Borel representation of We write
inf
G(x)
,
but it is not unique.
is a Lebesgue measurable
V
function such that if
?(x)
, which
for ess inf F(x,v)
5
then G(x)
F(x,v) 5
a.e.
Y v
ess infF(x,v)
a.e,
V
Note that
inf F(x,v)
when
F(x,v)
is Borel for any v
is not a
V
Borel function (cf. I. Ekeland - R. Temam [ I ] ) . 3 . The method of policy iteration was introduced by R. Bellman [ l ] ,
in the
general context of Dynamic Programming.
4 . For the study of degenerate Dynamic Programming equations (i.e., the matrix
-1
a
does not necessarily exist) we refer to P.L. Lions
-
J.L. Menaldi C11. 5. J.P. Quadrat has formulated a generalized martingale control problem,
which includes degeneracy (cf. J.P. Quadrat [ I ] ,
[Z]).
189
STOCHASTIC CONTROL
6. For numerical techniques t o solve the H.J.B. equation see J.P. Quadrat [ l l , P.L. Lions
- B.
Mercier [I].
7 . A s we have said in the general introduction, the most complete
treatment of the general Bellman equation is due t o P . L . Lions [ll,
[ZI. 8. The problem of semi group enveloppe was introduced by M. Nisi0 C21
9. Nisi0 has also introduced a problem of non linear semi group connected to stochastic control (cf. M. Nisi0
[ll).
10. In the context of Remark 5 . 3 . Under what conditions can we assert
that the solution u
of (5.55) coincides with that of ( 2 . 8 ) .
This Page Intentionally Left Blank
191
CHAPTER
FILTERING AND PREDICTION
V
FOR
LINEAR S.D.E.
INTRODUCTION We present here the classical theory of linear filtering, due to R.E. Kalman [ l ] , R.E. Kalman - R.S. Bucy [ l l . Xe have chosen a presentation which can be easily carried over to infinite dimensional systems, for which we refer to A. Bensoussan [ I ] , R. Curtain - P.L. Falb [ l ] , R. Curtain - A . J . Pritchard [ l ] . For filtering of jump processes For non linear filtering, cf. R. Bucy - P. Joseph (cf. P. Bremaud [ l ] ) . [ I ] , and the recent developments in E. Pardoux [ I ] , T. Allinger S.K. Mitter [ I ] .
1. SETTING OF THE PROBLEM
We consider a usual system
(S2,a,P,5t,w(t)), and
solution of the linear S.D.E.
where
(1.2)
F
E
L~(O,~$(R";R"))
G
E
L~(O,~;~(R";R"))
f(.)
E
L ~ ( o , ~ ; R ,~ )
Clearly the standard theory applies since
x(t)
to be the
192
CHAPTER V
g(x,t)
=
F(x)x
o(x,t)
=
G(t)
+ f(t)
.
5 is gaussian with mean x and covariance matrix
a
To the O.D.E.
corresponds a fundamental matrix
such that the solution of (1.4)
@(t,T)
can be expressed as x(t)
(1.5)
where
g
E
=
2
L (0,m;R").
@(t,O)x
+
Jt
The family
(1.6)
@(t,S)@(S,T)
(1.7)
@(t,t)
=
I
=
@(t,r)g(r)dT has the group property
@(t.T)
@(t,T)
d '
t,S,T
.
It is easy to check that the solution of (1.1) y(t) = @(t,O)c
(1.8)
+
Jt
can be expressed by
@(t,T)f(?)d?
+
It @(t,?)G(~)dw(r)
where the last integral is a stochastic integral. Formula (1.8) is a representation formula for the process
y(t).
It is also useful to
notice the following. Let (1.9)
h
E
Rn
and
-3 dt
=
F*(t)$
,
$(T)
=
h
FILTERING AND PREDICTION
193
then we have (1.10)
@(0).5 +
y(T).h
=
p(t)
O*(T,t)h
@(t).f(t)dt
+
iT @(t).G(t)dw(t)
.
Since
(1.11)
=
it is easy to deduce (1.8) from (1.10) and ( 1 . 1 1 )
It is clear from (1.8) or (1.10) that expectation y ( T ) (1.12)
i s a Gaussian variable with
y(T)
such that
Q(T,O)x + {T @(T,t)f(t)dt
y(T)
=
-dy- -
F(t)y
;(t)
=
i.e. (1.13)
dt
,
y(0) = x
.
Let
Y(t)
-
y(t)
then from (1.10) (1.14)
-Y(T).h
=
O(O).t
where (1.15)
Define next (1.16)
then
- b-'' dt
=
F* (t)+
+
iT @(t).G(t)dw(t)
CHAPTER V
194
hence from ( 1 . 1 4 ) , (1.18)
(1.17)
we deduce
E F(T).h
= Po @ ( O ) . $ ( O )
y(T).k
+
JT
G*(t)@(t).G*(t)$(t)dt
= II(T)h.k
where
II(T)
denotes the covariance operator of
y(T) (or
y(T)).
Hence we have the f o r m u l a (1.19)
n(T)h.k
(1.20)
n(T)
=
Po @ ( o ) . $ ( O )
=
+
O*(T,O) +
O(T,O)PO
J
T
G(t)G*(t)q(t).+(t)dt
iT O(T,t)G(t)G*(t)O*(T,t)dt
We will set for simplicity (1.21)
G(t)G*(t)
=
Q(t)
We can deduce from (1.20) that
.
II is solution of a differential
equation. We have (1.22)
TI(T)h.k
=
Po @*(T,O)h.O*(T,O)k
t
+ JT Q(t) @*(T,t)h.O*(T,t)k 0
The function s
1
O(s,t)h
E
H (t,T;Rn) , and
O*(s,t)h
E
H 1 (t,T;Rn)
+
hence
and
dt
.
.
195
FILTERING AND PREDICTION
(1.24)
d O*(s,t)h ds
=
O*(s,t)F*(s)h
We can approximate (1.22) with respect to
. T , using ( 1 . 2 4 ) .
We
obtain
dT
h.k
= Po
+
O*(T,O)F"(T)h.@*(T,O)k
+ Po O*(T,O)h.O*(T,O)F*(T)k
+
Q(T)h.k
+
+ JT (Q(t)@*(T,t)F*(T)h.@*(T,t)k
.
+Q( t ) @*(T,t ) h O*(T, t) F*( T) k)d t
and from (1.20) we get
We thus have proved
L e m a 1.1.
The process
y
soZution of (1.1) i s
whose mathematical expectation y(t)
covariance matrix n(t) (1.25)
We next define a process
where (1.27)
G
Gaussian process
is solution of 11.131 and whose i s so2zrtion of the equation
z(t)
by setting
+
196
CHAPTER V
(1.28)
is a
n(t)
Rn
and
(1.29)
R
Y 0 is
e.n(t)
Zt continuous martingale with values in E
Rp , the increasing process of
it R(s)B.Bds
,
where
R
is symmetric invertible and R-1
E
Lm(O,m$(Tn;RP)) bounded.
From the representation theorem of continuous martingales, we have q(t) =
Lt R”2(s)db(s)
,
where
b
is a standard 3
Wiener process. We also assume q(t)
(1.30)
It i s clear that
Z(t) (1.31)
is independant from
z(t)
5
and
w(.)
.
is a Gaussian process, whose expectation
is given by
i ( t ) = {t H(s)y(s)ds
.
Set
-z(t)
=
z(t)
- Z(t)
then (1.32)
-
(1.33)
E Y*(s,)?(s,)
z(t) =
it H(s)y(s)ds =
+ q(t)
.
a(s1,s2)~(s2)
if
s 1 2 s2
t
197
F I L T E R I N G AND P R E D I C T I O N
= @(sl,s2)~(s2) +
Jssl @(sl,s)G(s)dw(s) 2
hence
Let h , k
E
R”.
We have from (1.34)
E y(sl).h y(s2).k
=
E Y(s2).0*(s1,s2)h
=
iI(s2)O*(s1,s2)h.k
y(s2).k
therefore
from which we deduce (1.33).
It is easy to deduce from (1.33) and (1.20)
that
From (1.34) and (1.32) it is easy, although tedious to deduce the covariance matrix of
z(t)
We consider that the process be observed.
and the correlation function.
y(t)
cannot be observed, whereas
z(t)
can
The filtering problem consists in estimating the value of
y(t), knowing the past observations. More precisely, we are interested in
We note the following
198
Lema 1.2.
CHAPTER V
Ye have
(1.36)
a(z(s),O<s
- H*(t)
(q(t) + R-’(t)zd{t))
X(T) = h
with the payoff
-
2 x.X(O)
This problem i s the following
.
209
FILTERING AND PREDICTION
The optimal control 9 (3.5)
G(t)
is given by
=
-
.
R-'(t)H(t)y,(t)
Define next
Y
hence the pair
-
yT,pT
is solution o f
(3.7)
-
- H*(t)R-'(t)(zd
-YT(0)
=
- Po PT(0)
Now from (3.7) and (2.25) we deduce
and from (2.26) we see that
,
H(t)y)
-pT(T)
= 0
.
2 10
CHAPTER V
Y,(T).h
(3.8)
=
(zd
-
H(.)Y(.),?$)
-
H(.)?(.))
therefore
-yT(T)
(3.9)
=
CT(zd
which with ( 2 . 2 7 ) completes the proof of ( 3 . 2 ) .
0
We write (3.10)
r(T) = r(T;zd) = iTzd + y(T)
-
iT H(.)?(.)
then from ( 3 . 2 ) we see that (3.11)
yT(T)
=
-
r(T)
P(T)h
.
Our next task will be to obtain evolution formulas for the pair r(.) P(.).
3.2.
&arsjye-formulas
We will describe a decoupling argument on system ( 3 . 1 ) .
L e m a 3.1.
Functions
(3.12)
yT(t)
,
pT t )
are r e Z a t e d by t h e r e l a t i c n
yT(t) = r(t) - p t)pT(t)
v
t
E
C0,TI.
proos be convenient to denote by y t) ~ ~ , ~ ( the t ) solution of T,h to emphasize the dependance in h Consider next system ( 3 . 1 ) (3.11, on the interval (0,t) instead of (0,T) with final condition pT(t) It wil
instead of (s)
h.
More precisely consider the functions
with
Yt,PT(t)
(s)
,
s 2 t.
Since Y ~ , ~ ( s ), P ~ , ~ ( s )satisfy this system and since the solution is unique, we can assert that
21 1
FILTERING AND PREDICTION
(3.13)
Pt 'PT (t) But by definition of
(s)
=
pT(s)
for
s 5 t
r,P we have
which with (3.13) proves ( 3 . 1 2 ) .
;emu 3 . 2 .
he fitnetion
P(t) c H'(O,T~(R~;R"))
alzd is the unique solzrtion o f t h e eqzration. o f R i c c a t i type
(3.4)
1
P(0)
P 0
=
?roo _" By Lemma 3.1 applied to system ( 2 . 2 5 ) we may assert that
(3.5)
aT(t)
=
.
-
P(t)BT(t)
=
1 DaT.aTdt + PoBT(0).QT(O)
Now from ( 2 . 2 4 ) we have P(T)h.h
T
+
CT QBT.BTdt
where we have set (3.6)
D(t)
=
H*(t)R-'(t)H(t)
From (3.5) we obtain (dropping index T (3.7)
P(T)h.h
=
. on aT,BT)
T (PDP+Q)B.Bdt + PoB(0).B(O)
j 0
.
212
CHAPTER V
Next from the second equation (2.25) we get
- da = dt
(3.8)
(F*-Dp)a
B(T) = h
.
Define ( 3 9)
OP(t,S
9
F(t)
=
fundamental matrix associated to
-
Then the solution of ( 3 . 8) can be expressed as
which, going back to ( 3 . 7 ) yields
hence (3.10)
By analogy with (1.20), (1.25), formula ( 3 . 1 0 ) proves that differentiable in fact that i f c1
P
t
and that ( 3 . 4 ) holds.
is
Uniqueness follows from the
is a solution of ( 3 . 4 ) then defining
by ( 3 . 5 ) then the pair
in a unique way.
P(t)
6 by (3.8) and
a,@ is solution of (2.25) and thus defined
Applying (3.5) at time
T
shows that
P (T)h = P2(T)h
if
Pl,P2 are two solutions. Since h
uniqueness follows
is arbitrary, as well as
T, the
3
213
FILTERING AND PREDICTION
The f u n c t i o n
L e m a 3. 3.
r
E
1 H (0,T;R") and i s t h e unique s o l u t i o n o f
+ f(t) + P(t)H*(t)R-'(t)zd
dr = (F(t)-P(t)D(t))r(t)
(3.11)
r(0)
=
x
.
proos By (3.12) it is clear that
r
E
1 H (0,T;R").
Also
dt =
F~
-
Q~ + f + (FP+PF*
-
PDP+Q)~+
+ P(-F*~-D~+H * R-1 zd)
=
(F-PD)~+ (F-PD)P
P
*R-1
+ f + PH
zd
i.e. (3.11). Moreover r(0)
=
y(0) + P(O)p(O)
= x
which completes the proof of the desired result.
3.3.
Falr!ian-fjlter
Let us consider the observation process (1.26). z(t) where
b
is a
= Jt
H(s)y(s)ds
+
We have
it R''2(s)db(s)
Zt standard Wiener process with values in Rp. We
consider the S.D.E.
214
CHAPTER V
(3.12)
d9
((F(t)-P(t)D(t))?(t)
=
t
P(t)D(t)y(t)
+ f(t))dt
t
+ P( t)H*( t)R-’/’ (t)db (t) 9(0) = x
.
(3.13)
+
IT Op(T,t)P(t)H*(t)R-’/’(t)db(t)
.
Since dz = H y dt
R 1 ” db
t
we can rewrite (3.13) as (3.14)
?(T)
=
+
Op(T,O)x
+
JT
{T
O,(T,t)P(t)H*(t)R-’(t)dz(t)
Op(T,t)f(t)dt
t
.
Now from (3.11) we also can assert (3.15)
r(T)
=
Op(T,O)x +
T Oh(T,t) (f(t)+P(t)H*(t)R-’
(t)zd(t))dt
and (3.16) Take x
r(T).h =
0
,
f
(3.17)
=
0
= (i;h,zd) + y(T).h
then
7
=
0
-
(i;h,H(.)F(.))
, hence
^*
r(T1.h
=
(ZTh,zd)
r (T) =
iT O p ( T ,
and from (3.15) t) P (t) H* (t) R-’zd (t)dt
.
FILTERING AND PREDICTION
which compared to (3.17) yields (3.18)
23
=
.
R-l(t)H(t)P(t)O~(T,t)h
It follows from formula (3.19) that
Let us show that (3.21) Indeed let @(T)
Op(T,O)x
Op(T,t)f(t)
+
=
y(T)
- iTOp(T,t)P(t)D(t)y(t)dt
to be the right hand side of (3.21).
Differentiating in T
, we get
hence -do- - (F - PD)e + f dt O(0) = x
therefore O(T) which proves (3.21).
=
Qp(T,O)x
+
JT Op(T,t)f(t)
Therefore from (3.21) and (3.20) we see that
Qp(T,O)x + which with (3.14) shows that
T 0
Qp(T,t)f(t)dt
=
y(T) - IT H(.)Y(.)
215
216
CHAPER
v
This formula compared to (2.1) shows that of
Y(T)
is the best estimate
y(T).
W e thus have proved the
Theorem 3 . 2 . Ve make the asswnptims of Thecrem 2 # i . Then zi7.e conditional ezpectation 9(T) of y(T) given z ( s ) , O<s/e a s s m e ( 2 . 2 1 , ( 2 . 3 ) , (2.4), (2.5), ( 2 . 8 / , / 2 . 9 / ,
(2.211, (2.22).
(2.27)
inf
Then 7;e have
J(v(.))
V(.)&
A s the proof of Theorem 1 . 1 ,
=
.
inf*J(v(.)) vn,
using Lemmas ( 2 . 1 ) ,
We start with differentiability Lemmas as in denote by
y
tion) still by
§
(2.29)
Let
u
E
;3
2 S;(O,T)
and
the corresponding state defined by (2.1), and (to save nota-
z
the corresponding observation, defined by (2.7).
1%
=
F y 1 + B(t;u(t))
dz
(2.30)
( 2 . 2 ) , (2.3).
1.3.
We write
where
(2.161,
_ _1
dt -
1'
Z ] ( O )
= 0
.
24 1
VARIATIONAL METHODS
Let
v
E
control
2 L (O,T), we will denote by
3 u+5v
,
x,(t)
x I a defined by (2.12) with
the state corresponding u
replaced by
co
the
u+&.
We define
-
-dy- - F(t)y dt
(2.31)
Y(0) = 0
L e m a 2.4.
The :unctionaZ
+ Bv(t;u(t))v(t)
. J(v(.))
is Gateauz di;lr"ereqtisble on
and r e have t h e formula
?TOO t^
I
We have (2.33)
Now
From assumption ( 2 . 2 2 ) , it i s easy to check, like in Lemma 1.5 that
0 , Y'
1
C'(C0,TI).
of ( 3 . 1 3 1 s a t i s f i e s
8 r e g u l a r bcunded donain of Rn.
Set
z = u$
where
@ E B(Rn).
Then
z
satisfies the equation
(3.54)
From Remark 2 . 2 ,
( 3 . 5 0 ) and ( 3 . 5 1 ) , ( 3 . 5 2 ) it follows that the right hand
side of ( 3 . 5 4 ) is hb'lder in x,t
with adequate constants, and
h@
Rn.
E
C2+'
on compact subsets of
an initial condition which is
We can consider
z-h$
0. We can then assert that
z
to obtain is
twice continuously differentiable in x with second derivatives hb'lder, on Q$ , and once continuously differentiable in t, with hb'lder -
derivative on
.
Q@
This f o l l o w s from the result of Ladyzenskaya et alii, mentionned in Chapter 11, Since
q?
§
3.3.3.
is arbitrary we have in particular the property ( 3 . 5 3 ) .
0
269
VARIATIONAL METHODS
With the assumptions of Theorem 3.2, the function L"(x,v,p,t) continuously differentiable in x,v,p
and holder in
There exists a Borel function
4
Here we use 0 -
5 E C (8)I hence Thus by (2.17) we have defined a map
explicitely the assumption that Lm@).
to imply that
5 from Lm into itself. Let us show that it is a contraction. x Indeed take z1,z2 E Lm and c l = Sh(zl) , q 2 = S A ( z 2 ) . Then from S
(z) =
Lemma 2 . 3 we deduce that
which proves that
Sx
is a contraction. Since solution of ( 2 . 6 )
coincide with the fixed points of
Remark 2.2.
I3
S A , the proof is completed.
We say that problem ( 2 . 6 ) is a unilateral problem, since it
is defined by one sided conditions. The terminology comes from Mechanics where such problems are met frequently. Our motivation is completely different.
I7
3. VARIATIONAL INEQUALITIES
Assumptions ( 2 . 5 ) are two restrictive f o r the applicationsthat we have in mind.
We will relax them, but it will then not be possible to formulate
the problem like in ( 2 . 6 ) , since clearly regularity of the solution u requested in such a formulation.
is
288
LHAPTER V I I
We make the assumptions of Theorem 2.1, and the coercivity
Theorem 3.1. assumption
Then there exists one a d only one solution of the variational inequality
pi-ovided t h a t t h e set 1 K = { V E H Iv 0
some sufficiently
large constant. However since we do not want to rely on such an assumption, we will prove a result similar to Lemma 2 . 1 .
Lennna 3.1.
Without loss of generality, we may asswne
(3.6)
a o t y > O .
Proof Make the change of unknown function u = w z
.
Then we get the same problem with the following changes a
ij
a. 1
f
We recall that
1
+a
+
+
2
ijw
a.w2 1
wf
2 w S 2 ,
The result follows.
29 1
PROBLEMS OF OPTIMAL STOPPING
Let us now consider the problem a(u,v-u) + A(u,v-u)
(3.6)
B v X
For
1
,
Ho
v
5
$ ,
U E
I Ho
,u
5 $.
large enough, the assumptions of Theorem 3.1 are satisfied Therefore there exists one and o n l y one solution of
(namely (3.1)). (3.6).
E
(f,v-u)
2
We have the analogue of Lemma 2 . 3 .
Lema 3 . 2 .
Let
u
be t h e s o l u t i o n of (.3.6), and
corresponding problem w i t h
f changed i n t o
f + @
be t h e s o l u t i o n o f t h e
,
@ E
Lm.
Then we have
Since the penalized problems are the same as in section 2, we still have the estimate
Y
Hence E +
uE-UE
remains bounded in Lm.
+ u-u
0. Hence u"-';'
in Lm
However uE-EE
-+
u-u
in L2
0
( 3 . 7 ) holds.
Lema 3.3.
kssume
f,$ bounded.
I If1
L
+
L
-uLet $, 5
Then t h e s o l u t i o n of 13.61 s a t i s f i e s
/ / ~ / /m < A + yI
(3.9)
-u
lwLa *
be the solution of (3.6) corresponding to hence
as
weak star, from which it follows that
5
1 IqlI
a
L
.
f
=
0. Note that
292
Let
CHAPTER V I I
K
=
I~I) 1
We s h a l l prove that
m.
L (3.10)
-
u + K t O . Y
-
-
v = u+(u+K) as a test function in ( 3 . 6 ) , which is possible by the choice of K. We obtain
Take
or
K(ao+X,(u+K)-)
L
hence
(L+K)-
=
0
t
0
, which proves (3.10).
Using next Lemma 3 . 2 , we obtain ( 3 . 9 ) .
0
We can then obtain the
Theorem 3 . 2 .
and
f E LP@) the V . I . (3.11)
We assume 11.1), i1.2), 1 1 . 1 0 ) ,
,p
>
f.
v
provided t h a t t h e s e t
@
E
H K
1
~ v ,2 Q , u
0
,v)
E
H; n L~
u 5
q
defined i n 13.31 i s n o t empty.
Consider first the equation
a(u
E
Lm
Then t h e r e e x i s t s one and o n l y one s o l u t i o n of
* or
Let
a(u,v-u) t (f,v-u)
v
(3.12)
12.21.
=
(f,v)
,
293
PROBLEMS OF OPTIMAL STOPPING
AU
We know that
uo
0
+ aOuo
,
W2"@)
E
-
u = u-u
u
when
=
,
f
u
0
lT
hence since
.
o
=
p >
5 , uo
E
Now set
COG).
0
is a solution of (3.11).
;;
Then
is a solution of the
following problem
1 0
Y V E H
u
HA
E
n L~
,
v 2 $-u
,
-u
5
0
0 i-u
which is the same problem for data
f
=
0, and
$ = I$-u
O
E
L
m
.
Moreover
we set
KO
=
{v
since it contains vo-uo
1 0
H 1v
E
5
where v
0
I
$-u 0
5
9
is not empty
,
v
0
E
1
Ho.
Therefore without loss of generality, we may assume z
E
Lm, define
5
=
f = 0. Let next
as the solution of
Sh(z)
(3.13)
For
A
large enough, we may apply Theorem 3.1, to ensure the existence
and uniqueness of
5.
Moreover, from Lemma 3.3, we see that
Hence we have defined a map contraction. Indeed if
S,
z1,z2
from Lm
La
and
5
E
Lm.
into itself. This map is a c1,c2 are the corresponding
sohtions of (3.13), it follows from Lemma 3.2 that
294
But clearly the fixed points of (3.11),
when
Remark 3 . 1 .
S A coincide with the solutions of
0
f=O. Hence the existence and uniqueness.
When we make the assumptions of Theorem 2.2, we have one and
only one solution of (2.6) and also-of ( 3 . 1 1 ) .
These solutions
Indeed let us check that the solution u
coincides.
of ( 2 . 6 ) is a
solution of ( 3 . 1 1 ) . Indeed let v
E
1
Ho
,
v
5
$, we have
J0 (Au+aou-f) (v-u)dx and by Green's formula we see that u
=
J
0
(Auta u-f) (v-$)dx
0
satisfies ( 3 . 1 1 ) .
2
0
This justifies
the introduction of V.I. as a weaker formulation to ( 2 . 6 ) , when
$
Lema 3 . 4 .
Let
$,T
L m J and
E
corresponding t o them.
Let u s consider
' U
(3.15)
Au'ta
with (3.16)
X
is
0
not regular.
large and
Then one has
and
Q
u
u,u t o be the s o l u t i o n o f 13.121
0
E
LE
uE
Lm.
to be solutions of
+ AuE +
1 (u'-$)~
Then we have
=
f
, ~
€
=
10 ~
29 5
PROBLEMS OF OPTIMAL STOPPING Set K = Max(/
l$-@l
1
"
m,
L w
Y+X
J8
w+
.
l w E Ho
,
= uE--uE-K
We multiply (3.15) by (3.17)
lMl)
and (3.15)"
by
-w+
and add up.
CA(uE-UE) + a (uE-uE) + A(uE-GE)lw+dx + 0
-E1 X
+ dx
where
x Indeed assume
zE 2 ",)I w
-
= ((U"$)+
2
0
.
then "
C
(U"$)+,W+)
uE-$-K
,.,
< $-$-K
0
S
,
hence ((UE-qJ)',w+) which proves that
X
2
0
.
=
0
Now from (3.17) we deduce
a(w,w+) + X(w,w+) + J
8
hence
w+
=
0
.
[(a0+A)K+qlw+
Therefore uE-zE 2 K
.
By a reverse argument we conclude that (3.16) holds. Therefore considering the solution of
We obtain
dx < 0
=
296
CHAPTER VII
(3.18)
we can assert tha (3.19) Consider next the iterative process
n+l)
+ (f,v-u
When
. t , a ) I then the contraction argument mentionned in Theorem 3.2, guarantees that un Defining similarly
+
u
in L~
.
zn. It follows from estimate (3.19) that
(3.21) Letting a .
2
y
2
n
-t m
we deduce that (3.14) holds true, at least provided that
0. A s this stage, it is not useful to make the change of
unknown function u
= wz
, which changes
$
into $/w.
Indeed we will
only obtain estimate (3.14) with twice the right hand side, which is not the estimate we want.
One proceeds as follows. Consider the V.I.
291
PROBLEMS OF OPTIMAL STOPPING
6 > 0 , will tend to 0.
where
Let also &:
be the solution of the same problem with
1 4
changed into
$.
6 > 0 , we have
Since
And it is enough to show that
u6
-+
u
6
as
+
0
,
in some sense. For
such a result, we may consider the change of functions u therefore it is sufficient to assume
a .
2
y > 0
.
= uz
, and
Consider next the
iterative process
2
J,(un,v-un) 6 6
+
(f,"-Ut++
then we have
from which it follows that
>
k = h+y
with as
6
-t
.
In particular it follows that
u6
0. From this and the V.I., one deduces
is then enough to obtain
u
6
+
u
in
1
us
Ho weakly and
completes the proof of the desired result. We can then state the following regularity result
is bounded in bounded in Lw
Lm
1
Ho
.
It
weak star, which
c
298
CHPATER VII
Theorem 3 . 3 .
We make t h e a s s u m p t i o n s o f Theorem 3 . 2 and
(3.22) Then t h e s o l u t i o n
u
of (3.11) belongs t o
Co(&
.
Define
En = ll$n-j,l
I Lrn
,
En
-+
0
and
Clearly also
satisfy the assumptions of Theorem 2 . 2 .
The functions $n
Let
un
be the
It is also the solution of
solution of the V.I. corresponding to
@n. the unilateral problem ( 2 . 6 ) , hence in particular
u
E
Co(s).
But from
(3.14) we deduce that
lIun-uI Hence
u
E
0 -
.
C (0)
I L"
5
llQn-vl
I L"
*
0
*
0
299
PROBLEMS OF OPTIMAL STOPPING
Let u s prove to end that section, the useful result that the solution uE of the penalized problem converges to the solution
in
.
Co(s)
u
of the V.I.
This result will be generalized in section 5 for general
semi groups, with some slight changes in the assumptions.
We will
however need it in Chapter VII, section 3 .
Theorem 3 . 4 .
Under t h e assumptions ~f Theorem 3 . 3 , t h e n t h e s o l u t i o n
of 1 2 . 3 ) converges towards t h e s o l u t i o n
u
of t h e 7 . 1 . (S.11) i n
Let u s first remark that it is sufficient to prove this result when is regular.
Indeed let
as in Theorem 2 . 3 , @n
@n
Lemma 3 . 4 , we have (noting u
I /Un-U/I
(3.23)
+
$
in
the V.I. corresponding to
Co.
uE
Co(6)
.
y
From
on)
I lon-il I Lrn
5
L
But the proof of Lemma 3 . 4 , in particular estimate ( 3 . 1 6 ) , together with an iterative procedure for the penalized problem, like in ( 3 . 2 0 ) shows that the same estimate is valid for the penalized problem, namely
From ( 3 . 2 3 ) , ( 3 . 2 4 ) it is clear that if we have
1
lu:-unl
1
+
0
in
then the desired result will follow. a We may of course assume . Lemma 3 . 1 ) (1) Now for
)t
large, replacing
Lemma 2 . 2 , that
uE
2
C
0
, for n fixed We may thus assume
$
regular
y > 0 , without loss of generality (cf
a .
by
aO+X
(cf. ( 2 . 1 0 ) ) , we know fron
remains bounded in W2'p.
Now consider the iterative
sequence
(L) We have however to consider a penalized problem with EIUJ
2
.
E
changed into
300
CHAPTER V I I
hence
< - kn
-
k
where
NOW
=
A x+y .
1 /uE>OlI
I-k
Therefore letting p
+ m,
~< c ~, from , Lemma ~ 2.2; similarly
1
lu E, 1
< c .
Hence we have
IIu~-u~'~I I
(3.25)
2
C kn
.
Lrn We a l s o have
I lu-unt 1
(3.26)
< C
kn
L which follows from (3.25) and continuity and convexity of the norm.
It
also follows directly from (2.16) and an iterative scheme. Now for any fixed
since uE'n
remains in a bounded set of
depending on uE + u
in
n, we have by Lemma 2.2,
n).
COG) .
W2"
as
E
+
0
(a priori
From this and (3.25), (3.26) it follows that
0
30 1
PROBLEMS OF OPTIMAL STOPPING
4 . SOLUTION OF THE OPTIMAL STOPPING TIME PROBLEM
4.1.
Ihe-re9ular-case
We are going t o show the following
Theorem 4 . i .
We assume ( 1 . 1 1 , ( i . 2 ) , i l . i O / ,
(1.111, ( 2 . 2 ) , ( 2 . 5 1 .
the solution u
of 12.61 is given explicitely bg
(4.1)
u(x) = Inf Jx(6)
e
Then
.
Moreover there exists an optimal stopping time, characterized as follows. Define
and
then
6
If h
E
is an optimal stopping time.
Lp(&
,p >
5 , we know from Chapter 11, Theorem 4 . 1 ,
that
From this estimate follows that we can deduce the following Ito's the function u
integrated from formula
to
(4.4)
EX u(x(6A-c)exp
u(x)
=
+ EX
where
6
E
- J 6A.r
W2"
,
p >
ao(x(s))ds
4
+
JoAT (Au+aou) (x(s)) (exp-JS ao(x(X))dX)ds
is any stopping time.
302
CHAPTER V I I
Now using the relations ( 2 . 6 ) , it is easy to deduce from ( 4 . 4 ) that (4.5)
u(x)
5
V
JX(8)
8.
On the other hand we may assert that xc(x)
(Au+a u-f) = 0 p.p. 0
hence EX i8A?
xC(Au+aOu-f)
(x(s))
(exp-/’ ao(x(X))dh)ds
0
But for
s
0 such that
Let
u(x) < Y(X) - 6 and
Let
N,
such that
n t Nc:
implies
i
110,- $ 1 1 Therefore for
s S
6
e 6A T
tlx(s)
do1
.
is a random variable, we write
"e
=
c(etw)
.
is a stationary Markov process we have the property EX[€ltc~mtl= EX(t)
6
.
We are going to apply that formula to the R.V.
313
PROBLEMS OF OPTIMAL STOPPING
5
=
w(x(s 2 - s 1 )AT))
e
-a(s - s ) 2 1 +
% SAT)) emas ds
s2 t
Then
8 5
-a(s2-s 1) =
w(x(s A T
s1
)) e
-a(s-s
S
+
i12
xo(x(s~-rs ) ) e 1
f
and property (5.16) reads
But from (5.13) we have
hence we have proved that
-as 2 w(x(s,))
e
1
.
This implies that
-aS Z
+
s1
w(x(slAr)) e
1
,
)
ds
S]
314
Indeed l e t
CHAPTER VII
X
be t h e l e f t hand s i d e of ( 5 . 1 8 ) .
= T
T~
1 and
Similarly for
s 2 s
Theref o r e
and from (5.17)
which p r o v e s ( 5 . 1 8 ) . But t h e n t h e p r o c e s s
1
if
T > sl
We u s e t h e f a c t t h a t
315
PROBLEMS OF OPTIMAL STOPPING
is a sub martingale. By Doob's theorem, we deduce
EX[w(x(6A~))e-ae
+ J6 f 0
for any stopping time.
W(X)
x 8(x(XA?))e-aX
dhl
2
w(x)
But this is identical to
4
Ex[w(x(0))
x6,T
+ JeAT f
x
(x(s))e-as
dsl
8
and since w 4 $
Therefore w(x)
5
u(x).
0
This completes the proof of the desired result
As we have done in Chapter IV, for the problem of semi group envelope
we can now define a general formulation of ( 5 . 1 1 ) ,
(5.12).
in Chapter IV 5 5 . 2 , consider a topological space
(E,E) and spaces
B
and
(5.19)
C.
We consider a semi group satisfying @(t)
We also assume that
: B
+
B
@(O) = I
Namely, as
316
CHAPTER VII
:
(5.23)
@(t)
(5.24)
@(t)f
c
-+
c
- + f in
C
as
t 4 0 , Y f E C .
Let now
(5.26)
L
E
B
,
t
-+
O(t)L
is measurable from
Then we consider the set of functions u
into C
[O,m)
.
satisfying
U€C,U 0,
c o ( s ) : R"+
+
R ~ c, o n t i n u o u s ,
c o ( 0 ) = 0, non d e c r e a s i n g CO(Sl +
s2)
5
co(5,)
+
C0(L2)
Let a l s o :
357
IMPULSIVE CONTROL
1 . 2 . The model
An i m p u l s i v e c o n t r o l i s d e s c r i b e d by a s e t a s f o l l o w s :
(1.10)
w
= (5'
5
I
... < e n .. . .... cn ..
e2 , 52
0, c a l l e d t h e f i x e d c o s t ; co(S) i s c a l l e d t h e v a r i a b l e c o s t .
360
CHAPTER V I I I
2 . QUASI VARIATIONAL INEQUALITIES
Orientation Dynamic Programming l e a d s t o a n a l y t i c problem which resembles t h e t y p e o f problems d i s c u s s e d i n c h a p t e r V I I . This i s n o t s u r p r i s i n g , s i n c e a s t o p p i n g t i m e problem i s a v e r y p a r t i c u l a r c a s e o f impulse contro1,namely when we
impose
e2
=
e3
=
... =
fm.
T h i s w i l l be r e f l e c t e d i n t h e a n a l y t i c as w e l l
a s i n t h e p r o b a b i l i s t i c t r e a t m e n t . We w i l l e n c o u n t e r q u a s i v a r i a t i o n a l i n e q u a l i t i e s ( Q . V . I . ) i n s t e a d of V . I .
We i n t r o d u c e a non l i n e a r o p e r a t o r a s f o l l o w s :
(2.1)
MHx) = k + i n f [cp(x+S) + c o ( S ) l 620
x +
6 €8
which makes s e n s e f o r f u n c t i o n s (Pwhich a r e bounded below. We c a l l Q . V . I . t h e f o l l o w i n g problem :
where a s u s u a l :
(2.3)
+
a.
u v dx.
Remark 2 . 1 . The c o n d i t i o n u t 0 i m p l i e s t h a t Mu i s w e l l d e f i n e d .
I
Remark 2.2. Problem ( 2 . 2 ) i s a n i m p l i c i t V . 1 , s i n c e t h e o b s t a c l e depends on t h e s o l u t i o n . T h i s i s w h y , i t i s c a l l e d a Q . V . 1 , w i t h t h e V.I.
I t s h o u l d n o t be confused
36 1
IMPULSIVE CONTROL Vv 5 Mv,
u s Mu
I
which i s a t o t a l l y d i f f e r e n t problem.
Remark 2 . 3 The o p e r a t o r does n o t p r e s e r v e t h e r e g u l a r i t y o n d e r i v a t i v e s . I t does n o t map H 1 i n t o i t s e l f ' ] ) . However i t h a s a v e r y i m p o r t a n t p r o p e r t y which w i l l p l a y a fundamental r o l e i n t h e s e q u e l . I t i s monotone i n c r e a s i n g i n the following sense :
'Dl ( x )
:
+
1,l lu/
1 c.0 n +2.
LP
Consider now t h e e l l i p t i c P . D . E .
(3.32)
Au = f
Assuming p i s a s i n ( 3 . 3 1 ) , we can deduce from ( 3 . 3 1 ) , ( 3 . 3 2 ) and ( 3 . 5 ) , that : ~',T,T E u(y(8-T)) = E u(y(8'-T-Te))
+ E
ef ( y ( t ) )
dt.
'0 ,T
But n o t i n g t h e e s t i m a t e :
where t h e c o n s t a n t does n o t depend o n T , we deduce a b e t t e r e s t i m a t e t h a t ( 3 . 3 1 ) namely :
378
CHAPTER VIII
Proof of theorem 3 . l . We may c o n s i d e r t h e Q . V . I .
w i t h o b s t a c l e J1 = Mu. I t i s e a s y
(2.2) as a V . I .
t o check t h a t t h e assumptions o f theorem 4 . 2 ,
Qo,
W be a n i m p u l s i v e c o n t r o l . Consider
po,
p
t
c h a p t e r V I I a r e s a t i s f i e d . Let
, P;,
t h e n t h e p r o c e s s xn d e f i -
ned i n ( 1 . 1 4 ) s a t i s f i e s t h e e q u a t i o n :
We t a k e w i t h t h e n o t a t i o n of t h e b e g i n n i n g o f § 3.1
From ( l , l 7 ) ,
( I .18)
:
i t f o l l o w s t h a t assumption ( 3 . 5 ) i s s a t i s f i e d .
T h e r e f o r e from lemma 3 . 1 . ,
p r o p e r t y ( 3 . 8 ) a p p l i e d w i t h w = u we o b t a i n :
Let u s n e x t remark t h a t :
en
(3.38) Indeed i f 8""
(en,On+')
5
< T implies
r n , then x(s)
and x('J"),
x(e"-)
( 3 . 4 8 ) i s v e r i f i e d . I f 8"" x ( r n ) = x ( T ~ - )a @ ,
E
e n+ 1 - T n
d
for s
c &since
T.
If
en
= T~ T.
Ce n ,
1, x(s)
< T , therefore T 2
> T ~ t, h e n when
thus rn =
r n 2 T, which i s i m p o s s i b l e s i n c e 8"
I . We a p p l y t h e same method a s i n theorem n+ I 3.1, using the f a c t t h a t u i s a s o l u t i o n of a V.I. w i t h o b s t a c l e Mu". LetW
E
'Ikn+and l
take 0
2
j
2
n-l
We f i r s t have a s f o r ( 3 . 3 9 ) :
.
388
CHPATER V I I I
(3.66)
Operating a s f o r ( 3 . 4 5 ) , we o b t a i n :
(3.68)
We t a k e t h e mathematical e x p e c t a t i o n and add up t h e s e r e l a t i o n s , when j r u n s from 0 t o n-I. We o b t a i n :
n
389
IMPULSIVE CONTROL
n+ 1 =
Using t h e f a c t t h a t
(3.69)
un(x)
5
a . s . We o b t a i n :
+m
J ~ ( w ) ,f o r
w
c
V+'. 1
Let u s prove t h e e x i s t e n c e o f a n o p t i m a l c o n t r o l . Definet"(x)
Bore1 2 0
function such t h a t :
(3.70)
+ u n - l ( x + s"(x))
= k + co(;"(x))
Mun-l(x)
Vx
E
8, n
2 1
Consider xo a s
dx
0
=
0
U ( X )dwo
x (0) = x
T:
1
It
= inf
u n ( x o ( t ) ) = Mun-I(xo(t))}
then :
(3.71)
-1
'n -1
5,
=
0
-gn
We have a g a i n x 0 (en) -1 c @ when p r o c e s s x j-1 and -8:,'
if T:
Tn
5: f o r j
0 -1 ( x (en)) i f
;A
O
(4.5)
f
E
B (Bore2 bounded on
Then t h e s o l u t i o n u o f Q . V . I .
8)
(2.21 s a t i s f i e s
= 0, u 2 M u -a s
x6e
d s + $ ( t ) e-cYt u , V t 2 0.
of t h e s e t o f f u n c t i o n s s a t i s f y i n g ( 4 . 6 1 .
Proof u i s t h e s o l u t i o n of t h e V . I .
5.2, chapter V I I ,
w i t h o b s t a c l e $ = MU. T h e r e f o r e b y theorem
( r e l a t i o n s (5.11),
(5.12))
u s a t i s f i e s (4.6).
L e t us p r o v e t h a t i t i s t h e maximum element among t h e f u n c t i o n s s a t i s f y i n g %
( 4 . 6 ) . T h i s i s proved u s i n g t h e d e c r e a s i n g scheme. L e t u be an element o f ( 4 . 6 ) . We have
%
Assuming t h a t u
%
u
5
uo s i n c e :
%
un, t h e n M u 5 Mun, Therefore?; i s an element of t h e s e t
( 5 . 1 1 ) , (5.12) c h a p t e r V I I , w i t h $ = Mu". % n+l % then implies u < u T h e r e f o r e u S U.
But theorem 5 . 2 , c h a p t e r V I I ,
.
I
We c o n s i d e r now t h e a n a l o g u e of problem (5.27) c h a p t e r VII, w i t h an i m p l i c i t o b s t a c l e . Namely l e t $ ( t ) be a semi group s a t i s f y i n g ( 5 . 1 9 ) , c h a p t e r V I I . We a l s o c o n s i d e r :
..., ( 5 . 2 4 ) ,
393
IMPULSIVE CONTROL
L
(4.7)
E
B , t + $ ( t ) L i s measurable
from [ O , m l
into C
L t O
L e t now M be an o p e r a t o r such t h a t :
M :C
(4.8)
+
C i s L e p s c h i t z , concave, and monotone i n c r e a -
MYl
sing (i.e.
(4.9)
;
M(O)tk>O
5 M(D2 i f
Y1
5
(D2)
a > O
We c o n s i d e r t h e set of f u n c t i o n s : u
(4.10)
E
MU
C, u S e
Theorem 4.3.
We assume ( 5 . 1 9 ) ,
$(s)
Lds + e
..., (5.24)
-at
$(t) u
chapter VII and 1 4 . 7 1 ,
(4.81,
( 4 . 9 ) . Then the s e t of s o l u t i o n of ( 4 . 1 0 ) i s not empty and has a rnadmwn
element, which i s a p o s i t i v e f u n c t i o n .
Let z
E
C and c o n s i d e r 5 t o be t h e maximum s o l u t i o n o f :
(4.11)
S i n c e Mz
E
C, 5 =
Tz i s w e l l d e f i n e d , a c c o r d i n g t o theorem 5 . 3 , c h a p t e r V I I .
I t w i l l be c o n v e n i e n t t o w r i t e :
(4.12) where u(9)
T
=
u o
M
: C -+ C i s t h e maximum element of t h e s e t ( 5 . 2 7 ) , c h a p t e r V I I .
O n e e a s i l y checks t h a t u i s monotone i n c r e a s i n g and concave. From t h e assumpt i o n ( 4 . 8 ) on M i t f o l l o w s t h a t : (4.13)
T i s i n c r e a s i n g and concave.
394
CHAPTER VIII
L e t next
1,
m
uo =
(4.14)
e -at $ ( t ) Ldt
and uo E C , uo 2 0. L e t 1 > p > 0, s u c h t h a t :
I
(4.15)
! ~ I l u ~ l5 k
Then one h a s : T(0) 2 u u
(4.16)
0
Indeed by ( 4 . 1 5 ) and assumption ( 4 . 9 ) .
!J u0 9 M ( 0 )
and from ( 4 . 1 4 )
:
v
uo =
lot lo
e-as $ ( s ) vL d s + e
t
O
(1)
We can s t a t e t h e f o l l o w i n g :
Theorem 4 . 4 .
Assume (5.191,
. . ,,(5.24) chapter V I I and
14.S), (4.9), ( 4 . 1 9 ) .
Then problem 1 4 . 1 8 ) has one and o n l y one s o l u t i o n . Moreover h
-+
y,
-+
u in C, as
0, where C is the m d m u m element o f (4.10).
Let u s s e t oh($) t o be t h e s o l u t i o n o f ( 5 . 4 9 ) c h a p t e r V I I ( d i s c r e t i z e d V . 1 ) . Then uh : C
(4.20)
+
C , and :
ah i s i n c r e a s i n g and concave.
T h i s w i l l f o l l o w from t h e f o l l o w i n g f a c c :
(4.21)
if i
5
S
E
C v e r i f i e s 5 2 $ and
hL + e
-Uh
$(h)