Quantitative and Qualitative Games, Volume 58 (Mathematics in Science and Engineering)

Quantitative and Qualitative Games

This is Volume 58 in MATHEMATICS IN SCIENCE AND ENGINEERING A Series of monographs and textbooks Edited by RICHARD BELLMAN, University of Southern California A complete list of the books in this series appears at the end of this volume.

QUANTITATIVE A N D QUALITATIVE G A M E S AUSTIN BLAQUIERE FACULTY OF SCIENCES, PARIS UNIVERSITY OF PARIS, FRANCE

FRANCOISE GERARD FACULTY OF SCIENCES, PARIS UNIVERSITY OF PARIS, FRANCE

GEORGE LEITMANN DIVISION OF APPLIED MECHANICS UNIVERSITY OF CALIFORNIA BERKELEY, CALIFORNIA

@

ACADEMIC PRESS New York and London

1969

COPYRIGHT @ 1969, BY ACADEMIC PRESS, INC. ALL RIGHTS RESERVED NO PART OF THIS BOOK MAY BE REPRODUCED IN ANY FORM, BY PHOTOSTAT, MICROFILM, RETRIEVAL SYSTEM, OR ANY OTHER MEANS, WITHOUT WRITTEN PERMISSION FROM THE PUBLISHERS.

ACADEMIC PRESS INC.

111 Fifth Avenue, New Y&k, New York 10003

United Kingdom Edition published by ACADEMIC PRESS, INC. (LONDON) LTD.

Berkeley Square House, London W l X 6BA

LIBRARY OF CONGRESS CATALOG CARD NUMBER: 74-84246 AMS 1968 SUBJECTCLASSIFICATION 9075

PRINTED IN THE UNITED STATES OF AMERICA

Preface

This book, based on lectures given by A. Blaquikre at the Institute Henri Poincark, is a record of joint research in the field of two-person games. Having arrived at a reasonably comprehensive theory for a wide class of such games, we present it here in the hope that it will serve as the basis for further research and for application in the physical, biological, and social sciences. Thus, this volume is addressed to those interested in game theory for its own sake, as well as to those concerned with conflict situations in technology, economics, psychology, indeed, in any dynamical system. With a view toward increasing the scope of the theory, the treatment proceeds from the general to the particular. Results are obtained for systems whose dynamics are defined by a rather large class of mappings. Further results are derived then by restricting the dynamics to those described by ordinary differential equations (differential games) and by difference equations (multistage games). In Chapter 1, the notions of game and play are introduced, and the problem to be discussed is stated. Certain basic concepts, such as rules, objectives, and strategies, are defined. Chapters 2-4 are devoted to quantitative two-person, zero-sum games with perfect information of the state called games of degree by Rufus Isaacs. These chapters are essentially extensions of the geometric theory of optimal processes developed earlier by A. BlaquiBre and G. Leitmann.7 Differential as well as multi-stage games are treated. Chapters 5-7 deal with qualitative two-player games, called games of t

See Refs. 1-6.

vi

PREFACE

kind by Isaacs. In this type of game the players do not strive to minimize and maximize, respectively, a numerical payoff; rather they have conflicting aims in that they endeavor to terminate play on different target sets. In Chapter 5 additional geometric notions are introduced, such as the map of the game, and optimality for a player at a given state is defined. While the definitions and arguments are different from those employed in earlier chapters, the geometric results exhibit a strong similarity to the ones found in quantitative games. This similarity serves as a guide for the development of the theory. In Chapter 6, the theory is applied to differential games. The arguments used follow closely those employed in Chapter 3. While the framework of the theory of quantitative and qualitative games appears to be essentially the same, the similarities appear to be more formal than real. In Chapter 7, an attempt is made to establish a connection between these two types of games. We believe that geometric notions are well suited for a discussion of dynamical system theory, such as that of the games treated in this book. Geometric notions are intuitively appealing and, at the same time, capable of providing a modicum of rigor. It is our hope that the reader will come to share this point of view and that he will not be discouraged by the many assumptions introduced in the development of the theory. The reader whose interest lies primarily in the application of the theory may elect to forego some of the details and accept many of the arguments on an intuitive basis. It has been our aim to state all assumptions explicitly rather than to leave it to the reader to discover them (or not). The complications which have necessitated many of these assumptions appear to be inherent in the theory. In any event, this book is but a brief introduction to a geometric theory of games, and many questions remain unanswered. We should like to take this opportunity to thank the Office of Naval Research and the Delegation Generale A la Recherche Scientifique et Technique for supporting the research that has resulted in this book. A. BLAQUIBRE F. GBRARD G. LEITMANN

Paris and Berkeley September 1969

Contents

PREFACE

V

NOTATION AND TERMINOLOGY

I. Games and Plays 1.1 1.2 1.3 1.4 1.5 1.6 1.7

11.

Problem Statement and Assumptions Rules of the Game Trajectories and Paths Play, Terminating Play Playable Strategy Pairs Joining of Trajectories and Describing Curves Objectives of the Game, Qualitative and Quantitative Games

9

Some Geometric Aspects of Quantitative Games 2.1 2.2 2.3 2.4 2.5 2.6 2.7

Transfer of State and Performance Index Optimality Game and Isovaiue Surfaces Paths in Augmented State Space Joining of Paths Some Properties of Game Surfaces Uniqueness of the Value of the Game

11 12 13 15 17 17 20

111. Differential Quantitative Games 3.1 3.2 3.3 3.4

The State Equations and Strategies Transfer Time Rules of the Game Target e vii

21 22 23 23

Viii

CONTENTS

3.5 Integral Performance Index 3.6 A Variational Equation and Its Adjoint 3.7 Joining of Paths 3.8 Two Families of Paths 3.9 Regular Interior Points of a Game Surface 3.10 Transformation of a Tangent Plane 3.1 1 Regular Optimal Paths 3.12 Transversality Condition 3.13 A Min-Max Principle, Regular Case 3.14 Discontinuity Manifolds 3.15 Regular Portion of an Optimal Path 3.16 Jump Condition 3.;7 Constraints 3.18 A Min-Max Principle with Jump Condition 3.19 Example 3.1 3.20 Example 3.2 3.21 Example 3.3 3.22 Sufficiency Conditions

24 25 29 30 32 36 37 39 41 42 43 46 49 52 53 58 61 67

IV. Multistage Quantitative Games 4.1 4.2 4.3 4.4

4.5 4.6 4.7 4.8 4.9 4.10

4.1 1 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20

V.

Problem Statement State Equations, Strategies, and Target e Describing Curves and Paths in State Space Cost of Transfer Augmented State Space and Paths in E"" Some Properties of a Game Surface Sets fir and fiE 10-Directional Convexity Regular Points of a Surface &(C) Regular Optimal Paths Variational Difference Equation Linear Transformation and Inverse Transformation Some Properties of Linear Transformations A k and Transformation of a Tangent Plane Adjoint Equations Gradient and Adjoint Vector Transversality Condition A Min-Max Principle Constraints Example 4.1

'B

71 72 74 74 75 77 78 80 81 83 84 86 87 88 90 92 93 93 94 96

Some Geometric Aspects of Qualitative Games 5.1 Map of a Game 5.2 Optimal Strategies in a Qualitative Game, Sets of the Game

103 104

ix

CONTENTS

5.3 Some Properties of Sets of the Game 5.4 Surface of the Game 5.5 A Similarity between Qualitative and Quantitative Games 5.6 Problem Statement

VI.

106 107 110 111

Differential Qualitative Games 6.1 Rules and Objectives of the Game 6.2 A Basic Property 6.3 A Variational Equation and Its Adjoint 6.4 A Property of Boundaries of Sets of the Game 6.5 A Property of the Surface of the Game 6.6 Two Families of Paths 6.7 A Local Min-Max Condition 6.8 Gradient and Adjoint Vector 6.9 Transversality Condition 6.10 A Min-Max Principle 6.11 Autonomous Games with Time-Independent Targets 6.12 Constraints 6.13 Semipermeable Surfaces 6.14 A Property of the Gradient along a Path in a Semipermeable Surface 6.15 Transversality Condition for a Semipermeable Surface 6.16 A Min-Max Principle along a Path in a Semipermeable Surface 6.17 Example 6.1 6.18 Example 6.2

113 114 114 116 118 119 123 125 126 131 132 134 135 139 140 142 143 145

VII. A Connection between Qualitative and Quantitative Games 7.1 7.2 7.3 7.4

Problem Statement, Definitions of Games 1, 2, and 3 Local Saddle-Point Condition, Game 4 Surfaces of the Game A Connection between Paths in Games 1 and 4

150 153 157 159

Appendix

163

References

165

SUBJECTINDEX

169

ASSUMPTIONS, COROLLARIES, LEMMAS,THEOREMS INDEX

172

This page intentionally left blank

Notat ion and Terminology

Unless otherwise indicated mathematical symbols will be those in common use. Additional terminology will be introduced as needed. Vectors will be denoted by lower case letters. Subscripts will be used to denote the components of a vector; superscripts will be used to distinguish vectors. x = ( x , , x 2 , . . . x,) is an element of En.No special notation is used to distinguish between x E En and its column form; that is, unless otherwise specified, all vectors will be column vectors. The row form of x is xT = ( x , x 2 . . . x , ) . The Euclidean length of a vector x will be denoted by IIx11. The gradient of a function @: x @ ( x ) at a point xj will be denoted by grad @(xj). The transpose of a matrix M will be denoted by MT.The topological closure of a setoR in a topological space and the interior of R by R. will be denoted by The term domain will mean an open connected set of the appropriate space. A scalar-valued function defined on a domain will be said to be of class Ckif the function and its first k partial derivatives are continuous on the domain. A vector-valued function will be said to be of class Ce if each of its components is of class Ck. The notation V v 3 u will stand for: -+

a,

“for all v there exists u (which depends on v)” and 3 v V u will stand for: “there exists u (independent of v ) for all v” We shall write Definition whenever we define a new concept, and not for the specification of concepts introduced previously. We shall write Assumption whenever we introduce a new assumption of basic importance for the development of the theory. xi

This page intentionally left blank

Quantitative and Qualitative Games

This page intentionally left blank

I

Games and Plays

1.1 PROBLEM STATEMENT AND ASSUMPTIONS

We shall be interested in the behavior of a set of “persons,” called the players, each of whom strives to modify the state of a system in a most efficacious manner according to his own criterion. If there is only one player in the set, the theory reduces to the one of optimal control. In the following discussion we shall consider only the case of two players J , and JE whose interests are conflicting or at least different. We shall assume that the state of the system or, as we shall say, the state of the game, at any instant of time can be characterized by the values of a finite number of real variables xl, x 2 , . . . , x,, and that these values are known to both players. Hence we may think of the state as a point in an n-dimensional Euclidean space En, termed the state space of the game. In state space we select a rectangular coordinate system so that we may define a point by the state vector x = ( x l , x2, . . . , x,). We shall assume that the evolution in time of the state of the game can be influenced at all times by the two players. For example, as we shall see later, the behavior of the state may be described by a set of differential equations dxv/dt =f v ( x 1 ,

. .

x2,

*

7

x n , ~ 1 u2, ,

. . *

9

ur, ~ 1 ~,

. ..

2 ,

9

vs),

v = 1 , 2 , . . . , n (1.1)

where ul,u2, . . . , u,. and vl, v 2 , . . . ,v, are control variables which define vectors u = (ul, u 2 , . . . ,ur) and u = (vl, v2,. . . , v,) in r-dimensional and s-dimensional Euclidean spaces E’ and E’, respectively. Players J , and J E make their decisions through choosing the values of

2

I

GAMES A N D PLAYS

control variables u and v , respectively, at each instant of time. Later we shall discuss the case in which these choices are governed by functions of x,p , and e , which J,, and J,: select from two prescribed sets of functions. 1.2 RULES OF THE GAME

We shall assume that x belongs to the fixed subset G of En. Let there be given two prescribed sets R, and R E whose members will be denoted by r,. and r*;, respectively. Let T be the collection of all subsets of [O, a)containing 0. Let F be the set of all functions x: T, 4 G for all T, E T. Hereafter we shall w e simply X: T + X = X(T), T € T, to indicate a function x E F. Let .F be the collection of all nonempty subsets of F, and let y:

G X R,x

RE+g

be a mapping that associates with each point xi E G and with each pair (rl,, r,.;) E R , x R , an element y(xi,rIJrr E ) of .F, such that

x(0)

= xi

vx

E

y ( x i , r p , re)

Finally, let there be given two sets of points in G, namely 81. and 8,. We shall call these sets the targets of J , , and J E , respectively. Definition 1.1. A p i n e is defined by its rules and its objectives. The rules of the game consist of (i) the set G (ii) the sets R l , and R , (iii) the mapping y. The objectives will be defined later. Definition 1.2. Members rIJof RI,and rR of RE are strategies of players J , , and J,, respectively. Members (r,, r E ) of R , x R , are strategy pairs.

EXAMPLE 1 . 1 . Consider the following rules of a game: Let G be a domain of ETL.Let there be given the differential equation

dx/dr = f ( . ~ LI, , V) where u = (ul, u2, . . . , ur), u = (vl, u 2 , . . . , vs),.f=

(1.2)

(fi,fi,. . . ,fn>.

1.3

3

TRAJECTORIES AND PATHS

Let the choices of control variables u and u be governed by functions e: x - e ( x ) , x E G, respectively; that is, r , = p and rE = e being chosen from prescribed sets of functions R, and R E , respectively, let Eq. (1.2) be replaced by

p : x-p(x),

dXldT

=f ( X ,

p ( x ) , 4x1)

(1.3)

Equation (1.3) is said to have a solution if there exists a function x : T + X ( T ) defined and continuous on an interual A T , such that X ( T ) E G V T E A T , and dX(T)ldT = . f ( X ( T > > p(x(.)), e(x(.>>> is satisfied almost everywhere in AT. Now, for each initial point xi E G there may exist a solution of Eq. (1.3), x : T - x ( T ) , int [0, co) such that x(0) = xi. Let [0, T ~ be ) its interval of definition. We shall call x a maximal solution if X ( T ) is not defined for T = T ~ c‘r , if’imr-r3,r V*(xi)- V*(x)} - xgi < V*(xi)- V*(x))

A / C ( C ) = { x : xo - xgi

(2.10)

B/C(C) = ( x : x,

(2.11)

2.4 PATHS IN AUGMENTED STATE SPACE

Definition 2.6.

Paths n i s ( C )in 3 & G x {x,} are defined by

niS(C)& { x ' :

xgk

+ V ( Xxs; " , r p , rE, nks)= C , rkS c nis} (2.12)

where rris is a path in G from xi to xsgenerated by strategies r , rE E RE,and C is a constant parameter.

E

R, and

Thus, path nisis the projection on G of paths nis(C)in 9. Let 0 A 8 x {xo} be the target in augmented state space. One can

16

SOME GEOMETRIC ASPECTS OF QUANTITATIVE GAMES

I1

X’

Xi

FIG.2.3. Path in augniented state space.

easily verify that the above definition of a path in 3 agrees with the general definition of a path given in Section 1.3. By varying the value of parameter C in (2.12) one generates a oneparameter family of paths {IIis(C)}. This family belongs to an x,,cylindrical surface whose intersection with G is path ris. Definition 2.7. Three families of paths emanating from points in %*, (n;i,(C)},{np(C)},and (II;&(C)>, are defined by TI$(C) = { x k :

n ” , ” ~A) { x k :

xp + ~ ( x ” x, f ; rIJ*,TI.,.,

= C, r;;c

n-2) (2.13)

x:

+ y ( x k , xm; r I J ,rR;* , r R ) = C, rp c r p > km

(2.14)

nrE(C)= { x k : x:, + Y ( x k ,xQ;rlJ*,rP*, r:yI;) A

1 j = C , n-PE c r nq PE

(2.15)

2.6

17

SOME PROPERTIES OF GAME SURFACES

Paths T$, T:? and T;% are paths in G, emanating from points xi,xz,and xn of X * , generated by the strategy pairs (rp*, rE), (rl>,rE*)and (rp*,rE*) and ending at points x3,9" and xQ,respectively. Definition 2.8. Since (rE,*,r$;*) is playable at all points xn E X * , there exists a path n;f,(C) which reaches 0 at point x f . It is called an optimal path. Hereafter we shall denote an optimal path by n*(C). Definition 2.9. If (rI,*, rE) and (rp, rE*) are playable strategy pairs at points xz and xl,respectively, there also exist paths n$(C) and ng(C) which reach @.t They are called P-optimal and E-optimal paths, respectively. Hereafter we shall write simply lI,(C) and IT,(C) to indicate these paths. 2.5

JOINING OF PATHS

Let ni3be a path emanating from xi,generated by ( r p , rE), and ending at x3; and let xis be a path emanating from xi,generated by (Ep, i E ) and , ending at xs. Tn addition to Assumption 1.1, according to which rii u xis is a path riS generated by ( p r , p6), we shall introduce Assumption 2.2

+ V ( x j ,xs; E,,

F,,

= Y ( d ,xs; p p ,

PE,

Y ( x i , x3; rl,, r E , TP)

nlS) T i S )

(2.16)

We shall call this assumption an additivity property of the cost. 2.6 SOME PROPERTIES OF GAME SURFACES

In preparation for a fundamental theorem, let us now prove Lemma 2.1. No point of a path rIp(C') which emanates from x i is an A-point relative to the game surface through x i . iOf course, superscript f means that the terminal point of the path belongs to 0. However the terminal point need not be the same for paths IIYfC), rIL!(C), and IIFL (C).

18

11

SOME GEOMETRIC ASPECTS OF QUANTITATIVE GAMES

\

\

Frci. 2.4. Paths

I

I

I

I

n;(C’), n;(C”), n$B(C”) and

\

game surface through xz.

Lemma 2.2. No point of a path n$(C”)which emanates from x i is a B-point relative to the game surface through xi.

Let x 3 = (so3,xj) be a point of a path rlg(C’) generated by (rI,*, r E ) , where . K ~is its projection on G. Let r i j c r Ii,sbe the path in G emanating from s’,generated by ( r p * , r E ) and ending at xj, which has the same supporting curve as T:. First note that, since the family of game surfaces is defined on X * , xj cannot be an A-point relative to any member of the family if xj 6X * . Hence let us suppose that xj E X*. By condition (i) in the definition of an optimal strategy pair, there exists a path ryE which emanates from xj, is generated by ( r p * , r E * ) and reaches 8 at point sf. From Assumption 1.1 it follows directly that T$ U rjpfEis a path r);. Let ( r p * , pE) be the strategy pair that generates this path. Path r;: reaches 8 at point x f , and associated with it is a value of the cost of transfer ~ ( x ’ ,6 ; r p * , pE) = V(x2,

x’; rp*, p E ,

rg)

(2.17)

2.6

19

SOME PROPERTIES OF GAME SURFACES

In view of the saddle-point condition we see that V ( x 7 ,8; rp*, p E )

< V ( x Z 0;,

V*(xz)

rp*, YE*)

(2.18)

From the additivity property of cost (2.16) we have Y ( x z ,x f ;rI.*, pE,

TI?)

where

= Y ( x z ,x3;rp*, rE, +)

+ V ( x 3 ,x f ;r,,*, rE*, n;fE)

V ( x ’ , xf;rI,*, rE*, + ),

n

= V*(x’)

(2.19) (2.20)

From (2.17), (2.18), (2.19) and (2.20) we obtain Y ( x ’ , x’; rf,*, r E ,

+> V*(x’) Q

*(x3)

Then it follows from (2.13) and from the additivity property of cost that, along path IIY(C’), and hence

x,,’ - x: X”’

= Y ( x z ,x 3 ; rp*, rE, rg)

- xo” < V*(XZ) - V*(X?)

(2.21)

Finally, it follows from (2.21), (2.9) and (2.10) that x 3 I S not an A-point relative to the game surface through x’. Lemma 2.2 can be established by similar arguments. Lemmas 2.1 and 2.2 have the following corollaries. Corollary 2.1. All points in %* of a path nyE(C“’)whicfz emanates from x’ belong to the game surface through x’.

This corollary is a direct consequence of Lemmas 2.1 and 2.2 together with the definitions of A- and B-points relative to a game surface. Corollary 2.2. A path II$(C’) whose initial point is a B-point relative to C(C) has no A-point relative to C(C) and, of course, no point in C(C). Corollary 2.3. A path ng(C”) whose initial point is an A-point relative to C(C) has no B-point relative to C(C) and, of course, no point in C(C).

These corollaries are direct consequences of Lemmas 2.1 and 2.2, respectively, together with the translation property of game surfaces.

20 2.7

11

SOME GEOMETRIC ASPECTS OF QUANTITATIVE GAMES

UNIQUENESS OF THE VALUE OF THE GAME

Suppose there exist two optimal strategy pairs ( r l , * , rF;*)and (tp*,t E * ) ; that is, both satisfy conditions (i)-(iii) of Definition 2.2. Upon applying these conditions we arrive at once at Lemma 2.3. If and i j

XI

E

(r,>*,

then

X * , both (rl.*, rfi;*) and (FI,*, FK*) are optimal on X * , E .F(x') and

( F P * , r g * ) E F(.Y'),

V ( s ' , 8 ; r I , * ,rR;*) = V ( Y , 0;

FE*)

I n other \c~orrls,the Value of the game V *( s ' ) is independent of the choice of optimal straregy pair. We also have Corollary 2.4. If both (rl,*, r E * ) and (tl,*,Fx*) are optimal on X * , antlif(rp*, i,.*) and (F,,*, Y E * ) areplayable for allxi E X , then ( r I > *tE*) , and ( F T , * , rr:.*)are also optimal on X * .

I II

DifSerentiaI Quantitative Games

3.1 THE STATE EQUATIONS AND STRATEGIES

Thus far we have not specified the rules of the game. In this chapter we shall restrict the analysis to differential quantitative games. Let there be given the set of differential equations d.uJdt =fi.(x1,

x2,

. . . , x,

. . . , a,, v = 1,2, . . . ,n U l , u2,

01, 0 2 ,

.- .

I

Us),

(3.1)

termed state equations. Hereafter, x, = t will denote the time variable. It follows that f,(x1,

x,, . . . ,x n , U l , u2, . . . , u,,

01, u 2 ,

. . . , v,)

=1

We shall assume that state x = (sl, .uZ, . . . , x), belongs to the domain G of En, that vectors u = ( u l , u 2 , . . . , u,) and u = (ul, v2,. . . , v,) belong to the domains U and V of E'and E", respectively, and that the functionsf,, v = 1 , 2 , . . . ,I I - 1, are of class C1 on G x U x V. We shall consider strategies for players Jr, and J E to be functions of x , p : s--p(x) and e : x+e(x), x EG, respectively, belonging to prescribed classes of fu1ictions, and such that

p ( x )E K , ( X ) e(.x) E K,(x)

cu c

V

where K I L ( xand ) K,,(.u) are given subsets of E' and E", respectively, which may depend on s. These sets of strategies will be denoted by 9,and Y E respectively. ,

22

111

DIFFERENTIAL QUANTITATIVE GAMES

Of course, we can also use the notation of Chapter 1; that is,

Rp = yp, rp = p ,

R, = rF = e

The change in notation is introduced because we wish to emphasize that the sets of strategies are now sets of functions. Here we need not specify the classes of functions p and e . However, we shall require that Y,, and Y , satisfy Assumption 3.1. Whatever p‘, p” E Y;, and e’, e” E Y E ,and whatever s3E G, functions p”’: .Y 4 p’”(x) and e’”: x + e”’(x), x E G , where

p’(s)

/I”’(= .)

e’”(x) = e‘(x)

for x ,

< x,?

are strategies; that is, p”’ E Y,, and e’” E 9,. 3.2 TRANSFER TIME

Definition 3.1. In the following discussion we shall use a variable T , called transfer time, which is a function of time t , T : I + T ( t ) on an interval that depends on G, defined by two conditions; namely, dT/dt = 1 and, for xi E G, .(ti) = 0, where xni = ti.

FIG.3. I . Transfer time

T.

3.4

TARGET

23

0

3.3 RULES OF THE GAME

Equation (3.1) can be rewritten as (Ix/dT = f ( X ,

U , V)

(3.2)

For given strategies p and e , Eq. (3.2) is replaced by W d T = f ( x , p ( 4 , e(x),

(3.3)

x = x ( T ) , T E [0, T ~ ) is , a solution of Eq. (3.3), satisfying If x: T x i = x(O), then u and u are given by functions of T , u, and v; namely, ---f

u:

T + 1/

V:

T

-+

= u(T),

u = v(T), 7

E

u(T) = p(x(T)) V(T)

= e(x(-r))

(3.4)

10, T s )

There may be more than one solution of Eq. (3.3) that satisfies x(0) = x*. Of course, if p and e are C1 functions of x in a neighborhood of xi,it follows that there is a unique solution x of (3.3) satisfying x(0) = xi on an interval AT containing T = 0. But if xi is a point of discontinuity of p or e , or of both p and e , there may be more than one solution of (3.3) that emanates from xi. Furthermore, a solution that is unique in a neighborhood of the initial time may bifurcate at some later time, if the trajectory reaches a point of discontinuity of p or e , or of both. Also, there may be no solution of Eq. (3.3) in [0, 00) that satisfies x(0) = xi. The rules of the game consist of G, Y,, Y E and , mapping y defined as in Example 1.1 of Section 1.2. 3.4 TARGET 0

Assumption 3.2. Target 0 is closed and its boundary7

BAGncompO is defined by the equation 0(x) = 0

(3.5)

where function 0 is of class C' and grad 0 ( x ) # 0 on a domain containing

ae.

t Cornp 8 means the complement of 8 in G . 6 and comp 0 are the closures of 8 and comp 0, respectively, in the topology induced by En on G (see Appendix).

24

111

DIFFERENTIAL QUANTITATIVE GAMES

3.5 INTEGRAL PERFORMANCE INDEX

Now we shall introduce a functional which assigns a cost to a transfer of the state. Let fo: (s,u , v) 4 f o ( x ,u , v ) be a prescribed function of class C’ on G x U x V . I n addition to Assumption 3.1, we now introduce Assumption 3.3. The sets of strategies SP,, and Y Eare such that, for all G, for allp E Y,,,for all e E Y E and , for all paths risin G emanating from sz,generated by ( p , e), represented by x: T 4.Y = x ( T ) , T E [0, T 5 ] . the integral

s1 E

where

U(T)

= ~ ( x ( T ) )V ,( T ) = e(x(T)),

is defined.

Now let us consider a functional or performance index in integral form

T

1 ’(Y, u‘; p, e, r”) =

f0(x(7),u(T),v(T))di-

(3.6)

where Y’ = ~ ( 7 , ) . If T , = 0, then d sis a null path no, and Y-(.xz, Y’; p , e , no)= 0 ‘ifp E 9’,, Ve E 9, Hence, Assumption 2.1 is satisfied. If sSE 8, according to our earlier notation, we have T , = T~ and

T

Let us introduce a variable so and a function xo: [O, T ~ ] such , that

T

4

(3.7)

xo = x~(T),

E

(3.9) where C is any given constant. If T , # 0 we have dXu,/dT =fo(X, U , 0) for all T E (0, T ~ ) . Equations (3.2) and (3.10) constitute a set of n which we shall write in vector form dx/dT = f ( x ,

U,

v)

(3.10)

+ 1 scalar equations (3.11)

3.6

25

A VARIATIONAL EQUATION AND ITS ADJOINT

3.6 A VARIATIONAL EQUATION AND ITS ADJOINT

Next we introduce Assumption 3.4. There exists a strategy pair (p*, e*) that is optimal on a set X c G.

By (3.7), ( p * , e*) is optimal on X * X U 8. Consider a non-null optimal path II*(C) that emanates from xi E T * ,X * A X * x (x,}, is

Xf

n*u i I

I I

1

I

I I

I

I

I

I

I I

I I

I

I

FIG.3.2. Path n*(C)and its projection

X*

on G .

26

III

DIFFERENTIAL QUANTITATIVE GAMES

generated by ( p * , e*), is given by x * : T + x = x * ( T ) on [0,T ~ ] ,and reaches 0 2 0 x {x,,} at xf = x * ( T ~ ) . In other words, x * is a solution of d x / h = f(x,

(3.12)

p * ( x ) , e*(x))

Let v* be the projection of rI*(C) on G. Definition 3.2. If there exists a neighborhood of T * on which p* and e*, are of class C1, we shall associate with Eq. (3.12) the variational equation -= (17~ (

I

~

(af -+--+-af ap*

ax

all ax

afa.') aU ax

1

r

(3.13)

X=X*(T)

where v = 0,1,. cc = 0, 1 , .

ax

. . ,n ..,n

v = 0 , 1 , . . . ,n v . = 1,2, . . . , r v = 0,1, . . . , n a = 1 , 2) . . . )7.

av evaluated for u

= p*(x), u = e s ( x ) ;

ap. n ax -

ax

3P"*(X)

[

ax,

1

and v = 1,2, . . . , r v. = 0, 1 , . . . , I2 v = 1,2, . . . , s a = 0, 1, . . . , ri

For given initial condition q = ri at T = 0, the solution q: = T ( T ) of Eq. (3.13) is unique and continuous on [0, T ~ ] . Furthermore, ?(T), T E [O, Tp], is nonzero provided q i is nonzero. T + yi

3.6

27

A VARIATIONAL EQUATION AND ITS ADJOINT

Definition 3.3. The equation adjoint to Eq. (3.13) is (3.14)

where

For given initial condition 1. = 2 at T = 0, the solution A: T+ 3, = A ( T ) of Eq. (3.14) is unique and continuous on [0, Furthermore, A(T) is nonzero provided Az is nonzero. Equations (3.13) and (3.14) can be developed as follows:

T ~ ] .

and

v = 0 , 1 , 2 ) . . . ,n

It follows from (3.13) and (3.14) that (d/dT)(A(T)

so that

A(T)

-

Y-J(T) =

If the initial vectors 2

-

r1(T))

constant

= A(0)

p.qizo

=0

V T E [0, 7-53

(3.17)

and qi = q(0) are such that with

))3bi/)))$)I

f. 0

that is, if lbiis perpendicular to qi,then it follows from (3.17) that A(T) * q ( T )

=0

VT E

[o, 751

(3.18)

and since IIA(T)II llq(~)ll# 0, A(T) is perpendicular to ?(T) for all 7- E [O, 751.

28

111

DIFFERENTIAL QUANTITATIVE GAMES

Definition 3.4. Equation (3.13) defines a linear transformation .d(O,T ) of 7 j i such that ?(TI = d @ > T ) 7 ' (3.19)

Since Eq. (3.13) is linear and homogeneous, this transformation is nonsingular; that is, the inverse transformation ,d-'(O, T ) such that 7' = . d - l ( O , i5

detincd for all

T

E

T)?(T)

[O, T r ] .

F K . 3 . 3 . Transform of plane P(x*(O))due to . d ( O ,

7)

Next, let P ( x ' l ( 0 ) ) be an n-dimensional plane containing point x*(O) of T l i (C), and let P ( x * ( T ) )be its transform due to the linear transformat i o n d ( 0 , T); namely, f(X*(T))

== c d ( 0 , T ) P ( X * ( O ) ) = { x * ( T )

+ riz ~ P ( x " ( 0 ) ) ;

+d(0,

T)?7':

X*(o)

It follow\ from (3.18) that, if I' IS perpendicular to P ( x F ( 0 ) ) then , X(T) perpendicular to P ( x * ( T ) )for all T E [o, T r ]

ic

3.7

29

JOINING OF PATHS

3.7 JOINING OF PATHS

We shall now prove that Assumption 1.1 is satisfied. Consider a trajectory rij in G that emanates from xi, is generated by , ends at point ( p , e ) , is given by x: T + x = X ( T ) on [0, T ~ ] and xj = X ( T j ) .

Consider also describing curve Pp(xj,fi, 6) that emanates from xj and is given by S E y ( x j , $, 6). Of course, x ( T ~= ) S(0) = xj. If P, or rn(xi, 6,S), or both, reduces to a single point, Assumption 1.1 is trivially satisfied. Accordingly, let us suppose that neither rii nor r t ( x j , j , 6) reduces to a single point. Then, x is a solution of

dxld7 with initial condition x(0)

=f(.y,

= xi;

&/dT

(3.20)

p(x>,e(x>>

and S is a maximal solution of

=f(X,

(3.21)

f i ( x ) , e"(X))

with initial condition S(0) = xj. Now consider the (n-1)-dimensional plane P ( x i ) in En which contains point xi and is perpendicular to the time axis, namely the plane whose equation is x,, = xni. This plane separates E n into two half-spaces, the closed half-space D, {x: x, Q }x: and the open half-space D2 & {x:

x,

> X,j}.

From Assumption 3.1 it follows that functions e": x + e"(x), x E G, where

P(.4

=y

(4

e"(x) = e(x)

1

I

P(x) = fi(x) f?(.Y)

= @(X)

p:

for

XED, nG

for

,YE

x +P(x)

and

D, nG

are strategies. Moreover, if the players employ the strategy pair (jj,e"), the state equation becomes (3.22) d X / d T = f ( 9 , F(X), e"(x)) There exists a maximal solution 2:

T +x

= 2 ( ~of) Eq. (3.22) which

30

111

DIFFERENTIAL QUANTITATIVE GAMES

satisfies the initial condition %(O) = x i , namely the solution 2 such that a(T)

= X(T)

%(T)

=f (~T

~ )

for

T

E

for

T

E [ T ~ T~ ,

[0, T

~ ]

+

T+]

where T+ is the extreme future value of T associated with solution f. Solution 2 generates describing curve r Z ( x i p", , e"), and we have q x i ,

Furthermore, and

p, e") = rij u i ~ ~ p( , ~s)j , p=p*p=p e^= e a Z = e

Since these arguments apply whatever the trajectory and the describing curve I'%(xj,6,s), Assumption 1.1 is satisfied. One can readily verify that Assumption 2.2 is also satisfied. Hence, Lemmas 2.1 and 2.2 apply to differential games for which Assumption 3.1 is satisfied. 3.8 TWO FAMILIES OF PATHS

Again consider a non-null optimal path n*(C) that emanates from EF*, is generated by ('*, e*), and reaches @ at x f . Let T* be its projection on G. Next we introduce xi

Assumption 3.5. All points of T * , with the possible exception of its terminal point. are interior points of X * . Assumption 3.6. For all x t T * , and for all u E K,(x) and u E K,(x), there exist strategies cx, t 9,and @, E Y Ewhich are of class C' on some neighborhood of x in G, and which satisfy the conditions cx,(x) = u and D,(x) = 21, respectively. Let x 3 2 (xo3,x3)be any point of lJ*(C), different from terminal point x f . From Assumption 3.5 it follows that x3 is an interior point of %*. Let us also consider paths rI$(C') and n??(C"), which emanate from

3.8

31

TWO FAMILIES OF PATHS

x 3 and belonz to %*; namely,

II:(C')

{ x k : xgk

+ V ( x k ,x2;aZlre*, T:) = C', &; c (3.23) + V ( x k ,x"; p * , T:?) = C", T;? c n ; F } (3.24) T$}

IIi?(C'') A { x ' : X: where aZ9and BZ1are strategies satisfying Assumption 3.6 at point x3 E T * . Let BZ9,

a,,(x3) = t l b

E Ku(x3)

= ub

E Kv(X3)

PZ,(XJ)

where ub and ub are given vectors. According to Corollary 2.1, n*(C)belongs to the game surface through x ' ; hence it follows from the definitions of a game surface and of an optimal path that the game surface through x z is C ( C ) .

J

F"

FIG.3.4. Paths IIg(C') and nim(C").

32

DIFFERENTIAL QUANTITATIVE GAMES

111

Consequently, x’ E

X(C)

Now it followc from Lemmas 2.1 and 2.2 that ( I ) no point of I [$(C’)is a B-point relative to C(C); and ( 1 1 ) no point of I I ~ ~ z ( C is”an ) A-point relative to C(C). Finally, since II$(C’) and ng(C”)belong to X * , we have N

y X ( C ) A ( A / X ( C ) )U X(C)

(3.25)

N

Bz(c)

(3.26)

11 ;Y(C”), let us return t o the definition of game surface C(C) and recall its equation q x )

&

so

+ V * ( s ) = c,

sE

x*

(3.27)

Then it follows from the definition of A / X ( C ) and B / Z ( C ) that (i) a t every point x of A/C(C), q x >

>c

and (ii) at every point x of B/C(C),

+

@(x)

+

0, on which the functions E and 5 are continuous. Then, as a consequence of Assumption 3.8 we have PROPERTY 3.2. Every point x of game surface C(C), in a neighborhood in ?Z“* of x j , can be represented by x =

xj

+ €719 + a(.)

where lime+, [\lo(.)ll/c] = 0 and qj belongs to an n-dimensional plane, the tangent plane of the surface at x j . Definition 3.5. If a point x j of C(C) possesses a neighborhood that belongs to X * , it will be called an inferior point of C(C). A point x j of C(C) at which Property 3.2 holds will be called a regular point of C(C). An optimal path ll*(C) all of whose points, with the possible exception of the terminal one, are regular interior points of C(C) will be called a regular optimal path.

t One can verify that Assumption n*

is not tangent to 0.

3.8 is a consequence of Assumptions 3.5 and 3.7 if

34

111

DIFFERENTIAL QUANTITATIVE GAMES

Note that Assumption 3.8 is stronger than the assumption of regularity ; of course, according to Assumption 3.8 together with (3.27), the partial derivatives

. .-a v * f X ) ax, ax, are defined and continuous, and hence

a~.v *(x)

(3.30) is defined and continuous at every point x ’ , x 3 4 0 , of II*(C). However, at a regular point of C(C), some of the partial derivatives of V*(x)may be not defined, since the tangent plane of Z(C) at that point may be parallel to the x,-axis. Now let us return to path n$(C’), defined in Section 3.8, that emanates from x’ E n*(C), x3 4 0 , and is generated by strategy pair (a,,, e*). We shall assume that IIg(C’) is not a null path.? It follows from Property 3.1 that /lbExll -0

as

7-0

Since x 3 E C(C), and accordingly @ ( x 3 ) = C , it follows from (3.28) and from Assumption 3.8 that grad @ ( x 3 ) . d K x where

Upon dividing (3.31) by

T,

+ o(l16Exll) 2 0

which is positive for a K x # 0, we get

7

Then allowing

T + 0,

r

and consequently

(B”x/T)

we obtain

(3.31)

+

f ( x j , u’, e*(x?))

grad < P ( x 3 ) . f ( x 3 , ub, e*(x3))2 0

(3.32)

for all u” t KIC(x3), and for all x 3 E II*(C) with the possible exception of the terminal point of II*(C). Such non-null paths exist according to Assumptions 3.6 and 3.7.

3.9

35

REGULAR INTERIOR POINTS OF A GAME SURFACE

il XO

I ‘

\

FIG.3.5. Regular interior points of a game surface.

By invoking similar arguments for a non-null path IlF(C‘’) that emanates from point xi E ll*(C), xi $ 0 , and is generated by strategy pair ( p * , B,j), we obtain grad @(xj) . f ( x i , p * ( x j ) ,ub) Q 0

(3.33)

for all ub E &(xi), and for all xi E rI*(C) with the possible exception of the terminal point of II*(C). Moreover, since n*(C)c E(C),we have grad @(xi) * f(xj,p*(xi), e*(xi)) = 0

(3.34)

for all xi which belong to H*(C), with the possible exception of the terminal point of II*(C). Let n(xj) A grad @(xj), xi 4 0 It follows from (3.30) that no(xj) = 1

(3.35)

where no(xi) is the x,-component of n(xi). Note that n(xi) is normal to the

36

111

DIFFERENTIAL QUANTITATIVE GAMES

game surface X(C) and directed into region A / X ( C ) at point xi of X ( C ) ; that is, there exists a positive number 0 such that point xi myxi) belongs to A / X ( C )for all 6 on ( 0 , 0).

+

3.10 TRANSFORMATION OF A TANGENT PLANE

Again consider regular optimal path 1 I" ( C ) ,represented by x " : 7

T E

+X*(7),

[o, 751.

Let x i = ~ " ( 0and ) xj 2 x * ( T ? ) , T~ E [0, T~). Now let us consider a neighborhood in C(C) of point x i , denoted by A ( x i ) , such that A(X')

where

A

x(0)

and 1 . lim c +0

2.

yli

[y] =

x = x(0) E C(C)}

{x:

L x*(0)

+ Eq7 +

(3.36)

O(E)

0

E P,(x*(O)), the tangent plane of

A(xz), and hence of C ( C ) , at

x"(0).

We shall be interested in a transformation of A(x2) by means of state equation (3.12), that is in a(Xj) = {X:

X

=

X(7j),

Tj E

[o, 71))

where x : T -+x(7), 7 E [O, Tj]represents a trajectory riiwhose supporting curve is a solution of (3.12) with initial condition given by (3.36). It follows from Assumption 3.7 that, to each initial point x(0) E A ( x i ) ,

there corresponds a unique trajectory rij provided that I E ~ is sufficiently small. Moreover, since T? E [0, T ~ ) x, * ( T ) does not belong to 0 for all T E [o, Tj]. Since 0 is closed, X ( T ) does not belong to 0 for all T E [O, ~ $ 1 , which implies that trajectory rij is a path, no matter what the initial point is sufficiently small. x(0) E A(xi), provided that From the dependence on initial conditions of the solution x : T + x ( T ) , T E [0, T 3 ] ,it follows that X(7j)

=

X*(Tj)

fEqi

+ O(T,,

€)

where ~ / o ( TE ) ~I I /,~ tends to zero uniformly for all 7j as E -+ 0, and $ 4 ?(T~)where q: 7 q = ?(T) is the solution of variational equation ---f

3.1 1

37

REGULAR OPTIMAL PATHS

(3.13) on [0, T r ] , with initial condition ~ ( 0=) q i ; that is, rl(Tj)

=

d ( 0 , Tj)yli

It follows from Corollary 2.1 that X(Tj) = X*(Tj)

+ €7' +

E

ff(Tj, €)

where I ~ O ( T ~ , e)II/c tends to zero uniformly for all Since (3.37) is satisfied for all x(0) E A(xi), A(xj)

c(c)

T~

as

E +

(3.37)

0.

= C(C)

Since rli E P,.(x*(O)), and since d ( 0 , T ~ is) a linear nonsingular transformation, it follows that q ( ~ ?belongs ) to a n n-dimensional plane which contains point x*(T~).In view of (3.37) this plane is the tangent plane of C ( C ) ,P..(x*(T?)), at x * ( T ~ ) . Furthermore, the orientation of P c ( ~ * ( ~ j ) ) varies continuously with T ~ . 3.11 REGULAR OPTIMAL PATHS

Now consider the solution A: T + 1. = h ( ~ )T ,E [0, T j ] , of adjoint equation (3.14) subject to the initial condition A(0) = li. If we choose li perpendicular to the tangent plane of C(C) at point x*(O), that is, if A2 has the same supporting line as n(x*(O)), then, as a consequence of (3.18) together with the conclusion of Section 3.10, A(T,) is perpendicular to the tangent plane of C(C) at point x * ( T ~ ) ; namely, its supporting line is that of n(x*(Ti)). I t follows from Assumption 3.8, together with the definition of (D and (3.35), that p: T~ + p ( ~ ~where ), p ( ~= ~ n(x*(Tj)), ) is a nonzero continuous vector function of T~ on [o, T j ) . Furthermore, A : T~ + A ( T ~is) a nonzero continuous vector function of T ? on [0, T ~ ] .Hence, if we choose il" in the same direction as n(x*(O)), A ( T ~is) codirectional with n(x*(Tj)) for all T~ E [0, T ~ ) . These properties of A ( T ~ ) together , with (3.32), (3.33) and (3.34), result in

-

>0 f(X',p*(Xj), U b ) < 0

A ( T ~ )f(xj, ub, e*(xi)) h(Tj) *

A ( T ~* )f(xj, p*(x'), e*(sj)) VTj E

[o, T r ) ,

=

I

(3.38)

where

xj = x * ( T ~ ) (3.39)

0

VllbE

K,,(.X'),

(3.40) VVbE K?>(.Y')

38

DIFFERENTIAL QUANTITATIVE GAMES

111

Let x * : T + x = x*(T), T E [O, T f ] , denote the function that represents T*.? Since f is of class C1 on G x U x V , and p* and e* are of class C1 on a neighborhood of T*T, and x * : T ~ - + x * ( T ~ as ) well as A : T~+A(T~) are continuous functions of T~ on [o, 7 5 1 , it follows that

H: where H(Tj)

T~ -+

H(T~)

.

= A ( 7 j ) f(x*(Tj),p*(x*(Tj)),

e*(x*(Tj>))

is a continuous function of T~ on [O, T f ] . Hence (3.40) holds at T~ = T~ and xi = xf. One can prove that (3.38) and (3.39) also hold at T~ = T ~ .From (3.38) and (3.39) it follows that

A(Tj) .f(X*(Tj), %f(x*(Tj)>, e*(x*(Tj)))

>0 * f(x*(Tj), p*(x*(~j)),P z f ( ~ * ( ~ j ) ) )

(3.41) (3.42)

and PZf are strategies that satisfy Assumption 3.6 at point with M,f(xf) = ub and P=t(xf) = vb. Then allowing T~ T,, we obtain by continuity

where

~ , f

x f = x*(T,),

---f

-

>0 v b )< 0

A(T,) f(x*(Tf), ub, e*(x*(T,>)) x(Tf)

f(x*(Tf), p*(x*(Tf)),

Clearly, these inequalities are satisfied for all ub E K , ( x f ) and all vb E K,(xf). Hence, conditions (3.38), (3.39) and (3.40) hold for all T~ E [0, T f ] , for all uh E K,(.uj) and for all v b E K , ( x i ) . These relations can be rewritten as

-

min A(T) f(x,

I I E K,,( J )

11,

e * ( x ) ) = max A(T) f(x, p*(x), u )

-

h-,>( s)

i>E

-

= A(T) f(x, p*(x), e*(x)),

A ( T ) f(x, p*(.u), e*(x))

=0

V T E [0, T f ]

(3.43) (3.44)

where x = x * ( T ) and x = x*(T). Since the functionsf,,, Y = 0, 1, , . . , IZ, and the strategies p* and e* do not contain xg in their arguments, the right-hand side of the adjoint equation which is associated with the zeroth component of R is zero. It

7 Recall that

m* is

the projection of II*(C) on X *

3.12

39

TRANSVERSALITY CONDITION

follows that dAo/dT = 0, and hence A,(T) = constant (3.45) on [O, Tf). Furthermore, since 2 is codirectional with n ( x i ) , and since no(xi) = 1, A,(T)

on [0, T ~ ) . Since A: holds for all T E [0, T,].

T+

A

= constant > 0 = A(T)

(3.46)

is continuous on [0, T j ] , (3.46)

3.12 TRANSVERSALITY CONDITION

We shall now investigate conditions which must hold at the point where optimal path n*(C) reaches 0. The intersection of game surface C(C) with 0 is the set

0 n C(C) = {x: @ ( x ) = C,

x E O}

(3.47)

Because of (3.7), the condition x E O implies that V * ( x ) = 0 which, in turn, implies that @(x) A xo V*(x) = Xo Hence, 0 n C(C) = { x : x0 = C, x E O} (3.48)

+

That is, the intersection of C(C) with 0 coincides with the intersection of 0 and the plane Po, perpendicular to x,-axis, defined by xo = C. In other words, 0 n C(C) may be deduced from O by translation parallel to the xo-axis. Now let us consider again the optimal path n *(C), represented by x*: T X * ( T ) , T E [0, T ~ ]that , reaches 0 n C(C) at terminal point x f . Since A(T) is codirectional with grad @(x*(T))for all T E [0, T ~ )and , since A(T) and grad (D(x*(T)) possess zeroth components A,(T) and 1 , respectively, A(T) = A,(T) grad (D(x*(T)) VT E [0,T ~ ) ---f

Since the adjoint equation is homogeneous of degree one in A, and since A,(T) = constant > 0, we may set A,(T) = 1. Hence, A(T) = grad @ ( x * ( T ) )

VT E [0,7,)

40

I11

DIFFERENTIAL QUANTITATIVE GAMES

FIG.3.6. Transversality condition

Finally, since h ( ~is) continuous on [0,T/],grad @ ( x * ( T ) ) tends to a limit 17' as T -> T , , and h(T,)

=

Thus X(C) possesses a well defined tangent plane, P , ( x f ) , at point x f , namely the plane through x f which is perpendicular to izz.? Since, according to Assumption 3.2, 80 possesses a unique ( n - 1)-dimensional tangent plane, P,,(.u), at every point s of 80, so that the tangent plane, PXn c, (x),of 8(4 n C(C) is defined for all x E 8 0 n C(C), Pl;,,(xf)

= P,(x')

Since A('/) = d ,A(T,) is normal to P , ( x f ) , and hence to P x n e , ( x f ) . This orthogonality condition is called the tvansuevsa/ifj>condition. It This i s a consequence of the uniqueness of optimal path 11* ( C )

3.13

41

A M I N - M A X P R I N C I P L E , KEGULAR CASE

can be expressed as follows. Let 7 = (q0, q,, P , , , ( x f ) ; then A(7,j . q = 0

. . . , q n ) be a vector in (3.49)

Since xo = C and .Y E P,(xf) for points of PXnc-,(xf),and since a0 is defined by the equation B(x) = 0, it follows that 70

=0

(3.50)

3.13 A MIN-MAX PRINCIPLE, REGULAR CASE

Summarizing the results of the preceding sections, we arrive at a theorem of fundamental importance for the discussion of optimal strategies in differential quantitative games. Letting

*%?(Ib, x, u , u ) we have

A A. f(x, u , u )

Theorem 3.1. I f n*(C) is an optimal path, generated by strategies p* and e*, represented by x * : r + x = x * ( T ) , T E [0, r r ] , rf # 0 , and satisfying Assumptions 3.5-3.8, then there exists a nonzero continuous solution A: T A = A@), T E [0, rf],of adjoint equation (3.14) such that

-

8EKl'(X*(T))

- max ~ ( V Tx*(T), ) , p * ( x * ( ~ )u) ), L

=

(b)

ex,(x* ( r )

)

~ ( A ( T ) ,x*(T>,

p*(x*(T>>, e*(x*(T)>)

for all

T

E

[0, T p ]

-@(WT),x*(T), P*(X*(T)),e*(x*(T)))= 0

where x*(T) is the-projection of x * ( T ) on X * . Furthermore, if Assumption 3.2 is satis-ed, then vector (A,(T), & ( T ) , . . . , A,(T)) is normal to the terminal (d) manifold boundary 8 at T = T ~ .

42

111

DIFFERENTIAL QUANTITATIVE GAMES

Note that state equation (3.12) and adjoint equation (3.14) may be written as c_ i ~, a,n(i,, X,

21, V)

a,.

dT

nr., - - _-

a,x(J, x,

ax,

(IT

-2

7-1

u, u)

aM(lL, x, u , u ) +,*(x) au,

3%

where u = p * ( x ) , ZI = e*(x), x = x * ( T ) and = A ( T ) for T E [ 0 , T f ] . By using (3.49) in (a) and (b) of Theorem 3.1 we obtain the functional equation of dynamic programming:

3.14 DISCONTINUITY MANIFOLDS

Now let us suppose that X is domain, and consider Definition 3.6. We shall say that {XI, X,, composition of X , if

. . . , X K } constitutes a de-

(i) X,, CJ = I , 2 , . . . , K < a,is a domain of E n , and (ii) X , n X , = o k # I CJ= 1,2, . . . , K (iii) X , G X (iv) X c

xv c G ;

R

U X,

8=il

As a direct consequence of these conditions,

x = Uf=, x,.

Next we introduce an assumption that is weaker than Assumption 3.7, namely,

3.15

REGULAR PORTION OF AN OPTIMAL PATH

43

Assumption 3.9. X is a domain possessing a decomposition {XI, X,, . . . , X,) such that functions p', e* agree on each X , with functions-say p_",e*-of class C1 on a domain R, 3 F,, and on each nonempty Mkl A x k f3 k # 1, with functions-say p k l , ekl-of class C1 on a domain

xL,

RkL

Mkl*

FIG. 3.7. A decomposition of X ; discontinuity manifolds in En.

Furthermore, we shall make Assumption 3.10. A discontinuity manifold M,, is an (n-1)-dimensional surface which can be represented by m(x) = 0

where function m is of class C1and grad m(x) # 0 on a domain containing MkP

3.15 REGULAR PORTION OF AN OPTIMAL PATH

Definition 3.7. where

{C,(C), C,(C), . . . , C,(C)) is a decomposition of C(C),

+

6 = 1 , 2 , . . . ,K C,(C) A (x: O(X) & xo V*(x) = c, x E X,} And now we make an assumption that is weaker than Assumption 3.8, namely

44

Ill

DIFFERENTIAL QUANTITATIVE GAMES

Assumption 3.11. Function V* is continuous on X and agrees on each X,, 0 = 1 , 2 , . . . , K , with a function-say V'-twice continuously differentiable on R,. As a consequence of Assumption 3.1 1,

(3.53)

is defined for all x €3,& Xu x ( x , ] , and all points of C,(C), 0 = 1 , 2 , . . . , K, are regular interior points of Xo(C). Here we shall relax the assumptions of Theorem 3.1 ; that is, we shall be concerned with an optimal path n*(C), generated by strategy pair ( p * , e*), satisfying Assumptions 3.5 and 3.6, and we shall derive conditions (a)-(c) of Theorem 3.1 for points of a regular portion of n*(C), namely n * ( C ) n X,(C). By the same arguments as those employed in Section 3.8, it follows from Lemmas 2.1 and 2.2, and Corollary 2.1, that grad @(x) . f(x, p*(x), v ) Q 0

(3.54)

grad cD(x). f(x, u , e * ( x ) ) 2 0

(3.55)

grad @(x)

*

f(x, p * ( x ) , e * ( x ) ) = 0

(3.56)

for all x t II* ( C ) n X,(C), for all u E K , ( x ) and for all u E K,(x). Since ( p * , e*) is optimal on X,(3.56) holds for all x E C,(C), O =

1 , 2, . . . , K ,

and for all C ; that is, (3.56) is an identity in x on X u ,5 = 1 , 2 , . . . , K. Let us rewrite (3.56) as (3.57)

By differentiating (3.57) with respect to sa, p = 0, 1, . . . ,n, we obtain

3.15

45

REGULAR PORTION O F AN OPTIMAL PATH

Now, along a portion of optimal path n*(C)lying on C,(C), we have

From (3.58) and (3.59) we obtain

By developing form, we obtain

for x = x * ( T ) Letting

a),(.,

p * ( x ) , e*(x))/a.ua, and writing (3.60) in vector

E X,(C).

A ( T ) = grad @ ( x ) ,

x

= x*(T) E

&(C)

(3.62)

and substituting in (3.54), (3.55) and (3.56), we obtain

0 such that x * ( T ) E X,(C)

for all

T E

x * ( T ) E C,(C)

for all

T

(T, -

E (T,,

E , T,);

and

+ c);

and

T,

Ak, LA

grad m(xc) - f ( x C pk(xc)), , ek(xc))# 0 grad m(xc). f ( x C ,pz(xc),ez(xc))# 0 We shall now investigate conditions which must hold at a point xc = x*(T,) where optimal path Il*(C) crosses Akz.

Since p * , e* agree on X , with functions p k , ek, respectively, which are of class C' on a domain R , 3 and since A : T + il = A(T),

xk,

7-

IS

a solution of

E

(Tr

-

€2

7-A

A ( T ) tends to a limit A(T, - 0) as T + T,, T < 7,. Since A ( T ) = grad @ ( x * ( T ) ) for all T E (T, - E , T , ) , limr+Te,T0 ~ .

(3.123)

64

DIFFERENTIAL QUANTITATIVE GAMES

111

Condition (3.123) has two consequences. First, it follows from (3.1 17), (3.120), and (3.123) that (Jxz*(Tf) -

Lt.)Al(Tf)

>0

Hence, the subset of the target that can be reached by a path T * , that is the useable part of the target, must satisfy the condition A-2

> bt.2

(3.124)

Secondly, if there exist a strategy pair ( p * , e*) that satisfies the necessary conditions for optimality and a path X * that reaches 0 at point x * ( T ~ ) , then, in view of the continuity of h along every non-null portion of n* that does not intersect a discontinuity manifold, there exists an interval of time ( T ~- T,, T ~ ] , T, > 0, on which A,(T) and A,(T) A,(T) are strictly positive. So, let us consider Case (i) first; that is, let T E ( T ~ T,, Tf], and accordingly V ( T ) = 1 in Eq. (3.1 19). Equation (3.120) becomes

+

+

For

T

=T

1 - J X , * ( T ) JAi2(T) A;(T) we deduce from (3.125) that

+ M’hl(T)= 0

(3.125)

~ ,

A,(Tf) =

fJX,*(Tf)

- bL’Ip1

(3.126)

and so by the first of Eqs. (3.1 16), x,(T)

= constant = [Jx,*(Tr)

- wl-1

(3.127)

By substituting expression (3.127) for A,(T) in Eq. (3.125) one readily obtains (3.128) Equation (3.128) requires that (3.129) which implies the choice of the negative root in (3.128). For, if we choose the positive root, then dx,/cf.r < 0 in the second of Eqs. (3.121), which implies that x ~ * ( Tis ) a decreasing function Of T ; this, in turn, implies that x,*(T)

>x~+(T~)

for

T


0 , vuE x , ( X ) ;

where V * ( X )= V,*(X) yx E Mk. Consider a path rifthat is generated by a well-behaved strategy pair

( p , e). Since ( p , e ) is well-behaved, interval [0, -r5] contains a finite or at most a countable number of open intervals (0, T ~ ) (, T ~T,~ )., . . , ( T + ~ , = T ~ such ) that X ( T ) belongs either to an X , during any one such open interval. Thus, integral T~

I(x2)

'J

If

grad V * ( x ) . dx

E

D or to an M ,

cM

(3.138)

is composed of a finite or at most countable number of integrals; namely, by (i) and (ii) of the theorem, ilgrad V * ( x ) . d x

+ t

IT12

grad V *(x)

*

dx

+

*

..

This condition is implied by (b) and (c).

+I

+l.f

grad V * ( x ) . dx (3.139)

3.22

69

SUFFICIENCY CONDITIONS

where X(T) E nil c X(T) E

2’c

ndf

for

T E

nif

for

T

X(T) E niV--lvfc nif

(0, T J

E ( T ~ T, ~ )

for

E( T ~ - ~ ,

and where, according to (iii) of the theorem, V * ( x ) = V,*(x) if x E Mk. Then it follows from (i) and (iv) of the theorem that Z(Xi)

= -V*(Xi)

(3.140)

Consequently, integral Z(xi) is invariant with the choice of ( p , e). In view of this conclusion, together with condition (a) of the theorem and state equations (3.1), we have

Next let us consider a well-behaved strategy pair ( p * , e) generating a P-optimal path m$ represented by x: T + x = x(T), T E [0,T f ] , x(0) = x i q and form the cost difference

Jo

Now add the following null-term to the right-hand side of (3.142):

J:,*fgrad V * ( x ) . dx + V*(x*j

Since

J KP

if

(3.143)

grad V * ( x ) . d x =Jorfgrad V * ( X ( T ) )- ~ ( x ( T )~*(x(T)), , e(x(7)))

dT (3.144)

70

111

it follows that

A=

I”

[f( 0 X ( T h P*(X(T>)?

DIFFERENTIAL QUANTITATIVE GAMES

e(x(.)>>

+

(3.145) grad V*(X(T))- f ( x ( ~p*(x(~)>, ), e(x(~)))ldT Now, either T$ is a null path so that A = 0; or T$ is a non-null path so that X ( T ) E X - 19 for all T E [0, T ~ and, ) as a consequence of condition (b) of the theorem,

AGO

(3.146)

Similarly, by considering an E-optimal path and invoking (c) of the theorem, one can show that A 0 for a well-behaved ( p , e*). This concludes the proof of Theorem 3.3. It should be noted that Isaacs’ “Verification Theorem” (Ref. 16) constitutes a special case of Theorem 3.3. We shall leave it to the reader to verify that the strategies deduced in Examples 3.1-3.3 satisfy the conditions of Theorem 3.3 and hence are optimal in the sense of the theorem.

IV

Multistage Quantitative Games

4.1 PROBLEM STATEMENT

In this chapter we shall consider a multistage quantitative game, that is a quantitative game whose state is characterized by n - 1 real numbers xi, x z , . . . , x,-,, and by the stage k of the process, where k E (0, 1 , . . . ,K } It is convenient to think of the state of the game as a point

x

n =

( x l , xz, . . . , x,)

E

En

where x, E (0, 1 , . . . , K } . For x = xk A (xl, xz,. . . , xn-,, k ) we say that the state is at the kth stage. The state changes, that is, x may take on different values in En,as x, varies from 0 to K. The evolution of the state is governed by a set of difference equations; in particular, we shall suppose that the change of xk to xk+l depends on x k and some parameters, the control variables u,, uz, . . . , u, and u,, u 2 , . . . , us, which the players J , and J,, respectively, choose at each stage of play. Let u A (u,, u2, . . . , us)and u A (ul, u 2 , . . . , us). We shall assume that u E U and u E V , where U and V are fixed domains of E' and Es, respectively. With each change or transfer of the state we shall associate a real number, the cost. It is our purpose to discuss some geometric aspects of optimal play, and we shall illustrate some consequences of geometric properties by presenting a derivation of necessary conditions for a restricted class of problems. We shall assume, without loss of generality, that the multistage game which we are discussing now is the discrete version of a differential game which has been quantized, and that the stage k is the time variable. Of course, all the results will hold if k is not the time.

72

IV

2

1

0

I.:

k

MULTISTAGE QUANTITATIVE GAMES

K - l

,

FIG.4.1. The state space and the sets E,, E l , . . . ,Ex.

Since x, takes on integer values on [0, K ] , we shall consider

Ek

{x:

(4.1)

x, = k }

Let 9 be a fixed domain of E n , such that Gk 9 n Ek # o for k = 0 , 1 , ..., K . In what follows we shall assume that state x belongs to the set G 4 Uk Gk, k E (031,. - . K } . 3

4.2 STATE EQUATIONS, STRATEGIES AND TARGET

e

We shall consider the state of the game as a function of stage k ; namely on { i , i + 1, . . . ,j } , O < i < j < K , x(k)= ( x , ( k ) , . . . , X ~ - ~ ( ~ Ck). ) , If i # j we shall require that

x: k - + x k = x ( k )

x(k for k = i, i

+ 1) - x(k) = f k ( x ( k ) ,U W , v(k))

+ 1 , . . . ,j

- 1 , with u and v being functions of k ,

U: k + u = u(k),

V:

k-tu =~ ( k )

(4.2)

4.2

STATE EQUATIONS, STRATEGIES AND TARGET

0

1

2

k

73

8

K-1

K

x FIG.4.2. The sets X , and G,; 8 = X,.

where k belongs to {i, i + 1,

. . . ,j

- l}, and

We shall assume that the functions f:, Y = 1 , 2 , . . . , n - 1 , are of class C' on .9 x U x V , for k = 0, 1 , . . . , K - 1. Of course, since x,,, = k , we havef,,(x, u, u ) = 1. Let there be a fixed domain R G 9 such that X , 4R n E, # 0 for k = 0, 1 , . . . ,K. Let X A U , X , , k ~ ( 01 , . . . ,K - 1). We shall consider strategies to be functions of x, p : x - p ( x ) and e : x + e ( x ) , x E X , belonging to prescribed classes, such that

where K,(x) and.K,(x) are given subsets of (Iand V , respectively, which may depend on x. We shall be interested in transferring the state of the game from an initial state x ( i ) = xi to any one in prescribed set of states 8 2 X , by means of a strategy pair ( p , e ) , where u(k) = p ( x ( k ) ) and v(k) = e ( x ( k ) ) .

74 4.3

IV

MULTISTAGE QUANTITATIVE GAMES

DESCRlBlNG CURVES AND PATHS IN STATE SPACE

For given strategy pair ( p , e) and an initial state x(i) = xi in G, Eq. (4.2) defines a unique sequence of states

+

r = ( ~ ( i )x(i , + I), . . . , x(i + d ) ]

+

where i c/ is the largest value of stage k for which x(i d ) is defined. Of course, i cl K. if x(i) = x i does not belong to A ', or if i = K , or both, x(i 1 ) is not defined; that is, d = 0 and 1' reduces to the single point x i . We shallcall function x: k x7, = x(k) E r, k E {i, i I , . . . ,i d } the maximalsolution of Eq. (4.2) for given initial state x i and given strategy pair ( p , e). Of course, the value of d is defined by x i , p and e. For given xi and strategy pair ( p , e), there is a unique maximal solution. Replacing k by the new parameter T = k - i, the maximal solution x becomes 31: X k = x(T), T E (0,1 , . . . , d }

+
C} @(x) < C }

(4.14)

A/C,(C) = (x = (xo,x):

x E X,,

B/C,(C) = {x = (xo, x): x E X , ,

aqx)

(4.15)

As the value of parameter C is varied, Eq. (4.12) defines a one-parameter family of surfaces, namely {X,(C)}.

78

MULTISTAGE QUANTITATIVE GAMES

IV

Definition 4.3. For a given C, we shall say that xk E C,(C) is an interior point of C,(C), if there exists an n-dimensional ball A(xk) with center xk such that A(xL) = Fk.

Since R is open in E”, X , 2 R n Ek is open in E, and%, & X , x {xo> is open in E, x {x,}. Hence, every point of Z,(C) is an interior point of Xk(C),k = 0 , 1,. . . ,K.

N o w consider non-null optimal path lI*(C) which starts at point

X* 2 x* x

x2 = (so, xi) €$*,

is generated by ( p * , e * ) , and is given by x*: k

+ x’

= x*(k),

k

E

{i, i

{x,},

+ 1 , . . . ,K } ,

x * = (x,*, x*) = (x,*, XI*, . . . , x,*).

We shall prove first that x * ( j ) E%*

V j E {i, i

+ 1 , . . . ,K }

(4.16)

Indeed, by the definition of an optimal path, n*(C) reaches 0 at stage K , and hence x * ( K ) E 0 c %*. If we suppose that x * ( j ) 4 X* for j < K , then p * ( x * ( j ) ) , e*(x*(j)) are not defined, and hence x * ( k ) is not defined f o r k > j . Since x * ( k ) $ B f o r k Q j , we arrive at a contradiction, and so (4.16) is established. It follows from (4.16) that n*(c)c X* (4.17) Finally, it follows from (4.17) and from Corollary 2.1 that n*(C) belongs to the game surface through x i . Of course, according t o the definition of n*(C)and of a game surface, the game surface through xi is

C(C). Now let us consider non-null paths n$(C‘)and nF(C’’)which emanate from x * ( j ) , j < K , and which are generated by strategy pairs ( p , e*) and ( p * , e), respectively. Let us represent IIG(C’) and ni,m(C”)by xp;: k

and x,,: k respectively.

+ xk-

= x,(k),

+xk =

x,(k),

+ 1 , . . . ,l } , k E { j , j + I , . . . , m}, k

E

{j,j

4.7

SETS f i p AND

79

QE

As a consequence of Lemmas 2.1 and 2.2, lIiY(Cn)has no A-point and

IIg (C') has no B-point relative to C(C).

+

+

Since x * ( j ) belongs to %* and j < K , points x p ( j 1) and x,(j 1) of paths II;Y(C") and nj2(C'),respectively, are defined. From the properties of these paths, that BIS from Lemmas 2.1 and 2.2, we deduce that

Next let us define sets Q2,(j K - 1,by Definition 4.4.

+

a,(j 1) a E ( j-I- I )

{xp(j {xdj

+

(4.18) (4.19) 1,. .. ,

for all strategy pairs ( p * , e ) } for all strategy pairs ( p , e*)}

(4.20) (4.21)

+ 1) and Q,(j + I ) , j

+ 1) + 1)

FIG.4.4. Sets fi, and SZ,.

= i, i

80

IV

MULTISTAGE QUANTITATIVE GAMES

From (4.8) and (4.9) we have Q,(j

Q,(j

+ 1) = { x ( j + 1):

x ( j + 1) = x * ( j ) + f j ( x * ( j ) ,u * ( j ) , v ( j ) ) V v ( j ) E K,(x*(j))} (4.22) + 1) = { x ( j + 1): x ( j + 1) = x * ( j )

+ f j ( x * ( j ) , u ( j > ,v * ( j ) )

V u ( j >E K t 6 ( x * ( j ) ) } (4.23)

and from (4.18) and (4.19), sincej is arbitrary, it follows that

+ Qfi;(j+ 1) n (B/Xj+l(c)) = o

i . l I 1 ( j 1) n ( A / ~ ~ + ~= ( Co) )

and for al1jr.z {i, i

+ I , . . . ,K x*(j

-

+ 1)

E

(4.24) (4.25)

I}. Of course, Q,(j

+ I ) nQ,(j + 1)

4.8 x,-DIRECTIONAL CONVEXITY

Definition 4.5. A set S in En+l is directionally convex with respect to vector ivo, or equivalently it is x,+-directionally convex, if for all xl, x2 E S and all Y E [0, I ] there exists a K , --CO < K 0, such that X1

+

Y(X2

- X’)

+


0 such

x*(j

Since x * ( j that, for all E t ( 0 , o),

(T

x*(j

+ 1) +

E

6'x

E

Xk

Then (4.28) implies that

+ 1) +

N

L?/x,+l(c) A ( B / C ,,l(C)> u q+l(c> x*(j

N

E dPX

(4.29)

E

where BP,,,(C) J t follows from (4.29) with (4.13) and (4.15) that @(X*(j

+ 1) +

E

6"x) Q

c

(4.30)

I f grad @(xJ+l) is defined in a neighborhood point x * ( j can be rewritten as @(x*(j

+ 1)) + grad @(x*(j + 1)) -

E

+

6x'

O(E)

+ l), (4.30)

QC

where lim6+o [o(E)/E] = 0. 1) belongs to C,+,(C),andaccordingly cD(x*(j Since x * ( . j (4.31) becomes

+

grad O(x*( j Dividing by

E

E

8"x

+

O(E)

Q0

+ 1)) = C , (4.32)

which is positive, we deduce from (4.32) that

-

grad @(x*(J Then, upon letting that is

+ 1)) -

(4.31)

E

+ 1)) - 6"x +

Q0

0, we obtain

grad @(x*(j

+ 1))

O(E)/E

+ 1)) - BPx < 0


,v*(j))l 0 (4.33) This relation holds for all v(j) E K,(x*(j)). grad W x * ( j

*

4.10

83

REGULAR OPTIMAL PATHS

Finally, (4.33) can be rewritten as grad @ ( x * ( j

+ 1)) - f T x * ( j ) ,u * ( j ) , v(J)) S grad @ ( x * ( j + 1)) - f Y x * ( j ) , u * ( j ) , v*(j)) V V ( j )E K,(x*(j)) (4.34)

+

By similar arguments based on the fact that Q2,(j 1) is x,+-directionally convex and that Q , ( j 1) n (5/Xj+1(C))= ia , one obtains

+

grad @ ( x * ( j

+ 1)) - f Y x * ( j ) ,Nj),v * ( j ) ) 2 grad @ ( x * ( j + 1))

*

f i ( x * ( j ) ,u * ( j ) , v * ( j ) ) V 4 j ) E K , ( x * ( j ) ) (4.35)

4.10 REGULAR OPTIMAL PATHS

Definition 4.7. An optimal path n *(C) given by x * : k xk = x*(k), k E {i, i 1 , . . . , K } , is called a regular optimal path, if x * ( k ) is a regular point of C,(C) for all k E {it i 1, . . . , K - I}.

+

---f

+

Recall that all points of Ck(C)are interior points of C,(C), and hence that x * ( k ) is an interior point of Ck(C),k = i, i 1, . . . , K. Let x*: k + x k = x*(k), k E {i, i 1, . . . , K } , represent the projection T * of II*(C) on X*. Since Xk is open in E,, x * ( k ) is an interior point of X,, k = i, i 1 , . . . ,K . Next we introduce

+

+

+

+

Assumption 4.3. At every point x*(k), k = i, i 1 , . . . ,K - 1, there exists a neighborhood A(x*(k)) in Xk of x*(k) on which p * and e* are of class CI.

Since strategy pair ( p * , e*) is optimal o n X , it is optimal on A(x*(k)), k = i , i + 1 , . . . , K - 1. From (4.4) and Definition 2.3 we have A'-1

V*(xk) = zfOy(x(v),p*(x(v)),e*(x(v))) v=k

V x k E A(x*(k))

where x: v + x v = ~ ( vis) the solution of difference equation

+

x(v 1) - x(v> = f ' ( x ( v ) , p*(x(v)), e*(x(v))) with initial condition x(k) = x k , defined on { k , k 1, . . . , K } .

+

(4.36)

84

MULTISTAGE QUANTITATIVE GAMES

IV

Then we have

+

Lemma 4.1. A t points x * ( k ) , k = i, i 1, . . . , K, of optimal path n*(C)on game surface C(C) there exists a neighborhood in E , on which grad @ ( x k )is de$ned,t where

For k = i, i 4 1, . . . , K - 1, Lemma 4.1 is a direct consequence of Assumption 4.3, together with (4.36) and with the fact that the functions A”,v = k , k I , . . . ,K - 1 , are of class C‘ on 9 x U x V . For k = K , it follows from (4.5) that Y * ( x K )= 0 for all x K E 8, and hence grad @ ( x K ) = (1, 0, . . . , 0 )

+

4.1 1

VARIATIONAL DIFFERENCE EQUATION

Again let us consider points x * ( k ) , k path n*(C),and let us introduce Assumption 4.4.

i

+ 1 , . . . ,K , of optimal

Matrix M k given by Mk&

where

= i,

[I +

I 1

dfk(xk,p*(xk),e*(xk)) dX

xk=X*(k)

afk

d f k ( x k p*(xk), , e*(xk)) d f k afkap* -- - - + ---+-dx dx ax au

+

ax

afTcae* av ax

and I is the ( n 1) x (n x 1) unit matrix, is defined and nonsingular for k = i , i + 1 , . . . , K - 1. u)]

afk a - _-

ax

. . . )n

v = 1 , 2 , ..., r v. = 0,1,. . . ,n

ax ae* ax

v,v.=O,l,

ae,*(xk)

v=l,2,

...,s

-=[T vI . = O , I , ...,n

f Recall that x k 4(x,,xl,. . . ,~ , - ~ , k ) .

4.1 1

85

VARIATIONAL DIFFERENCE EQUATION

evaluated at xk = x * ( k ) , u = p * ( x * ( k ) ) ,ZI = e * ( x * ( k ) ) ,k = i, i K - 1. Now let us consider the bound vector

+ 1 , . .. ,

qk (qok,q I k , .. . , qnk)E En+l at point 2. Let q: k q k = q(k), k E { i , i + 1, . . . , K } , be defined in the following manner: -+

1. Let qz = q ( i ) be given at the initial point x * ( i ) of IT*(C). Then

2

x(i) = x*(i)

is a point i n a neighborhood in

+q(i)+

O(E)

X iof x * ( i ) , where lime+,,

[ ~ o ( E ) ~ / E= ]

0.

2. Let 2 be transformed by the state equation (4.7) with the same strategy pair ( p * , e*) that generates n*(C). The state equation can be written x(k

+ 1) - x ( k ) = f k ( x ( k ) , p * ( x ( k ) )e,* ( x ( k ) ) )

(4.37)

f o r k = i , i + 1, . . . , K - 1 One readily deduces from the state equation that x(i

+ I) = x*(i + 1)

+r[l+ that is x(i

d f 2 ( x 2p,* ( x i ) , e*(x2)) dx xt =

X*

(i)

+ 1) = x*(i + 1) + e M i q ( i ) +

O(E)

(4.39)

Letting

we have x(i

+ 1) = x*(i + 1) + q ( i + 1) +

O(E)

(4.40)

Likewise, if x ( k ) = x*(k)

+q ( k )+

O(E)

(4.41)

86

IV

MULTISTAGE QUANTITATIVE GAMES

$+I

A

then x(k

+ 1) =

X*(k

+ 1)

Letting q(k

we have

x(k

+ I) =

Mkq(k)

+ 1) = X * ( k + 1) + q ( k + 1 ) + 4 t )

(4.42)

and so

f o r k = i , i + 1, . . . , K - 1 . Equation (4.43) is the uariafional diference equation. Note that.fnk(x, t i , 2 ) ) = I , k = i, i I , . . . ,K - I , and hence

+

rll,(k

It follows that

+ 1) - q,(k)

(4.44)

=0

+ 1) = . . . = q A K )

(4.45)

q,(4 = qll(i

4.12 LINEAR TRANSFORMATION AND INVERSE

TRANSFORMATION

For given q(i) = q' we have q(k Let

+ 1) = M"(k)

= M'(MLp'q(k

q(k

*

. . M"(9

1.

Since matrices M h ,k = i, i matrices d h ,k = i, i 1,. linear transformarion

+

= MkMk-'

- M h M k - 1 . . . MZ

&h

fork = i , i + 1,... , K -

- 1))

+ 1 , . . . , K - 1 , are assumed nonsingular, . . ,K

+ 1) = e d k q ( i ) ,

- 1 , are nonsingular.

k = i, i

+ 1 , . . . ,K - 1

Hence the (4.46)

4.13

SOME PROPERTIES OF LINEAR TRANSFORMATIONS d kAND

gk

87

is nonsingular; that is, the inverse of dkexists, and we have q(i) = (d")-'q(k If q ( K ) = 71" is given, then

+ 1)

q(i) = ( d - - l q ( K )

and

q(k) = dk-'q( j ) = dk-'(LpylIC-'

where Let

&-l

& I.

.@

&k-1

)-lq(K)

(dK-')-l

where @ k (dK--l)-l. Then we have the inverse linear transformation q ( k ) = @q(K)

f o r k = i, i

(4.47)

+ 1 , . . . ,K - 1.

4.13 SOME PROPERTIES OF LINEAR TRANSFORMATIONS d k AND dc


(xZ) and since (rI,*(x'),r,) E F,,(x') V r , E R , implies that V r , E R, 3 r , E R , [namely, r,, = r p * ( x Z ) ]such that (r,, r E ) E F I , ( x ' ) , it follows that xi E A . By similar arguments one can prove that (5.2) implies that x z E B n comp A Definition 5.3. S, S,

A (xi:

(5.4)

3rp*(xi))

4 (xi: 3r,*(xi))

Sets S , and S , will be called the sets of the game. Let us note first that (8, n 8,) n ( S I , u S,) = o. If 8, n 8, = o and e p # o then 8, E S , ; and if 8,, n B E = o and 8, # o then OE SE. If target O,, (or 0,) is empty, the corresponding sets S , (or S,) may be empty or not. Furthermore, it follows from the remarks of Sec. 5.2 that S, G A

comp B

and

S E s BncompA Hence,

s,, n SE = 0

Definition 5.4. Let there be given a fixed set X , pp* E R , is optimal on X,, if Vxi. E X ,

Vr,

E

RE,

s S,.

(pI,*, rBJ E F P * ( x ' )

Likewise, let there be given a fixed set X , G S,. pE* E RE is optimal on X,, i f V x i E X E Vr,, E R,,

We shall say that (5.8)

We shall say that

(r,, pR*) E F e * ( x z )

(5.9)

106

V

SOME GEOMETRIC ASPECTS OF QUALITATIVE GAMES

In other words a stratesy for player J p that is optimal on X, is optimal at all points x2 E X I , ; that is, 3rz,*(xz) such that rI,*(xz)= pp* for all x' E X , . Likewise, a strategy for player J , that is optimal on X E is optimal at all points x2E X,; that is, 3rBT*(x2)such that rE*(xi) = pE* for all x7 E Xp:. 5.3 SOME PROPERTIES OF SETS OF THE GAME

Definition 5.5. A 9-path is a path that emanates from x' E S , and is generated by ( r p * ( Y ) , rl+:), r E E R, o

Furthermore there exists a

> 0 such that for all E , 0 < < a , E

xf

+ e q f E 8,

If Assumption 6.2 is satisfied along r g ethen , for E sufficiently small there exists a solution 2 : 7 + x = %(T), T E [O, T f ] , of trajectory equation (6.2) with end condition

n(7,)

= Xf

+Eqf

and there exists a point x' = n(0) such that x'

= xo

+ EqO +

O(E)

with qo = q(O), where q is the solution of variational equation (6.3) with end condition qf = q ( ~ ~ ) . We have

-

+

+

o(E)) = E grad g(xo) qo 4-O(E) g(x') = g(xo where limG+oO ( E ) / E = 0. For E sufficiently small, the sign of the righthand side is the same as the sign of E grad g(xo) qo, and hence g(xz) has the same sign as e l o qo. Since E > 0, since A(7) T ( T ) = constant along rDe and since ilf q f > 0, it follows that g(x') > 0 for E sufficiently small.

-

-

-

-

6.10

131

A MIN-MAX PRINCIPLE

Hence, for

E

sufficiently small, Xi E

s~ n h(X*)

Then, by the same arguments as those employed above, one obtains a contradiction with the fact that Xf

+ Erf E O p G s,

Finally, since K # 0, we conclude that K If x f E 8, and 8, is defined by

e(x)

> 0.

>0

one can again prove in similar fashion that

I f = Kgrad 8 ( x f ) ,

K

>0

6.10 A MIN-MAX PRINCIPLE

Now let

&?(A,

x, u , u)

L il . f ( X , u , u )

From Lemma 6.3 and from (6.28) one can deduce

x= Theorem 6.1. Let rrpe be the boundary path represented by x: T x ( T ) , T E [0, T,], rS# 0. If Assumptions 6.1-6.3 are satisfied, then there exists a nonzero continuous solution A : T + il = A(T), T E [0,T , ] , of adjoint equation (6.4) such that --f

(a)

min

&?(A, x , u , e f ( x ) ) = max &?(A, vEKy(x)

u€Ku(x)

x , p:o(x), u)

= # ( I , x , P ~ x )e X , x))

(b)

&?(A, x , P$(x), e>(x)) = o

f o r all T E [0, T J , where xo is the initial point of ripe, and (pzo,e,*.) is the strategy pair that generates ripe. Furthermore, i f 8, 8, = 0 ,if rrpe reaches 8, (or 8,) at point x f , if Assumptions 6.4 and 6.5 are satisfied, and if 8, (or 8,) is de$ned by 8,(x) 0 (or 8 E ( x ) O), then




I f = Kgrad 8 ( x f ) ,

where 8 = 8, or 8,.

K

>0

132

VI

DIFFERENTIAL QUALITATIVE GAMES

If xs X(TJdoes not belong to O p u ,O then Theorem 6.1 is a trivial consequence of Lemma 6.3 together with the conclusions of Section 6.8. If xs E Or U OE, we shall consider a value of T , say T‘ E [0, T ~ ) and , a point x’ = x(T’)E r g eat, which relations (a) and (b) above hold; namely, min A(T’)*f(x’, U , e:O(x’)) = max A(+) *f(x’, pzo(x’), U) UEK

I,

( x’)

vEKJx’)

= A(+) .f(x’, pEo(x’), e,*o(x‘))

and

.f(x‘, p*(x’), ezo(x’)) = o

~(7’)

Since f is of class C1 on G x U x V , and pz0 and e,*. are of class C1 on a neighborhood of rSer and x: T‘ + x’ = x(T’) as well as A: T’ + 1‘ = A(T’) are continuous functions of T’ on [o, T s ] , it follows that H , defined by H : T’ +H(T’) where

WT’)= VT‘) .f(x(~’),p*(x(~’)),eXT’))) is a continuous function of T’ on [0, T s ] . Hence, relation (b) holds at r = T , and xs X ( T J It is clear that relation (a) also holds at T = ‘rS,since A(Ts)

and

.f(X(Ts),

Ub,

e3(x(%>)> 20

A(T,) .f(X(Ts), P3(X(.,), can be deduced from

and

A(T’)- ~ ’ ( x ( T ‘~,.(X(T’>), ), e3(x(r’)))

Ub)

Q0

> 0,

azs(xs)= ub

/?Z’(X~) = Ub A(T’) ’f(X(T’), p;O(x(T’)), @z’(X(T’>)) Q 0, by continuity, for every given ub E Ku(xs)and every given v b E K,(xs). 6.11 AUTONOMOUS GAMES WITH TIME-INDEPENDENT

TARGETS

An autonomous game is one in which the function f,in the right-hand side of the state equation (6.1), does not contain the variable x, = t explicitly. This situation can arise only if p and e d o not contain x, in their arguments; that is, if they are functions of x l , . . . , x , - ~ only.

6.11

AUTONOMOUS GAMES WITH TIME-INDEPENDENT TARGETS

133

Let Y(T)& grad g(x(7)). From (6.23) we obtaint

or, taking account of (6.3),

or equivalently,

Now recall thatfn(x, u, v) = 1 , and so af,/ax, = 0 for a = 1 , . . . , n. It follows that the last row of the matrix

apz0

(aj- + - -af+ - -

ax

au ax

afae;or

av ax

is zero. Moreover, since

for v = 1, . . . ,n in an autonomous game, it follows that the last column of this matrix is also zero. On the other hand, if targets OP and 8, are time-independent7 that is, if OP and 8, in two target problems, and 8, or 8, in single-target problems,§ are x,-cylindrical sets in En,then the map of the game is composed of x,-cylindrical sets in En. It follows that S,, S , and E are x,-cylindrical sets in En. In this case, g is a function of x l , . . . , xnP1 only, which implies that the nth component of vector y(7) vanishes; namely, y(.) = (yl(.)7 Y2(7)?. . . Yn-l(T)> Since (6.30) is satisfied by n - 1 linearly independent vectors q(7) at point x = x(T), we arrive at 9

Y(4

t Here we require that g is twice continuously differentiable.

$ Here, p = p,*o and e = e,*o, andp, and e, are the vth components of vectorsp and e , respectively. 9 Of course, E may be empty; and, in one target games with 0, # 0 and 0, = S, may be empty. @,

134

VI

That is, function y:

T

+

y ( ~ )T,

E

DIFFERENTIAL QUALITATIVE GAMES

[ O , T,], where

Y ( 7 ) = grad g ( x ( 7 ) )

is a solution of adjoint equation (6.4). Finally, according to the definition of A(T), we have? (6.31)

A ( T ) = k grad g(x(T)) where k is a positive constant. 6.12 CONSTRAINTS

Let us consider constraint sets K,(x) and K,,(x), respectively, given by y a ( x ,u)

< 0,

Ya(X,D) s, at most r of the ya(x, u) or s of .the ya(x,u ) vanish at any point of G x U and G x V , respectively, and furthermore that the matrices

1 , 2 , . . . , k‘ cr = 1 , 2 , . . . , r

tc=

[”’$;

41

u = 1 , 2 , . . . , 1’ a = l , 2 , ...,s

have maximum rank at u = p z o ( x ) , D = e*,.(x), x = x*(k). By arguments similar to those employed in Chapter 3, one can verify that adjoint equation (6.4) may be written (6.33)

t At all points X ( T )

where grad g ( X ( T ) ) is defined.

6.13

135

SEMIPERMEABLE SURFACES

where

1

a p A ap,,(x,u) -ax= [ ax,

v = ~ 2 , . .. , k u = i , 2 , ..., n

ax

v = 192, * * , 1 u = l , 2 , ...,n

-

evaluated at x = x ( T ) , u = ~ ~ o ( x ( T ) )u, = e$(x(T)); and Lagrange multipliers p and v are k- and I- component vectors, respectively, given by

1 with p i 2 0, vi < 0.

lu=

Thus, if constraints (6.32) are independent of x, adjoint equation (6.4) reduces to daldr = -(aflax)Ta (6.34) 6.13 SEMIPERMEABLE SURFACES

We shall now complement Theorem 6.1 by

Theorem 6.2. I f g: x --t g(x), x E G, is such that grad g(x) is dejned and is diflerent from zero f o r all x E G , and fi there exist strategies p and e" such that (a)

min n(x) * f (x, u , t?(x)) = max n(x) . f ( x , p(x), u ) vCK,(x)

UEKU(Z)

= n(x> *f ( x ,j w , 44)

and (b)

n(x) *f(x,P(x), W ) )= 0

(6.35) (6.36)

136

VI

DIFFERENTIAL QUALITATIVE GAMES

f o r all x E G , where n(x) & grad g(x); and fi there exists a constant C such that g(x) < c vx~e, [or g(x) > C VXE~,]

and fi the surface %(C) dejned by %(C)

is not empty; then

V(C) c comp S,

[or Furthermore, $8,

(6.37)

{ x : g(x) = C }

g ( C ) c comp S,]

= 0 and. of course, 8 p # 0 , then %(C) C

s,.

Let xi E W(C) and consider describing curve r x ( x i , p ,Z), x E y ( x * , p ,i?), where p is any strategy for J,. If r x ( x 2 , pZ) , is a single point, then obviously rx(xi,p, Z) n 8, = 0 [or

r x ( x t , p ,2) n 8, =

01

Therefore let us suppose that r x ( x i , p ,Z) is given by x : T E [0, T ~ ) where , T , # 0. As a consequence of (6.35), grad g W > ) . f ( X ( ' ) > P ( X ( d ) ,

Z(x(4))

z0

T -+

x = x(T),

(6.38)

for all T E [0, T J ; and since the left-hand side of (6.38) is the time derivative of g ( X ( T ) ) , namely dg(x(T))/dT,for all T E (0, T ~ ) it, follows that

g:7- -g,(+ where g,(T) g(x(T)),is a non-decreasing function of Since xi = x(0) belongs to g ( C ) , and if g(x) < C for all x

E 8,,

T

on [O, T ~ ) .

then

rx(xi,p, e") n 8, for all x E y ( x i , p ,e") and for a l l p E 9,. Since (6.39) holds for all xi E %(C),

=0

(6.39)

6.13

137

SEMIPERMEABLE SURFACES

Consequently,

%(C) c comp A

which implies that %(C) c comp S , By similar arguments, one can prove that %(C) c comp SE if g(x) > C for all x E 8,. Hence, Theorem 6.2 is established. Now, if OE = m and, of course, 8, # 0, Equation (6.39) implies that

( p , e") E q g x i )

vp E Y

Hence

vxi E %(C)

p

%(c) sE

Definition 6.2. A surface %(C)defined by (6.37) is called a semipermeable surface,? if grad g ( x ) is defined and is different from zero for all x E G , and if there exist strategies and e" such that Conditions (a) and (b) of Theorem 6.2 are satisfied. We shall complement Theorem 6.2 by

Theorem 6.3. I f g,: x g,(x), v E I, where I is a set of indices, are functions dejned on G , such that grad g,(x) is deJined and is diflerent from zero f o r all x E G and f o r all v E I , and if there exists a strategy e" such that --f

n,(4

-f(X,

u,

tw)> 0

grad g,(x); and ifthere exist

f o r all u E K,(x), x E G , v E I , where n,(x) constants C,, v E I , such that 9 E #

where 9 E

g,(x) >, c,, v

A

= (x:

then 9 E

Furthermore,

if 0,

8,n9E=

and

0

0

E I>

c comp Sp

= 0 and, of course, 8, 9E

c sE

t According to the terminology of Isaacs.

# 0 , then

138

VI

DIFFERENTIAL QUALITATIVE GAMES

Let x z E g Eand consider describing curve r x ( x z , p ,e"), x E y ( x 2 , p ,&), where p is any strategy for J,. If r x ( x z , p e") , is a single point, then, obviously, r x ( x z , p &) , n 8,, = 0. So let us suppose that F x ( x z , p ,e") is given by x : T + x = x ( T ) , T E [0, T,), where T , # 0. We have grad g,(x(.)) .f(X('),P(X(4), &(x(.))) for all

T

E

E

>0

YE

1

(0, T ~ )from , which it follows immediately that

r x ( x z , p , e")

and hence for all x

YE1

[0, 7,); and hence

T E

dg,(x(T))/dT for all

>0

C 9

E

r x ( x z , p ,z) n e p = y ( x z , p ,Z), for all p

E

Y, and for all x'

( p , e") y p * ( x z ) and so

rn

v p E9 ,

E g E .Consequently,

v x zE 9

E

BE c comp S,,

Now, if 8, =

~5

and, of course, 8,, # 0 , then

(p, VpEYp

VX'EgE

Hence

9 E = SE which proves Theorem 6.3. By similar arguments as in the proof of Theorem 6.3 one can establish

If

Theorem 6.4. gy: x g,(x), v E I, where I is a set of indices, are functions dejined on G , such that grad g,(x) is defined and is different from zero for all x E G and for all v E I, and if there exists a strategy p" such that ---f

n,(x)

for all v

E

K,(x),x

E G,

-f(X,P"(X>,

4Q0

v E I , where nv(x)

grad g,(x);

and

if there

6.14

THE GRADIENT ; SEMIPERMEABLE SURFACE PATH

exist constants C,, v E I , such that

# 0 and 8, n g P= 0 ,where

$3,

g p Li {x: g,(x) Q

then

139

c,,v E r>

g P= comp S ,

Furthermore, if8, = 0 and, of course 8 , # 0 ,then g P

= SP

6.14 A PROPERTY OF THE GRADIENT ALONG A PATH IN A SEMIPERMEABLE SURFACE

Consider a non-null path 75. emanating from xi E % ( C ) , generated by strategy pair (p, Z) and represented by x: T -+ x = x ( T ) , T E [0, T j ] . As a consequence of (6.36) dg(x(T))= grad g(x(7)) ' ~ ( x ( TP(x(T)), ), ?(X(T))) = 0 dT

and so g ( x ( 7 ) ) = C for all Now, we introduce

T E

(6.40)

[0, T i ] . Hence, ii c %'(C).

Assumption 6.6. Strategiesp and Z are of class C1on a neighborhood of 71. Relation (6.36) is an identity in x on G . Let us rewrite it as

i ax, *)'.(X,

P(X),

qx)) =0

(6.41)

a=1

By differentiating (6.41) with respect to x p , /I= 1 , 2 , . . . ,n , we obtain?

a=l

ax, ax,

fa(&

p ( x ) ,q x ) )

Now, along a non-null path ii lying on %'(C),

(a,c,,) 5a " p O f , ( x , P ( x ) , ax, a=lax,ax,

--

dr

=

Z(x)),

8 = 1, 2, . . . , n

t Here we require that g is twice continuously differentiable.

(6.43)

140

VI

DIFFERENTIAL QUALITATIVE GAMES

Equations (6.42) and (6.43) lead to

By developing afa(x,p"(x),Z(x))/ax, and writing (6.44) in vector form, we obtain d (6.45) d-r

for x = X(T) E + If constraint sets K,(x) and K,(x) are given by (6.32), one can verify, by arguments similar to those used in Chapter 3, that adjoint equation (6.45) may be written as

(6.46) where ;1 = n(x(-r)), and p and v have the same meaning as in Sec. 6.12. If constraints (6.32) are independent of x, adjoint equation (6.45) reduces to

(6.47) Clearly, there are similarities between the properties of semipermeable surfaces and the properties of game surface E. One can easily find examples in which E is imbedded in a family of semipermeable surfaces. 6.15 TRANSVERSALITY CONDITION FOR A SEMIPERMEABLE.

SURFACE

Let g : x - g ( x ) be a function defined on G , such that gradg(x) is defined and is different from zero for all x E G , and let us suppose that there exist strategiesp" and e" such that Conditions (a) and (b) of Theorem 6.2 are satisfied. Furthermore, let us suppose that max,,8,g(x) is defined, and let M & max,.op g(x) g ( M ) A { x: g( x) = M } is a semipermeable surface. Clearly, we have n z 0

-

ww ep

6.15

TRANSVERSALITY CONDITION FOR A SEMIPERMEABLE SURFACE

141

Let 3, E V(M) n 8,. Since g ( f z , ) = M and grad g(4,) Z 0, 3 , is not an interior point of 8,; that is, 2, E 88,. Let E ae, 4, €7

+ +

+ + +

where 2, 7 belongs to the tangent plane T(fp) of 38, through f,; and limc+o I]o(E)]]/E = 0. Since 2, €7 O(E) E 8, we have g(fp

+ €7 + 4~)) = g(P,) + €7 . grad g(2,) + 4lk7 + 4e)II) Q M

and since g(f,) = M , we have

€7 * grad g(2,)

Dividing by (i) for

E, E

---f

E

7 * grad gG,)

0

0

and (ii) for

and allowing E

+4lE7 +

-

7 grad g(2,) = 0

+

Since this equation holds for all 2, q E T(2,) it follows that grad g(f,) is perpendicular to T(P,). Hence, 88, and % ( M ) have the same tangent plane at point 2,. It follows that grad g(fp) = K , grad O,(f,)


0 if target 8,

is defined by eE(x)

>0

142

VI

DJFFERENTIAL QUALITATIVE GAMES

Surface % ( M ) separates G into two disjoint sets, namely

B 2 {x: g(x)

and

8'

4 {x:

g(x)

>M) <M)

Likewise %(m) separates G into two disjoint sets, namely

B 4{x: g(x) < m>

and

8' A {x: g(x)

> m)

And, from Theorems 6.3 and 6.4 Q c comp S,

and

B c comp S,.

Furthermore (i) if 8E = ia and, indeed 8, # 0 ,then & c S,; and then 9 c S13. (ii) if Br. = ia and, indeed eE # 0 , 6.16 A MIN-MAX PRINCIPLE ALONG A PATH IN A SEMI-

PERMEABLE SURFACE

From the discussion of Sections 6.14-6.15 one can deduce at once Theorem 6.5. If.? is a non-null path emanating from a point xi of a semipermeable surface V ( C )& {x: g(x) = C } , generated by strategy pair (jj,e") and represented by x: T -+ x = x ( T ) , r E [0,T J , and if Assumption 6.6 is satisfied, then there exists a nonzero continuous solution A: T + A(T), T E [0, rs],of adjoint equation dil

such that

6.17

EXAMPLE

143

6.1

Furthermore ifAssumption 6.5 is satisjied and ife, is dejined byOp(x) and

(i)

M

4max g(x) is defined; seep

V(M); (iii) n- reaches 8 , at point x p at r f p , (ii) xi

then

0.

(c) h(7fP)= K p grad fJ,(x,), K p

Likewise, ifAssumption 6.5 is satisfied, and ifeE is deJned by eE(x) 2 0, and (i)' m = min g(x) is defined; XCeE

(ii)' xi E V ( m ) ; (iii)' reaches 8, at point x E at T

+

then

(d) h(TfE)= Kx grad e E ( X E ) , KE

/ ~ ,

> 0.

6.17 EXAMPLE 6.1

Let us consider a one target qualitative game described by

+

dx,/dr = u u dx21d7 = -2 dx,/dr = 1

with constraints given by and

e,

lul Q 1

= n { x = ( x l , x2, x3):

(6.48)

Ivl Q 1

(6.49)

qX) 2 x12 + x22 - ~2

< 01

(6.50)

Let us suppose that there exists a non-null boundary path ripe, emanating from xo E E, generated by strategy pair (pzo,e 4 ) , and represented by x: T -+ x = x ( T ) , T E [0, r,], which reaches at point x(T,), and that Assumptions 6.2-6.4 are satisfied.? The %-function is given by

ep

%(A, x , u, v) = I1(U

+ v) - 21, + 1,

(6.51)

t Assumption 6.1 is satisfied since the constraints are given by (6.49), and Assumption 6.5 is also satisfied since target 0, is given by (6.50).

144

VI

DIFFERENTIAL QUALITATIVE GAMES

The adjoint equations are so that

diluldT = 0,

a = 1,2,3 a = 1,2,3

A u ( ~= ) constant,

(6.52)

(6.53)

From condition (c) of Theorem 6.1 we have Xl(Tf)

= CXl(Tf)

h z ( T f ) = Cxz(.f) A3(7f)

C

(6.54)

=0

where, in view of (6.50), C2

>0

+

= R-'[Al'(T,)

&'(Tf)]

(6.55)

Then it follows from ( 6 . 5 3 ) and (6.54) that Al(4

= a1(7,)

= CX2(Tf)

A3(T)

(6.56)

0

In view of condition (a) of Theorem 6.1, U(T)

)'(V

= -sgn

xl(Tf)

= sgn XI(7,)

(6.57)

and so by condition (b), &(T)

3

0

which implies that X2(Tf)

(6.58)

=0

Along r p e(6.48) , may be rewritten as (6.59) Upon integration of (6.59) we find that a non-null boundary path rp , that reaches O,, at point x ( T ~is) a straight line whose projection on the xl-x2plane is given by xl(-r) X1(~f)= fR (6.60) X2(T) = -2(T - Tf)

6.18

EXAMPLE

145

6.2

The projection on the x1-x2 plane of the two families of possible boundary paths thus obtained is sketched on Fig. 6.6. One can prove that the closed shaded area of Fig. 6.6 is the projection of a set of E3 that contains S,, whereas the open unshaded area is the projection of a set of IP contained in S,. Indeed, S , and S, depend on 9,and 9,.

FIG.6.6. Example 6.1. Projection on the xl-x, plane of the two families of possible boundary paths; projection of a domain of E 3 that contains set S,. (op: projection of target Op.) 6.18

EXAMPLE 6.2

Next let us consider a one target qualitative game described by Eqs. (3.78), with constraints given by (3.79), and target 0, defined by

e,

2 tX = (xl,x2, xs): qX)A

x12

+

x22

- ~2 Q 01

(6.61)

Again let us suppose that there exists a non-null boundary path 7rpe emanating from-xOE S , generated by strategy pair (pzo,ez0), wherepz~= @zo,,p,*,),and represented by x: 7 -+ x = x(7), 7 E [0, T ~ ] which , reaches O p at the point x(T~),and that Assumptions 6.2-6.4 are satisfied.?

t Assumption 6.1 is satisfied since the constraints are given by (3.79), and Assumption 6.5 is also satisfied since target 0, is given by (6.61).

146

VI

The X-function is given by

S ( A , x, 2.4, u) = A1241

DIFFERENTIAL QUALITATIVE GAMES

+

AZ2.42

+ 11u +

A3

(6.62)

The adjoint equations are so that

dAJd7 = 0,

a = 1,2,3

(6.63)

).(,A

cc = 1 , 2 , 3

(6.64)

= constant,

From condition (c) of Theorem 6.1 we have &(7f)

= cx1(7f) = cxZ(7j)

A3(7j)

=0

A1(7j)

where, in view of (6.61), C2

= R-'[h12(Tf)

C

>0

+ AzZ((.rf)]

(6.65)

(6.66)

Then it follows from (6.64) and (6.65) that A1(7)

CX~(T~)

A2(7)

cxz(~f)

A3(7)

=0

From condition (a) of Theorem 6.1 we obtain

(6.67)

6.18

EXAMPLE

6.2

147

Consequently, (6.71) at a point where a non-null "oundary path Along rrge, (3.78) may be rewritten as

rrpe

reaches

D.

(6.72)

Upon integration of (6.72) we find that a non-null boundary path T~~ that reaches op at point x(T,) is a straight line whose projection on the xl-x2 plane is given by (6.73) x 2 = ax, b where

+

a =A b

Xd7-f)

-

~ ~ ( 7 (Rlw) ~ )

= f=

sgn xl(Tf)

-(R/w)a sgn X1(Tf) = f

W

41 - w2

d1R

w2

A straight line given by Eq. (6.73) intersects the xl-axis at point P: x1 = R/w, or at point Q : x1 = -R/w, depending on the sign of xl(Tf). Of course, it is tangent to the circle given by x12 xZ2= R2. The arguments above yield four families of possible boundary paths whose projections on the x1-x2plane belong to the four half-rays P'M' , P"M", Q'N', Q"N" sketched on Fig. 6.7. Let us consider the case for which a < 0 and b > 0, namely

+

a=-

W

,- - = b

G-2

In that case X1(Tf)

= Rw - I

R JI

- w2

148

VI

DIFFERENTIAL QUALITATIVE GAMES

and hence,

Consider a plane V,(C) given by We have

g ( x ) A x 2 - ax, - b = C g(x)

and

0

v x ~ e ,

n ( x ) 2 grad g ( x ) = (-a, 1 , 0 ) = [ ( R c J l - w2))-l]X(~) Conditions (a) and (b) of Theorem 6.2 are satisfied with p(X)

= U(T) =

z(X)

= V(T) = 1

(-W2,

-WJl

- W')

and hence, according to Theorem 6.2, U,(C) is a semipermeable surface such that Vl(C) c comp S , for all C>0

By similar arguments one can see that planes U2(C),U3(C),and g4(C) given by x2-axl-b=C a>O,b 0, C >

respectively, are semipermeable surfaces such that V2(C) E comp S , V3(C) E comp S ,

V4(C)E comp S ,

I

for all

C 0

0

6.18

EXAMPLE

149

6.2 \

x2

/\

M"

N"

FIG.6.7. Example 6.2. Projection on the x1-x2 plane of a domain of E3that contains set S,. (&: projection of target ep).

One can prove that the closed shaded area of Fig. 6.7 is the projection of a set of F that contains S,, whereas the open unshaded area is the projection of a set of E3 contained in S,. Indeed, S , and S, depend on 9 p and 9 ,. We recommend that the reader compare the treatment of this example to the one of Example 3.1.

7 A Connection between Qualitative and Quantitative Games

7.1 PROBLEM STATEMENT, DEFINITIONS OF GAMES 1, 2, AND 3

Definition 7.1. As in Chapter 6, let us consider a qualitative game in E n with state equation dx/dT = f ( ~ U,, U ) (7.1)

where x L2 ( x l , x,, . . . , xn). The definition of the game, the notation and the assumptions will be the same as those in Chapter 6 , except for the fact that we shall now focus our attention on the case where 8, # 0 and OE = 0.Such a game will be termed Game 1. Now let us consider a new variable x,, and the equation (7.2)

dX,/dT = f o ( x , u, v )

wheref, is of class C1 on G x U x V . We shall require that the sets of strategies 9,and 9,satisfy Assumption 3.3 and, as in Chapter 3, we shall write V(xi, xs; p , e, T?‘) f

I”

&(x(T), u(T),v(T))dT

where x: T x = x ( T ) , T E [ O , T , ] , represents a path risin G, x(0) = x * , X(TJ = xs, and U ( T ) = ~ ( x ( T ) V ) (, T ) = e(x(T)). Equations (7.1) and (7.2) constitute a set of n 1 scalar equations which we shall write in vector form ---f

+

dx/dT = f(x,

U,U )

(7.3)

7.1

PROBLEM STATEMENT, DEFINITIONS OF GAMES

where x

1, 2,

AND

3

151

(x,, xl,. . . , xn), and f(x, % 4 2 ( f o ( x ,u , ~ ) , f i ( X , u, v ) ,

Let En+l 2 En x {x,}, 9 A

o,(c) L {x:

. . . ,fn(x, a, u)). G x {x,}, 0 , A 8 , x {x,} and

x =

(x,, x),

E

e,,

xo

< c}

(7.4)

where C is a given constant. As the value of parameter C i s varied, Eq. (7.4) defines a one-parameter family of targets, namely {O,(C)}. 9 and 0, are x,-cylindrical sets whose projections on E nare G and O,, respectively. Definition 7.2. The set 9,the prescribed sets of strategies 9,and Y E , the state equation (7.3) and (i) Or in one case; (ii) 0,(C) in the other case; and the usual prescriptions for Jp and J E , define two single-target qualitative games in En+l.The games corresponding to (i) and (ii) will be termed Game 2 and Game 3 , respectively.

Note that the rules are the same for Game 2 and Game 3. Let xi be a point of 9,and xi its projection on G . We shall let II” denote a path in 9 from xi to xs, and risa path in G from xi to xs. Let IIif be a path in 9 that emanates from x i , is generated by ( p , e ) , and reaches 0, at point x f . Its projection on G is a path rifthat emanates from xi,is generated by ( p , e ) , and reaches 0, at xf,the projection on G of X f . Conversely, with each path rifin G that emanates from xi,is generated by ( p , e ) , and reaches O p at point xf, there is associated a path I I i f in 9 that emanates from xi,is generated by ( p , e ) , and reaches 0 , at point xf. As before, let rif be represented by function x : T + X = x ( T ) , T E [0, T ~ ] then ; to rifthere corresponds rIif

(

= x:

x = (x,

x),

x = x(7),

xo = xd + L f o ( x ( t ) , u(E), ~(6))d5,

where In other words,

7rif

u(7) = p ( x ( 7 ) ) , V(7) = e ( x ( 4 ) is the projection on G of nif.

7

E

LO, ~~1

152

vii

CONNECTION BETWEEN QUALITATIVE AND QUANTITATIVE GAMES

I

I I

@P

I

I I

X-f

I

L E"

FIG.7.1. Targets 0,,

a,,

and O,(C); paths

and

+f.

It follows that, if ( p , e ) is a playable pair (or a strongly playable pair) for Jp a t point x2 in Game 2, it is a playable pair (or a strongly playable pair) for J p at point x2 in Game 1, and conversely. We shall let T p ( x ' ) and Y I , ( x 2 )denote the sets of playable pairs for J p at point xz in Game 1 and at point xzin Game 2, respectively. In Game 3 , we shall let T p ( x e ; C ) denote the set of playable pairs for Jp at point x'. The sets of strongly playable pairs will be denoted by F p * ( x ' ) , Y p * ( x z ) , and Y p * ( x a ;C ) in Games 1, 2, and 3 , respectively. Likewise, if ( p , e ) is a playable pair for JE at point X~ in Game 2, it is a playable pair for J E at point x2in Game 1, and conversely. We shall let Y E ( x * )and F E ( x 2 )denote the sets of playable pairs for JE at point x' in Game 1 and at point x2in Game 2, respectively. We shall let F E ( x e ;C ) denote the set of playable pairs for J E at point xzin Game 3 , and Y E * ( x ' ) , Y E * ( x Z ) Y, E * ( x 2 ;C )the corresponding sets of strongly playable pairs in Games 1, 2, and 3 , respectively.

7.2

LOCAL SADDLE-POINT CONDITION, GAME

153

4

7.2 LOCAL SADDLE-POINT CONDITION, GAME 4

If (p,e)

E Tp(xi),we

shall write (7.6)

where x: T -+ x = x ( T ) , r E [0, T f ] , represents a path rifthat emanates from xi,is generated by (p, e) and reaches 8, at xf. The supporting curve of that path is r x ( x i , p ,e). Since, for given xi and (p,e)fYp(xi), rif need not be unique, V(xi,8,; p, e) need not be unique. We shalllet Vx(xi, 8,; p, e) designate the value of V(xi,O P ; p, e) defined by function x. Let xibe a point of S,, and 1etp;f be an optimal strategy for J p at point xi in Game 1 ; that is, Then

(j&

Ve E Y E

e) E Tp*(xi)

rx(xi,P : ~ ,e) n 8 , #

vx

0

E

y(xi,&, e),

(7.7) Ve E Y E

and, for x E y(xi,p& e), each rx(xi,p:g,e) is the supporting curve of one and only one &optimal path in G . Consider a member r$ of this set of paths. It follows from Lemma 5.1 that r$ c S,. The corresponding path IIif in S, x {xo}that emanates from xz = (xt,xi)and is given by (7.5) with

u ( 4 = p*,i(x(d),

v ( 4 = e(x(T))

reaches 0, at point x f = (xof, xf). By (7.5) and (7.6),

..

xgf = xi

+ vX(xi,8,;

p$, e)

It follows from the definition of O,(C) that

xof< c =+Xf

E 0,(C)

(7.9)

As a direct consequence of (7.7)-(7.9), the set of points xi,xi E S,, defined by

x6

+ Vx(xi,8,;

&, e)

max V(xi, 8,; ecYE

~ % i )

p 3 , e) > V(xi,0,; p z f , e )

for all xi E S,, for all p E w ( C z i ) , for all e E Y E and , for all values of V(xi,1 9 ~p ;, C,i) and V(xi,8,;p:t, e ) ; hence,

v(xi,8,; p>,

cZi)> max V(xi, 0,; eeY,

pzf, e)

> V(xi, 8,; p 3 ,

for all xi E S , and for all values of V(xi, 0 p ; p 3 , Css). We conclude that minDem(zzi) V(xi,8,; p , C,i) and V(xi,0,; p,*I,PZi) are defined for all xi E S,. In other words, we reach the following conclusions: 1. If there is more than one path in S, starting at point xi,generated by (p:;, Czi), and terminating on f J P ,then all the values of V ( x i , 0,; pzi, Z,i) are equal; and

These results are embodied in Lemma 7.1. Let p,*,be an optimal strategy for player J p at point xi E Sp in Game I . If maxeEYEV(xi, 0 , ; p 3 , e) is dejned for all xi E S,, and i f there exists Csi E 9,such that

> max V(xi, 0,;

V(xi, 8,; p , 2%;)

eEYE

pzi, e )

156

CONNECTION BETWEEN QUALITATIVE AND QUANTITATIVE GAMES

VII

for all x 2 E S p ,f o r all ( p , ZZt) E F,(xz) andfor all values of V(x*,6 ,; then

p , Ex.),

( I ) V ( X ' ,81,; p,*t,2,") is dejined f o r all xz E S,;

and

(ill v/(xZ, 0,; p:, el Q 1/(xZ,6,; P,*., p,,) Q v(xz, 6,; P , 2,a)

f o r all e E Y Ef,o r all p E d ( f ? , f*o)r, all values of V(x', 0,;p,$, e ) and V(xz,O,,;p, exC),and f o r all xz E S,; and (iii) min v(xZ,oI,; p , ),2

p d ~ , z )

= max ~ ( x '8 ,;, eE .YE

p,:, e) = ~(x',8 ,;

* -

p,., ezs)

Condition (ii) is a local saddle-point condition. Now, if there exists p* E 9,that is optimal on S, in Game 1, if maxPEy, V ( x i , BP;p*, e) is defined for all xi E S , , and if there exists 2 E 9,such that V ( x i , Or; p , 2)

> max V(xi,6,; ee.YE

p*, e)

for all (p,2) E FI,(xi),for all values of V(xi,8 ,; p , 2) and for all xi E S,, then (ii) is identical with the saddle-point condition given in Chapter 2; namely, it becomes

V(xi,O,,;p*, e ) Q V(x2, 6,;p*,

2)

< v(xi,OP;p,2)

for all xi E S,, for all ( p , 2) E Y P ( x i )and for all values of V(x1,BP;p*, e ) and V(xi,0,; p, 2). From the point of view of quantitative games,

vw, 8,; p , 4 ,

(p, e) EFp(xi),

is the cost functional associated with a terminating path in S , that emanates from x i , is generated by strategy pair ( p , e), and reaches target 6 ,. Definition 7.5. The quantitative zero-sum, two-player game which has the same rules as qualitative Game 1, and whose objectives are defined in terms of 8,;t and V(xi,O,;p, e), will be called Game 4.

t That is, 0 = Op.

7.3

157

SURFACES OF THE GAME

In Game 4,p* and P are optimal strategies on S , for players Jp and J E , respectively. 7.3 SURFACES OF THE GAME

Let us consider points xi = (x;, x i ) such that xb = c - max

eeYE

or equivalently X :

+ v(x~,8,;

v(x',;0,

p,*i, e), xi E S,

pzi, Czi) = C , xi E S,

(7.15)

Definition 7.6. Equation (7.15) is the equation of a set of points xi in S , x {xo); that is, (7,15) defines a surface which we shall call ageneralized surface of the game k(C) in Game 4.

This surface has properties which are similar to those of a surface of the game Z(C). When functions pz, and Czi do not depend on xi E S,, then i ( C ) and C(C) coincide for all xi E Sp. Surface % ( C )depends on parameter C . Variable:x is the cost variable. Since V(xi, 8,;p;i, Ezi) does not contain the cost variable, Eq. (7.15) defines a one parameter family of surfaces, {i(C)}, such that any two surfaces of the family can be deduced from one another by a translation parallel to the cost axis. Now consider a path IIifthat emanates from xiE S, x {x,}, is generated by (p, Zzi) where p E W(C%i), and reaches 0, at point x f = (x,f, xf). We have

xgf =

.:

+

qXi,

ep; p , zz{)

for allp E w ( P z i ) and for all values of V(xi,8 , ; p , Psi). From (7.16) and from the fact that

ep; pzi, czi) = max v(xi,ep; p:i, esYE

~(x',

we deduce that xd

> C - V(xi, 8,; xgi

+

el,

* -

psi, e,i) V(X' 8,; p, Czi)

>C

for allp E w ( C z i ) and for all values of V(xi,O,;p, Esi).

(7.17)

158

VII

CONNECTION BETWEEN QUALITATIVE AND QUANTITATIVE GAMES

FIG.7.2. Generalized surface of the game in Game 4.

*

Hence, at every point x2 of the set A / X ( C )defined by A/i;(C) L2 (xZ:

x2 = (xgz,

XZ),

xz E

s,, X :

> c - v ( ~8,;( , &, a,.)}

vp

E

(7.18) there exists a strategy e , namely gZt, such that whatever the strategy p , ( p , CX.) is a playable pair for JE at that point in Game 3 ; that is, ( p , a,)

E F ~ ( X ZC ; )

Y,,

VX'

E

A/2(C)

(7.19)

Since player JE has no target of his own, it follows from the definition of a playable pair that relation (7.19) coincides with E Y,,

(7.20) Equation (7.19), and hence Eq. (7.20), is a consequence of (7.17) if p E w(Cxt),and it is trivial ifp $ w(exz). ( p , Pxt) E TE*(x*; C)

Vp

Vx2E A&C)

7.4

A CONNECTION BETWEEN PATHS IN GAMES

1

AND

159

4

Next we shall let S,(C) denote the set of all points xz at which there exists an optimal strategy for player JE in Game 3. It follows from (7.20) that A / k

= %(C)

(7.21)

Similarly, consider B/$(C) 22 { x i : xi = (x;,

.

.

x”.

xi E sp, X :

< C - V(xi, 8,;

p:i,

c2i)}

(7.22)

Since V(xi, Op;pzi,C 2 i ) = maxeEYEV(xi, OP;p,*i,e ) , B/?(C)

= R(C)

where R ( C ) is the set defined by (7.12). Then it follows from (7.13) that &C) = SP(C) (7.23) Furthermore,

A$(C) Hence,

~& ( A / Z ( C ) )u 2 ( C ) 5 A / i ; ( C ) c S E ( C ) ~-

B 3 ( C ) A (B/T(C>)u Z(C) 5 B / Z ( C )c S,(C) Z(C) = A ~ Z ( Cn ) B ~ Z ( Cc) _ _ _ _

~

_

_

s,(c) n s,(c)

_

(7.24)

But E(C) = S,(C) n S,(C) is the surface of the game in qualitative Game 3. Consequently, Z(C)C B(C) (7.25) 7.4 A CONNECTION BETWEEN PATHS IN GAMES 1 AND 4

Now we introduce Assumption 7.3. Strategies p:i and t?%i do not depend on xi E S p . We shall denote these strategies by p* and 6. Then, as pointed out before, (i) ( p * , e) is an optimal strategy pair on S , in Game 4; and (ii) Z(C) and C(C) coincide for all x i E S,. Let my be a &optimal path in Game 1 , emanating from xi E S p and generated by strategy pair (p*, 6). By Lemma 5.1, mg

= SP

160

VII

CONNECTION BETWEEN QUALITATIVE A N D QUANTITATIVE GAMES

Let II*(C) be an optimal path in Game 4, generated by strategy pair (p*, d ) , whose projection on G is T* = T ~ From . Corollary 2.1, with %* = s, x {xo},n*(c)= C(C). And so, finally, we arrive at Theorem 7.1. If& is an optimal srrategy for player Jp at point xi E S , in Game 1; ifthere exist a cost V ( Y , 8,; p$, e ) satisfying Assumption 7.1 and a strategy dzt E 9,satisfying Assumption 1.2; fi Assumption 7.3 is satisfled, that is, if^:^ = p* and Fzt = Z do not depend on xz; then every 9-optimalpatlz xB in Game 1 , emanating from x a andgenerated by strategy pair (p*, d ) , is the projection on S , of an optimal path fI*(C) in Game 4 , generated by strategy pair (p*, e*), e+ = i?.

FIG.7.3. Connection between paths in Games 1 and 4. Game 1 is Example 6.2; Game 4 is Example 3.1 ; C(C) is game surface in the xo-xl-xz space, for Example 3.1. Sections of C(C) by planes parallel to x1-x2 are shown on this figure.

7.4

A CONNECTION BETWEEN PATHS IN GAMES

1

AND

4

161

Of course, there may be more than one quantitative Game 4 associated with a given qualitative Game 1 . Theorem 7.1 is illustrated by Examples 6.2 and 3.1. We shall let the reader verify that if Example 6.2 is Game 1, then Example 3.1 is a Game 4 associated with it. The corresponding game surface X ( C ) is pictured on Fig. 7.3 in the xo-x1-x2space. This figure should be compared to Figs. 3.9 and 6.7.

This page intentionally left blank

Appendix

Let G be a domain in En;that is, an open connected set in the topology of En. The topology induced by En on G (i.t.) is defined by the following

Axiom. An open set in the i.t. is a subset of G which is open in E" A subset of G is closed in the i.t. if its complement in G is open. It follows that 1'. G i s closed in the i.t. 2'. 0 is closed in the i.t. Let S be a subset of G . Point x is an interior point of S in the i.t. if S a n d there exists an open set in the i t . which contains x. The interior of S in the i.t. is S A {x: x is an interior point of S } It follows that

x

E

Sc

s

Let X be a subset of G, and let Y = comp X where comp X means the complement of X in G . The closure of X in the i.t. is L comp

x

f

164

QUANTITATIVE A N D QUALITATIVE GAMES

It follows that X G

x n

comp X = comp A'

comp Y = comp Y The boundary of X in the i.t. is

ax2XncompX We have

ax = X n comp X i u ax = i u (X n comp i ) = (2 u X) n (iu comp i ) =X

It follows that

comp

n ( X u comp X )

(iu 3x1= comp x u (comp i n comp comp i ) = comp X u ( X n comp X )

Furthermore, we have Hence Hence

XncompX=

o

comp ( X u ax)= comp X

x=xuax

165

REFERENCES

REFERENCES On the Geometry of Optimal Processes 1. A. BLAQUIERE AND G . LEITMANN, On the Geometry of Optimal Processes, Parts I, TI, 111, Univ. California, Berkeley, IER Repts. AM-64-10, AM-65-11, AM-66-1. 2. G. LEITMANN, Some Geometrical Aspects of Optimal Processes, SZAM J. Control 3, 53 (1965). 3. A. BLAQUIERE, Further Investigation into the Geometry of Optimal Processes, SZAMJ. Control 3, 19 (1965). 4. A. BLAQUIERE AND G. LEITMANN, Some Geometric Aspects of Optimal Processes, Part I : Problems with Control Constraints, Proc. Congr. Automatique Thioorique, Paris 1965. Dunod, Paris, 1967. 5. K. V. SAUNDERS AND G. LEITMANN, Some Geometric Aspects of Optimal Processes, Part 11: Problems with State Constraints, Proc. Congr. Automatique Thiorique, Paris 1965. Dunod, Paris, 1967. 6. A. BLAQUIERE AND G. LEITMANN, On the Geometry of Optimal Processes, in “Topics in Optimization” (G. Leitmann, ed.), pp. 265-371. Academic Press, New York, 1967. 7. H. HALKIN,The Principle of Optimal Evolution, in “Nonlinear Differential Equations and Nonlinear Mechanics” (J. P. LaSalle and S. Lefschetz, eds.). Academic Press, New York, 1963. 8. H. HALKIN,Mathematical Foundations of System Optimization, in “Topics in Optimization” (G. Leitmann, ed.), pp. 198-260. Academic Press, New York, 1967. 9. E. ROXIN,A Geometric Interpretation of Pontryagin’s Maximum Principle, in “Nonlinear Differential Equations and Nonlinear Mechanics” (J. P. LaSalle and S. Lefschetz, eds.). Academic Press, New York, 1963. 10. R. E. BELLMAN, “Dynamic Programming.” Princeton Univ. Press, Princeton, New Jersey, 1957. 11. G. LEITMANN, “An Introduction to Optimal Control.” McGraw-Hill, New York, 1966. 12. A. BLAQUIERE AND G. LEITMANN, Further Geometric Aspects of Optimal Processes: Multiple-Stage Dynamic Systems, in “Mathematical Theory of Control” (A. V. Balakrishnan and L. W. Neustadt, eds.). Academic Press, New York, 1967.

On The Theory of Games 13. J. VON NEUMANN AND 0. MORGENSTERN, “Theory of Games and Economic Behavior.” Princeton Univ. Press, Princeton, New Jersey, 1953. 14. J. MCKINSEY, “An Introduction to the Theory of Games.” McGraw-Hill, New York, 1952. 15. L. ZADEH,Optimality and Nonscalar-valued Performance Criteria, IEEE Trans. Automatic Control AC-8, 59 (1963). 16. R. ISAACS, “Differential Games.” Wiley, New York, 1965. 17. L. D. BERKOVITZ, A Variational Approach to Differential Games, in “Advances in Game Theory,” pp. 127-174. Princeton Univ. Press, Princeton, New Jersey, 1964.

166

QUANTITATIVE AND QUALITATIVE GAMES

18. L. D. BERKOVITZ AND W. H. FLEMING, On Differential Games with Integral Payoff, in “Contributions to the Theory of Games 111,” pp. 413-435. Princeton Univ. Press, Princeton, New Jersey, 1957. Necessary Conditions for Optimal Strategies in a Class of 19. L. D. BERKOVITZ, Differential Games and Control Problems, SIAM J . Control 5, 1 (1967). 20. D. L. KELENDZHERIDZE, A Pursuit Problem, in “The Mathematical Theory of Optimal Processes.” Wiley (Interscience), New York, 1962. 21. L. S. PONTRYAGIN, On Some Differential Games, SIAM J. Control 3, 49 (1965). 22. L. S. PONTRYAGIN, On the Theory of Differential Games, Uspehi Mat. Nauk 21, 219 (1966). 23. E. F. MISHCHENKO AND L. S. PONTRYAGIN, Linear Differential Games, Dokl. Akad. Nauk SSSR 174, 27 (1967) [English transl.]: Soviet Math. Dokl. 8, 585 (1967). 24. Y. C. Ho, A. E. BRYSON, AND S. BARON, Differential Games and Optimal PursuitEvasion Strategies, IEEE Trans. Automatic Control AC-10,385 (1965). 25. S. BARON,Differential Games and Optimal Pursuit-Evasion Strategies. Doctoral dissertation in Applied Mathematics, Harvard University, Cambridge, Massachusetts 1966. 26. E. N. SIMAKOVA, Differential Games (a survey paper), Automat. i Telemeh. 27, 161 (1966). 27. I. G. SARMA AND R. K. RAGADE, Some Considerations in Formulating Optimal Control Problems as Differential Games, Internat. J . Control 4, 265 (1966). 28. A. KAUFMANN, “Graphs, Dynamic Programming, and Finite Games.” Academic Press, New York, 1967. 29. G. LEITMANN AND G. MON, Some Geometric Aspects of Differential Games, 1.Astronaut. Sci. 14, 56 (1967). 30. G . LEITMANN A ND G. MON,On a Class of Differential Games, in “Proceedings of the Colloq. Advanced Problems and Methods for Space Flight Optimization, University of Liege, 1967” (B. Fraeijs de Veubeke, ed.) Pergamon Press, Oxford, 1968. 31. G . MON, Differential Game Theory: A Geometric Approach. Unpublished doctoral dissertation, Univ. California, Berkeley, 1967. 32. A . BLAQUIERE AND G. LEITMANN, “Jeux Quantitatifs, MCmorial des Sciences MathCmatiques.” Gauthier-Villars, Paris, 1969. 33. G. WANGAND G. LEITMANN, Necessary and sufficient Conditions for TwoPerson, Zero-Sum Multistage Games, J. Optim. Theory Appl. (1969, in press). 34. A. BLAQUIERE A N D G. LEITMANN, Multistage Quantitative Games, in “Proceedings of 2nd Hawaii International Conference on System Sciences, 1969,” p. 579. Western Periodicals Co., North Hollywood, California. AND N. GANI,Multistage Quantitative Games with Unprescribed 35. A. BLAQUIERE Terminal Stage, Comp. Rend. Acad. Sci. Paris, 268, 428 (1969). 36. A. BLAQUIERE A N D G. LEITMANN, Multistage Quantitative Games with Unprescribed Terminal Stage, Internat. J . Non-Linear Mech. (1969, in press).

On Qualitative Games 37. A. BLAQUIERE AND F. GERARD, “On the Geometry of Optimal Strategies in TwoPerson Games of Kind.” Mehanika, Moscow, 1968. 38. A. BLAQUIERE A N D F. GERARD, On the Geometry of Optimal Strategies in TwoPerson Games of Kind, J. Computer System Sci. Vol. 2, No 3, October 1968.

REFERENCES

167

39. F. GERARD, Theorie gkomktrique des jeux diffkrentiels qualitatifs a deux joueurs. Doctoral dissertation, Facultt des Sciences de Paris, 1968.

On Diferential Equations and System Theory 40. E. A. CODDINGTON AND N. LEVINSON, “Theory of Ordinary Differential Equations.” McGraw-Hill, New York, 1955. AND R. CONTI,“Non-Linear Differential Equations.” Pergamon 41. G. SANSONE Press, Oxford, 1964. 42. W. A. PORTER,“Modern Foundations of Systems Engineering.’’ Macmillan, New York, 1966.

This page intentionally left blank

Subject Index

A A point, 14, 17, 77, 79 Additivity of cost, 17, 19 Adjoint equation, 25, 27, 37-39, 41, 42, 52-54, 58, 62, 114, 115, 126, 131, 134, 135, 140, 142, 144, 146 difference equation, 90, 96, 97 Adjoint vector, 27, 92, 125 Autonomous game, 132

B B point, 14, 18, 77, 79 Boundary path, 119, 124, 131, 134, 143, 145, 147 C Constraint, 49, 52, 53, 58, 62, 94, 97, 134, 143, 145 Control variables, I , 23, 71, 97, 113 Convexity, 8 1 x, directional, 80 Cost, 11, 12, 18, 24, 58, 71, 74, 97, 156, 157 additivity of, 17, 19 Crossing of discontinuity manifold, 46, 60, 66

D Decomposition of domain, 42, 52, 67 of Z(C), 43 Describing curve, 4, 8, 29, 74, 104, 117, 136, 138

Differential games qualitative, 113 quantitative, 21 Discontinuity manifold, 42, 43, 47, 48, 52, 56, 60, 66 Dynamic programming, 42

E E optimal, 17, 70 & path, 106 optimal, 106 Extreme future time 3, 30 G Game, 1 autonomous, 132 game 1, 150, 155, 156, 159, 160 game 2, 150 game 3, 150, 154, 158, 159 game 4, 153, 156, 159, 160 map of, 103 multistage, 71 objectives of, 2, 9, 113 qualitative, 9, 103, 110, 113, 150 example of, 143, 145 quantitative, 9, 21, 71, 110, 150 example of differen~ial,53, 58, 61 of multistage, 96 rules of, 2, 3, 23, 74, 113, 151, 156 set of, 104, 105, 106, 116 value of, 12, 20, 56, 60, 77 zero sum, 10 surface in qualitative game, 107, 108, 112, 118, 123, 159 in quantitative game, 13, 17, 31, 32, 39, 76-78, 84, 93, 157

169

170

SUBJECT INDEX

H

0

%-function in differential qualitative games, 131, 143, 146 in quantitative games 41, 52, 543 583 62 multistage, 93, 97 Hamiltonian, see N-function

Objectives, 2, 9, 113, 156 Optimal pair in quantitative game, 12, 20, 25.. 44.. 52. 53, 67, 75, 83, 93, 145,

497

I Index, see also Cost performance, 11, 24, 58, 74 Jnduced topology, 163 Interior point, 32, 35, 44, 78, 83 lsovalue surface. 13

J Joining of paths, 8, 17, 29 of trajectories, 8 describing curves and, 8 Jump condition, 46, 47, 48, 60 min-max principle with, 52

K Kuhn-Tucker conditions, 49, 95, 135

I, Lagrange multiplier rule, 49, 95, 135 Linear transformation, 28, 37, 86, 87

M Manifold discontinuity, 42, 43, 47, 48, 52, 56, 60, 66 Map of game, 103 Maximal solution, 3, 29, 74 Min-max principle in differential qualitative games, 131 in differential quantitative games, 41 with jump condition, 52 local condition, 123 in multistage quantitative games, 93 along path in semipermeable surface, 142 Multiplier, Lagrange, rule, 49, 95, 135 Multistage quantitative game, 71

1