Singular Perturbations and Asymptotic Analysis in Control Systems

PREFACE

This collection of papers deals with Llw general role of singular perturbation techniques in control systems analysis and design problems. These methods have proven useful in tIle construction or "reduced order models" and the evaluation of cont-rol system designs based on those models. We have collected here the usc at' these techniques which will be informat;ive

1,0

11

representa,tive sampling of

t,lwse readers interested in

acquiring a taste ror the theory and its applications. We have also addressed those doing research in the subject matt;er by including some new results and methods not published elsewhere. The first paper in this collection,

Singular Perturbation Techniques ill Control

Theory is a survey of the role of singular perturbation ideas in engineering control sy:;tern design. The analysis and examples which it contains summarizes much of the work in the field prior to this volume. It sets the stage for the detailed treatment of more specialized topics in the subsequent papers.

In Part I we treat optimal control problems with small parameters. The paper Singular Perturbations for Deterministic Control Problems provides a comprehensive treatment of deterministic optimal control problems with "fast" and "slow" states. It is based on the asymptotic analysis of both necessary conditions and l;ho associated Hamilton-.Jucobi-Bellmun equation - that is, direct evaluation of the optimal cost runeLion. The treatment using a dlWlity for this equation is new. As a consequence, one can extend the concept of composite feedback involving "separation" or controls ror fast and slow states which had been derived earlier for quusi-linear systems the fuJI nonlinear case.

v larly Perturbed Systems nonlinear, non-autonomous singula.rly perturbed sysl,cms axe considered at I,lle outset. Tbe methods are I,hen exl,ended

1,0

treat multiparamctcr pcr-

turba.tion problems. In New Stability Theorems Jor Averagillg and Their Application to

lhe Convergence Analysis oj Adaptive Identification alld Oontrol Schemes multi-time scale methods are llsed

1,0

GreaL time varying nonlinear systems wHh applical,ions

1,0

csl,i-

mat,ion oj' the mLcs of convergence of adaptive identification a.nd conl,rol algoriLhms. These papers provide just a.pplied mathematics.

I'\'

sampling of the methods :lvILlla.ble

ill I,his

rich a.rea. or

Some of' the Impcrs indicate the broader range or mel,hods and

applications which lie outside con trol Uleory.

We trust that those readers who' have

round the papers in Irllis volume interesting will be motivated to explore the many important, contributions which treat relat,cd applica.tions in ellgineering and applied phy-

sics.

P.V. Kokotovlc A. Bcnsoussan G.L. Blankenship

SINGULAR PERTURBATION TECHNIQUES IN CONTROL THEORY

P. Y ](okotovic t

Abstract This paper discusses typical applications of singular perturbation techniques to control problems in the last fifteen years. The first three sections are devoted to the standard model and its convergence, stability and controllability properties. The next two sections deal with linear-quadratic optimal control and one with cheap (nearsingular) control. Then the composite control and trajectory optimization are considered in two sections, and stochastic control in one section. The last section returns to the problem of modeling, this time in the context of large scale systems. The bibliography contains more than 250 titles. Introduction For the control engineer, singular perturbations legitimize his ad hoc simplifications of dynamic models.

One of them is to neglect some "small" time constants.

masses, capacitances, and similar "parasitic" parameters which increase the dynamic order of the model.

However, the design based on a simplified model may result in a

system far from its desired performance or even an unstable system.

If this happens,

the control engineer needs a tool which will help him to improve his oversimplified design.

He wants to treat the simplified design as a first step, which captures the

dominant phenomena.

The disregarded phenomena, if important, are to be treated in

the second step. It turns out that asymptotic expansions into reduced (,touter") and boundary layer ("innerl!) series, which are the main characteristic of singular perturbation techniques, coincide with the outlined design stages.

Because most control systems are

dynamic, the decomposition into stages is dictated by a separation of time scales. Typically, the reduced model represents the slowest (average) phenomena which in most applications are dominant.

Boundary layer (and sublayer) models evolve in faster

tCoordinated Sciences Laboratory and Electrica.l Engineering Department, University 01 Illinois, 1101 W. Springlleld Avenue, Urbana, IL 61801. This paper Is based on the amhor's survey In Lhe SlAM Relliew, Vol. 6, No. -I, October Hl8·1, pp. 501-550,

3

the state space of (1), (2) reduces from n + m to n because the differential equation

(1.2) degenerates into an algebraic or a transcendental equation

o

g(x,z,u.o,t),

(1. 3)

where the bar indicates that the variables belong to a system with

E

= O.

He tll!ll

say that the model (1.1), (1. 2) is in the stctnda2,d fonn i f and only if the following crucial assumption concerning (1.3) is satisfied.

In a domain of interest equation (1.3) has k > 1 distinct ("isolated") real roots i

1,2, ... ,k.

(1.4)

This assumption assures that a well defined n-dimensional reduced model will dorrespond to each root (1.4).

To obtain the i-th reduced model we substitute (1.4)

into (1.1), (1.5)

In the sequel we will drop the subscript i and re,rrite (1.5) more compactly as

.:. x

= £(i.u,t).

(1. 6)

This model is sometimes called quasi-steady-state model, because z, whose velocity

z .s. EO:

is large when

E

is small, may rapidly converge to a root of (1.3). which

the quasi-steady-state form oE (1.2). of (1.1), (1.2) in the

na~t

i8

We will discuss this two-time-scale property

section.

The convenience of using a parameter to achieve order reduction has also a drawback:

it is not always clear how to pick the parameters to be considered as small.

Fortunately, in marty applications our knowledge of physical processes and components of the system suffice to be on the right track.

Let us illustrate this by examples.

Example 1.1

A well-known model of an armature controlled DC-motor is (1. 7)

ax

Lz

bx

Rz + u

(1. 8)

where x. z. and u are respectively, speed, current. and voltage, Rand L are armature resistance and inductance, and a and b are some

m~tor

is a Ilsmall parameter" which is often neglected. £:=L.

constants.

In most DC-motors L

In this case equation (1.3) is

5 network is

(1.17) = v

1

-

(1 +~)v + -R~ u.

u +

R

(1.18)

2

+

(0)

{b} Fig. 1.

System with a high gain amplifier:

(a) full model, (b) reduced model.

If this model were in the form (1.1). (1.2), both

and v

2

would be considered as

z-variab1es and (1.3) would be

o

(1.19)

(1. 20)

However, Assumption 1.1 would then be violated because the roots of (1.3). in this

v

case vI 2 , are not distinct. The question remains whether the model of this RCnetwork can be simplified by singular perturbation E = 0, that is, by neglecting the small parasitic resistance r?

Without'hesitation the answer of the electrical engi-

neer is yes, and his simplified model is given in Fig. 1.2b.

To justify this simpli-

fied model a choice of state variables must be found such that Assumption 1.1 be satisfied.

As will be explained in Section 10 a good choice of the x-variable is the

"aggregate" voltage

7 Most of the quoted singular perturbation literature assumes that model (1.1), (1.2) is in the standard form, that is, it satisfies Assumption 1.1.

The importance

of Example 1.3 is that'it points out the dependence of Assumption 1.1 on the choice of state variables.

In most applications a goal of modeling is to remain close to

original "physical" variables.

This was possible in our Examples 1.1 and 1.2, but

not in Example 1.3. where a new voltage variable (1.21) had to be introduced.

However,

few engineers, accustomed to the simplified "equivalent" circuit in Fig. 1.2b. would question the "physicalness" of this new variable.

On the contrary. physical proper-

ties of the circuit in Fig. 1.2a.are more clearly displayed by the standard form (1.22). (1.23).

Nevertheless the problem of presenting and analyzing singular per-

turbation properties in a coordinate-free form is of fundamental importance.

A

geometric approach to this problem has recently been developed by Fenichel (1979) Kopell (1979) and Sobolev (1984).

Nore common are indirect approaches which deal

with singular singularly perturbed problems. such as in

O'~mlley

the original "nonstandard l l model into the standard form (1.1)

t

(1979). or transform

(1.2). such as in

Peponides, Kokotovic. and ehol" (1982). or Campbell (1980, 1982).

He will return to

this modeling issue in Section 10.

Singular perturbations cause a multi-time-scale behavior of dynamic systems characterized by the presence of both slow and fast transients in the sy,stem response to external stimuli.

Loosely speaking, the slow response, or the "quasi-steady-state, II

is approximated by the reduced model (1.6), while the discrepancy between the response of the reduced model (1.6) and that of the full model (1.1), (1.2) is the fast transient.

To see this let us return to (1.1)-(1.6) and examine variable z which has been

e1lOcluded from the reduced model (1. 6) and substituted by its "quasi-steady-state" z. In contrast to the original variable z, starting at t quasi-steady-state z is not free to start from

ZO

from a prescribed zo, the o and there may be a large discrepancy

between its initial value ~(x(t

o

and the prescribed initial condition z.

(2.1)

),u(t 0 ),t 0 ) ZO

Thus

z cannot

be a uniform approximation of

The best we can expect is that the approximation

z

z(t)

+

(2.2)

a(e)

will hold on an interval excluding to' that is, for tE[tl,T] where tl > to' we can constrain the quasi-steady-state dition x

O

x to

However,

start from the prescribed initial con-

and. hence the approximation of x by

x may

be uniform.

In other lrords,

9 If this assumption is satisfied, that is, if

(2.8)

uniformly in x tl

>

to'

small.

O

to' then z will come close to its quasi-steady-state z at some time

,

Interval [to,t l ] can be made arbitrarily short by making £ sufficiently To assure that z stays close to z, we think as if any instant t E [tl,T] can

be the initial instant.

At such an instant z is already close to

z,

\~hich motivates

the following assumption about the linearization of (2.6). Assumption 2.2 The eigenvalues of

ag/cz

evaluated along x(t), z(t), u(t) for all t E [to,T]

have real parts smaller than a fixed negative number

ReA{~}

net)

of net) is bounded, IIO(t)!! < c

0 such that for all 0
0

f(x) - F(x)G

-1

where (3.19)

(x)g(x)

satisfies, for some differentiable C(x) - C(x)

= a(x),

a

x

>

D.

(3.20)

17-' (3.29) such that

a.(~z

- ip(x,t)ll)


t)~ + Ilxn,




0 the standard op-

27

o.

nand det B 1: 0 whic.h is a very special

This equation is in the standard form only if r and unlikely situation.

(6.12)

Por r ~

where

z

is a

z weakly

z(t;e}

(9.7)

0 const,~nt

Gaussian random vector t1lth covariance P snt.isfylng the Lyapunov

equati.on AP

+

o.

PAl +G\.](;1

Khalil (1978) assumes

11

(9.8)

colored noise disturbance in the fast subsystem to account for

situat Lons h1hen the correlation time of the input stochast il' pr-ocess is longer than the time constants of f

0 (an algebraic equation replaces a diffe-

'F'TP.IA. Domaine de Voluceau, nOcrttH!fIt:Ourt. rtp, 105. 78150 LE CllESNAY CEDEX. France and de Pa,i.~ - iJullphillc.

U/liVfr.~ilc

61

(cf. P. FAURRE. M. CLERGET, F. GERMAIN [IJ) and the structure of the set of solutions is interesting. We have presented it beyond what is strictly necessary to solve the boundary layer problems. The non linear case (often referred as the trajectory optimization in the litterature) has been considered in particular by P. SANNUTI [lJ, [2J. R.E. O'MALLEY [3J. [5J, P. SANNUTI - P.V. KOKOTOVIC [IJ, C.R. HADLOCK [lJ. M.I. FREEDMAN. B. GRANOFF [IJ. ~1.L FREEDtMN - J. KAPLAN [1], A.B. VASILEVA, V.A. ANIKEEVA [1], P. HABETS [l], M. ARDEMA [lJ, [2J ... ). In general the poi nt of view is to \'Jrite the necessary conditions of optimality and to find expansions. A pl'oblem \,Ihich is considered is to solve the necessary conditions of optimality for the E problem by perturbation techniques. ~Je do not treat this problem here. On the other hand the evaluation of the cost function for "good" controls does not seem very much considered in the litterature. nor the expansion of the optimal cost. The fact that the control Uo itself yields an approximation of order E was known at least in the L.Q case, although the proof given relies on the boundary layer analysis. ~Je show this fact in general \oJithout using the boundary layer. The presentation of the convergence in the "constraints" case (lack of regularity) has not either appeared in the litterature. The study of Bellman equations in duality seems also original. It should be interesting to study the complete structure of the set of solutions. In the Dynamic Programming approach, the main concept is that of feedback, due to J. CHOW - P.V. KOKOTOVIC [lJ. We extend this work and particular that the decomposition of the composite feedback as the sum feedback and a complementary term involving the fast state ;s general, tricted to a quasi linear structure of the dynamics.

composite prove in of the limit and not res-

63

1.2. The limit problem Consider first the algebraic equation (1.6)

g(x,y,v) = 0

in which x,v are parameters and we solve (1.6) in y. By virtue of (1.2), the equation (1.6) has a unique solution ~(x!v). Moreover differentiating formally (1.6) with respect to x,v we obtain (1. 7)

o

These formulas show that

~x' ~v

are continuous functions of x,v, and bounded.

Consider then the system. for v(.) (1. 8)

2 (O,T;R k)

c L

dx = f(x,y(x,v).v) at x(O) = xo'

- (l.B) has one and only one solution xC.) in H1 (O,T;R n ). By the properties of y. The limit problem consists in minimizing

(1.9)

faT 9-(x(t).~(t),v(t»dt

J(v(.»

+ h(x(T})

in which we have set ( 1.10)

~(t)

~(x(t),v(t)}.

We shall make assumptions on the limit problem. We shall assume basically that the necessary conditions of optimality (Pontryagin principle) are satisfied, as well as 2nd order conditions. This will imply, among other things, that the limit problem has a unique optimal solution. We shall define the Hamiltonian (1.11)

H(x.y,v,p,q}

= £(x,y,v)

+ p.f(x.y.v) + q.g(x,y,v).

65

and (1. 16)

9.

n.

xx - (9. xy Q, xv )

Conditions (1.15). (1.16) coincide with (1.12). (1.13). when Xo = x*. Therefore . It is possible to the constant control u* is optimal for (1.8). (1.9) when x show, at least v/hen Uad = Rk and for data sufficiently smo~th (cf A. BENSOUSSAN [lJ) that taking Xo - x'" sufficiently small. there exists a function wo(t) satisfying conditions (1.12), (1.13). L,3.

We can state the following convergence result Assume (1.1). (I.2) and the exi (1.13). (1.14) hold. Then one has (1.17)

Inf J€(v(.))

-+

uE - Uo

-+

a

in L2 (O.T;R k)

- Yo

-+

a

in

- Xo

-'r

a

in H1(O,T;R n )

of wo{t) such that (1.12),

inf J(v(.).

( 1.18)

(1.19 )

L2 (O.T;Rm)

The proof of Theorem 1.1 is done in several Lemmas

0

67

and from the 1st differential equation (1.21). we get also (1. 22)

T dxE: 2

Io Ierr I

dt ;::;

From the estimates sequence

Ko'

(1.21)~

(1.22)~

we can assert

that~

at least for a sub-

(1. 23)

Co ns i der; ng

nO\l1

from (1.23) it remains in a bounded set of L2(O~T;Rm}. But from the second differential equation (1.20)~ taking ¢ C~(O~T;Rm) rl~dt

f

To

hence (1.24)

. .;. 0

in

To proceed we use the classical technique of MINTY [1J (cf also J.L. LIONS [IJ). Let m 2 Z E L (O,T;R ). We have from (1.2)

hence (1. 25)

From the 2nd differential equation (1.20) we deduce

69

It is then possible to pass to the limit in the 1st differential equation (1.20) and to deduce

which together with (1.26) implies From the uniqueness of the limit we can assert that

and thus the desired result obtains

o

Lemma 1.2. The functions uE,yE remain bounded in L2(O,T;Rk) and L2 (O,T;Rm) respectively. The function xE remains bounded in H1(O.T;R n). --Proof. Let us set

It will be convenient to use the notation a = (x,y,v)(recalling that w = (x.y,v,p,q)). He thus write

Let us establish the formula (1. 27) 1

+

1

fTo fa fa

.a dtdAd~ -E-E • H (w,E }a

A

acr

IIIJ

71

Then

(H )-l() yy Hyv

+ (H

xx -

(H

H ) xy xv

H HVY vv

where y ;s a positive number independant of

Hyx

(XE)2

;:::

ylz E I2

Hvx

A,~.

Therefore we deduce from (1.27) and from the last condition (1.12) (1.29)

g(o

o

(note that

depends on

A,~).

Noting that

and the assumtion (1.18) on uE ,

we obtain

I2dtdAd)J. On the other hand

T [1 J1

Ja Ja a ~IZEI

))dt + Y

2

dtdAd~

73

and also

since yE is bounded. Therefore n (1.32) that

= O. Using this fact and Lemma 1.1, we deduce from

Using (1.31) and the definition of Since there exists always

• we easily prove (1.19).

such that

and

Taking account of (1.19). we have JE(U E) ~ J(u )' hence o

Since also

we also have

which completes the proof of (1.7).

o

1.4. Stronger convergence results in the case of regularity.

One can improve the convergence result of Theorem 1.1, when the following additional regularity is satisfied (1. 33)

dQo 0, Ii < T, \1 g, pll(x;.,.) E L a(8,T;H 1(lRI'I}) Cn exp(

-

Q'~

-

where

0'1' QI::J

>

0,

C I' C'J

>

0

(1.23)

I Y -'I I ~ ] l

and they depend only on the bound on g G

•

(1.2-:1)

Tn particular,

they do not depend on the particular feedback v (.). This result is due to D. Aronson [5]. Now note that; if ¢ is periodic, z is period ie, and we can write

z (y ,t ) =

f PD~(Y ,l,1/)¢(11)d'l l'

(1.25)

181

I z(y,l) Taking

11

=

J4>(t})TI(d'l) y

I

~ K

Iltbll

e

p.

[l), we deduce I z(y.t) -

JrfJ('1)TI (dry) 1


11 t!

-pl.

Y

Using the invariant measure m

Tn

11

(1.31 )

defined in (1.IG), we also see easily from

(1.113) and (1.21) that Jz(y,t) m(y) dy = J~6(y) m(y) dy. y

y

(1.32)

Using (1.31) in (1.32), we deduce J

Tn

(y ) dy J cP(77)

y

which proves th at.

n

(d '1) = J ¢(y)

y

Jm (y )dy

0,

'Tn

(y ) t/y

y

since

is not a. e. O. Normalizing the in tegral to be I,

111

r

we see that IT(dy)

=

m (y)dy

and th us {l.31} yields Izt>(y.i) -

Jq,(y}mtl{y)dy I


0 -

H (y ,D 4>0)

r/J in }[l{Jr) wellkly and a.c. -10

e In

L 2( Y) weakly.

Passing to the limit in (1.13) yields

-

~t/>

+ X=

e.

4> periodic.

Jy 4>dy

o.

(lAO)

,185

Considering m ~ thejnvariant pt'obability corresponding to v:!. we deduce by multiplying

Iy m~ I D (¢1

I ':!dy

- ¢:2) +

= 0

tP'J) + = o.

There(ore,

constant. 0

¢'J

1.0 The-ergodic control problem \Ve can now interpret the pair (X, ¢). \Ve already know

x= =

lim

Ci tl

o(y')

0 ..... 0

11m

Ci

0 .... 0

1111 Ky'Y(v (.».

In fact, one can,be slightly more ,precise. \Ve have

Theorem 1.5. 'Under the assumptions of Theorem

1.4

we have

00

'X = 'in] { 11m 0(0)

Ci

Ej'Ie -

0-0

= Illf{ 11m v (0)

r::;-oo

ot

I (y, (f).v' (t» tit }

(1..12)

0

1

T

1, EoVII(y(t),v(l» lit} 0

'Nloreover, choosing the undetermined constant for ¢ such ,that

J~ry(y )11~ (y)i/y

=,0,

we

y

have T

I

t/>Ur)=Inj {l!ill 'E!f(l(y(t),v{l)) - x)ut 7 .... 00

Proof.

v(.): E,J't/J(y(T»-o}

0

Let us simply prove (1.-13). For any control

11

(0) we have

T

¢(y)


periodic in y.

\Ve define u [ by u!=tt

Then

11

l

+tt/J-f-1tc;

(2.58)

satisfies (2.59) -I- H(x

203

V"(x .Du ,y ,DII q,)

v (x ,Y )

(2.63)

is an optimal feedback for the limit problem. In fact, this is the feedback Lo be applied on the real system as a surrogate for u «(x ,y) den ned in (2.13). One can show by techniques similar to those used in previolls paragraphs to obt.aJll Theorem 2.1, t,hat the corresponding cost function will converge as

tends

E

a

to

11

in Hl(O X V).

Note t.hat

unlike the deterministic situation the optimal feedback for the limit problem is not a function of x only.

In fact (2.63) corresponds to the composite feedback ot'

Kokotovic [8] (c.r. also

17\

Chow~

in the deterministic case).

3. Ergodic. Control for Reflected Diffusions 3.1 Assumptions and notation Our objective in this section is to describe another class or crgod ic con t,l'ol problems

and to consider stochastic can trol problems -with singular perturbations wll kh call be associated to them, as in section 2. \Ve shall consider ditTllSiolls with reOeetion. Let B be a smooth bounded doma.in in nd, whose boundary is denoted by DB. Lct g and

I be continuous funct;ions g (y,v):

Ii

X U

->

n. d

I (y .v):

iJ

X U ->

nd

(3.1 )

where U is as in (1.2) and Uall is as in (1.5). Consider (O,A. ,P ,FI ,6 (l)) as in section 1.1. and let y (/) represent t.he dilrllsion process reflected at the boundary of B ely

.J2 db

- XlJn (y (l ))/Ld lJ

yeo) =

(3.2)

y.

where II is t,he outward unit normal at. the boundary of B, and ,,(l) is an increasing process. Admissiblc con('rols are defined as in section 1.1. Lor; us consider next the process be (l) defined in (1.6). and t.he cbange of probabilit.y dotlned in (1.7). For the system

205 the coeJficicn ts are not smooth. These properties have not, been clearly stated in the literature. \Ve shall proceed differently using some ideas at' Y. Kogan [oj. Consider the parabolic problem 8z 8z a; I

IlB

= 0,

~z

-

Z

-

g II.D::

o

(3.7)

4>(1)), ¢ Borel bounded

(y ,0)

and we slla1l define the operator, as in (1.2S), PI/>(y) =

Let us write for r a Borel subset of A...~(r)

(3.8)

z (y ,1).

11

P Xr(x) - P Xr(Y). Y,z

E

B

(3.0)

\Ve have

Lemma 3.1 81t 1l

{ A;:~ (n

I

t}, X

,y ,f'}
0

207

Therefore, (1.33) also holds. This estimate implies in particular Lhat

In v

;:::

0 and can be

taken to be a probability. To prove the estimate (1.3-1), we rely on the foHowin g Lemma.

Lemma 3.2. The solution oj (3.5) (normalized as n probab£lilY) satisfies (3.17)

wilh a norm bounded in v (.).

Proof. Let us consider the problem o If

l/J E L' (8) then

tP

E

W~,'

(B).

:s .s

1


'1, . . . ,

1 8"

we proceed in the

finite number of steps suffice to imply (3.17).

2

J'

Since tl

SLime

3.

S I

is i.1l'bil,rary

way. For a.ny value of d, a

209 If h > k >

0,

then A (h )

c

A (k). and we have


d. then 1f;(t). ko ::; t


s

1.

\Ve use the following result from

[llJ (p. 63): Let

be nonnegative and non increasing, slIch that (3.21)

where C ,01, and f3 are positive constants with f3 >

1.

Then (3.22)

where (3.23)

It is clear that this result applies, and thus A1eas A (k) =

k

=

G 1111

I L'

I

0

where

J..

(3.2-1)

(Aleas B) d

The second estimate is thus proved. The proof of the second estimate is more involved. \Ve refer t,O [7].

Remark 3.1. The fUllction result in Lions -

~!lagenes

1)1

l12J

v

E

WI,IJ (B).

\1 p E

(I.OO).

T'his follows from a general

Teo. 6.1, p. 33. Indeed, we write (3.5) as follows

211 i)z at i)z -a IBB II

with tP E L l(B}. tP

~ O.

9~

.6.z =

0,

Z

.Dz

=

(y ,0)

(3.27)

0

4>(y}

Let us assume that (3.28)

where

Co

does nOL depend

011

tP, nor on

v (.).

v.

vVc then have

Proposition 3.1. The following estimate holds

where c is i'ndependenl of ¢. v (.). and y.

Proof. ''\Tc shall prove that lu! {:(y.l) ly,l)(·).I/I~O, ItPlLI

l,pk ILl =

if this is false. there exists a sequence I/Ik ~ 0, jng

=k

the solution of (3.27) corresponding- to 1/11:.

Vk (.).

I}

(3.30)

1. Vk.

tIl: (.)

such that, denot-

then one has (3.31 )

\Vriting i)ZI:

~

O.

I.IJI

lOB

=

O.

=1;

(y ,0)

tPJ: (y)

and making use of (3.28), we can asserL Lhat zdy ,I) is bOll nded in L =(fl X(8. T

then

ZJ:

remains bounded in

w~,llr (B

X (8,1')). \-f p.

2:::;

[J




T 11+1

=

, T

In! {t ~

u ~ 1

fl

Tn

I

y (t)

ED}.

n

~ 0

The process y (t) in the brackets is the process defined by (G.3), i.e. with initial condition y. Let us set Yn = Y (Tn

).

n ~

1.

Then Y n E PI and is a 1vlarkov chain with tmn-

sition probability defined by {B. IS)

\Ve define the following operator on Borel bounded functions on PI (B.IO)

We can give an analytic formula as follows. Consider the problem A ~ - g (y ,v (y)) • D ~ = 0 in D I •

d

I\

ifJ.

(6.20)

We first note that Eti= ifJ(y;: (O(a: »)) = Ev;: dy.l (0' (a:)

therefore taking account of (6.17) , we have P ¢(a:) = 17(a:)

(6.21)

where 17 denotes the solution of (6.16) corresponding to the boundary condition It Of course, in (6.21) x

E

rl

=~.

are the only relevant points. \'\'e then have

Lemma 6.1. The operator P is ergodic. Proof. \Ve proceed as in the proof of Lemma 3.1. Indeed, defining

\-f a:.y E r

l•

B Borel subset of P 1

everything amounts to showing that sup

U

,I.l/,e

>";:~(B)


o(Y) mtl(y) (ly

J I(y,v(y)) mfl{y) dy.

~

Hence. as a tends to 0, x ~

and since

11 ( • )

J I(y,v(y)) mil

(y) dy

is arbitrary X ~ X'. Therefore, (B.5\)) is proved .

.

Let v be a feedback associated to tP. where 4> is any solu tion of (6.53). Let us show that

(6.73)

Indeed call X the right hand side of (6.50). \Ve have A 4> - (} v

..

•

D 4> + X - X

=

-

f (y ,v) -

-

and Ji(y)

m~ (y) dy = o.

\Ve deduce as in (6.62)

r_

= -

J

Ef

l(y(t))tlt

Q

bence,

-

(X - X ) E;!'

TN

bounded in N.

However.

E;!'

TN -

+

00

as N -

since

= E; Z (Y:r (II and

(x))

I

:z

00,

X

-

= I (y)

251

D (rpt -

interchanging

,pl

4/2 ) +

o.

and ,p':l, leads ftnally to D (,p1 - ¢'2)

0,

and this completes the proof of

the uniqueness. 0

7. Singular perturbations with diffusions in the whole space 7.1 Setting of the problem Again we basically consider the setting of section 2, in which we shall drop the aBsurnptions of periodicity as far as the fast system is concerned. We consider / (x ,y ,v) : JRfl X n 4 X U _ 11111

U (x ,Y ,v) :

nil x ill:' X U _

I (x ,y ,v) : nil X n

J

X U -

(7.1)

JRd JR

continuous bounded

Uad compact of U (metrIc space).

On a convenient set (O,A ,Ft .PC) (c.f.

(7.2)

section 2.1), we deftne a dynamic system,

composed of a slow and a fast system described by the equations (2.7), with g replaced by Fy

+

g (x.y.v). The cost function is defined by (2.8). and we are interested in the

behavior of the value function u (x ,y). It is given as the solution of the H.J.B. equation (noting

Ay

= -

~y

-

Fy • D) A:t tiE -

.!.A f

II

U

It,

Uf

+ f3

tl ,

c = 0 for x

= If (x

,D,z

tI

,.y. -1 f

E r,

Dv u ,)

(7.3)

\1 y

E W 2,P,P(O X n d ). 2~p

d (c) then (5.10) is true Vn> dec). The theorem 3 is deduced

easily from this result.

Remark: In Tkiouat [16] a method is given to compute for each state x a bound q(x) on the size of vectors on which ,,,e have to make the vectorial minimization. 6-Example and application Let us show on a trivial 2 time scale example hO\II these results can be useful to design fast algoritlnn to solve stochastic control problem. Let us take the most simple example

305

using the particular structure of \Ii, and the results explained in Delebecque, that is \'1e compute solution of:

and P2 solution of :

. P,- ['] 1

1

=

Then \ye compute the aggregate \<Jhain transition matrix

0

P,

0

A= 0

0

0

0

A

Pz

0 0

and the aggregated cost

C, = P,

[CC']'

z

c3

z = Pz rtc

C

1

4

307

Then it is possible to improve the strategy by minimizing in u all the entries of the 4 - vector

\'Ie have seen that in this process we have never solved a linear of size 4 but

three systems of size 2, and 4 minimizations. generaly when the matrice mO has a block diagonal structure, this perturbation method ir.1prove the speed of the 1-I0Wllrd algorithn. In the best situation \<Je can obtain a [Z [2 algorithr.1 to solve the probler.l.

~bre

In this discussion we have only compute the first tem. of the expansion "there the vectorial minimization defines completely the control after computing only the two first terms of the expansion Wo and W . 1 The algorithm is complicated to be implemented. It is done inTkiouat [16]. This method can be applied for discrete version of the followinrr diffusion process : dX t

= b,(Xt,Yt,Ut ) ,

dt

+

dYt = £" b 2 (Xt ,Yy 'Ut ) dt

t

a 1 (Xt ,Yt , Ut ) ill1 1

+ ~

?

aZ(Xt,Yt,l't J d1'J~

that is diffusion process having b~o time scales. Some dam management problems can be described in this fOTQalism see Delebecque-Quadrat [19J

309

[11] R. PHILIPS, P. KOKafOVIC. A singular perturbation approach to modellin!) and control of

[12J H. SHDN, A.

~0 lJ-

(2.12)

'faking N

L

(x,

),

o

(2.13)

i=l

with unspecified e

I

1

s as a Lyapunov function candidate for (2.9),

it can be shown that the derivative of W along the trajectDry of

(2.9) satisfies dW

\'There (p T

0. i.e. wet) is persistently exciting. Thus persistent excitation of w is a necessary condition for exponential stability of (5.19).

(ii)

If

m(s)

is strictly positive real and w (t ) is persistently exciting, then R".m \4"m! (0) is

Hurwitz. Hence

m(s)

being strictly positive real is a sufficient condition for stabil-

ity of (5.19), given that w (t ) is persistently exciting.

It is intuitive that if w is persistently exciting and

m(s)

is close in some sense to

being strictly positive real that R".m Ii"tn! (0) will be Hurwitz (in particular. this is the case if Re m. (j v) fails to be posi~ive at frequencies where

n (j v)

is small enough). More specific

results in this context are in [11.17). In view of the results stated at the end of section 5. averaging can also be applied to the nonlinear system described by (5.10)-(5.11). with A {x

)=..1 +bx T Q.

Consequently.

the nonlinear time varying adaptive control scheme can be analyzed through the autonomous averaged system (a generalisation of the ideas of [24]). However. due to the nonlinearity of the system. the frequency domain analysis. and the derivation of guaranteed convergence rates are not straightforward.

409

e. -1.5 -3,

·~,S

·it

\ r

-7.5

\/

(a)

e, -l.:~

-2,~

-l,iS

-~.

-:.:: I

1

(b)

·7« S ~ O.

(c) Fig 5.3 Trajectories of parameter error q,i:= da-do ) and ¢'1J2 with three adapta.tion gaiDs (a) t==l (b) f=0.5 (e) f=O.l using log scale.

411

6. Concluding Remarks We have presented in this paper new stability theorems for averaging analysis of one and two time scale systems. We have applied these techniques to obtain bounds on the rates of convergence of adaptive identifiers and controllers of relative degree L We feel that the techniques presented here can be extended to obtain instability theorems for averaging. Such theorems could be used to study the mechanism of slow drift instability in adaptive schemes in the presence of unmodelled dynamics. in a framework resembling that of [10].

413

Thjs. in turn. implies that

(LIO) Clearly g(e)EK. From (Ll), it follows that:

aWE(t,X)

---::-,--- - d (t ,x ) = -e

at

W

E(t

.X )

(Lll)

so that both (2.6) and (2.7) are satisfied. If y(T )=a / T r , then the right-hand side of (L8) can be computed explicitly:

(L12) and. with

r

denoting the standard gamma function: co

Ja e

r

('J",)l-r e-r'd T';; a e r r(2-r) ~a e r

(L13)

o

Defining g(e)=2a e r • the second part of the lemma is verified.

Proof of Lemma 2.2 Define W E(t..x) as in Lemma 2.1. Consequently,

(L14) · ad OX (t .X) IS . zero mean. an d'IS b ound e. d Lemma. 2 1 can be app I"Ie d to ad aX (t .x) • an d Slnce

inequality (2.6) of Lemma 2.1 becomes inequality (2.10) of Lemma 2.2. Note that since ad ~t;) is bounded. and d (t ,0)=0 for all t

~O. d (t ..x)

is Lipschitz. Since d (t..x) is zero

mean, with convergence function yeT) Ox D. the proof of Lemma 2.1 can be extended. with an additional factor Ox H. This leads directly to (2.8) and (2.9) (although the function

gee)

od ~~,x) • these functions can be replaced

by a

may be different from that obtained with single

gee )).

415

+E ( f

(t .z .€)- f (t ,z .0) )

:= ef av (z)

+ Ep '(t

(L18)

,x ,z .e)

where. using the assumptions. and the results of Lemma. 2.2: (L19)

For E ~El' (2.10) implies that (J +E

OlY az) E

,

has a bounded inverse for all t ~O. z EBh .

r

Consequently. z satisfies the differential equation:

i = [I+e =

Ef

av

1l;:'

(z )

(e!,,(z)+ep'(t,z,e))

+ E P (t ,z .E)

Z

(L20)

(O)=xo

where: p(t.z,E)=

!

aWE

l+e-_a~

-1

OWe Ip'(t.z.e)-e-_-!a\'(Z) I a~

1

(L21)

and:

:= .pCe) II z I

(L22)

Generalized Bellman-Gronwall Lemma (cf. [71 P 169) If: x (t ). a (t ), u (r ) are positive functions satisfying: t

x (t ) :£;.

Jo a (

T )x ( T )d T

+ u (t )

(L23)

for all t e[O.T). and u(t) is differentiable.

Then: t

x(t) ~u(O)e

ja(a)dCT 0

t

+

z

J ti(r)c o

for all t E[O.T].

jaCa)da T

dT

(L24)

417

[15] Bodson. M. and S. Sastry. liSman Signal 1/0 Stability of Nonlinear Control Systems: Application to the Robustness of n MRAC Scheme." Memorandum No. UCB/ERL

M84/70. Electronics Research Laboratory, University of California. Berkeley, 1984. [16] Riedle. B. and P. Kokotovic. "Stability Analysis of Adaptive System with Unmodelled Dynamics." to appear in 1nt. 1. of Control. Vol. 41. 1985. [17] Kosut. R .. B.D.O. Anderson and I. Mnreels. "Stability Theory for Adaptive Systems: Method of Averaging and Persistency of Excitntion/' Preprint. Feb. 1985. [18] Nnrendra, K. and L Valavani ... Stable Adaptive Controller Design-Direct Control,'·

IEEE Trans. on Autonuztic Control, Vol. AC-23 (1978). pp 570-583. [19] Sastry. S., "Model Reference Adaptive Control - Stability. Parameter Convergence. and RObustness." IMA Journal of Mathemotical Control & Information, Vol. 1 (1984), pp 27-66.

[20] Hahn. W.o Stability of Motion, Springer Verlag. Berlin. 1967. [21] Luders. G. and K.S. Narendra. II An Adaptive Observer and Identifier for a Linear System," IEEE Trans. on Automatic Control, Vol. AC-18 (1973). pp. 496-499. [22] Kreisselmeier. G.

\I

Adaptive Observers with Exponential Rate of Convergence."

IEEE

Trans. on Automatic Control, Vol. AC-22 (1977). pp. 2-8. [23] Goodwin. G. C. and R. L Payne. Dynamic System Identification, Academic Press. New York. 1977. [24] Riedle B. and P. Kokotovic. "Integral Manifold Approach to Slow Adaptation," Report DC-80. University of Illinois. March 1985.