Extensions of linear-quadratic control, optimization and matrix theory

Extensions of Linear-Quadratic Control, Optimization and Matrix Theory DAVID H. JACOBSON National Research Institute ,...

Author: David H. Jacobson

81 downloads 687 Views 2MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Extensions of Linear-Quadratic Control, Optimization and Matrix Theory

DAVID H. JACOBSON National Research Institute ,for Mathematical Sciences Council for Scientijic and Industrial Research, South Africa (Honorary Professor in the University of the Witwatersrand)

1977

Academic Press London

*

New York

.

San Francisco

A Subsidiary of Harcourt Brace Jovanovich. Publishers

ACADEMIC PRESS INC. (LONDON) LTD 24/28 Oval Road London NWl United States Edition published by

ACADEMIC PRESS INC. 111 Fifth Avenue New York, New York 10003

Copyright 0 1977 by ACADEMIC PRESS INC. (LONDON) LTD

AN Rights Reserved

No part of this book may be reproduced in any form by photostat, microfilm, or any other means, without written permission from the publishers

Library of Congress Catalog Card Number: 77-76545 ISBN 0-12-378750-5

Printed in Great Britain by Galliard (Printers) Ltd, Great Yarmouth, Norfolk

PREFACE

Control, optimization and matrix theory are closelay 1 inked in many ways, perhaps most strongly by the linear-quadratic aspects they have in common. The present work seeks t o extend, develop and strengthen t h i s link by presenting a number of extensions of the we1 1-known 1 inear-quadratic theories. Consequently i t should prove t o be particularly useful t o graduate students, teachers and researchers in science and engineering. In a very definite sense t h i s i s a personal volume - i t reflects my attempts over the past f i v e years t o understand and analyse non-linear systems and t o contribute new develop-

ments. Inevitably some of the material presented has previously appeared i n one or another form elsewhere in the l i t e r a t u r e b u t many results are being made known here for the f i r s t time. Certain o f the results presented in Chapters 2 , 3 , 5 and 6 were developed j o i n t l y w i t h Drs. J.L. Speyer, W.M. Getz, M. Pachter and C.A. Botsaris; t h e i r contribution and cooperation i s gratefully acknowledged. Dr. D.J. Bell k i n d l y perused the d r a f t manuscript. Warm thanks are due t o the Executive of the Council for Scientific and Industrial Research and t o Professor R . E . Bellman who encouraged the project, t o Mr. F.R. Baudert and Dr. D. de Jongh who edited the d r a f t manuscript and proofread the typescript, and t o my secretary Elsa de Beer who expertly typed the volume. David H. Jacobson Pretoria, 1977 V

LYNNE

LlSA ' W E N

CONTENTS PREFACE

V

vi

DED ICATION 1.

INTRODUCTION 1.1 References

1 8

2.

NON-LINEAR-QUADRATIC CONTROL PROBLEMS 2.1 Exponential Performance Criterion Perfect Measurements 2.1.1 Discrete-time Formulation 2.1.2 Discrete-time Solution 2.1.3 Properties of Discrete-time Solution (a = -1) 2.1.4 Properties of Discrete-time Solution (a = +1) 2.1.5 Relation to Discrete-time Games 2.1.6 Continuous-time Formulation 2.1.7 Continuous-time Solution 2.1.8 Relation to Differential Games 2.1.9 Stability Properties 2.2 Exponential Performance Criterion Noisy Measurements 2.2.1 Discrete-time Formulation 2.2.2 Discrete-time Solution - Terminal Cost Only 2.2.3 Discrete-time Solution - The General Case 2.2.4 No Process Noise 2.2.5 Cont i nuous-t ime Formul ati on 2.2.6 A Note on Separation 2.2.7 A Further Separation Property 2.3 Non-linear Stochastic Systems 2.3.1 Discrete-time Formulation 2.3.2 Solution 2.3.3 Known Special Cases 2.3.4 Novel Special Cases 2.4 Infinite-time Optimal Control

9

vi i

9 9 11 15 17 18 19 20 21 22 24 25 27 32 35 38 40 41 43 43 45 47 49 50

viii

CONTENTS Continuous-time Formulation Bellman-type Theorems Certain Non-linear Problems Non-quadratic Performance Criterion Systems Homogeneous-in-the-input 2.5.1 Formulation 2.5.2 Stabilizability 2.5.3 A Bilinear Example Optimal Control 2.5.4 Non-homogeneous Extension 2.5.5 Optima Control of Quadratic Systems 2.6.1 Formulation 2.6.2 Solutions Conclusion References 2.4.1 2.4.2 2.4.3 2.4.4

2.5

2.6

2.7 2.8 3.

COPOSITIVE MATRICES, NON-CONVEX QUADRATIC FORMS AND QUADRATIC DIFFERENTIAL EQUATIONS 3.1 Introduction to Copositive Matrices 3.1.1 Properties of Copositive Matrices 3.2 Extensions 3.2.1 General Linear Constraints 3.2.2 An Example 3.2.3 Application to Quadratic Programming 3.2.4 Stochastically Copositive Matrices 3.2.5 Symmetric M-Matrices 3.2.6 Generalization of Finsler's Theorem 3.2.7 Application of Generalized Finsler Theorem 3.3 Quadratic Differential Equations 3.3.1 Finite Escape Times 3.3.2 Matrix Riccati Equation 3.3.3 An Example 3.3.4 Models in Population Dynamics 3.4 Invariant Sets 3.4.1 An Example 3.4.2 General Case

50 51 54 56 57 57 58 59 60 64 65 65 66 69 71 73 73 74 79 79 82 83 85 87 90 95 97 98 105 108 110 112 115 116

CONT E NTS 3.5 3.6

Conclusion References

4. NON-NEGATIVITY CONDITIONS FOR CONSTRA NED AND NON-QUADRATIC FUNCTIONALS 4.1 Linear-Quadratic Case 4.1.1 Non-negativity Conditions 4.1.2 Novel Non-negativity Cond tions 4.1.3 Exampl e 4.2 Constrained Case 4.2.1 Formul at ion 4.2.2 Sufficiency Conditions 4.3 Non-linear Systems 4.3.1 Formulation 4.3.2 Non-negativi ty Conditions 4.3.3 An Application 4.4 Conclusion 4.5 References 5. CONTROLLABILITY OF CONSTRAINED LINEAR AUTONOMOUS SYSTEMS 5.1 Introduction 5.1.1 Unconstrained Case 5.1.2 Zero Interior to R 5.1.3 Reachable Sets 5.1.4 Zero Interior to Convhull(R) 5.1.5 Global and Non-linear Results 5.2 Zero not Interior to Constraint Set or its Convex Hull 5.2.1 Oscillatory Systems 5.2.2 Bramer ' s Theorem 5.2.3 Global Results 5.3 Arbitrary-interval Null-controllabil t Y 5.3.1 Preliminary Results 5.3.2 Necessary and Sufficient Cond tions 5.3.3 Necessary Conditions 5.3.4 Necessary and Sufficient Cond tions Finitely Generated Cones

ix

119 120 123 123 124 129 133 134 134 134 141 142 143 144 148 149 151 151 152 153 155 157 160 161 161 163 166 167 169 173 177 179

CONTENTS

X

5.3.5 5.3.6 5.3.7 5.3.8

6.

183 186 188 194

5.4

Conclusion

195

5.5 5.A

References

196

Appendix

197

NEW APPROACHES TO FUNCTION M I N I M I Z A T I O N

199

6.1

Introduction

199

6.2

Homogeneous Model s 6.2.1 R e l a t i o n t o Newton's Formula 6.2.2 A1 g o r i thms based upon Homogeneous Functions

200

D i f f e r e n t i a l Descent

204

6.3.1

2 04

6.3

7.

Examples Minimum Time Function Further Necessary Conditions and S u f f i c i e n t Conditions Further Special- Cases

O u t l i n e o f t h e Approach

201 202

6.4

Conclusion

209

6.5

References

209

CONCLUSION

211

AUTHOR INDEX

213

SUBJECT INDEX

215

1.

INTRODUCTION

The treatments o f 1 inear-quadratic c o n t r o l problems given i n

ill a r e probably t h e most comprehensive a v a i l a b l e , though by now a l i t t l e dated i n some respects. Both d i s c r e t e - t i m e and continuous-time formulations a r e t r e a t e d i n t h a t reference, and v a r i a t i o n a l and dynamic programming techniques a r e used i n the generation o f s o l u t i o n s .

A perusal o f [l] thus

provides a r a t h e r good sample o f formul a t i o n s and techniques, i n a d d i t i o n t o a good l i s t of textbooks and o t h e r references. I n t h i s s h o r t chapter we do n o t very exhaustively review t h e m a t e r i a l i n [l] (most o f which i s a l s o a v a i l a b l e elsewhere), but r a t h e r describe l o o s e l y b u t adequately what, f o r t h e purpose of t h i s monograph, c o n s t i t u t e s a 1 inear-quadratic formulation i n c o n t r o l , o p t i m i z a t i o n o r m a t r i x theory. We then o u t l i n e t h e contents o f t h e f o l l o w i n g chapters i n some d e t a i l i n o r d e r t o e l u c i d a t e t h e magnitude o f t h e 'extensions' presented. Broadly speaking a 1inear-quadratic (Gaussian) c o n t r o l ( v a r i a t i o n a l ) f o r m u l a t i o n c o n s i s t s o f a finite-dimensional l i n e a r , d i s c r e t e - o r continuous-time, dynamic system which i s t o be c o n t r o l l e d i n such a way as t o minimize t h e value ( o r expected value) o f a performance c r i t e r i o n which i s t h e i n t e g r a l , o r sum, o f q u a d r a t i c f u n c t i o n s o f t h e system s t a t e and c o n t r o l v a r i a b l e s plus, perhaps, a quadratic f u n c t i o n o f the s t a t e a t t h e t e r m i n a l time. Stochastic formulations a l l o w a d d i t i v e Gaussian w h i t e noise t o d i s t u r b t h e dynamic system, and t h e o u t p u t s t h a t can be measured a r e assumed t o be l i n e a r f u n c t i o n s o f t h e s t a t e and Gaussian w h i t e noise. The most celebrated p r o p e r t y o f t h e s o l u t i o n o f t h e non1

2

L INEAR-QUADRATIC EXTENSIONS

s i n g u l a r 1inear-quadratic-problem ( o f t e n r e f e r r e d t o as an

LQP o r LQG) i s t h a t t h e optimal c o n t r o l i s a l i n e a r (timevarying) f u n c t i o n o f t h e s t a t e or, i n t h e s t o c h a s t i c case, the best estimate o f t h e s t a t e .

The m a t r i x R i c c a t i equation

and, i n t h e s i n g u l a r case [ Z ] , m a t r i x i n e q u a l i t i e s p l a y a special r o l e i n ensuring t h e existence o f t h e s o l u t i o n . Questions and assumptions r e l a t i n g t o s t a b i l i t y , c o n t r o l l a b i l i t y and o b s e r v a b i l i t y o f t h e system a r e a l s o important here. By a l i n e a r - q u a d r a t i c f o r m u l a t i o n i n m a t r i x theory we mean

t h e study o f t h e p r o p e r t i e s o f p o s i t i v e (semi-) d e f i n i t e quadratic f u n c t i o n s o f a f i n i t e number o f v a r i a b l e s and t h e i r r e l a t i o n t o l i n e a r e q u a l i t i e s and i n e q u a l i t i e s .

Here i t i s

simply t h e d e f i n i t e n e s s o f t h e quadratic f u n c t i o n t h a t makes the f o r m u l a t i o n standard. I f t h i s assumption i s relaxed we have immediately a non-convex quadratic f u n c t i o n which has p r o p e r t i e s n o t shared by t h e convex ( p o s i t i v e semi-definite) case. We use t h e term ' o p t i m i z a t i o n ' , as d i s t i n c t from 'optimal c o n t r o l ' , t o describe t h e problem o f f i n d i n g a minimum o f a f u n c t i o n of a f i n i t e number o f v a r i a b l e s subject t o e q u a l i t y and i n e q u a l i t y c o n s t r a i n t s ,

'Linear-quadratic'

i n t h i s con-

t e x t r e f e r s t o t h e f a c t t h a t algorithms f o r t h e s o l u t i o n o f t h e m i n i m i z a t i o n problem a r e almost always based upon a model based i n t u r n on t h e assumptions t h a t t h e f u n c t i o n t o be minimized i s a p o s i t i v e - d e f i n i t e quadratic form and t h a t the c o n s t r a i n t s a r e 1i n e a r . It i s f a i r l y evident from t h e foregoing d e s c r i p t i o n s t h a t the

1inear-quadratic thread t h a t runs through c o n t r o l , optimiza-

3

INTRODUCTION t i o n and m a t r i x theory forms a strong conceptual and operat i o n a l t i e between them.

Consequently i n t h i s monograph

c o n t r o l , o p t i m i z a t i o n and m a t r i x theory are n o t s t r i c t l y confined t o separate chapters: i n f a c t each o f them i s concerned w i t h these t h r e e subjects o f study t o a greater o r lesser extent. I n Chapter 2 we 'extend' t h e l i n e a r - q u a d r a t i c c o n t r o l problem by f i r s t r e p l a c i n g t h e quadratic performance c r i t e r i o n by the exponential o f a quadratic f u n c t i o n .

I n the d e t e r m i n i s t i c

case we g a i n nothing by t h i s move, as minimization of an exponential o f a f u n c t i o n i s equivalent t o minimization of t h a t f u n c t i o n a l , b u t i n t h e s t o c h a s t i c case a new, i n t e r e s t i n g formulation r e s u l t s . I f t h e s t a t e i s p e r f e c t l y measurable but Gaussian w h i t e noise enters l i n e a r l y i n t o the l i n e a r system, we f i n d t h a t the optimal feedback c o n t r o l l e r i s l i n e a r , as i n t h e l i n e a r - q u a d r a t i c case, b u t t h a t t h e c o n t r o l l e r depends upon t h e s t a t i s t i c s o f t h e noise, u n l i k e t h a t f o r the

1inear-quadratic case. I t t u r n s out, i n t e r e s t i n g l y , t h a t the c o n t r o l l e r i s equivalent t o t h a t obtained when the noise i s t r e a t e d as a ' b e l l i g e r e n t p l a y e r ' i n a two-person zero-sum l i n e a r - q u a d r a t i c game, and t h i s provides new j u s t i f i c a t i o n f o r t h i s type o f 'worst case' c o n t r o l l e r design.

Ift h e

measurement o f t h e s t a t e i s noise-corrupted, t h e optimal feedback c o n t r o l l e r r e t a i n s i t s l i n e a r character b u t i s i n the general case i n f i n i t e - d i m e n s i o n a l .

This i s another

s u r p r i s e when i t i s r e c a l l e d t h a t i n t h e l i n e a r - q u a d r a t i c Gaussian case t h e c o n t r o l l e r t u r n s o u t t o be the f i n i t e dimensional optimal c o n t r o l l e r f o r t h e d e t e r m i n i s t i c case simply w i t h t h e s t a t e replaced by the best (Kalman) f i l t e r e d estimate o f t h e s t a t e . An a p p l i c a t i o n due t o Speyer of t h e

4

L INEAR-QUADRAT1C EXTENSIONS

exponential f o r m u l a t i o n and s o l u t i o n t o homing m i s s i l e guidance i s a l s o mentioned. The next ' e x t e n s i o n ' i s obtained by g e n e r a l i z i n g t h e l i n e a r dynamic system t o a c l a s s o f r e s t r i c t e d non-1 i n e a r s t o c h a s t i c systems w h i l e r e t a i n i n g t h e quadratic performance c r i t e r i o n . The optimal c o n t r o l l e r i s here l i n e a r i n t h e system s t a t e b u t depends upon t h e n o i s e parameters.

Known r e s u l t s due t o

Wonham, McLane and Kleinmann f o r l i n e a r systems w i t h m u l t i p l i c a t i v e noise a r e generalized here. Next, we t u r n t o t h e c l a s s o f non-linear systems homogeneousin-the-input. We demonstrate t h a t such systems a r e asymptot i c a l l y s t a b i l i z a b l e under c e r t a i n c o n d i t i o n s which a r e almost necessary and s u f f i c i e n t .

Furthermore, we show t h a t s t a b i l i z -

i n g c o n t r o l l e r s a c t u a l l y minimize a wide v a r i e t y o f nonquadratic performance c r i t e r i a . We a l s o o b t a i n t h e s o l u t i o n t o t h e problem o f minimizing a c e r t a i n non-quadratic performance c r i t e r i o n s u b j e c t t o a l i n e a r dynamic c o n s t r a i n t .

This r e s u l t i s generalized by

Speyer t o a s t o c h a s t i c version which involves c o n t r o l of a 1 i n e a r s t o c h a s t i c dynamic system d r i v e n by a d d i t i v e and state-dependent white-noise processes. F i n a l l y i n t h i s chapter we study t h e c o n t r o l o f systems of quadratic and b i l i n e a r d i f f e r e n t i a l equations and o b t a i n some l i m i t e d r e s u l t s , v i z . t h a t f o r a c e r t a i n c l a s s o f problems the optimal feedback c o n t r o l l e r i s l i n e a r . Taken as a whole, Chapter 2 i l l u s t r a t e s t h a t t h e l i n e a r quadratic c o n t r o l problem has been extended i n n o n - t r i v i a l ways both by using performance c r i t e r i a more general than

INTRODUCTION

5

quadratic and by introducing classes of non-linear dynamic systems. These give rise to both linear and non-linear control 1 ers

.

In Chapter 3 we begin with matrix theory. First, copositive matrices are introduced. Quite simply a symmetric matrix is copositive if its associated quadratic form is non-negative for all vectors having non-negative elements. Interestingly, it turns out that all copositive matrices are sums of positive semi-definite matrices and matrices with non-negative elements (non-negative matrices) if and only if the dimensionality of the matrix is less than five. We show that this also implies that all positive semi-definite non-negative matrices have non-negative factorizations if and only if they are of dimension less than five. We show further that the representation for copositive matrices extends beyond dimension five if a more general type of copositivity, viz. stochastic coposi tivi ty, is defined. Closely related to copositive quadratic forms is the question of non-negativity of a quadratic form subject to equality and inequality quadratic constraints. I n the case of one constraint Finsler's theorem provides a complete answer, and in the case o f an arbitrary number of constraints we extend Finsler's theorem to provide a useful sufficient condition. We use this extension to yield insight into the properties of the inverse o f copositive matrices. We then turn to symmetric M-matrices which are in fact positive-definite, and whose inverses are both positivedefinite and non-negative. We show that these inverses have non-negat i ve factorizations .

6


Next we apply c o p o s i t i v e m a t r i x theory t o t h e non-convex quadratic programming problem t o provide s u f f i c i e n t c o n d i t i o n s f o r optimal i t y . The remainder o f Chapter 3 i s concerned w i t h a study o f t h e behaviour o f s o l u t i o n s o f systems o f autonomous quadratic d i f f e r e n t i a l equations.

S p e c i f i c a l l y we develop two sets o f

s u f f i c i e n t conditions f o r a solution t o e x h i b i t a f i n i t e escape time. The f i r s t s e t i s s i m i l a r t o c e r t a i n c o n d i t i o n s obtained by Freeman, w h i l e t h e second set, being based upon our r e s u l t s f o r non-convex q u a d r a t i c p r o g r a m i n g derived e a r l i e r , i s l e s s r e s t r i c t i v e owing t o our n o n - t r i v i a l use of the n o t i o n o f i n v a r i a n t sets. Chapter 4 c o n t a i n s what we b e l i e v e a r e s i g n i f i c a n t extensions of our r e s u l t s i n functional s .

PI

f o r t h e n o n - n e g a t i v i t y o f quadratic

F i r s t we review and r e f o r m u l a t e c e r t a i n impor-

t a n t s u f f i c i e n t c o n d i t i o n s f o r t h e n o n - n e g a t i v i t y o f uncons t r a i n e d q u a d r a t i c f u n c t i o n a l s and extend these t o t h e case where t h e c o n t r o l v a r i a b l e s a r e constrained.

A novel R i c c a t i

d i f f e r e n t i a l equation r e s u l t s from t h i s approach. Next we f u r t h e r extend these s u f f i c i e n t c o n d i t i o n s t o a general c l a s s o f non-quadratic, non-1 inear, constrained problems. Our r e s u l t s bear a resemblance t o c e r t a i n c o n t r o l l a b i l i t y condit i o n s d e r i v e d by Kunzi and Davison, and a l l o w us t o r e l a t e t h e n o n - n e g a t i v i t y o f non-quadratic f u n c t i o n a l s t o t h a t o f a c l a s s o f non-autonomous q u a d r a t i c f u n c t i o n a l s . Chapter 5 i s concerned w i t h t h e c o n t r o l l a b i l i t y o f autonomous l i n e a r dynamic systems i n which t h e c o n t r o l v a r i a b l e s a r e constrained t o l i e w i t h i n a c e r t a i n c o n s t r a i n t set. w e l l known t h a t ,

It i s

provided zero belongs t o t h e i n t e r i o r of t h e

INTRODUCTION

7

convex hull of the constraint set, such a linear system is null-controllable if and only if it is completely controllable when the constraint is removed. More recently Brammer has provided necessary and sufficient conditions for nullcontrollability when zero does not belong to the interior of the convex hull of the constraint set. Arbi trary-interval null-controllability, introduced in Chapter 5, requires that the system be controllable on any time interval, this being a more demanding requirement than null-controllability. As is well known, a system is arbitrary-interval null-controllable if it is null-controllable and if zero belongs to the interior of the convex hull of the constraint set, The main purpose of Chapter 5, then, is to provide necessary and sufficient conditions for arbi trary-interval nu1 1 -control 1 ability when the constraint set is of general type. Most interesting is the role of arbitrary-interval null-controllability as a necessary and sufficient condition for continuity of the minimum time function in time-optimal control of an autonomous 1 inear dynamic system.

In Chapter 6 we proceed to function minimization. We discuss the properties of a homogeneous model in comparison with a quadratic model and refer to a convergent algorithm for use on general functions. We also refer to the recent work of Kowalik who has further improved the effectiveness of the algorithm by introducing a highly stable numerical method in place of the Householder updating used in the first versions of the homogeneous algorithms. Next in Chapter 6 we introduce the differential descent approach presented in [3] and further developed extensively

LINEAR-QUADRATIC EXTENSIONS

8

by Botsaris. In this approach curvilinear, as opposed to linear search paths are used, which are developed by approximating the trajectories of steepest descent in appropriate ways. Such methods have considerable advantages in that they do not fail when Newton's method does, and automatically behave as gradient methods when far from the minimum of the function to be minimized, and as N wton's method when in the neighbourhood of the minimum. Chapter 7 briefly assesses the ear ier ones and indicates areas for further research. 1.1

References

111

IEEE Transactions on Automatic Control, vol. AC-19, December 1971, 'Special Issue on the Linear-QuadraticGaussian Problem'.

[2]

BELL, D.J. & JACOBSON, D.H. Singular Optimal Control Problems. Academic Press, New York and London, 1975.

[31

BOTSARIS, C.A. Differential Descent Methods for Function Minimization. Ph.D. Thesis, University of the Witwatersrand, Johannesburg, South Africa, 1975.

2.

NON-LINEAR-QUADRATIC CONTROL PROBLEMS

2.1

Exponential Performance C r i t e r i o n

-

P e r f e c t Measurements

We consider i n t h i s s e c t i o n t h e optimal c o n t r o l o f a l i n e a r discrete-time dynamic system d i s t u r b e d by a d d i t i v e Gaussian white noise.

I n place o f a quadratic performance c r i t e r i o n

we use an exponential one [11-[31.

We assume t h a t the s t a t e

o f t h e system can be measured p e r f e c t l y . The assumption o f Gaussian noise i s d e l i b e r a t e

-

indeed i t i s

the exponential nature of t h e Gaussian d e n s i t y f u n c t i o n which matches t h e exponential nature o f t h e performance c r i t e r i o n and r e s u l t s i n t h e l i n e a r form o f t h e optimal feedback c o n t r o l 1e r

.

2.1.1

Discrete-time Formulation

We consider a 1i n e a r d i s c r e t e - t i m e dynamic system described bY xktl

= Akxk t

B ku k

t

rkuk,

k=O,

...,N-1;

xo given (2.1.1)

where t h e ' s t a t e ' vector xk

E

R"', The known matrices Ak,

R ~ ,t h e c o n t r o l vector Uk

and t h e Gaussian noise i n p u t uk

E

Rq.

E

Bk, rk have a p p r o p r i a t e dimensions and may vary as a function o f t h e index k. The noise i n p u t i s a sequence o f independently d i s t r i b u t e d Gaussian random v a r i a b l e s having p r o b a b i l i t y d e n s i t y

9

10

L INEAR-QUADRAT I C EXTENSIONS

where

with

>0

Pk

(positive definite),

k=O

,...,N-1.

(2.1.4)

where E denotes t h e expected value operator. The performance c r i t e r i o n which we minimize i n order t o o b t a i n a d e s i r a b l e c o n t r o l l e r f o r (2.1.1),

i s s p e c i f i e d by

(2.1.6) where

Qk > 0 ( p o s i t i v e semi-definite),

Rk and where

>0 (J

(positive definite),

k=O,

k=O,

...,N

...,N-1

(2.1.7) (2.1.8)

= -1 o r t1.

We s h a l l r e f e r t o (2.1.6)

as a negative ( p o s i t i v e ) exponential

performance c r i t e r i o n when

(J

= -l(tl).

We wish t o determine a Borel-measurable f u n c t i o n C i such t h a t the control policy 0

4

Uk -

0

k ( x k ),

k=O

,...,N-1;

xk

A

= (Xo,XI,

...,X k } (2.1.9)

CONTROL PROBLEMS

11

Note t h a t f o r an a r b i t r a r y choice of

minimizes (2.1.6).

c o n t r o l s {uk}, V-(xo) and Vt(xo) a r e bounded according t o -1

< v-(xo) < 0,

The i n e q u a l i t i e s (2.1.10)

1G

v t (xo).

(2.1.10)

imply t h a t Vt(xo) may become

unbounded from above w h i l e V-(xo) cannot. Note t h a t i f t h e r e i s no noise present, i . e . Wk E 0, minimization o f (2.1.6)

k=O,.

..,N-1

(2.1.11)

i s e q u i v a l e n t . t o minimization of

subject t o xktl

= A x

k k

t Bkuk,

k=O,

...,N-1

(2.1.13)

which i s t h e standard l i n e a r - q u a d r a t i c problem (LQP). 2.1.2

Discrete-time S o l u t i o n

We d e f i n e J'(Xk;k)

A

=

N-1 T T min E u exp{a[ Z (x.Q.x.tu.R.u.)tx Q x 1) UkS***3UN-1 I x k i=k i i i i i i N N N (2.1.14)

so t h a t a t t i m e k + l

L INEAR-QUADRATIC EXTENS IONS

12

(2.1.15) Equations (2.1.14) and (2.1.15) y i e l d

= A x

xktl

k k

B u k k

t

+ rkok,

The c o n d i t i o n a l expectation i n (2.1.16)

x

k

given.

(2.1.17)

may now be w r i t t e n

explicitly to yield J"(Xk;k)

= min[exp{-(x a TQ X

2

+U T R U

k k k

k k k

)}IP(wk;k)Ja(Xk+l;k+l)dWkl m

-00

'k

(2.1.18)

and t h e boundary c o n d i t i o n a t k=N f o l l o w s from (2.1.14)

as

We now prove t h a t Ja(Xk;k) which i s defined f o r k=O, k=O,

...,N,

i s given by

U F i eXp(T o XkQk}, T

...,N,

solves (2.1.18)

(2.1.20) where W F > O ,

13

CONTROL PROBLEMS

(2.1.22) and where O

(2.1.23)

WN = QN.

I n a d d i t i o n , we have t h a t

and t h e optimal c o n t r o l p o l i c y i s Uk O

=

-cpk

(2.1.25)

where C i = (R

tBTkak+lBk)

-1 T-o BkWktlAk,

k=O

,...,N-1.

We need t h e f o l l o w i n g Lemma i n t h e proof t h a t (2.1.20) (2.1.18).

CQ

solves

Note t h a t the Lemna depends c r i t i c a l l y upon the

exponential nature o f p(wk;k).

LEMMA

(2.1.26)

I f (Pk-OrLWi+lrk)

> 0,

then

exp{-4wkPkwk} T .exp{-o 2 xTk + lw~k+lxk+l )dwk

14

L INEAR-QUADRAT IC EXT ENS IONS

PROOF OF LEMMA

The l e f t - h a n d s i d e o f (2.1.27) m

i s equal t o

1

where

The Lemma i s proved since t h e integrand i n (2.1.29)

is a

probability density function. Returning now t o equation (2.1.18),

and using (2.1.20)

t h e Lemma, we see t h a t i t s right-hand s i d e becomes

and

15

CONTROL PROBLEMS FE.min[o exp{-(x a TQ x +uTR u ) } 2 k k k k k k 'k

OF:

a T o expC- x w x I 2 k k k

(2.1.32)

which proves, by induction, t h a t Ja(Xk;k)

= aF;

exp@ x TWax )., 2 k k k

k=O

,... ,N.

(2.1.33)

I t i s easy t o v e r i f y by i n d u c t i o n t h a t under assumptions

(2.1.4),

(2.1.7)

and (2.1.8)

W i and ii a r e well-defined

and

p o s i t i v e s e m i - d e f i n i t e and t h a t

+BT$ B ) (Rk k k + l k

> 0,

k=O,.

..,N-1.

(2.1.34)

The case when CI = +1 i s discussed below. 2.1.3

P r o p e r t i e s o f Discrete-time S o l u t i o n (a = -1)

The optimal feedback s o l u t i o n

i s a l i n e a r f u n c t i o n o f t h e system s t a t e .

This c o n t r o l l e r ,

u n l i k e t h a t f o r t h e 1 inear-quadratic problem, depends upon

Pi',

t h e covariance m a t r i x o f t h e a d d i t i v e system noise.

16


It i s i n t e r e s t i n g t o i n v e s t i g a t e here two l i m i t i n g cases:

( ' n o n o i s e ' ) ; and t h e second + t h e f i r s t i n which Xmin(Pk) i n which Xmin(Pk1) + m ( ' i n f i n i t e l y w i l d noise'). I n the f i r s t case i t i s easy t o v e r i f y t h a t

ck

+

Dk, t h e optimal

feedback g a i n m a t r i x f o r t h e d e t e r m i n i s t i c 1inear-quadratic Droblem. As P i 1

+

I n t h e second case we s h a l l assume t h a t 0 we then have t h a t

---

'ktl

-t

-1 T rkWk+l'

T T 'ktl - Wk+lrk(rkWktlrk)

k=O,.

..,N-1 (2.1.36)

and, from (2.1.20),

(2.1.24)

it follows t h a t

J-(xo;O) Note a l s o t h a t i f

-P

rk has rank n f o r k=O,.

ik+l

-+

and Ck

-+

0,

k=O

..,N-l,

..,N-1

(2.1.38)

,...,N-1.

(2.1.39)

k=O,.

0,

(2.1.37)

0.

I n words, i f a l l components o f xk a r e d i s t u r b e d by ' i n f i n i t e l y w i l d a d d i t i v e noise' t h e r e i s no p o i n t (as f a r as performance c r i t e r i o n V-(xo) i s concerned) i n e x e r c i s i n g c o n t r o l t o t r y t o counteract these

'infinite

unpredictable disturbances'.

CONTROL PROBLEMS 2.1.4

17

Properties o f Discrete-time S o l u t i o n (a = +1)

Here t h e ( l i n e a r ) optimal feedback c o n t r o l l e r i s

+

Uk

=

-c +kx k'

k=O,

...,N-1

(2.1.40)

and again we note t h a t t h e c o n t r o l l e r i s d i r e c t l y a f f e c t e d -1 + + ~1 ('no by t h e noise covariance Pk v i a Ck. When Xmin(Pk) n o i s e ' ) we see t h a t C: + Dk, as i s t h e case when u = -1. + However, i f Amin(Pil) becomes large, J (xo;O) can cease t o e x i s t . To see t h i s , we assume t h a t

rkQk+1rk T > 0, k=O,.

..sN-~

(2.1.41)

and t h a t 'j

-

r>i+lrj > 0,

j=k+l,...,N-1.

(2.1.42)

It then f o l l o w s t h a t

rLw:+lrk > o so t h a t f o r Xmin(Pil)

(2.1.43)

s u f f i c i e n t l y large (2.1.44)

which c l e a r l y i m p l i e s t h a t t h e l e f t - h a n d side o f (2.1.27) i s ' i n f i n i t e ' . Clearly, then Jt(xk;k) i s ' i n f i n i t e ' , as a r e

+

+

J (xo;O) and V (xo).

18 2.1.5

LINEAR-QUADRATIC EXTENS IONS R e l a t i o n t o Discrete-time Games

When u = -1 t h e s o l u t i o n (2.1.25)

i s e a s i l y seen t o be iden-

t i c a l t o t h e s o l u t i o n o f t h e f o l l o w i n g cooperative determinist i c game ( a c t u a l l y , a 1 inear-quadratic c o n t r o l problem):

subject t o

Note t h a t t h e above f o r m u l a t i o n determines a 1 i n e a r optimal feedback strategy-pa ir ,

We now have a new i n t e r p r e t a t i o n f o r t h e 1 inear-quadratic cooperative game. I f p l a y e r Uk assumes t h a t p l a y e r Wk w i l l cooperate i n m i n i m i z i n g t h e quadratic c r i t e r i o n , t h e optimal feedback c o n t r o l l e r ( p o l i c y ) t h a t i s obtained f o r Uk, namely

-

uk = -ckxk,

i s a l s o optimal r e l a t i v e t o an exponential

performance c r i t e r i o n , if Wk i s Gaussian w h i t e noise. When u = +1, t h e a p p r o p r i a t e game i s non-cooperative, Min Max {uk) (Wk)

if

N-1 k=O

viz.

T

(XkQkxk+U:RkUk-W~Pkwk)+fX~Q~XN]

s u b j e c t t o (2.1.46). It i s w e l l known, and easy t o v e r i f y , t h a t i f

( 2.1.48)

CONTROL PROBLEMS pk

T t - rkwktlrk > 0,

19

(2.1.49) k=o,. ..,~-i

T + the value o f (2.1.48) i s given by ~xoWoxo. However, i f Pk r;wttlrk i s i n d e f i n i t e , (2.1.48) does n o t have a f i n i t e

-

value. Our i n t e r p r e t a t i o n o f t h e above non-cooperative d e t e r m i n i s t i c I f p l a y e r uk assumes t h a t uk w i l l be

game i s as f o l l o w s .

uncooperative i n m i n i m i z i n g t h e q u a d r a t i c c r i t e r i o n , t h e

+

t

optimal feedback s t r a t e g y f o r uk, namely uk = -Ckxk, i s a l s o optimal r e l a t i v e t o an exponential performance c r i t e r i o n , i f i s Gaussian w h i t e noise. When looked a t from t h i s viewp o i n t t h e min-max game s o l u t i o n f o r uk( 'worst-case design' )

Wk

does n o t appear t o be t o o p e s s i m i s t i c , s i n c e t h e exponential performance c r i t e r i o n i s r a t h e r appealing. 2.1.6

Continuous-time Formulation

Here t h e performance c r i t e r i o n i s

V U(xo)

= E

u exp$[

IT( xTQx+uTRu)dt+xT ( T ) Q T x ( T ) l )

IX0

0

(2.1.50)

s u b j e c t t o t h e 1 i n e a r s t o c h a s t i c d i f f e r e n t i a l equation dx = Axdt

t

Budt

+

rda

(2.1.51)

where, f o r n o t a t i o n a l simp1 i c i t y , t h e time-dependence o f t h e v a r i a b l e s has been suppressed. all t

E

[O,Tl,

I n (2.1.51)

and t h a t QT > 0.

Note t h a t Q 2 0, R

> 0,

for

a(.) i s a Brownian motion process o f zero mean

20


and satisfies E[a(t)aT(-r)l

=

min(t,-r) -1

J

P

(s)ds

(2.1.52)

0

where P

> 0 for all t

E

[O,T]

.

Formally, (2.1.51) may be considered as A = AX t BU

+ rw

( 2.1.53)

where w, the formal derivative of a , is zero mean Gaussian white noise and satisfies E[w(t)w T (T)] = P-'.6(t--r)

( 2.1.54)

and where 6 is the 'Dirac delta function'. We seek a Borel-measurable function Ca such that u'(X,t)

=

Ca(X,t),

t

E

[O,Tl

(2.1.55)

minimizes (2.1.50), where

x 2.1.7

A

=

EX(T)lO

Q-r

G t).

(2,. 1.56)

Continuous-time Solution

The continuous-time problem may be sol ved either by formal ly taking the limit of the solutions of the discrete-time versions or by solving the 'generalized' Hamil ton-JacobiBellman equation for the optimal value function Ja(x,t), viz.

CONTROL PROBLEMS

- -aaJu(x;t) t

21

o TQx+uT R u ) J a ( x , t ) + [ J ~ ( x , t ) l T (Ax+Bu) = min i2(x U

t

4 tr[Jix(x,t)rP

-1 T

r I).

(2.1.57)

Using e i t h e r method we f i n d t h a t u'(x,t)

= -R

-1 T u B S X,

t

E

[O,Tl

(2.1.58)

and

u T o = oFa expiq x S XI

J'(x,t)

(2.1.59)

where

-i'

= Q

+

SOA

+

-

A~SO

-1 T -1 T S ~ ( B R B -orP r 1s

, s~(T)=

Q,

( 2.1.60) and

-i'= 2.1.8

4aFatr(SurP

-1 T

r

),

F'(T)

= 1.

(2.1.61)

R e l a t i o n t o D i f f e r e n t i a l Games

It i s n o t hard t o v e r i f y t h a t (2.1.58)

w i t h u = -1 i s also

the optimal p o l i c y f o r c o n t r o l l e r u i n t h e f o l l o w i n g d i f f e r e n t i a l game, T T T T T Minimize [ I l ( x Qx+u Ru+w Pw)dt+ix (T)QTx(T)l 0 U('),W(.)

(2.1.62)

subject t o

i = Ax

+

Bu

+ rw,

xo given.

Note t h a t because o f our assumptions on Q,

(2.1.63)

R, P, QT t h e

22


solution S-(.) of the Riccati equation (2.1.60) exists for all t E [O,Tl

.

When u = +1 the appropriate game is non-cooperative, viz. T

Min Max [ 4 I(xTQx+uTRu-wTPw)dt+~xT(T)QTx(T)l U(') 4 . 1 0 (2.1.64) subject to (2.1.63). In this case the value of (2.1.64) is T equal to 4x (O)S+(O)x(O) provided that the following Riccati equation has a solution on [O,Tl:

-i+= Q

+ S+A + A ~ S +- S+(BR-1 B T-rp -1 r T 1s+ , s+(T)

=

Q,.

(2.1.65) Note that by standard results on Riccati differential equations, (2.1.65) has a solution for all t E [O,Tl if

(BR -1 BT-rP -1 r T ) 3 0 ,

for all t

E

[O,Tl.

(2.1.66)

If (2.1.66) is not satisfied (say for Amin(P-')

sufficiently large) the solution of equation (2.1.65) may exhibit a finite escape time, which would imply non-existence of a finite value of (2.1.50) and of (2.1.64). 2.1.9

Stability Properties

In this section we assume that all parameters are timeinvariant and we investigate the stability of the system A =

where C z is defined below.

(A-BC:)X

(2.1.67)

CONTROL PROBLEMS

23

F i r s t we consider t h e case when u = -1. We assume t h a t the p a i r (A,B) i s completely c o n t r o l l a b l e and t h a t Q > 0. Then, standard theory guarantees t h e existence o f Si, t h e unique p o s i t i v e - d e f i n i t e symmetric s o l u t i o n o f t h e algebraic R i c c a t i equat ion Q

+

SA

+

ATS

-

S(BR-'BT+yP -1r T ) S = 0

(2.1.68)

and we have t h e t i m e - i n v a r i a n t feedback gain

c;

.

= R-~B~s;

(2.1.69)

We now d e f i n e

L- 4 4xTs;x

(2.1.70)

which i s p o s i t i v e d e f i n i t e . Along s o l u t i o n s o f (2.1.67) we have = J~~(s;A+A~s;)~ x T ~ ;-1~BT~s i x (2.1.71)

i-

-

which, upon using (2.1.68),

i- = -

is

4 T~[Q+S;(BR

-1 T -1 T B -rP r )S;]X.

(2.1.72)

Now, if BR-'B~

-

-1 T rp r

>o

(2.1.73)

we have L-

0. Note that we can always define a matrix N by the formula

N N T A= BR - lB T-rp - lr T .

(2.1.75)

If we assume now that the pair (A,N) is completely controllable it follows that there exists a unique positive-definite symmetric matrix :S satisfying

Q

t

SA + A ~ -S S(BR -1 BT-rp -1 r T 1s = o

(2.1.76)

and

+ CL = R -1 B T Sm.

(2.1.77)

+ If we now define L+ fxT Smx, we can easily verify that L+ is a Liapunov function and (2.1.67) with controller CL is asymptotically stable in the large. Note the interesting point that condition (2.1.73) is sufficient, modulo complete controllability and definiteness of Q, to guarantee asymptotic stability of (2.1.67) with controllers ,C or CL. In the first case it is used to guarantee negativity of L-, while in the second it guarantees the existence of SL. 2.2

Exponential Performance Criterion

-

Noisy Measurements

In Section 2.1.2 the controller that minimizes an exponential performance criterion was derived from a simple backward

CONTROL PROBLEMS recursion (dynamic program).

25

I f , however, t h e system ' s t a t e '

i s not observed e x a c t l y , t h e m u l t i p l i c a t i v e nature o f t h e exponential performance c r i t e r i o n precludes a s i m i l a r approach. I n t e r e s t i n g l y , i t t u r n s o u t t h a t the optimal c o n t r o l i s a l i n e a r f u n c t i o n a l o f t h e smoothed h i s t o r y of t h e system 'state'.

The feedback gains depend upon t h e s t a t i s t i c s o f

both t h e system and t h e measurement noise.

2.2.1

Discrete-Time Formulation

The 1 i n e a r d i s c r e t e - t i m e dynamic system i s s p e c i f i e d by

'k+l

= A x

k k

+

where t h e ' s t a t e ' vector

B u t

k k

Xk E

rkwk, k=O,.

..,N-1

(2.2.1)

R ~ ,t h e c o n t r o l v e c t o r uk

E

R'"

Rq. The known matrices Ak, Bk, rk have a p p r o p r i a t e dimensions and may vary as a f u n c t i o n o f t h e index k. and t h e Gaussian white-noise i n p u t uk

E

The Gaussian w h i t e 'process noise' W k has s t a t i s t i c s Wk

>0

(2.2.2)

and t h e i n i t i a l ' s t a t e ' xo i s a Gaussian random v a r i a b l e w i t h statistics E[x,]

= 0,

E [ X ~TX ~=] Yo.

(2.2.3)

A t each stage, k, a n o i s y l i n e a r measurement zk i s made,

zk = H x + vk, k k

k=O,

...,N

(2.2.4)


26 where

zk

E

RS and t h e random v a r i a b l e v

k

has s t a t i s t i c s

The noise sequences {ukl, { v k l and t h e i n i t i a l ' s t a t e ' xo are assumed t o be independent o f each other. The performance c r i t e r i o n which we minimize i n order t o o b t a i n a d e s i r a b l e c o n t r o l l e r f o r (2.2.1) i s s p e c i f i e d by V'

=

EU

exp$ $1

(2.2.6)

where J,=

and where Qk 2 0, k=O,. I n (2.2.6),

..,N,

Rk

> 0,

k=O,.

..,N-1.

E denotes t h e expected value operator over t h e

space o f a l l t h e random v a r i a b l e s (i.e. expected value o p e r a t o r ' ) .

t h e 'unconditional

The problem, then, i s t o f i n d a n o n - a n t i c i p a t i v e c o n t r o l l e r ( i . e . one t h a t i s a Borel-measurable f u n c t i o n o f o n l y t h e

past measurements) t h a t minimizes V'. Defining

t h e c o n t r o l uk may be any Borel-measurable f u n c t i o n o f Zk. It f o l l o w s t h a t (2.2.6)

can be w r i t t e n i n terms o f nested c o n d i t i o n a l expectations as f o l l o w s :

CONTROL PROBLEMS va = E[EIZ0 [E IZ, [...E

27

I Z N [ ~ e @ / ~...I] ] 1.

(2.2.9)

I f we d e f i n e

(2.2.10) where i t i s assumed t h a t an admissible c o n t r o l sequence has been used, a r e c u r s i o n formula f o r Va(Zk) i s obtained from (2.2.9) and (2.2.10) as

2.2.2

D screte-time So u t i o n

-

Terminal Cost Only

It t u r n s o u t t h a t i t i s easier t o solve a terminal cost

...,

N-1) than t h e problem formulated problem (Qk = 0, k=O, above. We t h e r e f o r e f i r s t solve t h i s simpler problem and then show how t o convert t h e general problem i n t o t h i s form. We have t h a t

which may be w r i t t e n as follows, because t h e c o n t r o l Uk i Borel-measurable f u n c t i o n o f t h e measurement h i s t o r y ZkS ,N-1, k=O,.

S

a

..

(2.2.13) I n order t o perform t h e expectation i n (2.2.13)

we r e q u i r e

28


t h e c o n d i t i o n a l p r o b a b i l i t y d e n s i t y p(xNIZN).

Now i t i s

w e l l known, see f o r example 141, t h a t because o f t h e l i n e a r i t y o f (2.2.1) and (2.2.4) t h e c o n d i t i o n a l d e n s i t y p ( x k l z k ) i s A = E [x ] and i s propagated Gaussian w i t h c o n d i t i o n a l mean Izk by t h e Kalman f i l t e r equations

xk

where

and (2.2.16) The g a i n m a t r i x Kk i s g i v e n by K k = P H TV -1 k k k

where t h e covariance Pk

E[(xk-ik)(xk-ik)

and where Mk A= E[(xk-xk)(xk-")

(2.2.17) T

I

i s propagated by

T] i s propagated by

The e r r o r i n t h e estimate ek A= xk-xk i s s t a t i s t i c a l l y indepenThe dent of t h e measurement h i s t o r y Zk and t h e estimate

xk.

measurement r e s i d u a l Sk i s Gaussian and i s independent of t h e

CONTROL PROBLEMS

29

measurement history Zk-l. We therefore conclude that p(xNIZN) is Gaussian with mean iN and covariance PN. Fortunately, the exponential form of p(xNIZN) allows us to compute the expectation in (2.2.13) in closed form (cf. the Lemma in Section 2.1.2), so that

V'(ZN)

= uaN exp{;[

N-1

Z ukRk~kti$NiN1l

k=O

(2.2.20)

where

and where we have assumed that PN1-uQ, aN is given by

> 0. The coefficient

A backward step is now taken using (2.2.11), namely

V'(ZN-l)

=

E

IzN-1

V'(ZN).

As u k i s a Borel-measurable function of

(2.2.23)

Zk, (2.2.23) can be

written as

In (2.2.24) the conditional expectation is taken with respect to the measurement residual sk at k=N where, at any stage k , sk is Gaussian-distributed with zero mean and covariance

30

LINEAR-QUADRAT I C EXTENSIONS

C a l c u l a t i n g t h e expectation i n (2.2.24)

and where we assume t h a t Sil-~K$NKN a

N- 1

we o b t a i n

> 0.

The expression f o r

is

F i n a l l y , u s i n g (2.2.15)

Now u s i n g (2.2.9)

i n (2.2.26)

and t h e fundamental theorem i n [4, p. 7691

we have t h a t t h e minimum value o f V',

J U A=

we o b t a i n

min

EIV'(ZN-l)]

min

=

uo'

UO¶.**9UN-1

namely J',

' ' "N-2

Now,

J'(Z~-~)

A= min V ' ( Z ~ - ~ )

'N-1

i s g i v e n by

E[min VU(Zn-,)l. 'N-1 (2.2.30) (2.2.31)

CONTROL PROBLEMS

31

i s obtained by m i n i m i z i n g t h e r i g h t - h a n d s i d e o f (2.2.29) w i t h r e s p e c t t o u ~ - which ~ , yields

-

- -(RN-l

‘N-1

provided t h a t RN-l

+

-

+BT B ) -1BTA - 2 N 1 N N 1 N-1 N-1 N N 1

(2.2.32)

> 0.

Bi-lfiNBN-l

It then f o l l o w s t h a t

(2.2.33) where

-

“Q-1 -

AN-^

[o -0 N

a

B (BT B -1 T N N - i N - i N N - ~ + R N - ~ )BN-I~NIAN(2.2.34)

Applying t h i s procedure r e c u r s i v e l y backward i n t i m e y i e l d s a t each stage t h e l i n e a r o p t i m a l feedback c o n t r o l l e r

-’

T-

‘k

TA BkQk+l k k ’

= “Rk+BkQk+lBkl

k=O,

...,N-1 (2.2.35)

on t h e assumption t h a t Rk

+

Bitjk+lBk

> 0, where

and where .. Qk-l

-

Ak-l

T [o k-0 kBk-1 (BTk-1 0kB k - l + R k - l ) -1Bk-l’klAk-l (2.2.37)

on t h e assumption t h a t ’S;

-

oKiokKk

> 0.

The boundary

32


condition for (2.2.36) and (2.2.37) is provided by (2.2.21). The unconditional minimal expectation of the performance criterion V' is J'

= uao

exp(z u x-T~ Q- ~ x ~ }

( 2.2.38)

where a0 is obtained via the backward recursion "k -- "k+l I l-uKktl ktl K ktl Sktl I-'

(2.2.39)

with boundary condition provided by (2.2.22). 2.2.3

Discrete-time Solution

-

The General Case

When Qk, k=O ,..., N-1, are not all zero we see that (2.2.13) becomes

(2.2.40) where the terms in u k are legitimately taken ahead of the conditional expectation. Note that the expectation in (2.2.40), being conditional upon the whole measurement history, involves the smoothed estimate o f xk, k=O, ...,N. In order to convert this problem into the one solved in the previous section we first define (2.2.41) where x k iS an n(kt1)-dimensiona-1 vector. Next, we define the (Nt1)n x (N+l)n composite cost weighting matrix

33

CONTROL PROBLEMS

(2.2.42)

Using (2.2.41)

and (2.2.42),

e q u a t i o n (2.2.40)

becomes

C l e a r l y , p(XNIZN) i s r e q u i r e d here i n o r d e r t o c a l c u l a t e t h e F i r s t we see t h a t t h e recurrence e x p e c t a t i o n i n (2.2.43). r e l a t i o n f o r Xk i s

xk+l

.w

= A

x k k

t

-B u

k k

-

(2.2.44)

t rkWk

where

and where

$

E

R

(kt2)n

(k+l)n,

ikR ( k t 2 ) n x m

(2.2.45) Y

R(Wnxq rk The measurement e q u a t i o n i s ."

.,

zk=HX t v k k where

9:

=

j "i]

(2.2.46)

( 2.2.47 )

34

L I NEAR-QU ADRAT IC EXTENS IONS

and where -T Hk

E

R (kt2)n

'.

Note t h a t t h e d i m e n s i o n a l i t y o f

xk increases by n a t each stage. It may be v e r i f i e d t h a t t h e Kalman f i l t e r t h e o r y i s a p p l i c a b l e

t o (2.2.44)

and (2.2.46) and t h a t p(XklZk) i s Gaussian w i t h A c o n d i t i o n a l mean ik = E [X ] and covariance lzk Pk 9 E I ~ ~ r ( X k - i , ) ( X k - i k ) ~ ] . The g a i n m a t r i x o f t h e Kalman

#. .

f i l t e r i s then

-K k A=

(2.2.48)

iTV-l k k k

and t h e Kalman f i l t e r i s i d e n t i c a l t o (2.2.14)-(2.2.19)

with d. .

x 9 9 ik, !k, Aks Bk, pks Mk replaced by xk, fk, i k , Kk, ...k _ k Aks Bk, pk, Mk* We may now a p p l y t h e r e s u l t s o f Section 2.2.2 d i r e c t t o o b t a i n t h e l i n e a r optimal feedback c o n t r o l l e r as

where 'ktl

-

- -

Lktl

+

-T

- -

-1iT k+l k t l

uLktlKktl(S~~l-uKktlLk+lKk+l)

(2.2.50)

(2.2.51) (2.2.52) and where i t i s assumed t h a t (Rktii$ktlii,) -T .. ( S k ~ l - u K k t l L k t l K k t l > 0 and Pi1-,LN > 0.

-

> 0, Thus t h e optimal

CONTROL PROBLEMS

35

c o n t r o l i s a l i n e a r f u n c t i o n o f t h e smoothed estimate o f t h e e n t i r e s t a t e h is t o r y . Note t h a t t h e unconditioned expected value o f t h e performance criterion i s

J'

=

-

u -Tx L x I 2 0 0 0

U C Lexp{~

(2.2.53)

where a. may be computed from

-

i- i

T'

OLk - ak+lll-aKk+l

s

k+l k+l k+ll

-4

(2.2.54)

and (2.2.55) 2.2.4

No Process Noise

A s i m p l i f i c a t i o n r e s u l t s i f t h e r e i s no process noise W k = 0, k=O,. ,N-1) present. Indeed t h e problem can transformed q u i t e simply i n t o a terminal c o s t problem s t a t e vector o f dimension 2n. The optimal c o n t r o l a t

..

(i.e. be with a each

stage i s then a l i n e a r f u n c t i o n o f t h e En-dimensional s t a t e estimate

.

Owing t o t h e absence o f process noise, we may w r i t e N-1

'k

= mi:k(XN-

@N,j+l B.u.1 J J j=k

where t h e t r a n s i t i o n m a t r i x @ k-1

k,g

(2.2.56)

i s defined as (2.2.57)

36


The q u a n t i t y $ then becomes $ = xNCNxN T

+

-

N-1 N-1 2xT[ Z C;( Z @ .B.u.I k=O j = k N,J J J

N-1

( 2.2.58)

Z ukRkuk k=O

where

c;(

k -1 T -1 = (@N,k) Qk@N,k> ck = j =O

(2.2.59)

Equation (2.2.58) may be s i m p l i f i e d t o y i e l d Va i n t h e form

V

0

a

T

= aE exp(7[(xN

where

and Dk = @N,ktlBk' As nk i s a Borel-measurable f u n c t i o n o f Zk-l

(2.2.62) t h e terms

i n v o l v i n g qk and uk can be taken i n f r o n t o f t h e conditioned expectation E

IZN

t o yield

CONTROL PROBLEMS -

-T T [xk q k l

Y

37

N-1

% A T Dk = 10 DkI

9

Rk

Rk

T ’ DkCkDk. (2.2.64)

The e s t i m a t i o n equations f o r

ik are

,I:[ I0I], rk1.

where

rk4

zk 4

Ak

Ek =

(2.2.66)

‘kDk

As i n t h e p r e v i o u s s e c t i o n s we can r e c u r s i v e l y determine t h e

1 i n e a r o p t i m a l feedback c o n t r o l l e r as ‘k

-

where ‘ktl %

‘k

z

= ‘ktl

= TTc

-

+

X -

k ktl k

x

-- -(Rk ,FTck k t l k )-l(zTtfTc k k ktl k

“k+l

k

(2.2.67)

Fk t l ( S - l (2.2.68)

($E

r+zk ) ( Rk,PTck k t l k )-1($T,zTCk k k t lXk

k k+l k

(2.2.69) and where

38

L INEAR-QUADRATI C EXTENS IONS

'-

I n t h e above equations i t i s assumed t h a t P i l - d N OKkCk T= !?k

>0

and t h a t

kk +

qEk+lBk

%

> 0.

> 0,

The unconditional expected value o f t h e performance c r i t e r i o n i s given by

J' = uao exp{~(xo,O)Eo(xo,O) u -T -T TI

(2.2.71)

where a. may be computed from

ak

2.2.5

--

= T r I -1 ak+ll l-uKk+l k+l k+l k+l

(2.2.72)

Continuous-time Formulation

I n continuous-time t h e performance c r i t e r i o n i s V'

= UE exp{q[ ;(xTQx+uTRu)dt+x T (T)QTx(T)] 1 (2.2.74) 0

subject t o t h e 1 i n e a r s t o c h a s t i c d i f f e r e n t i a l equation dx = Axdt

+

Budt

+ rda

(2.2.75)

and t h e measurement equation dz = Hxdt

+ dB

(2.2.76)

CONTROL PROBLEMS

39

where a and 8 a r e Brownian motion processes w i t h zero mean that satisfy min( t,.r)

T

E[a(t)a (T)] =

I

E[B(t)BT(.r)1

I

and =

W( s)ds

(2.2.77)

V( s)ds.

(2.2.78)

0

min(t,-r) 0

We assume t h a t t h e i n i t i a l s t a t e xo i s Gaussian-distributed and t h a t xo, a ( - ) and

@ ( a )

a r e independent.

The r e s u l t s f o r t h e terminal c o s t problem (i.e. Q ( t ) = 0, t E 10,TI) may be found by t a k i n g t h e formal l i m i t o f t h e discrete-time equations. back c o n t r o l l e r i s u = -R

where

where

-6 = ~ i,

It t u r n s o u t t h a t t h e optimal feed-

-1 T-B Qx,

t

E

[O,T]

~ t i ij -~ j G(BR -1BT- ~ K V K ~ ) G

(2.2.79) (2.2.80)

t h e best estimate o f t h e s t a t e , i s propagated by the

continuous form o f t h e Kalman estimator which a l s o y i e l d s t h e Kalman g a i n K and P, t h e covariance o f t h e e r r o r i n t h e estimate

.

Although i n t h e general case an i n f i n i t e - d i m e n s i o n a l c o n t r o l l e r r e s u l t s , we have t h a t provided a( .) i s zero (no process noise) t h e r e s u l t s o f Section 2.2.4 become (2.2.82)


40

where

&

-C

=

%T% A C

w

= -1 =T =T%

-T

w

+ ?% - (CB+D)R (D +B C)+aCK VKC (2.2.83)

where ?(T) is given by (2.2.70) with CN+l replaced by QT + C(T) and where f?, ?, 8 and C are defined as

E,

C(t)

A

=

=T D 9- [O BTQT(T,t)1

(2.2.84)

I [@-1(T,.c)l Q( T)@-~(T,T)d.r.

(2.2.85)

0

The trans tion matrix @(t,T) sat sfies @-'(T,t)

= @(t,T),

(P(t,T)

=

-@(t,T)A,

@(T,T)

=

I. (2.2.86)

The 2n vector = [x-T n T1 i s propagated in part by the continuous-time Kalman filter and in part by

n 2.2.6

=

C(t)@(T,t)B(t)u(t),

n ( 0 ) = 0.

(2.2.87)

A Note on Separation

We note that there is a separation [5] between estimation and control in the problems treated in Sections 2.1 and 2.2, namely that the estimators (Kalman filters) can be constructed without having the optimal feedback controller. All that is The separation is, required i s the input sequence (uk). however, one-way, as the optimal controllers depend upon the estimation-error covariance. In contrast, the separation is

41

CONTROL PROBLEMS

both-ways in the Linear-Quadratic-Gaussian control prob em where the parameters of the optimal feedback controller are dependent upon only the deterministic system and criter on matrices A, B, Q and R. 2.2.7

A Further Separation Property

The solution for the terminal cost problem in cont exhibits a further type of separation which allows controller. This has been exploited [6] in an app the exponential-criterion theory to homing missile

nuous time an adaptive ication of guidance.

In this and other applications the measurement covariance matrix V is not known a priori and must be estimated on-line. In drder to determine 6 the Riccati (2.2.80) and the Kalman filter covariance matrix equations would have to be solved on-line after V i s estimated. However, the on-line backward integration of 0 may be avoided in the following way. First note that as V is estimated, the Kalman filter covariance equation

i)

=

AP

t

PAT + rwrT

-

PHT v -1 HP, p(to)

= P,

(2.2.88) must be integrated to yield P as a function of t. Next, note that U 4 satisfies

U

=

UA T

t

AU

-

BR -1 BT + uPH TV -1 HP

(2.2.89)

with boundary condition U(T) = ';Q

- aP(T) .

(2.2.90)

42


Now, d e f i n e a new m a t r i x v a r i a b l e M A= U t u P

(2.2.91)

so t h a t we have

ti = AM

wT -

t

BR-1BT

t

arwrT

(2.2.92)

w i t h boundary c o n d i t i o n M(T) = QY'. Letting S

4

M-l,

(2.2.93)

we see t h a t

T -S = SA + A S

-

S(BR

-1 T

B

-0rwr

T

)s,

S(T) =

Q ~ .

(2.2.94) It follows t h a t i f t h e process noise covariance W i s known,

S ( . ) can be c a l c u l a t e d o f f - l i n e and stored (note t h a t S i s

i d e n t i c a l t o S" i n Section 2.1.7).

Then

0

can be c a l c u l a t e d

o n - l i n e using t h e formula "

Q = (S-l-aP)-'

= (I-aSP)-lS

(2.2.95)

which i s obtained from (2.2.91). Thus t h e e f f e c t of t h e measurement noise covariance V on t h e i s o n l y v i a P. N a t u r a l l y t h e adaptive gain c o n t r o l gains i s optimal o n l y i f V i s estimated p e r f e c t l y , which i s u s u a l l y n o t t h e case; however, t h e above procedure a t l e a s t suggests

6

a r a t i o n a l approach. The above adaptive scheme has been a p p l i e d i n 161 and has been shown t o e x h i b i t c e r t a i n advantages over t h e standard

43

CONTROL PROBLEMS 1inear-quadratic design. 2.3

Non-1 inear Stochastic Systems

We t u r n now t o a d i f f e r e n t extension o f t h e l i n e a r - q u a d r a t i c theory. We r e t a i n t h e quadratic performance c r i t e r i o n b u t extend t h e c l a s s o f dynamic systems by adding a c e r t a i n type o f non-linear stochastic term 171 ( t h i s f o r m u l a t i o n includes t h a t o f l i n e a r systems w i t h m u l t i p l i c a t i v e noise [81, [91 ). It t u r n s o u t a l s o f o r t h i s c l a s s o f problems t h a t the optimal

feedback c o n t r o l l e r i s 1inear and t h a t i t s c o e f f i c i e n t s depend upon t h e s t a t i s t i c s o f t h e noise. 2.3.1

D i screte-time Formulation

We consider t h e problem o f minimizing w i t h respect t o i u k ) the performance c r i t e r i o n

E

N- 1 T B ( x ~ Q k X k t U k R k U k ) t ~ x ~ Q ~ x ~ l (2.3 .l) k=O

1

10'

subject t o t h e stochastic dynamic equation 'k+l

k k t Bku k

= A x

t

mk t fk(xk,uk,uk),

xo given (2.3.2)

E: R ~ ,uk E: R", uk E: ~ q ,f k : R~ x R ~ ~q X + R~ and k QN, Qk, Rk, Ak, ,"k are matrices o f appropriate dimension, and where mk c R

where x

.

The noise sequence {ukl i s assumed t o be independently d i s t r i b u t e d i n time and n o t necessarily Gaussian. Our assumpt i o n s on t h e random vector f u n c t i o n f k are as f o l l o w s .

44


ASSUMPTION 2 . 3 . 1 Fk(xk,uk) A= E[fk(xk,uk,Uk)] all X k E Rn, U k E Rm, k=O, N-1.

is zero for

. . a ,

Note that the results derived in this section hold also if i-k is a linear function of xk and uk. However, there is no loss of generality in assuming that 8 , = 0 because appropriate choices of Ak, Bk and mk will model any mean value of fk which is linear in xk and uk.

T

A

ASSfJMPJlON 2 . 3 . 2 Fk(xk,uk) = E[fk(xk'uk,wk)fk(xk,uk,wk)l exists and is a general quadratic function of xk, u k for k=O,. ,N-1. That is, Fk(xk,uk) has the representation

..

(2.3.3

i Wk,i Nk, i Mki are matrices of appropriate dimensions where Pk, i Wk, i Mki are symmetric, i=O ,...,n', gki E Rn, h ki E Rm, and Pk, where n' = n(ntl)/2.

ASSUMPTION 2.3.3 Representation (2.3.3) is such that Fk(xk,uk) > o , for all xk E R ~ ,u k E R ~ . This assumption i s actually necessary and not at all restrictive because Fk must @e a covariance matrix for all xk, Uk. We seek a closed-loop optimal feedback controller of the form Uk =

ck(xk)

3

xk

A

=

{xoIxl,

* * SXk}

where Ck is a Borel-measurable function of Xk, k=O,.

(2.3.4)

..,N-1.

45


Solution

The main r e s u l t i s stated as

THEOREM 2.3.1

Define t h e f o l l o w i n g q u a n t i t i e s : A

'

n'

=

4

tr(Sk+lPi)Wl i=1

Nk(sk+l) a =

3

n'

ik(sk+l)

'

(2.3.5)

tr(Sk+lpl)Nl

(2.3.6)

i=l k

(S

) k+l

4$

n'

Z tr(Sk+lPk)Mi i=1

(2.3.7) (2.3.8)

T dk = A S k k + lm k

+

ek = dtr(Sk+lPi)

T Akdk+l

+

+ ik(Sk+l)

3m:Sk+lmk

+

-

--IAkRk mk, dN = 0 (2.3.14) -T--ldL+lmk + ek+l .@kRk mk'

-

-

eN = 0.

(2.3.15)

46


Then, i f w

Rk

> 0,

.. ,N-1

k=O,.

(2.3.16)

the optimal closed-loop c o n t r o l l e r i s l i n e a r and i s given by

and t h e minimum value of (2.3.1)

-.1

3x S x

0 0 0

t

is

..-I

doxo

+

eo.

(2.3.18)

The p r o o f f o l l o w s by i n d u c t i o n and proceeds as PROOF F i r s t define f o l 1ows

.

T x.tu.R 1

1

(2.3.19)

so t h a t

-

-

Jk(xk) = m i n [t(XkQkXktUkR~uk)tEIXkJk+l(Xk+l)j I I 'k

. (2.3.20)

It i s then easy t o show by i n d u c t i o n t h a t

The next r e s u l t demonstrates t h a t i n e q u a l i t y (2.3.16) ., under c e r t a i n reasonable c o n d i t i o n s on Qk, Rk and Q

holds In

f a c t these c o n d i t i o n s a r e u s u a l l y assumed t o h o l d i n t h e standard 1inear-quadratic c o n t r o l problem formulation.

47

CONTROL PROBLEMS

THEOREM 2.3.2

Suppose t h a t

(2.3.22)

Then

.,

Rk

PROOF

>0

and

sk+l > 0,

k=O,

...,N-1.

The proof i s a g a i n by i n d u c t i o n . i m p l y t h a t i f Sk+l

and e q u a t i o n (2.3.3)

(2.3.23)

Assumption 2.3.3

2 0, t h e n

(Sk t l ) > 0, and i n e q u a l i t i e s (2.3.22) i m p l y t h a t 0 'ktl N we have and SN > 0. Now assuming t h a t Si > 0, i = k + l , that

6i-l > 0,

i=k+l,

...,N.

...,

T h i s p e r m i t s t h e computation of

Sk, dk and ek v i a (2.3.13)-(2.3.15)

and a l l o w s us t o w r i t e

the equation T T $xSx t d x +ek= k k k k k N-1 T m in E [ S $(xiQixitu.R.u.)+ix 1 1 1 Uk,. ,u N- 1 I x k i = k

.

Now because o f t h e i n e q u a l i t i e s (2.3.22), o f (2.3.24) Sk

> 0.

T NQ Nx NI

. (2.3.24)

t h e right-hand side

i s non-negative f o r a l l Xk which i m p l i e s t h a t

This i n t u r n implies t h a t

ik-l > 0 and

so t h e proof

by i n d u c t i o n i s complete. 2.3.3

Known Special Cases

The standard 1 inear-quadratic-Gaussian p r o b l ern emerges as a s p e c i a l case i f we s e t

48

LINEAR-QUADRATIC EXTENSIONS f k ( x k Yu k YW k ) = rkWk

with E[Wk] =

(2.3.25)

Y

T E[WkWk] = Ak

O y

.

(2.3.26)

Here

and i t t u r n s o u t t h a t t h e optimal feedback c o n t r o l l e r i s

T

independent o f rkAkrk. The next known case i s t h a t o f m u l t i p l i c a t i v e noise 191 f o r continuous-time treatment.

-

see [81

Here

where

(2.3.29)

1 w i t h uk E :l,

W:

E:

2:.

The j - t h component o f xk i s denoted

1 2 Note t h a t i t i s n o t necessary t o assume t h a t Wk and uk a r e

uncorrelated although t h i s i s o f t e n assumed i n t h e l i t e r a t u r e . It i s n o t hard t o c a l c u l a t e

(2.3.30)

gk, 8,

and

A,

u s i n g (2.3.29)

and

and i t i s c l e a r t h a t t h e optimal c o n t r o l l e r depends

upon t h e noise covariance Ak.


49

Novel Special Cases

In t h i s section we display certain novel special cases t h a t have not appeared i n the l i t e r a t u r e hitherto. The f r s t case i s that of norm-dependent noise; t h a t i s , the dynam c system i s described by 'rkwk (2.3.31)

where D1

> 0,

D3

>0

and D1

-

T -1 D2D3 D2 2 0, and where E[ukI = 0 ,

T E[w w 1 = A k k k ' Note t h a t i f D1 = 21, D 2 = D3 = 0 we have the noise rkwk multiplied by IlxkII.

For the above case we have 7k(xkYuk) =

(2.3.32)

and

so t h a t t h i s non-linear system s a t i s f i e s our assumptions. Other non-1 inear examples t h a t s a t i s f y our assumptions are those in which

(2.3.35) and

50


0 where ak,

. ..,akn are uncorrelated.

Note t h a t t h e i n t e r e s t i n g r e s u l t t h a t a c e r t a i n class o f non1inear s t o c h a s t i c systems subject t o quadratic performance c r i t e r i a e x h i b i t s 1 inear optimal c o n t r o l l e r s i s i n c o n t r a s t t o t h e d e t e r m i n i s t i c non-linear cases (obtained by r e p l a c i n g (ak} by a known sequence) f o r which there do not appear t o be

closed-form s o l u t i o n s .

2.4 I n f i n i t e - t i m e Optimal Control Solutions o f i n f i n i t e - t i m e optimal c o n t r o l problems can be obtained a t l e a s t conceptually w i t h t h e a i d o f Bellman's p a r t i a l d i f f e r e n t i a l equation.

I n t h i s s e c t i o n we f i r s t prove

a well-known b u t n o t widely documented theorem on Bellman's equation.

We then assume a form f o r t h e s o l u t i o n and show

t h a t t h i s enables c e r t a i n non-1 i n e a r i n f i n i t e - t i m e problems t o be solved.

We mention here t h a t Pearson [lo] has previously

suggested t h e use of t h e form o f s o l u t i o n o f Bellman's equation.

However, he f a i l e d t o s t a t e t h e need f o r checking

a c r u c i a l symmetry condition, so t h a t h i s r e s u l t s , except i n t h e case o f h i s s c a l a r problem, a r e i n v a l i d .

2.4.1

Cont inuous-t ime Formulation

We consider t h e problem of minimizing w i t h respect t o u(.) t h e performance c r i t e r i o n

(2.4.1)

CONTROL PROBLEMS

51

where

R where L:Rnxm

R1,

+

= f(x,u),

f:RnXm

x(0) = xo

+

x

Rn,

E

(2.4.2)

Rn, u

E

Rm.

We make t h e f o l l o w i n g assumptions:

ASSUMPTION 2 . 4 . 1 A U = {u(.)lu(.) t

where R

E

The c o n t r o l f u n c t i o n u ( * )

52,

c Rm. The d i f f e r e n t i a l equation (2.4.2) [O,m)

has a

f o r each

u.

ASSUMPTION 2 . 4 . 3 f(x,u)

E

(2.4.3)

[O,m))

unique s o l u t i o n defined on t h e i n t e r v a l E

U, where

i s piecewise continuous i n t and u ( t )

ASSUMPTION 2 . 4 . 2 u(.)

E

Along a l l s o l u t i o n s o f (2.4.2),

a r e piecewise continuous i n t, t

E

L(x,u) and

[O,m).

ASSUMPTION 2 . 4 . 4 u(-) 2.4.2

E

We wish t o determine a c o n t r o l function U which minimizes (2.4.1) and causes x ( t ) + 0 as t +

m.

Bellman-type Theorems

THEOREM 2 . 4 . 1

Suppose t h a t t h e r e e x i s t s a continuously

d i f f e r e n t i a b l e f u n c t i o n J:Rn

+

and which s a t i s f i e s f o r a l l x

E

R1 which i s p o s i t i v e d e f i n i t e Rn t h e steady-state Bellman

equation min [L(x,u)tJx(x)f(x,u)l us52

= 0.

(2.4.4)

Furthermore, suppose t h a t t h e s o l u t i o n o f R = f(x,uo(x))

goes

52


t o zero as t

+ m

and t h a t along t h i s s o l u t i o n u o ( x ( . ) )

E

U

where uo(x)

a r g min [L(x,u)+Jx(x)f(x,u)l.

(2.4.5)

U E R

Then, (2.4.1)

has a minimum i n t h e c l a s s o f c o n t r o l f u n c t i o n s

f o r which u ( . )

E

U and which causes x ( t )

+

0 as t

-+

m.

The

minimum v a l u e o f V i s J ( x ) and t h e m i n i m i z i n g c o n t r o l l e r i s 0

U0(X),

PROOF

Along t r a j e c t o r i e s o f (2.4.2) we have J(x(t))

-

J(x(0))

dJ ~ ( x , u ) ~ =T 0

(2.4.6)

J x ( x ) f ( x , u ) d T = 0.

(2.4.7)

-I

0

which may be w r i t t e n as J(x(t))

-

J(x(0))

t

-J

0

S u b t r a c t i n g t h i s and t h e o t h e r i d e n t i c a l l y zero q u a n t i t y t

I

min [L(x,u)+Jx(x)f (x,u)l d.r o u s R

(2.4.8)

from V y i e l d s m

V = J(x0) + JL(x,u)dT 0

t

+ I C J x ( x ) f ( x , u ) - min [ L ( x , u ) + J x ( x ) f ( x , u ) l 1d.r 0

UER

-

J(x(t)). (2.4.9)

Now, by Assumption 2.4.4 we must c o n s i d e r o n l y those c o n t r o l s

CONTROL PROBLEMS

u(

53

U which cause x(t) + 0 as t OJ. Consequently restricting attention to this subset of U and noting that as t + =, J(x(t)) + 0 (by assumption, J is positive definite) we obtain a )

E

-+

03

V = J(xo)+

I( [L(x,u)+Jx(x)f(x,u)l 0

-min [L(x,u)tJx(x)f(x,u)J )dt. U En

(2.4.10)

Clearly the integrand is non-negative and takes on its minimum value when the feasible control uo(x) is used.

THEOREM 2 . 4 . 2 Suppose we assume that Jx(x) = xTS(x), S:Rn + R n x n . Then this form is permissible if the matrix a -[S(x)xl is symmetric. ax PROOF It is well known that x T S(x) is the derivative of a scalar function J(x), J:Rn syrmetric.

+

R1 if and only if ax[S(x)xl a

is

Using the form Jx(x) = xTS(x) it is possible to obtain positive definite solutions to certain non-quadratic Bellman equations which are of interest in optimal control. It is worth noting that Pearson [lo] has previously used this form for Jx(x), though he suppresses, notationally, the dependence of S on x. However, Pearson fails to check the important a condition of symmetry of -[S(x)x] and this invalidates the ax results obtained for his examples of dimension two (and higher). The following theorem combines into one those given above. Suppose there exists a function S:Rn + Rn a such that =[S(x)xI is symmetric and I' Ty S(y)dy is positive

THEOREM 2 . 4 . 3

0

definite, that satisfies the Bellman equation

54

LINEAR-QUADRATIC EXTENSIONS min [L(x,u)

t

x T S(x)f(x,u)l

= 0.

(2.4.11)

U E f l

Furthermore, suppose t h a t t h e s o l u t i o n o f (2.4.2) zero as t

-f

m

when u s i n g t h e c o n t r o l l e r uo(x) obtained from

uo(x) = a r g 0

Then, u ( x ( t ) ) , controls u ( + )

t E

min [L(x,u)+x uefl

T

S(x)f(x,u)l.

[ O p ) minimizes (2.4.1)

E

U which causes x ( t )

minimum value o f (2.4.1) 2.4.3

goes t o

+

0 as t

(2.4.12)

i n t h e subset of +

03,

and t h e

i s J(xo).

C e r t a i n Non-1 i n e a r Problems

Consider t h e b i l i n e a r dynamic system A = Axu

where x E Rny u

E

R

1

(2.4.13)

and A i s a constant n x n m a t r i x .

The f o l l o w i n g performance c r i t e r i a y i e l d Bellman equations t h a t can be solved e x p l i c i t l y :

o o

V = where

Taking (2.4.14)

I"

T j x QXdty Q > O ,

0

n A= ( u l l u l

1).

1

(2.4.14)

(2.4.15)

t h e Bellman equation i s

1 T 2 2 T min [-(x Qx) +ju t x S(x)Axul = 0. 8 U

(2.4.16)

CONTROL PROBLEMS

55

Assuming, f o r convenience, t h a t S(x) i s symmetric y i e l d s =

U'(X)

and

- f T~[S(x)A+ATS(X)] x

1 T T 2 - B{x [S(x)AtA S(x)Ixl

1 T -(x 8 Qx)'

This equation i s s a t i s f i e d f o r a l l

(2.4.17) = 0.

(2.4.18)

x i f S (a constant matrix)

i s chosen t o s a t i s f y

Q 5 [SAtAT Sl

(2.4.19)

= 0.

A are a l l negative T ( p o s i t i v e ) t h e n Q + ( - ) [SA+A S] = 0 y i e l d s a unique p o s i t i v e T T d e f i n i t e s o l u t i o n . Hence J(x) = fx Sx and uo = t x Qx T (u0 = -X Qx). I f t h e r e a l p a r t s o f t h e eigenvalues o f

I f (2.4.15) i s minimized t h e a p p r o p r i a t e Bellman equation i s

min [fxTQx+xTS(x)Axul = 0

(2.4.20)

U E R

which y i e l d s

xT [S(x)A+ATS(x)lx

(2.4.21)

flxTIS(x)AtATS(x)lxl = 0.

(2.4.22)

T

(2.4.23)

-

0

u (x) = and

$xTQx

-

sign

Here 0

u (x) = s i g n

x Qx

= t1

i f t h e r e a l p a r t s o f t h e eigenvalues o f

A are a l l negative,

and

u 0 (x)

=

-1

(2.4.24)

56

L INEAR-QUADRAT I C EXTENS IONS

i f they a r e a l l p o s i t i v e .

Note t h a t i n both cases a quadratic J ( x ) solves Bellman's equat ion. 2.4.4

Non-quadratic Performance C r i t e r i o n

Suppose t h a t t h e system t o be c o n t r o l l e d i s

R

= AX t BU

where t h e constant p a i r (A,B)

(2.4.25)

i s completely c o n t r o l l a b l e .

The performance c r i t e r i o n i s

0

+ where Q

> 0,

T u Ruldt R

>0

(2.4.26)

and P i s t h e unique p o s i t i v e d e f i n i t e

symmetric s o l u t i o n o f Q

+

PA

t

AT P

-

PBR

-1 T B P = 0.

(2.4.27)

It i s easy t o check t h a t t h e choice

S(x) = (l+xTPx)P

(2.4.28)

results in J(x) = and

4[(lt~~Px)~-ll

T -1 T u0(x) = - ( l t x Px)R B PX

(2.4.29) (2.4.30)

CONTROL PROBLEMS

57

and t h a t a l l s o l u t i o n s o f d = Ax t Buo(x) go t o zero as t

+a.

Note l a s t l y t h a t

a

T

-[S(x)x] ax

T

= ( l + x Px)P t 2Pxx P

(2.4.31)

which i s symmetric. For small x, u o ( x )

= -R -1BT Px

which i s t h e standard c o n t r o l l e r

o f l i n e a r - q u a d r a t i c theory, w h i l e f o r l a r g e x, uo(x) = -x T PxR-1BTPx which i s cubic i n x. f o r an extension o f t h i s We r e f e r t h e reader t o Speyer [ll] example t o a s t o c h a s t i c s e t t i n g . 2.5

Systems Homogeneous-i n-the-input

T h i s i s a c l a s s of systems which permits a complete a n a l y s i s o f i t s s t a b i l i z a b i l i t y p r o p e r t i e s and a l s o y i e l d s e x p l i c i t c l osed-1 oop c o n t r o l 1e r s which minimize a wide v a r i e t y of performance c r i t e r i a [121. 2.5.1

Formulation

S p e c i f i c a l l y , a system i s s a i d t o be homogeneous-in-the-input i f i t i s o f t h e form

m

d = X Bi(x)ui, x(0) = x0 (2.5.1) i=1 1 where x E Rn, ui E R , i=l, m, and where t h e Bi(x), Bi:Rn -F Rn a r e assumed t o be continuous f u n c t i o n s o f x. We s h a l l f u r t h e r assume t h a t t h e c o n t r o l s a r e chosen i n feedback form as ui(x) = gi(x), i=l,..,,m where, unless otherwise stated, gi:Rn+R1 i s a continuous f u n c t i o n o f x.

...,

58 2.5.2

L I NEAR-QUADRAT IC EXTENS IONS Stabilizability

In this section we provide necessary conditions and sufficient conditions for stabilizability of (2.5.1). The gap between these conditions is small. THEOREM 2.5.1 Suppose that ui(x) = gi(x), i=l ,...,m and that g i and Bi are once continuously differentiable with respect to x. A necessary condition for x = 0 to be an asymptotically stable equilibrium point of (2.5.1) is that there exists a positive definite function V(x), V:Rn -t R 1 , which is continuously differentiable, such that there is no non-zero x E Rn for which Vx(x)Bi(x),

(2.5.2)

i=l ,...,m

are all zero. PROOF

As the right-hand side of (2.5.1) with the above choice of controls is independent of t and is continuously differentiable with respect to x, an inverse theorem of Liapunov 1131 applies direct. This theorem guarantees the existence of a positive definite V(x) with i(x) = Vx(x)k negative definite. Now, along trajectories of (2.5.1)

and it follows immediately that this expression can be negative definite only if the theorem is true. The following theorem provides sufficient conditions for stabil izabil ity.

THEOREM 2.5.2

A sufficient condition for x

=

0 to be made

CONTROL PROBLEMS

59

into an asymptotically stable equilibrium point o f (2.5.1) is that there exists a positive definite function V(x), V:Rn + ,'R which is once continuously differentiable, such that there is no non-zero x E Rn for which

Vx( x)Bi (x) ,

i =1,.

..,m

(2.5.4)

are all zero. If in addition V(x is radia ly unbounded, then ui(x) = -Vx(x)Bi(x) causes x = 0 to be a globally asymptotically stable equilibrium point

.

PROOF

Let us set ui(x) = -Vx(x)Bi(x), the ui(x) are continuous and

i=l,

...,m.

Then,

The proof now follows from a standard Liapunov theorem [131. 2.5.3

A Bilinear Example

Theorem 2.5.2 is especially useful in those cases where (2.5.1) cannot be stabilized by means of constant controls u i = ki, i=l, m. For example, let

...,

0

1

B1 = [1

(J

' B2

and 2 =

r: j 0

B 1X U 1

t

=

B2xu2.

(2.5.6)

(2.5.7)

In this case it is easy to see that there are no real numbers

60


m

kl and k2 such t h a t

’’

Z B.k.

I n other

i s a s t a b i l i t y matrix. i=l words , c o n s t a n t c o n t r o l s cannot s t a b i l i z e (2.5.7). T i f we l e t V(x) = f x x we see t h a t

However ,

Vx(x)Blx

(2.5.8)

= 2x1x2

and Vx(x)B2x =

-

2 x1

2 x2

(2.5.9)

which do n o t v a n i s h simultaneously f o r any non-zero x

E

R

2.

As V(x) i s r a d i a l l y unbounded we have t h e r e s u l t t h a t ul(x)

=

-

(2.5

2x1x2

2

u2(x) = x1

- x22

.lo)

(2.5.11)

g l o b a l l y a s y m p t o t i c a l l y s t a b i l i z e (2.5.7). Note t h a t i t i s a t r i v i a l m a t t e r t o prove t h a t i f t h e r e e x i s t m 1 ki E R , i = l ,...,m such t h a t Z Biki i s a s t a b i l i t y m a t r i x , i=l then t h e r e e x i s t s V(x) = f x Sx, S > 0, such t h a t t h e s c a l a r s T T Vx(x)Bix = f x (SBi+BiS)x, i=l,. ,m a r e n o t zero simultaneously

..

f o r a non-zero x 2.5.4

E:

Rn.

Optimal C o n t r o l

I n S e c t i o n 2.5.2 we showed t h a t s u b j e c t t o v e r y reasonable c o n d i t i o n s , ui(x) s t a b i l i z e (2.5.1).

=

-

Vx(x)Bi(x),

i=l,...,m

asymptotica 1Y

It f o l l o w s e a s i l y t h a t

1 U i ( x ) = - “ J x ( ~ ) B i ( ~ ) l2p+l, i = l ,...,m

(2 5.12)

CONTROL PROBLEMS

61

where p i s a non-negative i n t e g e r , a l s o accomplishes t h i s . This r a i s e s t h e question o f whether o r n o t an ' o p t i m a l ' s t a b i l i z i n g c o n t r o l l e r can be designed.

It t u r n s o u t t h a t

t h e answer t o t h i s question i s i n t h e a f f i r m a t i v e and t h a t s t a b i l i z i n g c o n t r o l l e r s which minimize a wide v a r i e t y of performance c r i t e r i a can be constructed f o r (2.5.1). We d e f i n e t h e performance c r i t e r i o n m

i = ,(q(x)

+

0

& =1 2( i

ui P + l ) ) d t

where q ( x ) i s a p o s i t i v e d e f i n i t e f u n c t i o n o f x

E

Rn and p

We then have the f o l l o w i n g

i s a non-negative i n t e g e r . theorem.

THEOREM 2 . 5 . 1

(2.5.13)

Suppose t h a t t h e r e e x i s t s a r a d i a l l y unbounded,

p o s i t i v e d e f i n i t e f u n c t i o n V(x) which i s once continuously d i f f e r e n t i a b l e , which s a t i s f i e s t h e Be1lman equation

ul,.

min

. .,u m

[q(x)+

A

m ui2(p+1)+ Z Vx(x)Bi(x)ui1 i=1 i=1

= 0.

(2.5.14) Then, t h e c o n t r o l s (2.5.12) g l o b a l l y a s y m p t o t i c a l l y s t a b i l i z e

(2.5.1) and minimize (2.5.13) i n t h e c l a s s o f c o n t r o l functions which causes x ( t )

-+

0 as t

-f

a.

PROOF Carrying o u t t h e m i n i m i z a t i o n i n (2.5.14) y i e l d s (2.5.12) which, when s u b s t i t u t e d back i n t o (2.5.14) y i e l d s

62


Note that in order for (2.5.15) to hold, Vx(x)Bi(x), i=l,...,m cannot all be zero for a non-zero x E Rn - this is precisely a sufficient condition for the controller (2.5.12) to be globally asymptotically stabilizing. An application of Theorem 2.4.1 then completes the proof. Because of the special form of the Bellman equation we can, without loss of generality, replace q(x) in (2.5.13) by (2.5.16)

In other words, instead of specifying q(x) and solving the Bellman equation for V(x) we can choose a suitable V(x) and hence specify q(x). The next theorem is a statement of this approach .

THEOREM 2 . 5 . 2

Suppose there exists a radially unbounded, positive definite function V(x), V:Rn -+ R1, which is once continuously differentiable, such that there is no non-zero x E Rn for which Vx(x)Bi(x), i=l , . . . , m are all zero. Then, the controls (2.5.12) globally asymptotically stabilize (2.5.1) and minimize (2.5.13) in the class of control functions which causes x(t) -+ 0 as t -+ m where q(x) is given by (2.5.15). An appropriate choice of V(x) will yield a desired q(x). In many cases the class of q(x) defined by (2.5.15) with V(x) a quadratic function, is adequate. Indeed, returning to the example of Section 2.5.3 we have, for p = 0, n

q(x) = ZL 3[x TBixI2 i =1

=

2 2 2I $[(2x1x2) 2+(x1-x2)

=

$(x 2+x2 ) 2 . 1 2

(2.5.17)

CONTROL PROBLEMS The c o n t r o l s t r a t e g i e s (2.5.10),

(2.5.11)

t h e r e f o r e minimize

2 2

(2.5.18)

2 2 2

OD

63

= I{$(x1+x2) +$(ul+u2)1dt. 0

Note t h a t f o r l a r g e p, (2.5.13)

i s approximately

which leads us t o t h e f o l l o w i n g r e s u l t .

THEOREM 2.5.3

Suppose t h a t t h e r e e x i s t s a r a d i a l l y unbounded, p o s i t i v e d e f i n i t e f u n c t i o n V(x), V:Rn + R1, which i s once

continuously d i f f e r e n t i a b l e such t h a t t h e r e i s no non-zero x E Rn f o r which Vx(x)Bi(x), the controls u.(x) = 1

-

i = l ,...,m

signIVx(x)Bi(x)l

a r e a l l zero.

,

g l o b a l l y a s y m p t o t i c a l l y s t a b i l i z e (2.5.1)

i=l,...,m

Then,

(2.5.20)

and minimize

00

J = .f q ( x ) d t

(2.5.21)

0

i n t h e c l a s s o f c o n t r o l f u n c t i o n s which causes x ( t ) t

+

00

+

0 as

and which s a t i s f i e s t h e c o n t r o l c o n s t r a i n t s -1 < u i

< 1,

where, w i t h no l o s s o f g e n e r a l i t y

i=l,...,m

(2.5.22)


64

F i n a l l y , we mention t h a t i t i s n o t hard t o show t h a t

...,m g l o b a l l y a s y m p t o t i c a l l y

ui(x) = - [Vx(x)Bi(x)]2pt1, i=l, s t a b i l i z e s (2.5.1) and minimizes OD

(2.5.24)

i=1

0

where, w i t h no l o s s o f g e n e r a l i t y

2.5.5

Non-homogeneous Extension

We extend t h e r e s u l t s o f Section 2.5.4 t o systems o f t h e type

A = f(x) + THEOREM 2.5.4

m

x Bi(x)uiY i=1

~ ( 0 =) x0.

(2.5.26)

Suppose t h a t t h e r e e x i s t s a r a d i a l l y unbounded,

p o s i t i v e d e f i n i t e f u n c t i o n V(x), V:Rn

-+

R’,

which i s once

continuously d i f f e r e n t i a b l e , such t h a t V x ( x ) f ( x ) i s negative semi-definite. Suppose f u r t h e r t h a t t h e r e i s no non-zero n x E R f o r which V x ( x ) f ( x ) and Vx(x)Bi(x), i=l, m are a l l

...,

zero. Then, ui(x)

=

-

Vx(x)Bi(x),

i=l,...,m

g l o b a l l y a s y m p t o t i c a l l y s t a b i l i z e s (2.5.26) OD

m

0

i=1

(2.5.27) and minimizes

= / { q ( x ) t $ Z u:ldt

0 as Here q ( x ) i s , w i t h o u t l o s s o f g e n e r a l i t y , given by t h e

i n t h e c l a s s o f c o n t r o l f u n c t i o n s which causes x ( t ) t

+

00.

(2.5.28)

p o s i t i v e d e f i n i t e expression

-P

CONTROL PROBLEMS

-

Vx(x)f(x)

in

t

4z

i=1

65

[Vx(x)Bi(x)12.

(2.5.29)

Note t h a t t h e r e i s o f t e n considerable f l e x i b i l i t y i n choosing Indeed i n many cases

a V function t o obtain a suitable q(x).

For example, i n t h e case

a quadratic f u n c t i o n i s adequate.

A = AX

m

t

Z Bixui i=1

(2.5.30)

where A has eigenvalues w i t h non-positive r e a l p a r t s , t h e r e T e x i s t s an S such t h a t V(x) = &x Sx s a t i s f i e s t h e conditions o f I t then t u r n s o u t t h a t

t h e theorem.

ui(x) m i nimi zes

=

-

j x T (SBi+BiS)x, T

T J = wII&xT Q x t i 2m [x T(SBitBiS)xl i=1

0

where Q = - ( S A t A TS )

(2.5.31)

.i=l,...,m 2

t4

m

2

(2.5.32)

X uildt

i=1

> 0.

2.6

Optimal Control o f Q u a d r a t i c Systems

2.6.1

Formulation

We consider here t h e optimal c o n t r o l o f systems o f t h e type

T

d d t f .1( x ) where fi:Rn Bi

E

Rnxm,

= $x Aix

-+

R

1

i=l,

T

t x Biu,

xi(0)

= xo

, i=l,

i

i s a continuous f u n c t i o n , Ai

...,n.

E

We assume t h a t (2.6.1)

f o r each piecewise continuous c o n t r o l f u n c t i o n .

R

...,n

nxn

(2.6.1)

,

has a s o l u t i o n The special

case where fi = x

was f i r s t t r e a t e d i n [141, w h i l e t h e i’ above more general f o r m u l a t i o n was suggested by W.M. Getz.

The c o n t r o l problem i s t o design a c o n t r o l l e r f o r (2.6.1)


66

which minimizes the performance criterion

V

=

- T Qx+u TRu)dt Ih(x

(2.6.2)

0

where Q E Rn ,' R E Rm x m are symmetric, positive definite matrices. Note that without loss of generality Ai can be assumed to be symmetric.

An alternative, finite-time, problem is to design a controller which minimizes

V

=

T

I $(xTQx+uTRu)dt

(2.6.3)

0

subject to the terminal constraint (2.6.4)

x(T) = b f 0 where T is given. 2.6.2

Solutions

We first state as theorems the solutions of the above two problems. We then prove one of these theorems. Suppose that there exists a vector c TUEOREM 2.6.1 which satisfies the Riccati-1 ike equation Q

t

n

Z ciAi

i =1

-

n

n

i=l

i=l

( Z ciBi)R-1 ( Z ciBi)T = 0.

E

Rn

(2.6.5)

Suppose, further, that each solution of the differential equations

CONTROL PROBLEMS

67

(2.6.6)

...,

i=l, n, i s defined f o r t t

+ m.

E

[ O p ) and t h a t x ( t )

Then, t h e minimum value o f (2.6.2)

c o n t r o l s which causes x ( t )

J

+

0 as t

+ m

+

0 as

i n t h e class o f

is

= cT [ f ( x o ) - f ( 0 ) l

(2.6.7)

and t h e optimal c o n t r o l l e r which achieves t h i s minimum i s u =

-

-1 n R ( Z c.B.) TX . i=1 1 1

(2.6.8)

THEOREM 2 . 6 . 2 Suppose t h a t t h e r e e x i s t s a vector c E Rn which s a t i s f i e s (2.6.5) and which causes each s o l u t i o n of (2.6.6) t o have t h e value b a t t = T; i.e. x(T) = b. Then, t h e minimum value o f (2.6.3) i s given by

J = cT [ f ( x o ) - f ( b ) l

(2.6 9)

and t h e optimal c o n t r o l l e r which achieves t h i s minimum va ue and which causes x(T) = b i s u =

-

n R - l ( Z ciBi) i=1

Tx.

(2.6 . l o )

As t h e p r o o f s o f t h e two theorems a r e very s i m i l a r we prove o n l y Theorem 2.6.1.

P R O O F O F THEOREM 2.6.7 that

Along a s o l u t i o n o f (2.6.1)

we have

L INEAR-QUADRAT IC EXTENS IONS

68 C

T

- cTf ( X ( t ) ) + Jt

f(Xo)

d

z [ C

0

T

f ( x ( ~ ) ) l d=~0. (2.6.11)

This, s u i t a b l y r e w r i t t e n , i s

-

cTf ( x o )

+

cTf ( x ( t ) )

n J($xT Z n ciAix+xT Z ciBiu)d-r o i=l i=1

= 0.

(2.6.12) Adding t h i s i d e n t i c a l l y zero q u a n t i t y t o (2.6.2) t

+ 00

and l e t t i n g

yields

v

= cTf ( x o )

-

Lim c T f ( x ( t ) ) t+-

03 n J & { x T [ Q t Z ciAi-( 0 i=l

t

n

Z ciBi)R-l(

i=1

n

Z ciBi)

i=1

n n T T [u+R-l( Z ciBi) XIR[u+R-'( ciBi) i=1 i=l

+

T

lx

Tx l l d t (2.6.13)

which, upon using (2.6.5)

v

= cTf ( x o )

-

becomes

Lim c T f ( x ( t ) ) t + w

W

+ J$[utR-'( 0

n

Z ciBi)

i=1

n

T T

XIR[u+R-l( Z ciBi) i=1

T xldt. (2.6.14)

As we o n l y draw our c o n t r o l s from t h a t c l a s s which causes x(t)

+

0 as t

T c f(x(t))

we see t h a t because o f t h e c o n t i n u i t y of f, T = c f ( 0 ) . Now, by assumption, ( 2 . 6 . 8 ) Y i e l d s a

+

03

Lim t+c o n t r o l f u n c t i o n which causes x ( t )

+

0 as t

+

00

so t h a t , by

CONTROL PROBLEMS inspection, (2.6.8) o f (2.6.14)

minimizes (2.6.14)

69 and t h e minimum value

i s g i v e n by (2.6.7).

A1 though t h e r e s t r i c t i o n s o f t h e theorems, p a r t i c u l a r l y t h e s o l v a b i l i t y o f (2.6.5),

a r e q u i t e s t r i n g e n t we nevertheless

have e x h i b i t e d here a c l a s s o f h i g h l y non-linear systems whose optimal c o n t r o l 1e r s r e l a t i v e t o quadratic performance criteria, are linear.

Note t h a t except f o r t h e s t a b i l i t y

c o n d i t i o n on (2.6.6),

t h e optimal c o n t r o l l e r i s independent

o f the function f ( x ) . 2.7

Conclusion

I n t h i s chapter we covered a broad ,class o f non-linearq u a d r a t i c c o n t r o l problems.

Sections 2.1 and 2.2 d e a l t w i t h

t h e m i n i m i z a t i o n o f exponential performance c r i t e r i a subject t o l i n e a r dynamic systems, i n a s t o c h a s t i c s e t t i n g .

We showed

t h a t t h e optimal c o n t r o l l e r i s l i n e a r b u t t h a t , i n the presence o f measurement noise, i t may be i n f i n i t e - d i m e n s i o n a l . I n t h e case o f no measurement noise we demonstrated a close t i e between t h e s t o c h a s t i c c o n t r o l problem and a class of d e t e r m i n i s t i c d i f f e r e n t i a l games.

One o f t h e games, t h e non-

cooperative one, i s o f t e n used t o design a type o f 'worstcase c o n t r o l l e r ' f o r l i n e a r systems.

The t i e between t h i s

game and t h e s t o c h a s t i c c o n t r o l problem seems t o imply t h a t t h i s type o f design i s n o t as conservative as i t s name would imp1y

.

We note here t h a t exponential performance c r i t e r i a have i m p l i c a t i o n s a l s o f o r fuzzy s e t theory [151.

I n f a c t the

exponential c r i t e r i a can be e a s i l y i n t e r p r e t e d as membership f u n c t i o n s o f fuzzy sets and t h i s determines t h a t t h e n o t i o n

LINEAR-QUADRATIC EXTENTIONS

70

o f 'confluence o f fuzzy goals and c o n s t r a i n t s ' , introduced by Bellman and Zadeh, be i n t e r p r e t e d i n novel ways. Section 2.3 was concerned w i t h non-1 inear s t o c h a s t i c systems o f special s t r u c t u r e . The f o r m u l a t i o n includes c o n t r o l and state-dependent noise i n l i n e a r systems, which have received a t t e n t i o n elsewhere,

Our general f o r m u l a t i o n y i e l d s a number

o f i n t e r e s t i n g f u r t h e r special cases.

We remark t h a t again

here t h e optimal feedback c o n t r o l l e r i s l i n e a r and depends upon t h e s t a t i s t i c s o f t h e system noise. Section 2.4 t r e a t e d i n f i n i t e - t i m e c o n t r o l problems v i a t h e Bellman equation.

T h i s allowed t h e s o l u t i o n o f some special

non-linear c o n t r o l problems and paved t h e way f o r Section 2.5 which was devoted t o t h e s t a b i l i z a b i l i t y and optimal c o n t r o l of non-linear systems homogeneous-in-the-input.

This i s one

o f t h e few classes o f non-linear systems which a l l o w the determination o f 'closed form' non-1 i n e a r feedback c o n t r o l l e r s . F i n a l l y , Section 2.6 was devoted t o the optimal c o n t r o l of a c l a s s o f h i g h l y non-linear systems.

The r e s u l t s , though

r a t h e r r e s t r i c t i v e , provide i n s i g h t i n t o quadratic and more non-linear systems. I n t h e next Chapter we t u r n t o m a t r i x theory

-

i n particular

t h e p r o p e r t i e s of c o p o s i t i v e and r e l a t e d matrices which p l a y a r o l e i n t h e understanding o f non-convex quadratic forms a r e studied.

C e r t a i n m a t r i x - t h e o r e t i c r e s u l t s a r e then

e x p l o i t e d i n o b t a i n i n g c o n d i t i o n s f o r t h e s o l u t i o n s o f quadrat i c d i f f e r e n t i a l equations t o have f i n i t e escape times.

CONTROL PROBLEMS 2.8

71

References

[l]JACOBSON, D.H. Optimal Stochastic Linear Systems w i t h Exponential Performance C r i t e r i a . and t h e i r R e l a t i o n t o D e t e r m i n i s t i c D i f f e r e n t i a l Games. IEEE Trans. Automatic Control , AC-18, 1973, pp. 124-131.

[2]

JACOBSON, D.H. On a Result i n Stochastic Optimal Control. I E E E Trans. Automatic Control , AC-18, 1973, pp. 411-412.

[3]

SPEYER, J.L , DEYST, J. 81 JACOBSON, D.H. Optimization o f Stochas t c Linear Systems w i t h A d d i t i v e Measurement and Process Noise Using Exponential Performance C r i t e r i a . IEEE Trans. Automatic Control , AC-19, 1974, pp. 358-366.

[4]

IEEE Trans. Automatic Control , AC-16, December 1971. Special i s s u e on t h e Linear-Quadratic-Gaussian Problem.

[5]

WITSENHAUSEN, H.S. Separation o f Estimation and Control f o r Discrete-Time Systems. Proc. IEEE, 59, 1971, pp. 1557-1566.

161

SPEYER, J.L. An Adaptive Terminal Guidance Scheme Based on an Exponential Cost C r i t e r i o n w i t h A p p l i c a t i o n t o Homing M i s s i l e Guidance. IEEE Trans. Automatic Control , AC-21, 1976, pp. 371-375.

[7]

A General Result i n Stochastic Optimal JACOBSON, D.H. Control o f Non-linear Discrete-Time Systems w i t h Quadratic Performance C r i t e r i a . J. Math. Anal. Appl., 47, 1974, pp. 153-161.

[8]

WONHAM, W.M. Optimal S t a t i o n a r y Control o f a Linear System w i t h State-dependent Noise. SIAM J. Control, 5, 1967, pp. 486-500.

[9]

McLANE, P.J. Optimal Stochastic Control o f Linear Systems w i t h S t a t e and Control Dependent Disturbances. IEEE Trans. Automatic Control , AC-16, 1971, pp. 793-798.

1101

PEARSON, J.D. Approximation Methods i n Optimal Control I. Sub-optimal Control. J. E l e c t r o n i c s and Control, 13, 1962, pp. 453-469.

72

[ll]

LINEAR-QUADRATIC EXTENSIONS SPEYER, J.L. A Non-linear Control Law for a Stochastic Infinite Time Problem. IEEE Trans. Automatic Control , AC-21, 1976, pp. 560-564.

[121 JACOBSON, D.H. Stabilization and Optimal Control of Non-linear Systems Homogeneous-in-the-input. In Proceedings o f a Conference on Directions in Decentralized Control, Many-Person Optimization and Large-Scale Systems, held in Boston, Mass., 1-3 September 1975 and published by Plenum Press, New York, 1976, pp. 389-399. [131 HAHN, W. Stability of Motion. Heidel berg , New York, 1967.

Springer Verlag, Berlin,

1141 JACOBSON, D.H. On the Optimal Control of Systems of Quadratic and Bilinear Differential Equations. Proc. 6th IFAC World Congress, Boston, Mass., August, 1975. [151 JACOBSON, D.H. On Fuzzy Goals and Maximizing Decisions in Stochastic Optimal Control. J. Math. Anal. Appl , 55, 1976, pp. 434-440.

.

3.

COPOSITIVE MATRICES, NON-CONVEX QUADRATIC FORMS AND QUADRATIC DIFFERENTIAL EQUATIONS

3.1

Introduction t o Copositive Matrices

I t t u r n s o u t t h a t much o f t h e t h e o r y o f c o n t r o l , o p t i m i z a t i o n

and s t a b i l i t y depends h e a v i l y upon m a t r i x t h e o r y

-

i n parti-

c u l a r upon eigenvalue t h e o r y f o r general square m a t r i c e s and upon t h e p r o p e r t i e s o f q u a d r a t i c forms.

U s u a l l y one assumes

t h a t q u a d r a t i c forms a r e p o s i t i v e s e m i - d e f i n i t e , t h a t i s , T It i s t r i v i a l t o show t h a t p o s i t i v e x Qx > O f o r a l l x E Rn. s e m i - d e f i n i t e q u a d r a t i c forms a r e convex f u n c t i o n s ( t h e converse i s a l s o t r u e ) , and t h i s i s an added a t t r a c t i o n f o r t h e i r use.

Furthermore, t h e S y l v e s t e r - t e s t f o r p o s i t i v e

d e f i n i t e m a t r i c e s i s w i d e l y known and r e l a t i v e l y easy t o apply, a t l e a s t i n a few dimensions.

Last but not least,

Liapunov s t a b i l i t y t h e o r y i s based s o l i d l y upon t h e n o t i o n of a p o s i t i v e d e f i n i t e ( q u a d r a t i c ) form. Though u s e f u l , t h e n o t i o n o f a p o s i t i v e s e m i - d e f i n i t e m a t r i x For example, ifQ T i s p o s i t i v e s e m i - d e f i n i t e we have a f o r t i o r i t h a t x Qx > O ( o r q u a d r a t i c form) i s v e r y r e s t r i c t i v e .

f o r a l l x > 0. However, t h e s e t o f m a t r i c e s f o r which T x Qx > O f o r a l l x > 0 i s l a r g e r t h a n t h e s e t of p o s i t i v e semi-definite matrices.

T h i s c l a s s , which i s c a l l e d t h e

c l a s s o f c o p o s i t i v e m a t r i c e s , i s c l e a r l y important i n problems

o f c o n s t r a i n e d m i n i m i z a t i o n such as t h a t o f m i n i m i z i n g xTQx subject t o Ax where x

E

>b

Rn.

73

(3.1 . l ) (3.1.2)

74

L INEAR-QUADRATI C EXTENSIONS

The two major e a r l y works on c o p o s i t i v e matrices are, i n our opinion,

[ll and 121.

As pointed o u t above, any n x n p o s i t i v e

semi-definite m a t r i x i s a l s o c o p o s i t i v e as i s any n x n m a t r i x having non-negative elements.

I n [ll Diananda t h e r e f o r e

conjectured t h a t t h e c l a s s o f c o p o s i t i v e matrices c o n s i s t s o f matrices which a r e sums of p o s i t i v e semi-definite matrices and m a t r i c e s having non-negative elements. was a b l e t o prove t h a t t h i s i s t r u e f o r n

Q

E

Rnxn),

O , x > 0 has a nonof Q, the system Q Q

trivial solution.

Gaddum's theorem i s useful in t h a t i t relates the properties of a non-convex quadratic form t o those of systems of linear inequalities. Such systems of equations may be tested f o r solutions via a linear program as the following theorem indicates.

NON-CONVEX QUADRATICS The system o f l i n e a r i n e q u a l i t i e s Q

THEOREM 3 . 7 . 4

x

>0

kl...k

sx>o,

has a n o n - t r i v i a l s o l u t i o n i f and o n l y i f t h e f o l l o w i n g

l i n e a r program has a n o n - t r i v i a l s o l u t i o n

> 0:

and E

79

max

(X,;)

with

f O (3.1.14)

E

X 9E

subject t o the constraints

Q

kl...k

sx

> Ee

e>x>O

(3.1.15) (3.1.16)

where e i s a v e c t o r o f ones o f t h e same dimension as x .

PROOF

I f t h e l i n e a r i n e q u a l i t i e s have a n o n - t r i v i a l s o l u t i o n

x t h i s q u a l i f i e s as a s t a r t i n g s o l u t i o n f o r t h e l i n e a r program, with

E > 0.

Consequently, t h e l i n e a r program has a non-

t r i v i a l solution.

On t h e o t h e r hand, i f t h e l i n e a r program

has a n o n - t r i v i a l s o l u t i o n , so do a l s o t h e l i n e a r i n e q u a l i t i e s , 3.2

Extensions

I n t h i s s e c t i o n we extend c e r t a i n o f t h e above r e s u l t s . b e g i n b y g e n e r a l i z i n g t h e c o n s t r a i n t s e t from x 2 0 t o Ax 3.2.1

We

> 0.

General L i n e a r C o n s t r a i n t s

T We here c h a r a c t e r i z e m a t r i c e s Q f o r which x Qx > 0 f o r a l l x such t h a t Ax > 0, A E Rm x n

.

T TffEOREM 3 . 2 . 7 Suppose t h a t x Qx > 0 f o r a l l x such t h a t Ax > 0 where Rank(A) = m, and where A = [A1 A2], A1 E R m x m


80

and is non-singular and A,

[:' :-I

E

R

(n-m).

Then,

T

=

where E

cll

R m x m is copositive, (n-m)

c22

C12y T

E

(3.2.1)

(n-m) is positive semi-definite,

Range (CZ2) for all y

> 0, y

-C C+ CT is copositive, where

cll12 22 12

E

'+I

Rm , CI2

R m x (n-m)

denotes pseudo-

(3.2.2)

inverse.

Note that the rank assumption on A guarantees that there is an arrangement of variables such that A1 is non-singular. nxn COROLLARY 3.2.7 A usefu special case arises when A E R has full rank n. In this case

Q where C

E

R

- A TC A = O

(3.2.3)

is copositive.

COROLLARY 3.2.2

Q

If A = C22

f

0 then (3.2.1), (3.2.2) imply that

> 0 (positive semi-definite).

(3.2.4)

There is a further corollary which is sufficiently important

81

NON-CONVEX QUADRATICS t o warrant i t s statement as a theorem.

T

A s u f f i c i e n t c o n d i t i o n f o r x Qx > 0 f o r a l l TffEOREM 3 . 2 . 2 x such t h a t Ax > 0, A E Rm i s t h a t t h e r e e x i s t s a coposit i v e m a t r i x C E Rm x m such t h a t

Q

- ATCA

0 (positive semi-definite).

(3.2.5)

Note t h a t t h e r e i s here no assumption on t h e rank o f A. A i s square and non-singular, c o n d i t i o n o f Theorem 3.2.2

P R O O F O F THEOREM 3 . 2 . 1

We separate t h e v e c t o r x i n t o two

x=[

E

Rm, x 2

E

C o r o l l a r y 3.2.1 shows t h a t t h e

i s b o t h s u f f i c i e n t and necessary.

pieces as f o l l o w s

where x1

If

:;I

(3.2.6)

We t h e n see t h a t

R.”’

A y = AX = Alxl

t

A2x2

,

y

E:

Rm,

y

>0

(3.2.7)

whence x1 = A;l(y-A2x2)

(3.2.8)

and i t then f o l l o w s t h a t

We then have t h a t

AT1

-A’1A2]

0

In-m

[ I]. (3.2.9)

82

L INEAR-QUADRATIC EXTENSIONS x;]ATQA[

1 2 0 for all y > O ,

x2 E R n-rn

(3.2 . l o )

where (3.2.11)

Defining (3.2.12)

we see that (3.2.10) holds if and only if conditions (3.2.2) are satisfied. Equation (3.2.1) follows upon noting that the inverse of A is (3.2.13)

3.2.2

An Example

Here we illustrate Theorem 3.2.2. 0

1

We wish to know whether xTQx Ax > 0.

Let -1

> 0 for all x

E

R 2 such that

Clearly Q is not positive semi-definite. However, choosing

83

NON-CONVEX QUADRATICS C to be the copositive matrix

I'

(3.2.15)

0

yields Q - A TC A =

'0

2.16) which is positive semi-definite, implying the non-negat vi ty of xTQx for all x such that Ax > 0. 3.2.3 Application to Quadratic Programming A typical quadratic programming problem is the following. Minimize the quadratic function f(x)

=

3~T QX

(32.17)

subject to the inequal ity constraints Ax > b where x

E

Rn, A

E

Rmxn, b

E

Rm and Q

(32.18) E

Rn x

is symmetric.

It is well known that a necessary condition (Fritz John, Kuhn-Tucker) for to minimize (3.2.17) subject to (3.2.18) is that there exists h Z 0, A E Rm, such that

84

L INEAR-QUADRATI C EXTENSIONS

Qx and Ai - b

-

T A X = 0

(3.2.19)

> 0 , XT ( A i - b )

= 0.

It i s a l s o known, and e a s i l y shown, t h a t i f

(3.2.19)

and (3.2.20)

then

(3.2.20)

i satisfies

i a c t u a l l y minimizes (3.2.17)

(3.2.21)

Q 2 0 ( p o s i t i v e semi-definite). The case Q

>0

if

i s r e f e r r e d t o as convex because t h e f u n c t i o n

f ( x ) i s convex i f (3.2.21) holds. I t t u r n s o u t t h a t t h e convex case i s r a t h e r r e s t r i c t i v e as t h e r e a r e i n t e r e s t i n g q u a d r a t i c problems where Q

3” 0, and t h i s has s t i m u l a t e d

research on t h e non-convex case [71-[131.

We a r e now a b l e

t o provide the f o l l o w i n g useful s u f f i c i e n t condition f o r o p t i m a l i t y i n t h e non-convex case. Suppose t h a t

THEOREN 3 . 2 . 3 .

s a t i s f i e s (3.2.19),

(3.2.20).

Then a s u f f i c i e n t c o n d i t i o n f o r i t o m i n i m i z e (3.2.17) that there exists a copositive matrix C

Q

-

T A CA

>0

E

is

Rm x m such t h a t

(positive semi-definite)

(3.2.22)

C(Ai-b) = 0.

(3.2.23)

and

PROOF

If

x minimizes (3.2.17)

then

-

( ~ + i ) ~ Q ( x +>i )x-T Qx f o r a l l x such t h a t

(3.2.24)

a5

NON-CONVEX QUADRATICS

A ( ~ + x ) - b 2 0.

(3.2.25)

From (3.2.22), (3.2.23) we have

x TQx

-

[A(x+x)-bl TC[A(i+x)-bl

2 0, for all x

E

Rn (3.2.26)

and from (3.2.19), (3.2.20) that xTQx

= A

T [A(x+x)-bl , for all x

E

Rn.

(3.2.27)

T T From (3.2.26) and (3.2.27) we see that both x Qx and x Qx are non-negative for all x such that (3.2.25) holds. Inequality (3.2.24) now follows easily. Note that if we further restrict C to be a matrix with all its elements non-negative, equation (3.2.23) becomes a complementary slackness condition similar to the second inequality o f (3.2.20). 3.2.4

Stochastically Copositive Matrices

Theorem 3.1.1 is remarkable in that it characterizes copositive matrices only for n 0 for all x such that xTAx = 0, where Q, A are syrrmetric real matrices, then there exists a real scalar u such that

Q

-

uA

>0

(positive definite).

(3.2.45)

In this section we generalize Finsler's theorem by treating the more general case of an arbitrary number of quadratic equalities and inequalities, Furthermore we apply the general ized result to deduce certain properties of the inverse of a copositive matrix. We consider the following formulation Minimize xTQx subject to and

(3.2.46)

...,r

(3.2.47)

..,s

(3.2.48)

x TAix = bi

i=l,

xTC.x 2 d

j=1,.

J

...

j'

...,

where Q y Ai, i=l, ¶ r and C j=1, s are n x n symnetric jy matrices and x E Rn. We assume that there is an x E Rn which satisfies (3.2.47), (3.2.48).


THEOREM 3.2.7

[16]

91

Suppose t h a t there e x i s t s a non-singular

m a t r i x F which simultaneously diagonal i z e s Q, Ai, i=l,. Cj, j=1,. ,s. Then t h e non-1 i n e a r programming problem

..

formulated above has a s o l u t i o n j; e x i s t scalars ai, matrix

fi

i=l,

..,r,

Rn i f and only i f t h e r e r, p j Z 0, j=1, s and a diagonal E.

...,

...,

having non-negative elements, such t h a t Q

-

r X uiAi i=l

-

S

X pjCj j=1

-

(F-l)TflF-l

= 0

S

(3.2.49)

2 p . [iTC.;(-d .] = 0 j=lJ J J

(3.2.50)

-T -1 T- -1x ( F )NF x = O

(3.2.51)

= bi

,

i=l, r

...,

(3.2.52)

-T x C . i >dj J

,

j=1,

...,s.

(3.2.53)

-T x AiX

COROLLARY 3.2.5 Under t h e s t a t e d assumption o f simultaneous T d i a g o n a l i z a b i l i t y we have t h a t x Qx Z 0 f o r a l l x E Rn such T ,r, xTCjx > 0, j = l , . ,s i f and o n l y t h a t x A.x = 0, i=l,. 1 ift h e r e e x i s t r e a l scalars ai, i=l, r and p . 3 0 , j=1, s J such t h a t

..

Q

r

..

...,

...,

S

- i 2= laiAi - j 2= l p.C.J Z 0 ( p o s i t i v e s e m i - d e f i n i t e ) . -

J

(3.2.54)

PROOF OF THEOREM 3.2.7

Let y = F'lx. Then, because 0 FTQF, j=l,,, ,s a r e diagonal, the

IiA= FTA.F, i=l,. ..,r, E . 4- FTC.F, 1

J

J

.

4

EXTENS IONS

LINEAR-QUADRATI C

92

above f ormul a t ion becomes Minimize

2

Y1¶ subject t o

n

Z

( A .i)

k= 1

. ..

2 k=l

2

('j)kkYk

2 ,...,yn 2 as z1

Re-naming y1

$&yk

2

(3.2.55)

9Yn

,

y 2 = bi kk k

n k=l

zn

'

dj 9

> 0,

problem i s

(3.2.56)

i=l,...,r

j=1,.

. .,s.

...,z n 2 0 we

(3.2.57)

see t h a t t h e

(3.2.58) subject t o

n

Z: (Ai)kkzk

k=1

,

= bi

(3.2.59)

i = l ,...,r

(3.2.60)

>0 ,

zk

k=l,

...,n .

Well-known necessary and s u f f i c i e n t c o n d i t i o n s f o r

(3.2.61)

21,.

t o s o l v e t h i s l i n e a r program a r e t h a t t h e r e e x i s t aiy i=l,.

..,r

and p 2 0, j = l , . j r

'i -

Z

OiAi

i=1

., ,s

-

. .,? n

such t h a t S

Z pjtj

j=l

-R

= 0

(3.2.62)

NON-CONVEX QUADRATICS where the diagonal matrix where

93

has non-negative elements and

(3.2.63)

n

2

k=o

Rkkik =

0.

(3.2.64)

Pre-multiplying (3.2.62) by (F-l) and post-multiplying by F-l yields (3.2.49), and (3.2.63) and (3.2.64) become (3.2.50), (3.2.51) respectively

.

We now establish the relationship of the above results to that of Finsler (3.2.45). First we note the important fact that any matrices Q and A which satisfy Finsler's conditions also satisfy our assumption of simultaneous diagonal izabil ity. This follows from the fact that Q and (Q-aA), which is positive definite, are simul taneously diagonalizable by a nonsingular matrix [17]. Next we note that (3.2.45) i s recoverable from Corollary 3.2.5. For setting A1 = A, A2 = ... = Ar = 0, C1 = = Cs = 0 we have that there is a al such that

...

Q

- alA 2 0.

(3.2.65)

It is easy to verify that the inequality in (3.2.65) can be strengthened to strict inequality. From the above discussion it is clear that Theorem 3.2.7 and Corollary 3.2.5 are non-trivial generalizations of Finsler's Theorem. We now investigate the degree of restrictiveness of the

94


assumption o f simultaneous diagonal i z a b i l i t y .

The n o t i o n of

copositiveness a s s i s t s us here. Theorem 3.2.7 does n o t n e c e s s a r i l y h o l d if THEOREM 3.2.8 t h e assumption o f simultaneous d i a g o n a l i z a b i l i t y i s removed.

PROOF

The p r o o f i s by a counter-example. Suppose t h a t T i s c o p o s i t i v e . Then t h e statement t h a t x Qx > 0 Q E R5' f o r a l l x > 0 can be re-phrased as

T x Qx>O

(3.2.661

subject t o X.X.

1 J

IfTheorem 3.2.7

2 0,

i=l,

j

...,5.

j=itl,

(3.2.67)

were here v a l i d we would have

-

Q

where t h e C

...,4,

, j=l,...,10

10 Z p.C. 2 0 j = lJ J

(3.2.68)

a r e t h e matrices w i t h non-negative

elements which r e a l i z e (3.2.67).

However, (3.2.68)

would

then imply t h a t every c o p o s i t i v e m a t r i x i n f i v e dimensions i s expressible as t h e sum o f a p o s i t i v e s e m i - d e f i n i t e m a t r i x and a m a t r i x w i t h non-negative elements. But the example of (3.1.6)

disproves t h i s and so we conclude t h a t Theorem 3.2.7

i s n o t n e c e s s a r i l y v a l i d i n t h e absence o f t h e s t a t e d assumption. The next theorem provides necessary c o n d i t i o n s and s u f f i c i e n t c o n d i t i o n s f o r (3.2.54)

t o hold;

t h e gap between necessity

95

NON-CONVEX QUADRATICS and sufficiency is small.

THEOREM 3 . 2 . 9 ( i ) Necessary Condition. Suppose that (3.2.54) holds with strict inequality (i.e. it is positiver S definite). The Q and ( Z aiAit I: p.C.) are simultaneously i=l j=l J J diagonalizable and x Qx > 0 for all x E Rn such that S xT ( Ir: u . A . t Z p.C.)x > 0. i =1 j=1 J J

’’

(ii)

Sufficient Condition.

aiy

Suppose there exist i=l,...,r r S and , j=l,... ,s such that Q and ( I: 3 i A i tj=l Z 6.C .) are J J j i =1 simultaneously diagonalizable and that. xTQx > 0 for all x E Rn S T r such that x ( I: aiAit I: p.C.)x 2 0. Then inequality (3.2.54) i=l .i =1 J J holds. The above theorem, which is easy to prove, indicates the extent to which the diagonalizabil ity assumption of Corollary 3.2.5 may be weakened

.

3.2.7 Application of Generalized Finsler Theorem We establish here certain properties of inverses of copositive and other matrices. First we present necessary and sufficient conditions for XT Q-1 x > O for all x

E

(3.2.69)

Rn such that x TQx 2 0.

(3.2.70)

96


A necessary and sufficient condition for (3.2.69) to hold for all x such that (3.2.70) holds is that there is a scalar p 2 0 such that

THEOREM 3.2.10

Q

-

pQ3

0.

(3.2.71)

Written in terms of the eigenvalues A k , k=l, ...,n of Q this is 3 (3.2.72) A k - PAk 2 0. COROLLARY 3.2.6 A sufficient condition for the inverse of a copositive matrix Q to be copositive is that (3.2.71) holds for some p > 0. PROOF OF THEOREM 3 . 2 . 1 0

The problem as posed is clearly T equivalent to that of the non-negativity of x Qx for all T 3 x E Rn such that x Q x 2 0. Because of the fact that Q and Q3 comnute, there exists an orthogonal (invertible) matrix which simultaneously diagonal izes them. Corollary 3.2.5 therefore applies, yielding (3.2.71).

A further interesting observation is that Theorem 3.2.10 implies that necessary and sufficient conditions for the set R n IxTQx >, 0) to be identical to the set { x E R n (xTQ-1x>O) are that there exist p1 2 0, p2 2 0 such that (x E

and

3

Q

(3.2.73) -p2Q >O.

These inequalities imply that Q( 1-plp2) > 0 which imp1 ies

97


t h a t Q i s p o s i t i v e d e f i n i t e o r negative d e f i n i t e , o r t h a t p1p2 = 1. This last-mentioned case i m p l i e s t h a t

Q

-

3

P1Q

-

0, P1Q

(3.2.74)

Q>O

or that

9-l

= plQ,

p1

> 0.

(3.2.75)

2 An example o f such a m a t r i x i n R i s 3.3

Quadratic D i f f e r e n t i a l Equations

I n t h i s s e c t i o n we i n v e s t i g a t e t h e p r o p e r t i e s o f a general c l a s s o f quadratic d i f f e r e n t i a l equations.

More s p e c i f i c a l l y ,

we o b t a i n s u f f i c i e n t c o n d i t i o n s f o r t h e existence o f f i n i t e escape times and provide upper bounds f o r these times.

The

c l a s s o f problems studied i s o f i n t e r e s t i n i t s own r i g h t b u t appears t o be o f importance a l s o i n optimal c o n t r o l theory ( c f . Section 2) and i n t h e mathematical modelling o f c e r t a i n chemical engineering and predator-prey problems. It t u r n s o u t t h a t t h e r e e x i s t c e r t a i n Liapunov-type theorems which provide s u f f i c i e n t c o n d i t i o n s f o r t h e existence of f i n i t e escape times i n s o l u t i o n s o f non-1 inear d i f f e r e n t i a l equations [181 b u t these theorems have n o t d i r e c t l y y i e l d e d u s e f u l r e s u l t s i n t h e case o f quadratic d i f f e r e n t i a l equations. The primary reason f o r t h i s i s t h a t t h e t i m e d e r i v a t i v e o f a s c a l a r f u n c t i o n o f even order i s odd, and t h e r e f o r e i t i s d i f f i c u l t t o comment upon i t s s i g n w i t h o u t having some f u r t h e r knowledge o f t h e t r a j e c t o r i e s o f t h e quadratic d i f f e r e n t i a l equations.

Furthermore t h e known general theorems mostly

98


assume positivity and radial unboundedness of a Liapunov-1 ike function, which are properties usually associated with even functions. Thus few explicit results are known for quadratic differential equations. Our approach to the problem is to use a 'linear Liapunov function' whose time derivative is then a quadratic function of the system state. Requiring this quadratic form to have a definite sign for all values of the system state or, less restrictively, for those values of the state which lie in certain invariant sets (cones) allows us to obtain our sufficient conditions. We apply our conditions to the wellknown matr x Riccati equation to obtain results which complement those of Kalman. We demonstrate also that the conditions can be app ied to a general class of quadratic equations used to model interacting populations. The Lotka-Volterra model of predator-prey dynamics is a special case of this class. Note that the idea of using a linear Liapunov function and requiring that its time derivative be definite in sign for all values of the state has occurred also to Frayman [191 , but our first results pre-date his [201. 3.3.1

Finite Escape Times

Consider the general non-1 inear system of ordinary differential equations >i. = f(x,t),

x(0) = xo

(3.3.1)

where x E Rn and where f:Rntl Rn is sufficiently wellbehaved to guarantee a unique solution x(t;xo) on [O,t) for some > 0. It is known that either x(t;xo) may be extended -f

99

NON-CONVEX QUADRATICS f o r a l l t beyond t o r t h e r e e x i s t s a t i m e te

such t h a t

I n t h e l a t t e r case t h e s o l u t i o n i s e' s a i d t o have a f i n i t e escape t i m e a t te.

Ilx(t;xo)ll

-+

mas t

+

t

I n t h i s c h a p t e r we d e r i v e s u f f i c i e n t c o n d i t i o n s f o r t h e s o l u t i o n o f t h e q u a d r a t i c equations

x i = xTA.x 1

T + b.x + c i' i = l , 1

...,n,

x ( 0 ) = xo (3.3.2)

t o have a f i n i t e escape time. i=l,

...,n a r e

i=l,

E

R

nxn

,

constant, and w i t h no l o s s o f g e n e r a l i t y a r e

assumed t o be symmetric. ci,

Here, t h e m a t r i c e s Ai

...,n a r e

The v e c t o r s bi

E

Rn and t h e s c a l a r s

constant.

Owing t o t h e f a c t t h a t t h e r i g h t - h a n d sides o f t h e d i f f e r e n t i a l equations (3.3.2)

a r e q u a d r a t i c i n x, i t f o l l o w s from standard

e x i s t e n c e t h e o r y t h a t f o r each xo

E

Rn t h e r e e x i s t s a unique

o f (3.3.2) d e f i n e d on t h e i n t e r v a l [O,i(xo)) where i ( x o ) > 0 f o r each xo 8 Rn. Furthermore i t i s known s o l u t i o n x(t;xo)

t h a t even t h e s i m p l e s t q u a d r a t i c d i f f e r e n t i a l e q u a t i o n can e x h i b i t f i n i t e escape times.

For example t h e s c a l a r q u a d r a t i c

d i f f e r e n t i a l equation j = Y

2

(3.3.3)

has a unique s o l u t i o n which has a f i n i t e escape t i m e te = i f and o n l y i f xo

> 0.

xO

Before proceeding t o an a n a l y s i s o f (3.3.2) d e t a i l t h e s c a l a r q u a d r a t i c equation.

1 -

we c o n s i d e r i n

The r e s u l t s so obtained

a r e t h e n used t o bound t h e s o l u t i o n s o f (3.3.2).


100

Given constants kl

LEMMA 3 . 3 . 7

> 0,

k2, k3 t h e s o l u t i o n

y(t;yo) o f the scalar d i f f e r e n t i a l equation

i=

klY

2

+ k2Y + k3'

Y(0) = Yo

(3.3.4)

goes t o i n f i n i t y i n a f i n i t e t i m e under t h e f o l l o w i n g conditions: (i) (ii)

f o r a l l yo, whenever k i

> (-k2 + - 4klk3 > 0.

f o r yo ki

-

4klk3

B be t h e r e a l r o o t s o f kly

+

( i ) i s given by t

=

m

2

-

arctan[2kl(yo+

k2

-2T;-)/fl]) (3.3.5) 1

and i n case ( i i ) i s g i v e n b y

and

PROOF Local e x i s t e n c e t h e o r y guarantees t h a t t h e r e i s a > 0 such t h a t t=J-

Y(t) Yo

dY klYZ+k2Y+k3

,

t E t0,i).

(3.3.8)

101


T h i s expression can be i n t e g r a t e d t o y i e l d y ( t ) , b u t t h e r e a r e two d i s t i n c t cases. (i)

0

fies

O we have

102


Note t h a t i f (yo-a) t

E

< 0,

then y ( t ; y o ) can be extended f o r a l l

>0

However, i f (yo-a)

[O,m).

t h e s o l u t i o n can o n l y be

extended as f a r as te where te s a t i s f i e s (Y0-C) which y i e l d s (3.3.6).

-

(Yo'a>e

te

Equation (3.3.7)

= o

(3.3.13)

i s obtained upon

d i r e c t i n t e g r a t i o n o f (3.3.8). The next lemma r e l a t e s a c e r t a i n d i f f e r e n t i a l i n e q u a l i t y t o (3.3.4).

LEWA 3.3.2 V(x) of x

Suppose t h a t t h e r e e x i s t s a s c a l a r f u n c t i o n E

Rn and s c a l a r s kl

> 0,

k2 and k3 such t h a t along

s o l u t i o n s o f (3.3.1)

-> dt

klV 2 ( x )

Suppose f u r t h e r t h a t I l x I I + s o l u t i o n x(t;xo)

m

+ k2V(x) + k3

whenever V(x)

+

.

m.

(3.3.14) Then t h e

o f (3.3.1) has a f i n i t e escape t i m e whenever

the s o l u t i o n o f

goes t o i n f i n i t y i n a f i n i t e time.

Furthermore Lemna 3.3.1

provides upper bounds f o r t h e escape time of (3.3.1).

PROOF that

I t f o l l o w s immediately from (3.3.14)

and (3.3.15)


103

so t h a t V goes t o i n f i n i t y i n a f i n i t e t i m e i f y does, and t h i s i m p l i e s t h a t Ilx(t;xo)ll

-+

00.

We may now use Lemmas 3.3.1 and 3.3.2 t o deduce s u f f i c i e n t cond it ons f o r t h e so u t i o n o f (3.3.2) t o have a f i n i t e escape time. For any w E Rn w i t h i - t h element mi, we define A

w

b

w

A

=

A =

cw

n

Zw.A i i

(3.3.17)

n Zw.b i=l i i

(3.3.18)

n 2 WiCi i=1

(3.3.19)

i=1

(3.3.20) where Aw i s t h e s m a l l e s t eigenvalue o f t h e symmetric m a t r i x

THEOREM 3 . 3 . 1 [21] Suppose t h e r e e x i s t s a v e c t o r w E Rn such t h a t Aw > 0 ( p o s i t i v e d e f i n i t e ) . Then t h e s o l u t i o n o f (3.3.2) has a f i n i t e escape t i m e te under t h e f o l l o w i n g c o n d i t i o n s f o r a l l xo, whenever Aw

(i) (ii)

< 0;

f o r xo s a t i s f y i n g

T w x0 whenever A

w

I n case ( i )

> 0.

> - $wTAw-1bw + 3%.

r I

w

(3.3.21)

104

L INEAR-QUADRATIC EXTENSIONS teG -

q

{x -

arctan[Zw T (xo+tAw-1b )- 'w

2

wTw

- -I ) . q (3.3.22)

I n case ( i i )

. We o b t a i n from Equation (3.3.2)

PROOF

d T T -d(at X ) = x Awx

(3.3.23)

that

T + bwX +

(3.3.24)

cw

which may be w r i t t e n as d

-(w dt

T

-1 T -1 T -1 X ) = (x+&Aw bw) Aw(x+4AW bw) + cW - abwAw bw

- [wT (x+gA-'b

>T'w

)I 2 +

W

w w

cw

-

abwAw T -1bW

O

(3.3.25)

by t h e Cauchy-Schwartz i n e q u a l i t y . Re-arranging (3.3.25) d

=(w

T

x)

we o b t a i n

T 2

'w

>T

W W

*(w x)

-

t

T w w

T -1 bw

TAw-1bw ( W TX )

+'w - ( $ w TAW -1bw)2 T w w

.

(3.3.26)

Hence t h e s o l u t i o n o f (3.3.2)

has a f i n i t e escape t i m e under

+

c

w

aA b,

t h e c o n d i t i o n s provided i n Lemma 3.3.1 i f we i d e n t i f y kl, k2 and k3 as f o l l o w s


kl A= -

x

>o

T

w w

(3.3.27)

k q L . w ATw-1 b w 2 T w w k3

4

whence

A = k2 2 3.3.2

x

T -1

w w

-

(3.3.28)

*(&I Aw bw)2

T

105

t cw

-

T -1 ,bwAw bw

4klk3 = ‘w (.bA , , T -1bw-4cw) = Am. T w w

(3.3.29)

(3.3.30)

M a t r i x R i c c a t i Equation

We now apply Theorem 3.3.1 differential

t o t h e autonomous m a t r i x R i c c a t i

equation P = Q t PF t

FTP

-

where a l l t h e matrices a r e i n R

PRP,

nxn

P(0) = Jl

, and

(3.3.31)

Q, R, ll and hence

P(t;II) a r e symmetric. Many s u f f i c i e n t conditions a r e known f o r t h e s o l u t i o n o f (3.3.31) t o e x i s t on Kalman 1221 , v i z .

lo,-),

R>0,

b u t those known best are due t o

Q>O,

II20.

The s u f f i c i e n c y r e s u l t s f o r the s o l u t i o n o f (3.3.31)

(3.3.32) to

escape i n a f i n i t e time, presented below, complement the existence conditions. I n order t o convert (3.3.31)

t o a s u i t a b l e vector quadratic

106


equation we use t h e f o l l o w i n g m a t r i x o p e r a t i o n s . mxn (i)The Kronecker p r o d u c t A 0 B of two m a t r i c e s A E R and B E. R p x q i s d e f i n e d 1171 as a m a t r i x C E R m p x n q whose elements i n t h e ( i - l ) p + l t o i p rows and ( j - l ) q + l t o j q columns

i = l , ...,m, j=1,

a r e (aijB),

...,n ,

aB l C= aB ml

viz.

... anlB (3.3.33)

... amnB

I t i s easily verified that ( A 0 B)T = AT 0 BT

(3.3.34)

and

(A

o ~ 1 - =l A - ~o B - ~ ,

A, B non-singular.

(3.3.35)

d e f i n e a stacking operator ( i i ) For any m a t r i x A E Rm a ( A ) as an mn v e c t o r formed by s t a c k i n g t h e columns o f A under each o t h e r , i n o r d e r , as f o l l o w s U(A)

A = (all,.

..,aml;a12,.

.,am2;. .. ;aln,.

.

..,a mn lT (3.3.36)

C l e a r l y , u i s a l i n e a r o p e r a t o r from R " ' ~ A, B E R m x n and c E R 1 u(A+B) = a ( A )

+

a(B)

u(cB) = cu(B).

+

R"'~,

i.e. f o r

(3.3.37) (3.3.38)


107

( i i i ) For any m a t r i c e s A, B and P f o r which t h e m a t r i x product APB i s defined, i t can be shown t h a t u(APB) = (BT O A ) u ( P ) .

(3.3.39)

We now use t h e above operations and t h e symmetry o f P, Q, R and II t o prove t h e f o l l o w i n g lemma. Let W

LEMMA 3 . 3 . 3

E

R

be a g i v e n constant m a t r i x .

Then along s o l u t i o n s o f (3.3.31) we have t h a t &(UTP,

= PTRwP + fwP T

+

wTq

(3.3.40)

where w

= u(W)

P

= 4P)

q

= u(Q)

(3.3.41)

b

fa = [F 0 I + I 0 Flu(W)

R

w

PROOF

= - WOR.

I

We have t h a t d

=[U

T

T T (W)a(P)] = u (W)u(Q+PF+F P-PRP)

.

(3.3.42)

Now uT(W)o(PRP) = uT(W)( I0 PR)u(P) = uT(RPW)a( P) = uT(P)(W O R ) a ( P )

.

(3.3.43)

108


Similarly a(PFtFTP) = ( F T O I t I O F T ) a ( P ) . Using these i d e n t i t i e s i n (3.3.42)

(3.3.44)

y i e l d s (3.3.40).

We can now apply Theorem 3.3.1 t o y i e l d

THEOREM 3.3.2

[211

The s o l u t i o n o f (3.3.31)

has a f i n i t e

escape t i m e i f t h e r e e x i s t s a symmetric m a t r i x W that

-

W 0R

>0

E

Rn x n such

and

t

f

JmsuT(W)a(w)/h

(3.3.45)

where

A

aT (W 1 t

4w 1

[aT ( W) ( F 0 ItI0F ) ( bl"

0R-l) ( F 0 ItIOF )a( W)

4oT(W)a( Q)l

(3.3.46)

and where h>O i s t h e smallest eigenvalue o f 3.3.3

-

W OR.

An Example

We apply Theorem 3.3.2

t o an i l l u s t r a t i v e example i n R

L e t P E R Z x 2 , R = 2F = 12, and

2x2

.

109


Q

]I: [

%l

=

,

=12

n 22

II = rI 12

and l e t In denote t h e n x n i d e n t i t y m a t r i x .

(3.3.47)

With t h e above

values of R and F we have P = - P

2

+ P + Q ,

P(O)=II.

(3.3.48)

The Kalman c o n d i t i o n s II

> 0,

Q 2 0 would guarantee t h e exis-

tence o f P f o r a l l t

(0,m).

However, l e t us assume t h a t

E:

these c o n d i t i o n s do n o t hold, and l e t us apply Theorem 3.3.2. We chose W =

-

I2so t h a t

+ F

I0F T

T

(J

(W)o(n)

I4

0 I =

(J

(W)a(W) = 2

=

-(II 11+R 22 )

=

(3.3.50) (3.3.51)

-

tr(n).

(3.3.53)

.

(3.3.54)

Hence A(W,F,R,Q)

=

1 + 2WQ)

Theorem 3.3.2 then i m p l i e s t h a t t h e s o l u t i o n o f (3.3.48) has a f i n i t e escape t i m e


110 (i) (ii)

for all TI, whenever tr(Q) < -4 for tr(l[) < 1

-

/1+2tr(QY

, whenever tr(Q)

2

-4.

From these and Kalmanls conditions we can deduce that (a) when Q = 0 the solution exists on [O,m) and escapes in a finite time whenever tr(TI)

whenever II < 0;

>0

(b) when TI = 0, the solution exists on [O,C~) whenever Q > 0 - 1 < 0 or and escapes in a finite time whenever tr(Q) < - 5 , i.e. tr(Q) < 0. As it is much less restrictive to stipulate that the trace of a matrix should have a definite sign than it is to require the matrix 'itself to be definite, the finite escape time conditions in (a) and (b) are less restrictive in form than the Kalman existence conditions. Note that in ( i i ) above the condition on tr(n) is dependent on tr(Q) so that we cannot state conditions on TI and Q which are independent of one another. This restriction in stating the conditions could be minimized by selecting a W other than I which would in some sense maximize the variety of Q's and Ills which would satisfy the inequality (3.3.45).

3.3.4 Models in Population Dynamics A popular model for describing the dynamic behaviour of n

interacting populations xi, i=l,...,n

ii

=

n xi( z a..x.+~.), 1J J 1 j=1

is

x.(o) 1

= x io

>o.

(3.3.55)

This set of differential equations can be put in the form of

111

NON-CONVEX QUADRATICS (3.3.2)

.. .,n

by d e f i n i n g the f o l l o w i n g parameters f o r i = l ,

A. = 1

T b. = (0 1

,...,0,szi,O ,...,0)

(3.3.57 )

( 3.3.58)

ci = 0. From (3.3.58)

i t i s c l e a r t h a t cw = 0 so t h a t Aw

t h e assumed p o s i t i v e d e f i n i t e n e s s o f Am.

>0

owing t o

The c o n d i t i o n f o r

a f i n i t e escape time becomes

xo Furthermore, as

>-

1 w

>0

T -1

I w Aw bw

t

$

T T -1 wbwAw bw/Aw'.

(3.3.59)

i s the l a r g e s t eigenvalue of A - l we

have, by t h e Cauchy-Schwarz i n e q u a l i t y , t h a t wTwbw(A;l/Aw)bw T

2 wToboAw T -1Aw-1bw 2 ( w TAw-1bw)2

. (3.3.60)

I n e q u a l i t y (3.3.59)

thus becomes wTxo

>0

(3.3.61)

which, i n c i d e n t a l l y , i s t h e s u f f i c i e n t c o n d i t i o n f o r a f i n i t e

112


escape time valid for all equations (3.3.2) for which cw = 0. 3.4

Invariant Sets

In Section 3.3 we obtained sufficient conditions for the existence of finite escape times of solutions of quadratic differential equations for which there exists w E Rn such that n

wiAi > 0. i =1

(3.4.1)

Z

This is a rather stringent condition which precludes, for example, an analysis of the system

il

=

2x1x2 X1(O) =

i,

=

2x1x2 X2(O) = 1 , 5 ~> 0

E > O

(3.4.2)

whose solution obviously has a finite escape time. Therefore instead of insisting upon the existence of w E Rn such that T n (3.4.1) holds we require only that x ( Z wiAi)x is definite i=l on a suitable invariant set of (3.3.2). For simplicity we shall assume that bi = 0, c.1 = 0, i=l,...,n so that (3.3.2) becomes

Xi

=

i=l, ...,n xT Aix, ~ ~ ( =0xi0, )

.

(3.4.3)

The set (x E RnIDx 2 0 1 where 0 E R n x n is an invariant set associated with (3.4.31, that is x(t) lies in this set for all t for which x(t) exists, if

LEMMA 3 . 4 . 7


113

>0

Dxo

(3.4.4)

and T n x ( Z D. .A.)x j = l 1J J

>0

for all x

E

Rn such t h a t Dx

> 0,

i=l,..

.,n.

(3.4.5)

PROOF

For each i we have t h a t n ( Z D..x.) j = l 1J J

T

n j = l 1J J

= x ( Z D..A.)x

.

(3.4.6)

Rn such t h a t Dx > O we have t h a t t h e components o f Dx a r e mono-

Now i f t h i s t i m e d e r i v a t i v e i s non-negative f o r a l l x t o n e non-decreasing i n t h i s s e t . f a c t t h a t Dxo

> 0,

E

This, t o g e t h e r w i t h t h e

i m p l i e s t h a t Dx

>0

f o r a l l t f o r which x

exists. I n e q u a l i t y (3.4.5)

concerns t h e n o n - n e g a t i v i t y o f a n o t

n e c e s s a r i l y convex q u a d r a t i c f u n c t i o n on a subset of Rn defined b y l i n e a r i n e q u a l i t i e s .

We t h e r e f o r e use Theorem

3.2.2 t o o b t a i n t h e n e x t lemma. LEMMA 3.4.2

The s e t { x

E

R"Dx

> 01

i n v a r i a n t s e t a s s o c i a t e d w i t h (3.4.3) Dxo

>

E

i s an

Rnx

if

(3.4.7)

0

and t h e r e e x i s t c o p o s i t i v e m a t r i c e s Ci

.. . ,n

where D

E

Rn

such t h a t for

i = l,

n

Z D..A

j = 1 1J j

-

T D CiD

>0

(positive semi-definite). (3.4.8)

114

PROOF

L I NEAR-QUADRAT I C EXTENS IONS

The p r o o f f o l l o w s from Lemna 3.4.1 and Theorem 3.2.2.

We can now s t a t e t h e main theorem.

THEOREM 3 . 4 . I

R n x n and c o p o s i t i v e i=l,...,n, and w E Rn such t h a t Ci

W

E

z D..A j = l 1J j

-

n

Z w.A

j =1 J j

nxn D E R , R n x n and c o p o s i t i v e ,

> 0,

Suppose t h e r e e x i s t kl

E

Jxo

>0

(3.4.9)

Dxo

>0

(3.4.10) i=1,...,n

D'C~D>O,

-

T

D WD

Then, t h e s o l u t i o n o f (3.4.3)

-

klu

T

>O.

(3.4.11) (3.4.12)

has a f i n i t e escape t i m e te

where (3.4.13 )

PROOF Lemma 3.4.2 s t a t e s t h a t { x E Rn\Dx > 01 i s an i n v a r i a n t We have a l s o t h a t s e t associated w i t h (3.4.3). T n (wTx) = x ( Z w.A.)x

aT so t h a t

&(wTx) for a l l x

E

Rn such t h a t Dx

j = lJ J

> kl(w >0

TX )2

i f (3.4.12)

(3.4.14 )

(3.4.15 ) holds f o r some

NON-CONVEX QUADRATICS c o p o s i t i v e W. escape t i m e a t

Now

$ = kly 2 , y(0)

klyo

e q u a l i t y t h a t (3.4.3)

T

= w xo

115

>0

has a f i n i t e so i t f o l l o w s from t h e preceding i n has a f i n i t e escape t i m e te which

s a t i s f i e s (3.4.13). 3.4.1

An Example

Consider a g a i n equations (3.4.2). A1 = A2 =

1: :]

Here we have t h a t (3.4.16)

so t h a t t h e r e i s no l i n e a r combination o f A1 and A2 which i s p o s i t i v e d e f i n i t e . Theorem 3.3.1 t h e r e f o r e cannot be used and so we t u r n t o Theorem 3.4.1. We s e t

r: -:]

kl = 1,

D =

,

wT = (1

0)

2 C2 = W = [l

C1 = 0,

0) 1

(3.4.17) (3.4.18)

so t h a t

T

w xo = Xl0

2

Z DljAj

j =1

-

>0

DTCDI

= 0

(3.4.19) (3.4.20)

2

=

[; ;]

>0

(3.4.21)

116

L I NEAR-QUADRAT IC EXTENS IONS

2

w.A. j=1 J J Z

- klwT- DT WD

'1

=

=

0

[:,

20

(3.4.22)

[:::]>o.

(3.4.23)

and =

The conditions of Theorem 3.4.1 are therefore satisfied and (3.4.2) has a finite escape time te =GF1

.

3.4.2

General Case

Both Lemna 3.4.2 and Theorem 3.4.1 can be extended to the general quadratic equation (3.3.2), but the results are more cumbersome, especially in so far as a proper generalization of Theorem 3.4.1 is concerned. However the proofs follow easily from a n application of the matrix theory of Sections 3.1 and 3.2.

LEMMA 3.4.3 The set {x E Rn1D(x-2) > 0) is a n invariant set associated with (3.3.2) if there exist C i E R n x n and copositive, ii, h i E Rn, i=l, ..., n such that n 2j=1 Z D..A.x. 1J J 1

t

n j=1D..b. 1J J Z

T h i =G 0, hiD(ii-2)

= 0,

t

DT h.1 = 0, i=l, ...,n (3.4.24)

D(ii-2)

> 0,

i=l,

...,n (3.4.25)

NON-CONVEX QUADRATICS D(x0-Z) n

C D..A j = l1J j

-

and -T xi(

n

C D..A.)xi

j =1 1J J

t

-T xi(

>0

(3.4.26)

...,

T D CiD 2 0, = 0,

CiD(xi-X)

...,n

n

t

(3.4.27)

i=l, n

i=l,

n

C D..b.) j = l13 J

117

Z D..c

j = 1 1J j

(3.4.28)

...,

i=l, n.

>O,

(3.4.29)

PROOF

L e t Di denote t h e i - t h row of D.

d =[Di(x-Z)]

T n = x ( C D..A.)x j =1 1J J

t

Then

T n x ( Z D..b.) j = l1J J

n

t

Z Dijcj.

j=1

(3.4.30) Well -known necessary c o n d i t i o n s f o r t h e right-hand side t o be minimized w i t h respect t o x subject t o D(x-2) Z 0 a r e t h a t the minimizer

xi

s a t i s f y (3.4.24),

t h a t i n e q u a l i t i e s (3.4.27)

(3.4.25).

and (3.4.28)

We prove now

a r e then s u f f i c i e n t

conditions f o r iit o be t h e global minimizer o f t h e r i g h t hand side o f (3.4.30). We d e f i n e $I~(x) as t h e right-hand s i d e o f (3.4.29). $i(iit6x)

= $i(ii)

t

t

n [2 Z D..A.;. i=1 1J J

T n bx ( Z D..A.)Gx j = l1J J

1

t

Then

n T Z: D..b.] 6~ j = 1 1J J (3.4.31)

where 6x i s an a r b i t r a r y p e r t u r b a t i o n which must s a t i s f y


118

(3.4.32)

D(Xi+6x-%) 2 0.

Now the second term on the right-hand side of (3.4.31) is non-negative by virtue of (3.4.24) and (3.4.25). Therefore a sufficient condition for X i to be a global minimizer is that the third term on the right-hand side of (3.4.31) should be non-negative for all 6x which satisfy (3.4.32). That conditions (3.4.27) and (3.4.28) guarantee this, is proved in Theorem 3.2.3. Inequality (3.4.29) allows us to conclude that

X [D i (x-jz)] > 0,

(3.4.33)

i=1,. ..,n

and this together with (3.4.26) yields the lemma. Suppose that {x E Rn1D(x-X) > 0) is a n THEOREM 3.4.2 invariant set associated with (3.3.2), and that there exist n kl > 0, k2, k33 W E R n x and copositive, v, i , w E R such that n n T 2 [ Z w A - k w wT] E t Z w . b - k 2 W + D v = 0 j=l j j 1 j=l J j (3.4.34) (3.4.35) v < 0, v TD(i-3) = 0, D(i-Z) > 0

n

Z

w.A

j=l ~j

- kiwT

-

DTWD 2 0

(3.4.36)

WD(i-iC) = 0 n x-T ( Z w.A.-k wT)i+ j=l J J

1

(3.4.37)

n xT( jZ=1n w.b.) + j=1 Z J j J J

W.C

-T - k3 2 0 .

k2X w

(3.4.38)

119

NON-CONV EX QUADRAT I CS Then, the solution of (3.3.2) has a finite escape time according to conditions ( i ) and ( i i ) of Lemma 3.3.1, and T Lemma 3.3.2, where yo is set as w xo. It is easily verified that conditions (3.4.34) PROOF (3.4.38) imply that d

T

Z(U X) > kl(w for all x E {x E RnlD(x-X) Lemnas 3.3.1 and 3.3.2. 3.5

T

X)

2

+ k 2 (T~X ) + k3

(3.4.39)

> 01, which allows the use of

Conclusion

In this chapter we presented a number of matrix-theoretic results relating to non-convex quadratic forms. I n particular, copositive matrices and their role in quadratic programming were emphasized. A n extension of Finsler's theorem was also presented. Quadratic differential equations were studied with a view to obtaining conditions for the existence of finite escape times in their solutions. On the assumption that a linear combination of certain matrices is positive definite, a set of conditions was obtained which was applied to the matrix Riccati and certain population equations. The notion of invariant sets was introduced and exploited to relax the restrictive positive definiteness requirement on the linear combination of the system matrices. Here too copositive matrices played an important role.

In Chapter 4 we turn to the development of conditions for the


120

non-negativity of non-quadratic functional s , thus generalizing certain of the results of Chapter 4 of P31. References DIANANDA, P.H. On Non-negative Forms in Real Variables Some or All of which are Non-negative. Proc. Cambridge Philos. SOC., 58, 1962, pp. 17-25. GADDUM, J.W. Linear Inequalities and Quadratic Forms. Pacific J. Math., 8, 1958, pp. 411-414. HALL, M. & NEWMAN, M. Copositive and Completely Positive Quadratic Forms. Proc. Cambridge Philos. SOC., 59, 1963, pp. 329-339. BAUMERT, L.D. Extreme Copositive Quadratic Forms. Pacific-J. Math., 19, 1966, pp. 197-204. BAUMERT, L.D. Extreme Copositive Quadratic Forms 11. Pacific J. Math., 20, 1967, pp. 1-20. MARKHAM, T.L. Factorizations of Completely Posit ve Matrices. Proc. Cambridge Philos. SOC., 69, 1971 pp. 53-58: COTTLE, R.W. & MYLANDER, W.C. Ritter's Cutting P ane Method-for Non-convex Quadratic Programming. In: Integer and Non-linear Programming, edited by J. Abadie, 1970, pp. 257-283. COTTLE, R.W., HABmLER, G.J. 81 LEMKE, C.E. Quadratic Forms Semi-definite over Convex Cones. Proceedings of the Princeton Symposium on Mathematical Programming, edited by H.W. Kuhn, Princeton University Press, 1970, pp. 551-565. COTTLE, R.W. On the Convexity of Quadratic Forms over Convex Sets. Operations Research, 15, 1967, pp. 170172.

COTTLE, R.W. Note on a Fundamental Theorem in'Quadratic Programming. SIAM J. Applied Mathematics, 12, 1964, pp. 663-665.


121

1111

MARTOS, B. Quadratic Programing with a Quasiconvex Objective Function. Operations Research, 19, 1971, pp. 87-97.

1121

COTTLE, R.W. & FERLAND, J.A. On Pseudo-convex Functions of Non-negative Variables. Math. Programing, 1, 1971, pp. 95-101.

[131

KARAMARDIAN, S. The Complementary Problem. Programming, 2, 1972, pp. 107-129.

1141

JACOBSON, D H Factorization of Symmetric M-Matri ces Linear Algebra and its Applications, 9, 1974, pp. 275278.

11 51

FINSLER, P. Uber das Vorkommen definiter und semidefiniter Formen in Scharen quadratischer Formen. Comnentarii Mathematici Helvetici, 9, 1937, pp. 188-192.

Math.

..

.

[161 JACOBSON, D.H. A Generalization of Finsler's Theorem for Quadratic Inequalities and Equalities. Quaestiones Mathematicae, 1, 1976, pp. 19-28. [171

BELLMAN, R. Introduction to Matrix Analysis. Hill, New York, 1970.

McGraw-

[181 LA SALLE, J.P. & LEFSCHETZ, S. Stability by Liapunov's Direct Method with Applications. Academic Press, New York, 1961, Chapter 4. FRAYMAN, M. Quadratic Differential Systems: A Study in Non-linear Systems Theory. Ph.D. thesis, Department of Electrical Engineering, University of Maryland, 1974. JACOBSON, D.H. Conditions for Existence of Finite Escape Times and Divergence of Solutions of Quadratic Differential Equations. Internal Report, Department o f Applied Mathematics, University of the Witwatersrand, Johannesburg, Republic of South Africa, 1972, 20 pages. GETZ, W.M. & JACOBSON, D.H. Sufficiency Conditions for Finite Escape Times in Systems of Quadratic Differential Equations. J. Inst. Math. Applic., to appear.

122


[221 JACOBSON, D.H. New Conditions for Boundedness of the Solution of a Matrix Riccati Differential Equation. J. Differential Equations, 8, 1970, pp. 258-263. [231 BELL, D.J. & JACOBSON, D.H. Singular Optimal Control Problems. Academic Press, New York and London, 1975.

4.

NON-NEGATIVITY CONDITIONS FOR CONSTRAINED AND NONQUADRATIC FUNCT IONALS

4.1

Linear-quadratic Case

Necessary and s u f f i c i e n t c o n d i t i o n s f o r t h e non-negativity o f q u a d r a t i c f u n c t i o n a l s have been derived d u r i n g t h e past several years [ll-[41 and have found a p p l i c a t i o n i n s i n g u l a r optimal c o n t r o l and e s t i m a t i o n t h e o r y [51.

Chapter 4 of [ll

and [21-[41 t o g e t h e r provide a complete treatment o f t h e problem o f d e c i d i n g whether t h e q u a d r a t i c performance criterion T T T T T J [ u ( . ) ] = J ( ~ xQX+U CX+$U R u ) d t + h x (T)QTx(T) (4.1.1) 0

i s non-negative f o r a l l piecewise continuous f u n c t i o n s u(

0

)

where

R

= AX

+

BU

,

~ ( 0 =) x0 = 0

(4.1.2)

Dx(T) = 0

(4.1.3)

and where t h e m a t r i c e s Q E R n x ', C E Rm n, R E R m x m , A E R n x n and B 8 R a r e assumed t o be continuous f u n c t i o n s o f t i m e on [O,Tl.

R are constant. With no l o s s o f g e n e r a l i t y t h e matrices Q, R and QT a r e assumed t o be symmetric and D i s assumed t o have rank s. The m a t r i c e s QT

E

R n x n and D

E

I n t h e f o l l o w i n g pages we present c e r t a i n r e s u l t s on t h e above-mentioned l i n e a r - q u a d r a t i c problem and i n t e r p r e t these i n new ways which enable us t o o b t a i n s u f f i c i e n t c o n d i t i o n s f o r n o n - n e g a t i v i t y o f f u n c t i o n a l s which a r e non-1 i n e a r quadratic.

123

124 4.1.1


Non-negativi t y Conditions

A c l a s s i c a l necessary c o n d i t i o n f o r non-negativity o f J [u( .)I i s provided by t h e f i r s t theorem. A necessary c o n d i t i o n f o r J [u( . ) I t o be nonTHEOREM 4 . 7 . I negative f o r a1 1 piecewise continuous c o n t r o l f u n c t i o n s u( .) which cause (4.1.3) t o be s a t i s f i e d i s t h a t R(t) 2 0 f o r a l l t

E

(4.1.4)

[O,Tl.

I n the Calculus o f V a r i a t i o n s c o n d i t i o n (4.1.4)

i s referred

t o as t h e Legendre-Clebsch necessary c o n d i t i o n . I n f a c t , an even stronger statement i s possible, as i s confirmed by t h e next theorem.

THEOREM 4 . 7 . 2 A necessary c o n d i t i o n f o r J [ u ( . ) l > kllu(.)Il k > 0 ( i . e f o r J [ u ( . ) I t o be s t r o n g l y p o s i t i v e ) where Ilu(.)Il 2 h - /u t T (t)u(t)dt i s that

2

,

0

R(t) Note t h a t i f (4.1.5)

>0

for a l l t

E

(4.1.5)

[O,Tl.

holds, t h e f u n c t i o n a l (4.1.1)

i s referred

t o as non-singular, w h i l e i f (4.1.4) holds, i t i s r e f e r r e d t o as p a r t i a l l y s i n g u l a r . I n t h e non-singular case we have t h e f o l l o w i n g well-known r e s u l t which i s proved i n

ill.

THEOREM 4 . 1 . 3 A necessary and s u f f i c i e n t c o n d i t i o n f o r J [ u ( . ) l t o be s t r o n g l y p o s i t i v e when D 5 0 i s t h a t (4.1.5) holds and t h a t there e x i s t s f o r a l l t E [O,Tl a symmetric m a t r i x f u n c t i o n o f t i m e S(.) which s a t i s f i e s t h e m a t r i x Riccat i d i f f e r e n t i a l equation

NON-NEGATIVE FUNCTIONALS

-s

=

Q + SA + ATS

-

125

(C+BTS)TR- 1 (C+BTS)

(4.1.6)

In the singular case (4.1.6) fails, owing to the noninvertibility of R. However, the following sufficient condition is valid. TffEOREM 4 . 1 . 4

161 A sufficient condition for J[u(-)I to be

non-negative for all piecewise continuous functions u( .) is that there exists for all t E [O,T] a continuously differentiable, symmetric matrix function of time P(.) such that

'i + Q +

PA

t

ATP

,C + BTP for all t

E

(C+BTPITj

2 0

(4.1.8)

R

[O,Tl and

ZT [QT-P(T)lZ > 0

(4.1.9)

where

Z=

-0;' D* T

1

n-s

D = [Dl

D21 , D1

E

Rsxs, D2

E

Rs x (n-s) (4.1 .lo)

Note that the assumed rank condition on D guarantees that it is always possible to re-label variables so that D1 is invertible. In the totally singular case (R = 0) we have that condition (4.1.8) becomes

126


i) +

Q

+ PA + ATP > O

(4.1.11)

T

(4.1.12)

C + B P = O for all t

E

[O,Tl.

Though the above conditions (4.1.8)-(4.1.12) a r e not i n general necessary, they d i f f e r from necessary and s u f f i c i e n t conditions only i n small d e t a i l [l] , [ 7 ] . As t h e r e s u l t s t o be presented i n this chapter a r e mainly of a sufficiency nature, we prefer t o s t a t e Theorem 4.1.4 r a t h e r than the more a b s t r a c t necessary and s u f f i c i e n t conditions.

The following theorem r e l a t e s (4.1.11), (4.1.12) t o a matrix Riccat i equation. THEOREM 4 . 7 . 5 [ll Suppose t h a t R :0 f o r a l l t t h a t (general ized Legendre-C1 ebsch condition)

(-1)

& Hu > 0

- aH au

for a l l t

E

[O,Tl and

(4.1.13) (4.1.14)

[O,Tl where H

and

- = O u

E

4

jxTQx

-

A

+ u TCx +

= QX

T A (Ax+Bu)

+ AT A + CT U .

( 4 . l . 15) (4.1.16)

Suppose f u r t h e r t h a t t h e r e e x i s t s a function S(-) which satisfies f o r a l l t E [O,Tl the matrix Riccati equation

127

NON- N EGAT IVE FUNCT IONALS

..

+ [(AB-B) ' T StBT Q-CA-Cl' T [% a Hul-l1(AB-B)' T S+BT Q-CA-Cl C(T) + B~(T)s(T)=

(4.1.17) (4.1.18)

o

Then there exists a P(-) which satisfies (4.1.11), (4.1.12) and (4.1.9). Note that it is easy to prove that the conditions of Theorem 4.1.3 imply satisfaction of those of Theorem 4.1.4. The results summarized above indicate the close relationship between the existence of a matrix function of time which satisfies certain matrix differential and algebraic inequal ities and equalities, and the non-negativity of J[u( ) I . It is this relationship which we exploit later in this chapter to develop sufficiency conditions for the non-negat vity of non-1 inear-quadratic functional s. Our next observation concerning the preceding theorems is also of some importance. Let us define

51

t

Mil'

Q

t

PA

t

lC + BTP

A21

ATP

(C+BTP)T) R

I

(4.1.20)

R

A = zT [QT-P(T)lZ.

We can then state and prove the following theorem.

(4.1.21)


128

TifEOREM 4 . 1 . 6 Suppose t h a t t h e c o n d i t i o n s o f Theorem 4.1.4 are s a t i s f i e d . Then t h e r e e x i s t a continuous, p o s i t i v e semid e f i n i t e symmetric m a t r i x f u n c t i o n o f t i m e M(.) and a p o s i t i v e semi-definite symmetric m a t r i x N such t h a t

(4.1.22) for a l l

u(.) such t h a t (4.1.3) holds, where (4.1.23) T T Add t h e i d e n t i c a l l y zero i n t e g r a l J$x P(AxtBu-R)dt

PROOF

0

t o J [ u ( ' ) I and i n t e g r a t e by p a r t s t o o b t a i n J [u( * ) I =

T

I [$xT(P+Q+PAtA T P)xtuT( C+BTP)x+$uTRuld t 0

Now because u ( . ) i s chosen so t h a t (4.1.3) that

holds, we have

where y ( T ) i s defined by (4.1.23).

As a consequence of t h e f a c t t h a t xo = 0, and using (4.1.20) (4.1.21)

J [u(

*)I

JIu(.)l

becomes = J$[x 0

which i s (4.1.22)

T T T T T u Iq[x u I dt

+

holding w i t h e q u a l i t y .

&yT(T)Ny(T] (4.1.26


129

i s s t r o n g l y p o s i t i v e i f and

Note t h a t t h e f a c t t h a t J [ u ( . ) I

o n l y i f t h e c o n d i t i o n s o f Theorem 4.1.3 hold, implies t h a t

(4.1.22) holds w i t h Ml = 0, M21 = 0, M22 = k I f o r some k > 0, N = 0. Consequently t h e s u f f i c i e n t c o n d i t i o n presented i n t h e next theorem i s imp1 i e d by the known s u f f i c i e n t conditions f o r t h e non-singular, p a r t i a l l y s i n g u l a r and s i n g u l a r l i n e a r quadratic cases.

THEOREM 4. I .7 A s u f f i c i e n t c o n d i t i o n f o r J [u( - ) I 2 0 f o r a l l u ( * ) f o r which (4.1.3) holds i s t h a t there e x i s t a c o n t i nuous, p o s i t i v e s e m i - d e f i n i t e symmetric m a t r i x f u n c t i o n of time M(.) and a p o s i t i v e semi-definite symmetric m a t r i x N such t h a t

(4.1.27) f o r a l l u ( * ) f o r which (4.1.3) 4.1.2

holds.

Novel Non-negativity Conditions

F i r s t we note t h a t t i o n s hold.

R >0

i f and o n l y i f t h e f o l l o w i n g condi-

Either

A,

20

(4.1.28)

A22 > 0

(4.1.29)

-+

(4.1.31)

-T

- 821MllM21


130

where It1 denotes pseudo-inverse,

or

A,

2 0

(4.1.32)

,w,

2 0

(4.1.33)

m;,)

N(A22)

- R;RA,;

A,

z

(4.1.34) 0.

(4.1.35)

If we use t h e second s e t o f c o n d i t i o n s when R > 0 we s e t

,A,

=

>0

(4.1.36)

(CtBT P)

(4.1.37)

(C+BT P ) T R- 1 (C+B T P)

(4.1.38)

,A, fil

= R =

which c l e a r l y s a t i s f y (4.1.34),

and r e s u l t i n t h - well-known

R i c c a t i equation

-

P =

Q + PA + A T P

-

(CtBT P)TR- 1 (CtBT P )

ZT [QT-P(T)l Z 2 0 .

(4.1.39) (4.1.40)

In t h e p a r t i a l l y - s i n g u l a r case we could hope t o replace R - l by Rt b u t we have no cause t o suspect t h a t (4.1.34) w i l l be In f a c t , i n general i t w i l l n o t be s a t i s f i e d ; i n satisfied. p a r t i c u l a r , i n t h e t o t a l l y s i n g u l a r case t h e c o r r e c t form f o r

R,

i s g i v e n i n (4.1.17).


131

However we can state and prove the following theorems. J [ u ( . ) I is non-negative for all u(-) for which (4.1.3) holds if there exist for all t E [O,T] a continuously differentiable symmetric matrix function of time P(.) and a positive definite symnetric matrix function of time W(.) such that

THEOREM 4 . 7 . 8

-

P

t PA

Q

=

+ ATP

-

MIl

(4.1.41)

ZT [QT-P(T)l Z > 0 R

-

where

(C+BTP)Mil( CtBTP)T > 0 for.all t

All

(4.1.42) E:

(CtBTP)TW- 1(CtBTP).

=

[O,Tl (4.1.43) (4.1.44)

Here All i s given by (4.1.44) and I,, = C t BTP, fi22 = R. With this choice, conditions (4.1.28)-(4.1.31) are satisfied

PROOF

.

THEOREM 4 . 7 . 9

sat i sfi ed PROOF

.

If R

>W

inequality (4.1.43) is always

First we note that

fill

=

fill

can be written as

[i(CtBTPl]' [W(C+BTP)l

where the symmetric matrix

w

i s defined by

e w-4. The pseudo-inverse

(4.1.45)

Qil is then

(4.1.46)

132


[i(C+BTP)l '[8(CtBTP)] Tt

(4.1.47)

so that the left-hand side of (4.1.43) is R

-

( C+BTP)

[i(CtBTP)l '[8(C t BT P)] Tt ( CtBTP)T.

Pre- and post-multiplying (4.1.48) by root of W - l , yields GRi

(4.1.48)

the symnetric square

- [i( C+BTP)l [i(CtBTP)] '[w( CtBTP)] Tt [8(C t BT p)] T (4.1.49)

which is iRV

Setting R = W

t

- [i(CtBTP)] [8(CtBTP)] '.

4, where It

k$i -

r$

(4.1.50)

is positive semi-definite yields

[ i ( C t B T P ) l [6(C+BTP)l+

(4.1.51)

which, by the properties of pseudo-inverses and the symnetry of is positive semi-definite. Pre-multiplying and postmu1 tiplying (4.1.51) by i-' preserves this positive semidefiniteness, and (4.1.43) fol lows. Theorem 4.1.8 can be generalized via (4.1.28)-(4.1.31) read as follows.

to

THEOREM 4 . 1 . 1 0 J[u(-)] is non-negative for all u(*) for which (4.1.3) holds if there exist for all t E: [O,Tl a continuously differentiable symmetric matrix function of time P( and a positive semi-definite symmetric matrix function of time A,, such that 0

)


-

P = Q

t

PA

133

+ ATP - ill

(4.1.52) (4.1.53)

N ( C + B ~ P2 ) N ( A ~ ~ )

R 4.1.3

-

(CtBTP)R;l(C+BTP)T

>0

(4.1.54)

.

(4.1.55)

Example

Suppose that A, B, C, Q, R are constant matrices and that there exist constant symmetric matrices P, fil such that Q + P A + A PT= R l l > O .

(4.1.56)

Such a P will exist for arbitrary Q, ill if and only if

Xi(A) + Aj(A) f O , i , j = l ,

...,n

(4.1.57)

where Xi(A) denotes an eigenvalue of A. Condition (4.1.54) is automatically satisfied owing to the assumed positivedefiniteness of Rll . 0 for all u( - ) such that Sufficient conditions for J [u( . ) I (4.1.3) holds are therefore that there exists a constant symmetric matrix P such that

ZT [QT-Pl Z > 0 Q

t

PA t ATP

>0

(4.1.58) (4.1.59)

134

L I NEAR- QUADRATI C EXT ENS IONS R

-

( c + B ~ P ) ( Q + P A + A ~ P ) - ~ ( c + B ~> P )0. ~

4.2

Constrained Case

4.2.1

Formulation

(4.1.60)

We suppose here t h e same f o r m u l a t i o n as i n Section 4.1 b u t T T T t h a t t h e (n+m)-vector ( x u ) i s constrained t o belong t o Rn+m. S p e c i f i c a l l y , we wish t o i n v e s t i g a t e t h e nonnxu negativity o f

T T T T J [u(.)] = / ( h x QX+U C X + ~ U Ru)dt 0

t

T f x (T)QTx(T)

(4.2.1)

subj ec t t o

A = AX

+

BU

,

~ ( 0 =) 0

Dx(T) = 0

(4.2.2) (4.2.3)

and

(4.2.4) where t h e c o n t r o l f u n c t i o n u ( - ) i s admissible if i t i s piecewise continuous on [O,Tl

.

N a t u r a l l y we assume t h a t t h e r e

e x i s t s an admissible c o n t r o l f u n c t i o n such t h a t (4.2.3) and

(4.2.4) a r e s a t i s f i e d . 4.2.2

Sufficiency Conditions

Our f i r s t s u f f i c i e n t c o n d i t i o n i s an immediate g e n e r a l i z a t i o n o f Theorem 4.1.4.

NON-N EGAT IVE FUNCT I0NALS

135

A sufficient condition for J [u( *)I to be non-

TffEOREM 4 . 2 . 7

negative for all piecewise continuous control functions u( for which (4.2.3) and (4.2.4) hold is that there exists for all t E [O,Tl a continuously differentiable, symmetric matrix function of time P(.) such that a )

i) + Q C

t

t

PA

t

ATP

BTP

(C+BTP)T] R

[ 1]

2 0

(4.2.5) E

Qxu, for all t

E

[O,Tl, and

ZT [QT-P(T)l Z > 0.

(4.2.6)

PROOF

The proof is very similar to that of Theorem 4.1.6 and we refrain from presenting it here. Although Theorem 4.2.1 is not easy to use, in that a constructive method for generating P(.) is not suggested, it is important because the constraint set Qxu is arbitrary. Before going on to consider some special constraint sets which do permit the development of constructive theorems, we state an appropriate generalization o Theorem 4.1.7.

THEOREM 4 . 2 . 2 A sufficient condition for J [u(.)l 2 0 for all u(.) for which (4.2.3), 4.2.4) hold, is that J[u(.)l

T . -u'lM[x >jf[x' T uT1 Tdt

t

fyT(T)Ny(T) (4.2.7)

0

where N>O

(4.2.8)


136

:]

and the symmetric matrix function of time M(.) satisfies

[CT nTI M[

> 0 for all

[ ]

E

ax,, , for all

t

[O,Tl

E

.

(4.2.9) The first special constraint set which we consider is nXU

E

QXxRm , where ax

Rn

(4.2.10)

or x(t)

E

ax , u(t)

E

Rm , for all t

E

[O,Tl. (4.2.11)

THEOREM 4 . 2 . 3

J[u(.)l 2 0 for all t i ( - ) for which (4.2.3), (4.2.11) hold, is that there exists for all t E [O,Tl a continuously differentiable symmetric matrix function of time P(.) such that A sufficient condition for

R>O

gT[b+Q+PA+ATP-(C+B TP) TR+(C+BTP)lE > O for all 5

ZT [QT-P(T)l Z > 0. PROOF

(4.2.12)

E

ax (4.2.14) (4.2.15)

As r) is unconstrained we can minimize the left-hand side of (4.2.5) with respect to r), provided that (4.2.12) and (4.2.13) are satisfied. This yields (4.2.14) and the theorem i s proved.

NON- NEGAT I V E FUN CT I ONALS

137

The corresponding case when x(t)

E

Rn, u ( t )

flusRm f o r a l l t

E

[O,Tl

8

(4.2.16)

i s covered by t h e f o l l o w i n g theorem which may be proved i n a s i m i l a r way t o Theorem 4.2.3. A sufficient condition f o r J[u(*)l 2 0 for a l l u ( - ) f o r which (4.2.3), (4.2.16) hold, i s t h a t t h e r e

THEOREM 4 . 2 . 4

exists f o r a l l t

E

[O,T] a c o n t i n u o u s l y d i f f e r e n t i a b l e

symmetric m a t r i x f u n c t i o n o f t i m e P ( . ) such t h a t

w, ( C t B T P) Trl T

rl [R-(CtB

T

-t

P)Mll(CtB

E

>0

@(w,,) T

(4.2.17)

f o r a l l rl

E

T P) 10 > O f o r a l l rl

ZT [QT-P(T)]Z

(4.2.18)

flu E

nu

>0

(4.2.19) (4.2.20)

where

8,, 4 i)

t

Q

t

PA

t

AT P.

(4.2.21)

Perhaps t h e most u s e f u l special case i s t h a t where

(4.2.22)

(4.2.23)

138


In this case we succeed in constructing P( . ) . A sufficient condition for J [u( a)] > 0 for THEOREM 4.2.5 all u(-) for which (4.2.3), (4.2.23) hold, is that there exist for all t E [O,Tl a continuously differentiable symmetric matrix function of time P(.) and a continuous symmetric matrix function of time G( . ) such that

i) + Q + PA + ATP

(C+BTPIT]

C + BTP

R

where G(t) is copositive, t

-

F~G> F o

(4.2.24)

[O,Tl.

E

COROLLARY 4.2.1 Suppose we assume that F is block diagonal and we partition the matrix F and the copositive matrix G as F =

0

F11

, G =

(4.2.26)

F22

0

where Fll , Gll E R n x n and FZ2, GZ2 for all t E [O,Tl

R

-

F

~

E

Rm x m , and suppose that

~ > 0. G

~

~

F

~(4.2.27) ~

Then the conditions of Theorem 4.2.5 are satisfied if there exists for all t E [O,T] a solution to the matrix Riccati d i ffe r e n t i.: 1 equ a t i on

NON-NEGATIVE FUNCT IONALS

- S

= Q

-

-

T FllGllFll t SA

t

139

ATS

( CtBTP-F~2G21F11)T(R-Fi2G22F22)-1( CtBTP-F~2G21Fll)

ZT [QT-P(T)l Z

2 0.

(4.2.28) (4.2.29)

PROOF

From (4.2.24) we see that the copositivity of G implies that

+Q

t

PA + ATP R

for all

[ :] E

2

[tT qTl F TG F (

20

(4.2.30)

.QXu defined by (4.2.23), and the conditions

of Theorem 4.2.1 are satisfied. The proof of Corollary 4.2.1 is straightforward and follows that of Theorem 4.3 of [ll. Theorem 4.2.5 and Corollary 4.2.1 are important in that they answer the question as to whether or not J[u(-)l is nonnegative subject to linear constraints on x and u, at least as far as sufficiency is concerned. Conditions for this case which are both necessary and sufficient are not yet available and it seems that their discovery will be a non-trivial task. In particular, necessity usually requires an assumption on the controllability of the dynamic system [ll and this is a complex concept when constraints are present - see Chapter 5.


140 Theorem 4.2.6

i s a d i r e c t consequence o f C o r o l l a r y 4.2.1 and

Theorem 4.1.3.

THEOREM 4.2.6

A (necessary and) s u f f i c i e n t c o n d i t i o n f o r T T T I d t t o be s t r o n g l y p o s i t i v e i s t h a t u I F GF[ J[u(.)l 0 t h e r e e x i s t s a symmetric m a t r i x f u n c t i o n o f time P(.) such

- T[x

t h a t (4.2.28), 4.2.3

(4.2.29)

are s a t i s f i e d .

Induced State Constraints

A c o n t r o l c o n s t r a i n t s e t Ru may induce a s t a t e c o n s t r a i n t s e t RX even though t h e r e i s no e x p l i c i t c o n s t r a i n t on t h e s t a t e .

For example, suppose we have t h e system

i = AX

> 0,

where Aij

i f j and Bij

>O).

Ru = {u E R y u

x(t)

>0

Rx = { X

t BU

,

>0

~ ( 0 )= 0

(4.2.31)

f o r a l l i,j and where

I n t h i s case i t i s w e l l known t h a t

f o r a l l t E [O,TI and so t h e s t a t e c o n s t r a i n t E Rn[x > 0) s induced by Ru and t h e dynamic system.

A g e n e r a l i z a t i o n o f t h i s r e s u l t i s provided by t h e f o l l o w i n g

theorem.

THEOREM 4.2.7

Consider t h e l i n e a r dynamic system = AX

where u ( t ) E

+ BU ,

~ ( 0 )= 0

(4.2.32)

R~ = { u E R m I F 2 2 ~ > 0).

Suppose t h a t t h e r e e x i s t s a constant n x n m a t r i x F1I that (FllAFi:)ij

>0 ,

if j

,

f o r a l l t e [O,Tl

such

(4.2.33)

NON-NEGAT IVE FUNCTIONALS and

FllBu > 0 for all u

E

nu,

Then the state x(t) E fix for all t constraint set ax is given by

ax PROOF

= {X

E

4.3

E

E

[O,Tl. (4.2.34)

[O,Tl where the induced

Rn[Fllx> 0).

(4.2.35)

Defining y = Fllx, we see that j , = F ~ ~ A F ; : t~ F

and y

for all t

141

~

, ~y(o)B

= ~o

(4.2.36)

0 if (4.2.33) and (4.2.34) hold.

Non-1 i near Systems

The results of Section 4.2 followed rather easily from those of Section 4.1 for the unconstrained 1 inear-quadratic formulation. The question that arises now is whether or not similar extensions are possible to (constrained) non-1 inear systems. Some results which hint that this may indeed be feasible are contained in [8], [91. In particular in 181 the controllability of the non-linear system =

A(x,t)x

t

B(x,t)u

(4.3.1)

is related to that of the linear time-varying system = A(y,t)x

+ B(y,t)u

(4.3.2)

where Y(.] belongs to a certain set of continuous vector


142

functions. Loosely speaking, the sufficiency result is obtained that (4.3.1) is 'controllable' if (4.3.2) is 'controllable' for all continuous y( .). It turns out that we can indeed make statements about the non-negativi ty of non-quadratic functionals subject to the dynam i cal constraint (4.3.1 ) by considering a re1 a ted (constrained) 1 inear-quadratic problem. 4.3.1

Formulation

We consider the question of whether or not the functional T

J [u(-11

=

J' [JxTQ(X,U,t)X

+ u TC(x,u,t)x+$u T R(x,u,~)u]dt

0

(4.3.3) is non-negative subject to the constraints

D(x(T))x = 0

(4.3.5)

and where u(.] is piecewise continuous on [O,T]. The matrices A, B, Q, C, R are assumed to be continuous functions of their arguments. Furthermore Q, R and QT are assumed to be symmetric for all values of their arguments. The dynamic system' (4.3.4) is assumed to have a solution defined on [O,Tl for each piecewise continuous input u( -1.

NON-N EGAT IVE FUNCTIONALS 4.3.2

143

Non-negativity Conditions

Gle first relate the question of the non-negativity of (4.3.3) subject to (4.3.4)-(4.3.6) to that of the non-negativity of a set of linear-quadratic problems. THEOREM 4.3.1

A sufficient condition for J[u(-)l

for all piecewise continuous u(-) for which (4.3.5) and (4.3.6) hold, is that j [u( . ) ; y ( .),z( 2 0 for all piecewise continuous u(.) and for all y(-),z(.) E YZ where 20

a)]

and =

A(y,z,t)x

[ :] YZ = {y(.),z(.)ly(-)

+ B(y,z,t)u

nxu for all t

; x(0) = 0

E

[O,Tl

[ :] nxuL

(4.3.8)

(4.3.10)

is absolutely continuous, z ( - ) is piece-

wise continuous on [O,q , and

€

(4.3.11)

Clearly all solutions of (4.3.4) together with their corresponding inputs belong to YZ. It follows, then, that we can set y ( - ) = x(*) and z ( - ) = u(-) in (4.3.7)-(4.3.9) so 20that for this particular choice J[u(-)I = J[U(-);X(.),U(.)I PROOF

Theorem 4.3.1 may be stated differently, as follows.

L IN EAR-QUADRATI C EXTENSIONS

144

THEOREM 4.3.2 A sufficient condition for J[u(-)] > 0 for all piecewise continuous u(-) for which (4.3.5) and (4.3.6) hold, is that (4.3.12)

subject to (4.3.8)-(4.3.11). Note that for fixed y(-), z(-) the formu ation spec fied by (4.3.7)-(4.3.10) is identical to that of (4.2.1)-(4 2.4) so that all the sufficiency conditions of Section 4.2 which hold also if Q, C, R, A, B are piecewise continuous, are applicable. The complication that arises is that the conditions must be tested for all the Q, C, R, QT, A, B, D realized by (4.3.11). Note further that the class of y(-), z(-) which must be tested where can be reduced in size by replacing YZ by the set

r7 = {y( D(Y

(4.3.13)

which however requires the repeated integration of the nonlinear dynamic equation (4.3.4) and the satisfaction of (4.3.5).

4.3.3 A n Application We here demonstrate an application of Theorem 4.3.2 which was suggested by a note of Speyer and Shaked [lo]. However

NON-NEGAT I V E FUNCT IONALS

145

t h e i r t h e o r y appears t o c o n t a i n some q u e s t i o n a b l e steps. We c o n s i d e r t h e n o n - n e g a t i v i t y o f

T

J [u( . ) I = It(xTQx+uTRu)dt

+

&xT(T)QTx(T)

(4.3.14)

0

where

X B(x)

E

R

nxm

= AX

+ B(x)u , ~ ( 0 =) 0

(4.3.15)

i s g i v e n by

X

\

kl

0

Xk2 X

B(x) =

, ki

k3

(1

E

,...,n) , i=l, ...,m

0

"km ............... 0

(4.3.16)

I

and where

'i min

<Xi(t)

<Xi

max

.

(4.3.17)

The m a t r i c e s Q, R and A a r e assumed t o be f u n c t i o n s o n l y o f t i m e and R = I> 0 f o r a l l t

Theorem 4.3.2

Inf y ( * ) EYZ

states t h a t J[u(.)] Inf U(')

E

[O,Tl.

(4.3.18)

i s non-negative i f

T

14(xTQx+uTu)dt t 4xT(T)QTx(T) 2 0 0

(4.3.19)

L I NEAR-QUADRAT IC EXTENSIONS

146

subject to

X

AX

=

B(Y)u

t

, X(0)

=

(4.3.20)

0

where x(t) satisfies (4.3.17). If we drop the last-mentioned requirement, which then imp1 ies a more restrictive version of the theorem, we have that J[u(.)I 2 0 for all piecewise continuous controls u(-) which cause (4.3.17) to hold if

-

Inf Y(.)€YZ

I

Inf Il(xTQx+uTu)dt + &xT(T)Q,x(T) U(')

0

>0 (4.3.21)

subject to =

AX

t

B(y)u

3

X(0) = 0

(4.3.22)

where x and u are unrestricted, YZ = (y( . ) ]y(

a )

and xi

is absolutely continuous,

< yi(t) < xi

.

(4.3.23)

We then have the following theorem. THEOREM 4.3.3

A sufficient condition for (4.3.21) to hold,

is that there exists for all t E [O,Tl a continuously differentiable symmetric matrix function of time S( .) which satisfies

-

S = Q t SA

+ AT S

-

T SB(i()B ( i ( ) S , S(T) = QT ( 4.3.24)

NON- NEG AT I V E FUNCT I 0NA LS where 'ki

PROOF

-

max(lxkiminl 'lXkimax

147

I ) , i=l,...,m.

(4.3.25)

By dynamic programming we have t h a t t h e optimal value

function V(x,t)

= min

Y(T> T E

[t,Tl

T

m in

J'$ ( x TQx+uTu)dT

U(.)

t

T E [t,Tl

+ $xT(T)QTx(T) (4.3.26)

satisfies

Xmin

m in Q Y(t)

av - at

(x,t)

'

m i n [$ ( x ' Qx+u u)+Vx ( x t )(Ax+B(y)u)l u(t) T

Q Xmax

= T

(4.3.27) with V(x(T) ,T) = 4xT(T)QTx(T).

(4.3.28)

Upon m i n i m i z i n g (4.3.27) w i t h r e s p e c t t o u ( t ) we g e t

(4.3.29) and t h i s s u b s t i t u t e d back i n t o (4.3.27) y i e l d s

(4.3.30)

148


Now because of the special form of B(y) - see (4.3.16) - we have that the quantity in square brackets in (4.3.30) is minimized by setting

irrespective of the value of Vx(x,t).

- g(x,t)

TQX

= &X

t

V,(x,t)Ax

-

This then yields

T hVx(x,t)B(x)B T (?)V,(x,t) (4.3.32)

which is solved by V(x,t)

=

&xTS(t)x , t

E

[O,Tl

.

(4.3.33)

Satisfaction of (4.3.21) follows upon noting that V(0,O)

=

0.

4.4 Conclusion

In this chapter we first stated the more important theorems for non-negativity of linear-quadratic functionals developed in [ll and elsewhere. Certain of our theorems are novel statements of non-negativity conditions. Section 4.2 was devoted to extensions of these theorems when the state and control vectors are required to belong to general constraint sets. When the constraint sets are specified by linear inequalities, a generalized Riccati equation emerges. In Section 4.3 we formulated a general class of non-linear constrained functionals and showed how the theorems of Section 4.2 are applicable. The main complication that arises in the non-linear case is that the conditions have to be

NON- N EGAT IVE FUNCT IONALS

149

tested for all time functions belonging to a specified set. This seems to be a common complication in the analysis of general non-linear systems and can only be obviated by suitable bounding techniques, an example of which is provided by an application of Theorem 4.3.2 to a non-linear problem of special structure. In the next chapter we investigate the question of controllability of a linear system when the control vector is required to belong to a general constraint set. References BELL, D.J. & JACOBSON, D.H. Singular Optimal Control Problems. Academic Press, New York and London, 1975. COPPEL, W.A. Linear-quadratic Optimal Control. Proc. Royal SOC. of Edinburgh, 73A, 1974, pp. 271-288. MOLINARI, B.P. Non-negativity of a Quadratic Functional. SIAM J. Control, 13, 1975, pp. 792-806. ANDERSON, B.D.O. Partially Singular Linear-quadratic Control Problems. IEEE Trans. Automatic Control , AC-18, 1973, pp. 407-409. KRASNER, N.F. Semi-separable Kernels in Linear Estimation and Control. Information Systems Laboratory, Center for Systems Research, Stanford University, California, U.S.A., Technical Report 7001-7, 1974, 319 p. JACOBSON, D.H. A General Sufficiency Theorem for the Second Variation. J. Math. Anal. Appl., 34, 1971, pp. 578-589. JACOBSON, D.H. & GETZ, W.M. Remarks on a Paper by V.B. Haas and Partially Singular Extremals. CSIR Special Report, WISK 208, May, 1976.

150 [81

LINEAR-QUADRATIC EXTENSIONS DAVISON, E.J. & KUNZE, E.G. Some Sufficient Conditions for the Global and Local Controllability of Non-linear Time-varying Systems. SIAM J. Control, 8, 1970, pp. 489-497.

[91 WERNLI, A. & COOK, G. Sub-optimal Control for the Non-1 inear Quadratic Regulator Problem. Automatica, 11, 1975, pp. 75-84. [lo] SPEYER, J.L. & SHAKED, U. Minimax Design for a Class of Linear Quadratic Problems with Parameter Uncertainty. IEEE Trans. Automatic Control, AC-19, 1974, pp. 158-159.

5.

CONTROLLABILITY OF CONSTRAINED LINEAR AUTONOMOUS SYSTEMS

5.1

Introduction

I n optimal c o n t r o l and g e n e r a l l y i n t h e c o n t r o l o f dynamic systems t h e r e a r e c e r t a i n system-theoretic concepts which play v i t a l r o l e s . One o f these i s s t a b i l i t y , but perhaps the most important i s c o n t r o l l a b i l i t y . a dynamic system iC = f(x,u)

Loosely speaking we c a l l

c o n t r o l l a b l e i f t h e r e e x i s t s an

admissible c o n t r o l f u n c t i o n u( . ) which t r a n s f e r s t h e s t a t e of t h e system from an a r b i t r a r y i n i t i a l value xo t o t h e o r i g i n o f t h e s t a t e space i n f i n i t e time.

I n t u i t i v e l y t h e concept

o f c o n t r o l l a b i l i t y i s basic t o t h e problem o f designing minimal time c o n t r o l l e r s , v i z . c o n t r o l l e r s which s t e e r xo t o the o r i g i n o f t h e s t a t e space i n minimum time. This chapter i s devoted t o a study o f t h e c o n t r o l l a b i l i t y p r o p e r t i e s of t h e 1i n e a r autonomous dynamic system iC=AxtBu where A u

E

E

nxn R ,B

E

(5.1.1)

R n x m a r e constant matrices and x

E

Rn,

Rm and where t h e c o n t r o l v a r i a b l e u i s required t o s a t i s f y

the constraint u(t)

E

R

= Rm,

t

E

lo,-).

(5.1.2)

The f o l l o w i n g sections review c e r t a i n well-known conditions f o r c o n t r o l l a b i l i t y o f (5.1.1) t o the i n t e r i o r o f R (0 (5.1.1)

E

when 52 = Rm and when 0 belongs

Int(R)).

The c o n t r o l l a b i l i t y o f

when 0 L I n t ( R ) i s then i n v e s t i g a t e d i n d e t a i l and

new r e s u l t s a r e presented p e r t a i n i n g t o t h e ' a r b i t r a r y 151

152


i n t e r v a l n u l l - c o n t r o l l a b i l i t y ' o f (5.1.1) 5.1.1

subject t o (5.1.2).

Unconstrained Case

When s2 E Rm, c o n d i t i o n s f o r t h e c o n t r o l l a b i l i t y o f (5.1.1) a r e much s i m p l i f i e d .

F i r s t we d e f i n e p r e c i s e l y t h e concept

o f c o n t r o l 1a b i 1ity. DEFZNZTZON 5 . 1 . 1

The l i n e a r autonomous system (5.1.1)

c o n t r o l l a b l e i f f o r each xo

E

is

Rn t h e r e e x i s t s a bounded

measurable c o n t r o l f u n c t i o n which steers xo t o t h e o r i g i n of t h e s t a t e space i n f i n i t e time. Necessary and s u f f i c i e n t c o n d i t i o n s f o r c o n t r o l l a b i l i t y were supplied by R.E. f o l 1ows

.

Kalman d u r i n g t h e e a r l y 1960s and a r e as

A necessary and s u f f i c i e n t c o n d i t i o n f o r i s that t o be c o n t r o l l a b l e when Q 5

THEOREM 5 . 1 . 1 (5.1.1)

Rank(Q) = n where

A Q = [B,AB,.

. .,A

(5.1.3)

n-1

Bl

.

(5.1.4)

Equivalently, W(0,t)

t

4 J@(t,.r)BB

T T

cp (t,-r)d.r

0

for a l l t

>0

>0

(positive-definite) (5.1.5)

where cp(t,.r) s a t i s f i e s

aT @(t,.r)

= A@(t,.r),

@(T,T) =

and i s t h e t r a n s i t i o n m a t r i x associated w i t h

I

(5.1.6) = Ax.

CONTROLLABILITY CONDITIONS

153

Inequality (5.1.5) actually allows one to compute a continuous control function which steers xo to 0. THEOREM 5.7.2

u(t)

=

The control function

-BT@ T(T,t)W-l(O,T)@(T,O)xo,

t

E

[O,Tl

(5.1.7)

steers xo to the origin in time T. PROOF

The solution of (5.1.1) at time T is given by

x(T)

=

@(T,O)xo

T

.

+ .f@(T,T)Bu(T)dT 0

(5.1.8)

Substituting (5.1.7) into (5.1.8) and using (5.1.5) yields

x(T)

=

@(T,O)x0

- @(T,O)x0

=

0

.

(5.1.9)

Note that the rank condition (5.1.3) is easy to check, especially when n is small. On the other hand (5.1.3) is not the correct condition when A and B are time-varying, but (5.1.5) remains valid. Note also that controllability here implies not only that there is a control function u( .) which steers xo to 0 in finite time but also that there is a control function u(.), given by (5.1.7) which performs this function in an arbitrary time interval [O,Tl. Thus when a z Rm controllability actually implies 'arbitrary-interval controllability', and the converse is trivially also true. 5.1.2

Zero Interior to

0

We assume here that R 5 Rm is arbitrary but that

154


0

E

Int(R).

(5.1 .lo)

In general, whether or not (5.1.10) is satisfied, it would be too much to expect that (5.1.1) is controllable subject to the restriction (5.1.2). Therefore we introduce the following definition. D E F I N I T I O N 5.1.2 The dynamic system (5.1.1) is nullcontrollable if there exists an open set V in Rn which contains the origin and for which any xo E V can be controlled to the origin in a finite time by a bounded, measurable control function.

Clearly null-controllability is really controllability in a sphere which surrounds the origin and so controllability implies null-controllability but the converse is not usually true. We can now state the relevant null-controllability theorem. THEOREM 5.1.3

System (5.1.1) is null-controllable if (5.1.10) holds, if and only if Rank(Q)

=

n.

(5.1.11)

PROOF

If (5.1.1) is null-controllable it is easy to show that (5.1.1) is controllable if the restraint (5.1.2) is removed: condition (5.1.11) then follows. On the other hand, (5.1.11) implies that (5.1.7) is well defined and so for xo E CxlIlxII < E ~ Iit follows that Ilu(t)ll < c2(c1) for t E [O,Tl where E ~ ( E ~-+) 0 as cl 0. As 0 E Int(R) it is clear that for E~ sufficiently small, u(t) E R , t E [O,Tl. -+

Note that as T is arbitrary in the above construction there


155

exists for each T > 0 a s e t V(T) as specified in Definition (5.1.2). Therefore when 0 E I n t ( R ) null-controllability i s , actually, equivalent t o 'arbitrary-interval nu1 1-controllability'

.

5.1.3

Reachable Sets

The notion of a reachable s e t and certain of i t s properties play important roles in controllability theory, as we shall see i n the following pages. We now define the reachable sets See Section 5.A, the Appendix, for R R ( t ) , R C H ( n ) ( t ) and R,. certain set-theoretic definitions. DEFTNITION 5. I . 3 The reachable s e t Rn( t )( RCH(n) ( t ) )a t time t i s defined t o be the s e t of a l l points i n Rn t o which the origin can be steered a t time t by a bounded measurable control function u ( - ) which s a t i s f i e s U ( T ) E 0 ( u ( T ) E Convhull(f2)) for a l l T E [ O , t l . The reachable s e t R, i s the u n i o n over positive t of the s e t s R R ( t ) .

[ll

Consider the linear process (5.1.1) w i t h constraint s e t Rc, Rm. The reachable s e t R R ( t ) a t time t i s convex. If in addition R i s compact then R R ( t ) i s convex, compact, and varies continuously with t on t 2 0 and

TffEOREM 5 . 7 . 4

RCH(Q) t > = R ~ ( t ) *

The f i r s t part of the theorem and the compactness of R Q ( t ) f Q i s compact follow d i r e c t from Lemma 4A [l, p. 1631 on the range of a vector measure. The equivalence of R R ( t ) , RCH(Q) ( t ) i s proved in Theorem 1 A [l, p. 1641 by n o t i n g t h a t R R ( t ) E R C H ( n ) ( t ) and t h a t these s e t s are convex and compact if R i s compact and by proving t h a t R n ( t ) i s dense in PROOF

RCH( Q ) ( t,

L I NEAR-QUADRATIC EXTENSIONS

156

The n e x t theorem shows t h a t t h e n a t u r e o f t h e reachable s e t s o f (5.1.1)

s u b j e c t t o (5.1.2)

i s d i r e c t l y related t o the

c o n t r o l l a b i l i t y of t h e n e g a t i v e o f system (5.1.1).

TtlEOREM 5 . 7 . 5 t o (5.1.2)

o f system (5.1.1)

The reachable s e t ,R

subject

c o n t a i n s a neighbourhood o f t h e o r i g i n i f and o n l y

i f t h e system

iC s u b j e c t t o (5.1.2)

=

-

AX

-

BU

(5.1.12)

i s null-controllable.

PROOF

Suppose t h a t u ( t ) , t E [O,Tl s t e e r s (5.1.12) t o 0 a t t = T. Then i t i s easy t o show t h a t u ( T - t ) ,

from 0 t o xo a t t = T.

s t e e r s (5.1.1) eas i1y v e r i f i e d

.

from xo t

[O,Tl

E

The converse i s a l s o

The n e x t theorem i s o f fundamental importance i n s t u d i e s of null-controllability.

THEOREM 5.7.6

[2]

such t h a t Bu = 0. reachable s e t ,R zero v

E

Suppose t h a t t h e r e e x i s t s a v e c t o r u

E

R

Then, t h e o r i g i n i s i n t e r i o r t o t h e o f (5.1.1)

i f and o n l y i f t h e r e i s no non-

Rn such t h a t T At v e Bu G O f o r a l l t

>0

and f o r a l l u

E

R

. (5.1.13)

PROOF

By Theorem 5.1.4,

RR(t) i s convex f o r a l l t

Furthermore t h e f a c t t h a t t h e r e i s a u i m p l i e s t h a t RR(tl) all t

> 0.

E

R such t h a t Bu = 0

5 R R ( t 2 ) f o r tl S t 2 and 0

T h e r e f o r e i t f o l l o w s t h a t ,R

> 0.

E

RR(t) for

i s t h e u n i o n of

nested i n c r e a s i n g convex s e t s which c o n t a i n t h e o r i g i n , and

CONTROLLAB I LITY CONDITIONS

157

is thus a convex set which contains the origin. If in fact the origin is interior to ,R there exist n + 1 points in ,R whose convex hull contains the origin as an interior point. It follows that these n + 1 points must be contained in RO(t) for some t > 0 since these sets are increasing. Therefore it is clear that the origin is interior to ,R if and only if the origin is interior to RO(t) for some t > 0. Consequently if the origin is not interior to R, there exists a v E Rn such that

v Tx(t) g o

(5.1.14)

for all t > 0 and all admissible controls u(*). (5.1.14) is just t

vT JeA(t-T)Bu(T)dr

Inequality

(5.1.15)

G0

0

and it follows by continuity and a special choice of u(.) that

v TeAt Bu G O for all t > O and for all u

E

R.

(5.1.16) On the other hand, if the origin is interior to ,R there cannot exist a vector v E Rn which satisfies (5.1.16), as the existence of such a vector would imply inequality (5.1.14). 5.1.4

Zero Interior to Convhull(0)

The last theorem and Theorems 5.1.4 and 5.1.5 may now be combined to yield the following result which is similar to Theorem 5.1.3.

158


THEOREM 5 . 1 . 7 0

E

Suppose t h a t f2 i s compact and t h a t

Int(Convhull(f2)).

Then, system (5.1.1)

s u b j e c t t o (5.1.2)

i s n u l l - c o n t r o l l a b l e i f and o n l y i f Rank(Q) = n.

PROOF

(5.1.17)

i s compact, Theorem 5.1.4 i m p l i e s t h a t f2 can be replaced by Convhull(f2) as f a r as reachable s e t s a r e concerned. As z e r o belongs t o Convhull(f2) we have t h a t t h e r e e x i s t s a v e c t o r u E Convhull(f2) such t h a t Bu = 0. Theorem 5.1.6 t h e n i m p l i e s t h a t z e r o i s i n t e r i o r t o ,R i f and o n l y i f t h e r e i s no non-zero v E Rn such t h a t As

$2

T At v e Bu G O f o r a l l t

> O and f o r a l l u

E

Convhull(f2). 18) ( 5 .l.

We now prove t h a t t h i s i s t r u e if and o n l y i f (5.1.17) holds. Suppose f i r s t t h a t Q does n o t have r a n k n. Then, t h e r e e x i s t s a v e c t o r v E Rn such t h a t T vQ=O.

(5.1.19)

Now i t i s w e l l known t h a t t h i s i m p l i e s t h a t T At v e B = O

(5.1.20)

so t h a t T At v e Bu = 0 f o r a l l t > O and a l l u

E

Convhull(0) (5.1.21)

which c o n t r a d i c t s t h e statement t h a t z e r o i s i n t e r i o r t o . R,


159

Suppose now that Q has rank n but that vTeAtBu = 0 for all t and all u E Convhull(R). Then by setting t = 0 we obtain

v TBu

=

0 for all u

E

Convhull(S2)

(5.1.22)

and successive differentiations of vTeAtBu and evaluations at t = 0 yield

vTAi Bu

=

0 for all u

E

Convhull(R),

i=l ,... ,n-1. ( 5.1.23)

As zero is interior to Convhull(R), Equations (5.1.22) and (5.1.23) imply that Q has rank less than n, which i s a contradiction. Therefore (5.1.17) implies that there exists u E Convhull(R) such that vTeAtBu is not identically zero for all t. Now either v TeAtBu changes sign as a function of t for this fixed u E Convhull(R) or it is of one sign. Suppose in fact that vTeAt Bu G O (not identically zero, as we have proved). As zero is interior to Convhull(S2) we can switch the sign of vTeAtBu to be positive (whenever it is non-zero) by reversing the sign of u so that (5.1.18) does not hold for any v E Rn. This completes the proof that zero is interior to R, if and only if (5.1.17) holds. The observation that Rank[B,AB, ...,A n-1Bl

=

Rank[-B,AB,-A 2B,...,(-l) nAn-1B l (5.1.24)

then yields the theorem.

160 5.1.5

LINEAR-QUADRATIC EXTENSIONS Global and Non-linear Results

As remarked prior to Definition 5.1.2 controllability (as distinct from null-controllabil ity) cannot in general be expected of ( 5 . 1 . 1 ) when it is subjected to the constraint ( 5 . 1 . 2 ) . However, the following theorems provide conditions which guarantee that controllability (null-controllability in the large) can be achieved by (5.1.1) subject to ( 5 . 1 . 2 ) . THEOREM 5 . 1 . 8

[l]

Consider the system ( 5 . 1 . 1 ) subject to the control constraint ( 5 . 1 . 2 ) . Suppose that zero belongs to the interior of R , that Rank(Q) = n, and that A is stable (Re()\) < 0). Then ( 5 . 1 . 1 ) is controllable. The stability of i = Ax ensures that any initial point xo E Rn can be steered by u(-) E 0 until x(t) approaches 0 and therefore enters the domain of null-controllability of ( 5 . 1 . 1 ) . But then x(t) can be steered to the origin in a finite time. PROOF

When the control function is scalar the strict inequality on Re()\) can be relaxed, but then compactness of R is assumed. THEOREM 5.1.9

[l]

Suppose that m = 1 and that R is a compact set which contains zero in its interior. Then (5.1.1) is controllable if and only if Rank(Q) = n and every eigenvalue X of A satisfies Re(A) GO. When m > 1 we have the following theorem.

THEOREM 5.1.10

[ll

Suppose that R is a compact set which contains zero in its interior. Assume that no two Jordan canonical blocks of A contain equal eigenvalues of A. Then (5.1.1) is controllable if and only if Rank(Q1 = n and every

161

CONTROLLABILITY CONDITIONS eigenvalue A of A satisfies Re(A) G O .

Finally, we remark that null-controllability of a non-linear system can sometimes be deduced from the null-controllabil ity of its linearization. This is made precise in the following theorem [ l l .

THEOREM 5.7. I 7

Consider the non-1 inear autonomous system i

=

(5.1.25)

f(x,u)

where f:Rn+' Rn is once continuously differentiable in x and u. Suppose that zero is interior to the constraint set R c _ Rm, and assume that -f

f(0,O)

=

( 5 , l . 26)

0

Rank[B,AB, ...,A n-1 Bl

=

where A

=

-(O,O), af ax

B

=

-(O,O) af

au

(5.1.27)

n

.

(5.1.28)

Then (5.1.25) is null-controllable. 5.2

Zero Not Interior to Constraint Set or its Convex Hull

5.2.1

Oscillatory Systems

In this section we devote attention to recent work on the controllability of (5.1.1) subject to (5.1.2) when 0 k Int(R). Initially research on this problem was motivated by the question [31: can the motion of a simple pendulum be brought to rest in a finite time by the application of a unit force acting only in one direction? In mathematical terms one asks

162


whether t h e system Xl

=

x2

x2

=

- x1 + u

(5.2.1)

is ( n u l l - ) c o n t r o l l a b l e when u(t)

E

[OYlI

y

t

E

[O,..).

(5.2.2)

Clearly (5.2.1) i s c o n t r o l l a b l e when u ( t ) E R 1 and i s not r e s t r i c t e d according t o (5.2.2). Furthermore i f u ( t ) E [ - ~ , 1 1 E > 0 Theorem 5.1.3 guarantees t h a t (5.2.1) i s null-controllable. Saperstone and Yorke were t h e f i r s t t o prove t h e following r e s u l t when R = [0,11 . Suppose t h a t m = 1 and R = [0,11. The system (5.1.1) is null-controllable i f and only i f a l l the eigenvalues o f A have non-zero imaginary p a r t s and Rank(()) = n .

THEOREM 5.2.7

T h i s interesting theorem implies t h a t i f n i s odd (implying a t l e a s t one real eigenvalue) (5.1.1) i s not null-controllable. Note t h a t the conditions of the theorem a r e s a t i s f i e d by (5.2.1) and (5.2.2) so t h a t (5.2.1) i s indeed null-controllable subject t o the c o n s t r a i n t t h a t u ( t ) E [0,11. Note a l s o t h a t Theorem 5.1.4 allows one t o replace the interval [ O y l l i n Theorem 5.2.1 by the s e t ( 0 , l ) consisting o f t h e two end points of the compact i n t e r v a l . I t turns out t h a t Theorem 5.2.1 i s not t h e best t h a t can be obtained when 0 k Int(R) and 0 k Int(Convhull(R)). The


163

a p p r o p r i a t e g e n e r a l i z a t i o n o f Theorem 5.2.1 which we i n v e s t i g a t e f u l l y i n t h e n e x t s e c t i o n i s due t o Bramner [2]. Brammer ' s Theorem

5.2.2

The f o l l o w i n g theorem i s a major g e n e r a l i z a t i o n o f Theorem 5.2.1 though i t i s simple i n form. However, Brammer's proof i s exceedingly l e n g t h y and i n t r i c a t e so t h a t we p r e f e r t o p r e s e n t H a j e k ' s s h o r t p r o o f [4].

THEOREM 5 . 2 . 2

[21

Suppose t h a t

there i s a u

E

Sl such t h a t Bu = 0

(5.2.3)

and t h a t t h e s e t Convhull(C2) has non-empty i n t e r i o r i n Rm. (5.2.4) Then (5.1.1)

s u b j e c t t o (5.1.2)

i s n u l l - c o n t r o l l a b l e i f and

only i f Rank(Q1 = n

(5.2.5)

and t h e r e i s no r e a l e i g e n v e c t o r v o f A T v Bu G O f o r a l l u

E

Sl.

T satisfying (5.2.6)

C l e a r l y (5.2.6) i s t r i v i a l l y s a t i s f i e d i f A has no r e a l eigenvalues so t h a t Theorem 5.2.2 reduces t o Theorem 5.2.1 when m = 1 and Sl = [0,11

.

HSjek's p r o o f o f t h e above theorem r e q u i r e s t h e f o l l o w i n g lemmas, which we o n l y s t a t e .


164

LEMMA 5.2.7 Suppose t h a t A E R n x and V i s a closed convex cone i n Rn such t h a t v and -v belong t o V only i f v = 0. If At V i s invariant under eAt, v i z . e V 5 V f o r a l l t > 0 then V contains an eigenvector of A. At The l i n e a r hull of {e B u l t > 0, u E Q) i s Rn LEMMA 5 . 2 . 2 i f and only i f Rank(Q) = n (provided t h a t (5.2.3), (5.2.4) hold)

.

The system A = Ax t Bu s a t i s f i e s (5.2.5), LEMMA 5 . 2 . 3 (5.2.6) i f and only i f the negative system d = - Ax - Bu does. PROOF OF THEOREM 5 . 2 . 2 As a consequence of Theorems 5.1.5, 5.1.6 and Lemma 5.2.3 we have t h a t (5.1.1) i s null-controllable subject t o conditions (5.2.3) and (5.2.4) i f and only i f there is no non-zero vector v E Rn such t h a t

v T eA t Bu G O f o r a l l t > O and f o r a l l u

E

Q. (5.2.7)

If (5.2.5) f a i l s i t is easy t o show via Lemma 5.2.2 t h a t there is a non-zero vector v E Rn such t h a t

v T eA t Bu = 0 f o r a l l t and a l l u

E

If (5.2.6) is s a t i s f i e d by a real eigenvector v we have t h a t

0. E

T At T At v e Bu = v Bu.e where A i s the real eigenvalue associated w i t h v . from (5.2.6) and (5.2.9) t h a t

(5.2.8)

T Rn of A (5.2.9) I t follows

CONTROLLABILITY CONDITIONS At vTe Bu GO f o r a l l t > O and a l l u E fl.

165 (5.2.10)

We conclude from t h e above t h a t t h e c o n d i t i o n s (5.2.5) (5.2.6)

and

a r e necessary.

Next we assume t h a t (5.1.1) t h e c o n d i t i o n s (5.2.5),

i s not null-controllable but that

(5.2.6)

hold;

t h i s leads t o a contra-

diction. L e t W be t h e s e t o f a l l l i n e a r combinations w i t h non-negative c o e f f i c i e n t s o f p o i n t s eAtBu f o r t > 0, u E fl. Then from (5.2.7) we have t h a t i f (5.1.1) t h e r e i s a v E Rn such t h a t

i s not null-controllable

T v w G 0 for all w

E

W.

(5.2.11)

Now W i s a closed convex cone i n Rn which by Lemma 5.2.2 and c o n d i t i o n (5.2.5)

satisfies

W

+

(-W)

and i s o b v i o u s l y i n v a r i a n t under eAt, eAtWs

w

(5.2.12)

= Rn

viz.

f o r a l l t 2 0.

(5.2.13)

Next we note t h a t t h e s e t V c o n s i s t i n g o f those vectors v

E

Rn which s a t i s f y

T v w

0 application of t h e non-negative control u ( t ) E [0,11 only increases the value of x 2 ( t ) away from i t s s t a r t i n g value of x2(0). More precisely, we have t h a t

xl(t)

= x,(O)cos(t)

t

+ x 2 ( 0 ) s i n ( t ) t ./

sin(t-T)u(T)dT

0

(5.3.1) and x2(t) =

-

t x , ( O ) s i n ( t ) + x,(o)cos(t) + ./ cos(t-T)u(T)dT 0

(5.3.2)

so t h a t when x l ( 0 ) x2(t)

= 0 we have

= x2(0)cos(t)

t

+ ./ cos(t-T)u(T)dT 0

(5.3.3)

which i s positive f o r x2(0) > O and t 0 there exists an open set V(T) in Rn which contains the origin and for which any xo E V(T) can be controlled to the origin at time T by a bounded measurable control function. The next theorem relates the properties of the reachable sets of (5.1.1) to arbitrary-interval null-controllability. The system (5.1.1) subject to (5.1.2) is arbitrary-interval nu1 1-control lable if and only if for each T > 0 the reachable set RR(T) of (5.1.1) contains the origin in its interior. THEOREM 5.3.7

PROOF

For fixed but arbitrary T we have

RR(T)

=

T

U eA(T-T)BU(r)drlu(T) 0

E

R , O < T GT)

(5.3.4)

If (5.1.1) is arbitrary-interval null-controllable we have for each T > 0 that -r

V(T)

=

;eA(T-T)Bu(r)dTlu(r) 0

E

R, 0

< T < T). (5.3.5)

It follows from (5.3.4), (5.3.5) and the invertibi ity of eAT that RR(T) contains the origin in its interior because V(T) does. The converse, namely that V(T) exists f RQ(T) contains the origin in its interior, follows in the same way via the invertibility of eAT . Note that the relationship between arbitrary-intervul nu1 1-


170

controllability and reachabil ity is simpler than that between null-controllabil ity and reachabil ity (cf. Theorems 5.1.5, 5.1.7).

The following theorem exploits the relationship.

THEOREM 5.3.2 A necessary condition for arbitrary-interval null-controllability o f (5.1.1) subject to (5.1.2) is that there should exist for each non-zero v E Rn and for each T > 0 a time T , 0 < T < T and a u E R such that vTeATBu > 0.

(5.3.6)

Suppose that (5.1.1) subject to (5.1.2) i s arbitraryinterval null-controllable. Theorem 5.3.1 then implies that for each T > 0, Rn(T) contains the origin in its interior. This implies that there i s no non-zero v E: Rn such that PROOF

T

vTJeA(T-T)Bu(-r)d-r 0. Now because R is bounded Cl(Convhull(Bfi)) is compact so that there is a sub-sequence (2,) of {z,,) which converges to .? E Cl(Convhull(BS2)) as k 03 and 'rk +. 0. Hence Lim v T eATk zk = vT? 2 0, which contradicts (5.3.14). Now R -+

k-+m

compact implies that Convhull(R) is compact and (5.3.11) follows from (5.3.10). The following necessary and sufficient condition is the counterpart of Theorem 5.1.6.

THEOREM 5.3.4 Suppose that there exists a vector u E R such that Bu = 0. Then, (5.1.1) subject to (5.1.2) is arbitrary-interval nu1 1-control lable if and only if there exists for each non-zero v E Rn and for each T > 0 a time T , 0 G T G T and a u E R such that v T eAT Bu > 0. Equivalently, there should exist no v E Rn such that for some T > 0, T AT v e Bu GO, 0 G T G T for all u E R . The assumption that there is a u E R such that Bu = 0 implies that Rn(tl) C Rn(t2), tl < t2 d T. Furthermore Rn(t) is convex (Theorem 5.1.4) and 0 E RR(t), t E (0,Tl , T > 0 and arbitrary.

PROOF

We already showed (Theorem 5.3.2) that the condition of the theorem is necessary for arbitrary-interval nu1 1 -controll ability. To prove sufficiency we show that if the system is not arbitrary-interval null-controllable then the condition of the theorem is violated. If the system is n o t arbitraryinterval null-controllable then 0 tf Int(RR(T)) for some T > 0 (Theorem 5.3.1). As a consequence of the convexity and


173

t h e n e s t i n g o f t h e reachable sets we see t h a t t h i s i m p l i e s t h e existence o f a non-zero v

E

t vTleA(t-T)Bu(T)d.r

Rn such t h a t

< 0,

0

< t < T,

0

f o r a l l u ( - ) such t h a t

U(T)

E

.

R

(5.3.15)

For a p a r t i c u l a r choice o f u ( - ) t h i s i m p l i e s t h a t

T At v e Bu

< 0,

0

< t 0 there is a T~ 0 < T < T and a u E R such that vTeAT Bu > 0. NOW, trivially, vTeATBu = uTBTeA ' T ~so PROOF

T

that the preceding statement is equivalent to the statement that for each non-zero v E Rn and T > 0 there is a T~ 0 G T < T such that BTeA ~v TL R', where (see Section 5.A) Q' is the polar of R. Now Q' is a closed, convex cone so that

AT^

the separating hyperplane theorem implies that BTe v f R' is equivalent to the existence of a non-zero a E Rm such that T

aTw ' < 0 for all w ' E R' and aTBTeA v' > 0. It follows from this that (A, By R) is arbitrary-interval nu1 l-controllable if and only if for each non-zero v E Rn and T > 0 there is a

T

0 < T < T and a E R" such that aTBTeA v' = vTeAT Ba > 0 where n'' A= (R')'. Thus (A, By R) is arbitrary-interval nullcontrollable if and only if (A, By ,") is arbitrary-interval null-controllable. T,

Now it is clear from the definition of the polar of a set that 5 R" and Q" is a closed, convex cone. Therefore it follows that Convhull(R) C- Q " , Conichull(Convhull(C2)) E and Cl(Conichull(Convhull(Q))) ~ n " .Hence il c C1 (Conichull(Convhull(s2))) E so that arbitrary-interval null-controllability with the constraint set C1 (Conichull(Convhull ( a ) ) ) imp1 ies the same with R" and hence with R, and the theorem is proved.

,"

,I'

The above theorem and its proof indicate that one' may work with the 'nicer' closed, convex, cone C1 (Conichull (Convhull(0)))

175

CONTROLLABILITY CONDITIONS which is equivalent to Cl(Convhull(Conichull(i2))) R S Rm.

for any

The following theorem shows that if (A, B, Q ) is arbitraryinterval null-controllable so is (A, B, 6) where 6 is a suitable, bounded, constraint set. THEOREM 5.3.6

Let K

= {z

E

RmIIlzll G 1).

Then , (i) the system ( A , B, Q ) is arbitrary-interval nullcontrollable if and only if the system (A, B, Conichull(Q)n K) is arbitrary-interval null-control lable; (ii) if 0 E Convhull(Q) then the system (A, B, Q ) is arbitrary-interval null-controllable if and only if the system (A, B, Convhull(0) n K) is arbitrary-interval nullcontrol 1 able. PROOF

(i)

If we can show that

Conichull (Q) = Conichull (Con chu

i(n) n

then the theorem follows from Theorem 5.3 5. would allow us to write Cl(Conichul1 (Convhull (Conichull( R )

n

K)

(5.3.17)

Indeed (5.3.17)

K)))

=

Cl(Convhull(Conichull(Conichull(R) n K ) ) )

=

C1 (Convhull (Conichull (0)) ) = C1 (Conichull (Convhull(52))) (5.3.18)

176

L INEAR-QUADRATIC

EXTENS IONS

and an application of Theorem 5.3.5 would yie d the desired result We thus proceed now to prove that ( 5 3.17) holds. First, it is obvious that Conichull ( a ) fl K so

E

Conichull (f2)

that Conichull(Conichull(f2) fl K)

Next, suppose that z a

E

E

(5.3.19)

Conichull(i2).

Conichull(Q), z Z 0. Then

1

y = z E Conichull(i2) so that y E Conichull(R) fl K. It follows then that IlzlLy E Conichull(Conichull(f2) fl K) so that z E Conichull(Conichull(i2) fl K), and

Conichull(S2) c Conichull(Conichull(i2) fl K ) . (5.3.20) The inclusions (5.3.18) is proved.

and (5.3 19) yield (5.3.17) and (i)

(ii) Here Theorem 5.3.5 wou d be again applicable if one could show that

Conichull (Convhull(i2))

=

Conichull(Convhull(f2) fl K ) . (5.3.21)

Clearly we have that Convhull(f2) 2 Convhull(i2) fl K so that Conichull(Convhull(i2) fl K) c Conichull(Convhull(S2)) ( 5 3.22)

Now let z E Conichull(Convhull(R)), z Z O . Then there s a A > 0 such that Az E Convhull(i2) and we have that if IIAz 11 1 then Az E Convhull (f2) fl K so that z E Conichull(Convhull(i2) fl K). If it turns out that IIAz II > 1 then since 0 < 1 and 0 E Convhull(i2) we have

<m


1

177

m

Az E Convhull(i2) r l K. Convhull (a) and Hence Az always belongs t o Conichull(Convhull(i2) rl K) so t h a t

that

Xz

E

Conichull(Convhull(i2)) 5 Conichull(Convhull(i2) rl K ) . (5.3.23) Inclusions (5.3.22) and (5.3.23) yield (5.3.21) and ( i i ) i s proved. 5.3.3

Necessary Conditions

In this section we provide a necessary condition f o r arbitraryinterval n u l l - c o n t r o l l a b i l i t y which has an appealing geometric character. First we need the following lemma.

Let C be a closed cone i n Rn and l e t v

LEMMA 5.3. I

E

Rn be

such t h a t

IlvII

=

T 1, v c < O , f o r a l l c

E

C , c f 0.

(5.3.24)

n I V Tx = -1, x

E

c}

(5.3.25)

Then the set A A =

(X

E

R

is compact. As A is the intersection of two closed sets i t i s closed. W e now prove t h a t A is bounded. We have t h a t x E A implies t h a t x = - v + N(v T ), where N ( . ) denotes ' t h e null space o f ' . Let us assume t h a t A i s unbounded, which implies T t h a t there i s a sequence xk E N(v ) such t h a t - v + X k E C and II X k II 03 a s k + 03 Clearly the sequence

PROOF

+

.

178


T A Xk ' k = mK c - N(v ) fl (xlIlxII G 1 ) which is a compact set, so that yk -+ j E N(V T ), viz. vTi = 0. NOW since c is a cone we k' have that zk = E C and as C is closed it follows

,f$-,, + T i p

that Lim zk = j E C, so that vTj < 0, a contradiction. k-+a A necessary condition for arbitrary-interval THEOREM 5 . 3 . 7 null-controllability of (A, B, L?) is that the cone C 4 Cl(Conichull(Convhull(BR))) is not pointed. If the cone C is pointed there exists a v E Rn such that the hypothesis of Lemma 5 . 3 . 1 holds, which implies that the set A is compact. Clearly A is also convex and non-empty and is such that 0 A. Theorem 5 . 3 . 3 then implies that the system (A, I, A) is not arbitrary-interval null-controllable. Noting that Conichull(A) = C we see that Theorem 5.3.5 implies that the system (A, I, C) is not arbitrary-interval nullcontrollable. A further application of Theorem 5 . 3 . 5 yields the fact that (A, B, L?) is not arbitrary-interval nullcontrollable. PROOF

The above theorem has the following implications. If u E R E R' and 0 f Int(Convhull(R)) then COROLLARY 5 . 3 . 1 the system (A, B, L?) is not arbitrary-interval null-controllable. Note, however, that such a system can be null-controllable, cf. Section 5.2.1. COROLLARY 5 . 3 . 2 If B has full rank then the cone C is not Cl(Conichull(Convhull(R))) pointed if and only if the cone is not pointed. Note also that regardless of the rank of B we always have that


179

Conichull (Convhull(BQ)) = Conichull (B(Convhul1 (Q))) = BConichull(Convhull(Q)). If B has full rank then BC1 (Conichull(Convhull(Q))) = C1 (Conichull (Convhull (BQ))). Finally, Theorem 5.3.7 and Corollary 5.3.2 imply the following result.

If B has full rank then a necessary condiCOROLLARY 5 . 3 . 3 tion for arbitrary-interval null-controllability of the system (A, B, Q ) is that the cone t = Cl(Conichull(Convhull(Q))) is not pointed. 5.3.4

Necessary and Sufficient Conditions Generated Cones

-

Finitely

When c = Cl(Conichull(Convhul1 ( a ) ) ) is a finitely generated cone we can deduce a necessary and sufficient condition for arbitrary-interval null-controllabil ity of the system (A, B, 0 ) . We sha 1 present an example which illustrates the use of the theorem and another which illustrates that in general c" cannot be replaced by R in the theorem. A sufficient condition in the spirit of Theorem 5.1.3 is also presented. TffEOREM 5.3.8

Suppose that = Cl(Conichull(Convhull(Q))) is a finitely generated cone. Then the system (A, B, Q ) i s arbi trary-interval nu1 1 -control1 able if and only if there exists for each non-zero vector v E R n (in fact, for each v E (Bt)', where is the polar o f ( . ) ) an integer i(v), 0 < i(v) < n-1 and a u E A(v,i(v)) such that v ~ A ~ ( ~ ) B>u0, where ( - ) I

A(v,i) A= (u

E

tlvTA jBu=O, j=O ,..., i-1)

(5.3.26)

180

LINEAR-QUADRATIC EXTENSIONS A(v,O)

PROOF

6

c.

( 5.3.27)

Sufficiency follows easily upon noting that

T AT i(v)Bu i(v> v e Bu=vA T(-qT + higher-order terms in

T

(5.3.28)

for all u E A(v,i(v)). Indeed this equation and satisfaction of the conditions of the theorem imply that for each v there is a t(v) > 0 sufficiently small and a u E t such that vTeAT Bu > 0 for all T, 0 < T < t(v). Theorems 5.3.4 and 5.3.5 then imply that the system (A, B y a ) is arbitraryinterval null-controllable. Suppose now that the system (A, By a ) is arbitrary-interval null-controllable. Then for a given v E Rn either there is a u E such that v TBu> 0, in which case the conditions of the theorem are satisfied with i(v) = 0, or v TBu < 0 for all u E C. If in addition to this last-mentioned case A(v,l) = (0) or v TABu < 0 for all u E A(v,l) we have by the continuity in T of vTeATBu that there exists for each generating vector uk of t, k=l, ...,p, a time tk(v) such that v TeAT Buk < 0 for all T, 0 < T < tk(v). Then, since each vector u E t is a non-negative combination of the generating vectors uk, k=l, ...,p we have vTeATBu S 0 for all u E and 0 G T St(v), where t(v) = min tk(v). Hence Theorems 5.3.4 k and 5.3.5 imply that the system is not arbitrary-interval null-control lable, a contradiction. This then imp1 ies that either v TABu > 0 for some u E A(v,l) in which case i(v) = 1, or i(v) > 1. A similar argument may be used to investigate Clearly i(v) < n-1 as the larger possible values of i(v).

c


181

otherwise the Cay1 ey-Hami 1 ton theorem would imply that v TAk Bu = 0 for all u E A(v,k) and all k and A(v,k) = A(v,n) for all k > n. This in turn would imply, using a similar argument to that developed above which is based upon the finite generation of the cone t , that there is a time t(v)>O such that v T eAT Bu < 0 for all T, 0 < T < t(v) and all u E t, contraditting the assumed arbitrary-interval null -controll ability of (A, B, 52). The following remarks which relate to Theorem 5.3.8 are important.

(i) When checking the conditions of Theorem 5.3.8 on a particular system (A, B, 52) we need only consider those vectors v E Rn such that v E (BConvhull(52))' where denotes the polar of ( - ) I

( a ) .

(ii) If the cone C = Cl(Conichull(Convhull(B52))) is finitely generated, then Theorem 5.3.8 is applicable because the arbitrary-interval null-controllability of system (A, I, C) is equivalent to that of (A, B, t ) and (A, B, 52) - see Theorem 5.3.5. ( i i i ) If m = 2 or n = 2, Theorem 5.3.8 is always applicable because every cone in R 2 is finitely generated.

(iv) If the control constraint set 52 is a polyhedron, is finitely generated and Theorem 5.3.8 can be used. The following theorem provides us with a set of sufficient conditions for arbitrary-interval null-controllabil ity.

THEOREM 5 . 3 . 9 Suppose there exist * B E R" such that

QS ~ 5 2 ,P E

R"x m and

t

182

LINEAR-QUADRATIC EXTENSIONS = Epus f o r a l l us

BU,

t h e s e t (Puslus

E

E

os

(5.3.29)

;

GSl c o n t a i n s t h e o r i g i n Pus = 0 i n i t s interior;

Rank[i,Ai,

...,A n - l -B]

(5.3.30) = n.

(5.3.31)

Then, t h e system (A, B y R) i s a r b i t r a r y - i n t e r v a l n u l l controllable.

PROOF

Naming t h e v e c t o r Pus as z we see t h a t t h e system

R = AX

t

EZ

(5.3.32) (5.3.33)

s a t i s f i e s t h e c o n d i t i o n s o f Theorem 5.3.8; c o n d i t i o n on t h e p a i r (A,

i) and

i.e.

t h e rank

t h e f a c t t h a t Rz c o n t a i n s

zero i n i t s i n t e r i o r a r e e a s i l y seen t o be s u f f i c i e n t . We remark t h a t a s p e c i a l case o f t h e c o n d i t i o n s o f Theorem 5.3.9

has been used p r e v i o u s l y i n t h e l i t e r a t u r e .

Specifi-

c a l l y i n [7] t h e minimum-time o p t i m a l c o n t r o l problem i s considered and t h e f o l l o w i n g assumptions a r e made, n R c R i s a polyhedron.

(5.3.34)

The s o - c a l l e d P o n t r y a g i n c o n d i t i o n o f general p o s i t i o n i s assumed, namely i f b i s a v e c t o r c o - l i n e a r w i t h an edge o f t h e polyhedron R then [b,Ab,.

. .,A

n-1

bl = n

(5.3.35)

CONTROLLABILITY CONDITIONS 0

183

R and 0 k vertex(f2).

E

(5.3.36)

We now show t h a t these assumptions imply the existence of an as, P and a s specified i n Theorem 5.3.9. Indeed, i f 0 E Int(R) we can s e t Rs = R, P = I , B = I . If 0 I Int(R) e can then then by (5.3.36) i t must belong t o an edge of R . W t o be b (a vector co-linear w i t h define Rs t o be t h a t edge, bT t h a t edge) and P t o be bTb

5.3.5

.

Examples

Our f i r s t example i l l u s t r a t e s t h a t i n Theorem 5.3.8 the s e t e consider the following system: cannot be replaced by R. W

A =

$2 =

[: :] [: 7 ’

(5.3.37)

=

{u

First we note t h a t

-C = Cl(Conichu

1 Convhull(R))) = { u l u l

E

1 R , u2

> 0). (5.3.39)

Now T

v BU = vlul + v2u2 I

(5.3.40)

so t h a t f o r each v E R2 f o r which v1 f 0 o r v2 > 0 there i s T a u E so t h a t v Bu > 0. The only case t h a t remains i s v1 = 0, v2 < 0. Here we note t h a t

184

LINEAR-QUADRATIC EXTENSIONS T v Bu = 0 f o r u2 = 0,

u1 a r b i t r a r y

(5.3.41)

so t h a t A(v,l]

= {ulul

and t h e r e c l e a r l y e x i s t s a u

E

1

, u2

A(v,l)

E

T v ABu =

R

-

= 0)

(5.3.42)

such t h a t

u1 > O .

(5.3.43)

Therefore t h e c o n d i t i o n s o f Theorem 5.3.8 a r e s a t i s f i e d and t h e system ( A , B y $2) i s a r b i t r a r y - i n t e r v a l n u l l - c o n t r o l l a b l e . However, i f we r e p l a c e v2

0 and A ( v , l ) i s

j u s t t h e s e t (01, so t h a t t h e c o n d i t i o n s o f t h e theorem cannot be s a t i s f i e d . Our n e x t example i s one which s a t i s f i e s t h e c o n d i t i o n s o f Theorem 5.3.8 b u t n o t those o f Theorem 5.3.9.

The system

equations a r e

A3 =

-

x1

+

x2

-

x3

+ u3 - u4

and R = iulu.

1

so t h a t

> O , i=1,...,41

(5.3.45)


1 A =

-1 -1

y]

-

1 - 1

,

B = [ l

0

o

o

1 -1

185 0

0

0 0 1 -11

.

(5.3.46)

We have t h a t

v TBU = v1(u1-u2) T C l e a r l y v Bu

+

v2U1

+ v 3 ( U 3-U 4 ).

(5.3.47)

>0

i f v3 f 0 and i f v1 f 0 and f o r some u E T I f v2 f 0 then v Bu f a i l s t o be p o s i t i v e f o r some

v2 = 0. u E o n l y i f v1 = v3 = 0, o r v1 = -v2 and v3 = 0, o r v1 + v2 and v1 a r e o f opposite s i g n and v3 = 0. The s e t A(v,l) f o r these t h r e e cases i s , r e s p e c t i v e l y iUIUl

= 0, u2

u3

> 0,

u4 2 01

( 5.3.48)

hlUl

2 0 , u2 = 0, u3

> 0,

u4

> 01

( 5.3.49)

iUIUl

= u2 = 0, u3

> 0,

>o,

u4

> 01.

(5.3.50)

Furthermore,

v TABu = v1(u1-u2)

+

v2(-2ul+u2+u3-u4)

+

v3(u2-u3+u4) (5.3.51)

and as

v2 f 0 and v3

= 0 i n a l l t h e above cases, (5.3.51)

be made p o s i t i v e f o r a u

E

(ulul

=

can

u2 = 0, u3 > 0, u4 > 01.

Thus t h e c o n d i t i o n s o f Theorem 5.3.8

a r e s a t i s f i e d and t h e

system i s a r b i t r a r y - i n t e r v a l n u l l - c o n t r o l l a b l e . Now we note t h a t (5.3.29)

and (5.3.30)

change s i g n as a f u n c t i o n o f u

S

E

as

imply t h a t Bus must where as i s a subset of

186 s2.


But

I n our example t h i s can happen o n l y i f u1 = u2 = 0.

then (5.3.44)

i s n o t even n u l l - c o n t r o l l a b l e ,

l e t alone

a r b i t r a r y - i n t e r v a l nu1l - c o n t r o l l a b l e , and t h e s u f f i c i e n c y conditions o f Theorem 5.3.9 5.3.6

cannot be s a t i s f i e d .

Minimum Time Function

One o f t h e most important and i n t e n s i v e l y studied problems i n optimal c o n t r o l theory i s t h a t o f s t e e r i n g t h e s t a t e o f a l i n e a r dynamic system t o t h e o r i g i n i n minimum time.

Usually

one places enough r e s t r i c t i o n s and assumptions on t h e dynamic system and t h e c o n s t r a i n t s e t R t o ensure t h a t t h e minimum time f u n c t i o n T(x), i.e.

t h e minimum t i m e t o reach t h e o r i g i n

from s t a t e x, i s continuous i n an open neighbourhood of t h e origin.

I n t h i s s e c t i o n we show t h a t t h e c o n d i t i o n o f

a r b i t r a r y - i n t e r v a l n u l l - c o n t r o l l a b i l i t y o f t h e autonomous l i n e a r dynamic system i s necessary and s u f f i c i e n t f o r t h e c o n t i n u i t y o f t h e minimum t i m e f u n c t i o n . We need t h e f o l l o w i n g theorem which we quote i n s l i g h t l y modified form from [ll

.

THEOREM 5.3.10

Suppose t h a t R i s compact.

a measurable c o n t r o l f u n c t i o n ul(-) steers t h e s t a t e xo

E

w i t h ul(t)

Ift h e r e e x i s t s E s2 which

Rn t o t h e o r i g i n o f t h e s t a t e space i n

time tl then t h e r e e x i s t s a measurable c o n t r o l f u n c t i o n u ( ' ) with u(t)

E

Q which steers xo t o t h e o r i g i n i n minimum time T.

I n c i d e n t a l l y , note t h a t t h e converse o f Theorem 5.3.10

is

t r i v i a l l y true, v i z . i f xo can be steered t o t h e o r i g i n i n minimum t i m e T, then i t can be steered t o t h e o r i g i n i n f i n i t e time tl 2 T!


187

The above theorem implies that the minimum time function T(x) is defined in an open neighbourhood of x = 0 when fl is compact if and only if (5.1.1) is null-controllable. However, nullcontrollability is not sufficient to ensure that T(x) is continuous in an open neighbourhood of x = 0. Suppose that fl is compact. A necessary and sufficient condition for the minimum time function T(x) to be continuous in an open neighbourhood of the origin of the state space is that (5.1.1) subject to (5.1.2) is arbitraryinterval null-control lable.

TffEOREM 5 . 3 . 1 7

PROOF Note that T(0) = 0 and suppose that T(x) is continuous in an open neighbourhood of the origin. Then the definition of continuity implies that for each t > O there is an open neighbourhood of the origin, say V(t), such that all states in V(t) can be steered to the origin in time t, which is arbitrary-interval null-controllability. Suppose now that (5.1.1) subject to (5.1.2) is arbitraryinterval null-controllable. Then by Theorem 5.3.10 and subsequent remarks the minimum time function T(x) is defined in an open neighbourhood o f the origin. Furthermore arbitraryinterval null-controllability implies that for each t > 0 there is an open neighbourhood of the origin, say V(t>, such that all states in V(t) can be steered to the origin in time t. Owing to the continuous dependence of the solutions of the differential equation on the initial values we know that given a point x in the domain of definition of T(-) there is a neighbourhood of x, say i , such that all points y E can be steered in time T(x) to V(t). This then implies that all points y E can be steered in time T(x) + t to the origin.

188


It follows that for each t > 0 and x in the domain of T(.) there is a neighbourhood v" of x such that T(y) - T(x) < t for all y E which implies that T(x) is upper-semicontinuous in an open neighbourhood of the origin. Now the set of states R(t) which can be steered to the origin in time t > 0 is compact, convex, and varies continuously with t on t > 0, and ,.. R(tl) c R(t2), tl < t p (cf. Theorem 5.1.4, the proof of Theorem 5.1.6 and Theorem 5.3.1). Moreover, if the minimum time to steer x to the origin is T(x) we have that x E ai(T(x)). Also such an x cannot belong to the sets fi(T(x)-t), t > 0 as T(x) is the minimum time required to steer x to the origin. Therefore there exists for each t > 0 a neighbourhood of x such that T(y) 2 T(x)-t for all y E i, or T(x) - T(y) G t for all y E &, which implies that T(x) is lower-semicontinuous in an open neighbourhood of the origin. The fact that T(x) is both upper- and lower-semicontinuous in an open neighbourhood of the origin implies that it is continuous in an open neighbourhood of the origin.

-

The above theorem implies that Pontryagin's conditions (5.3.34) - (5.3.36) which are sufficient for arbitraryinterval null-controllability also guarantee the continuity of the minimum time function T(x). It also follows that our necessary and sufficient conditions for arbitrary-interval null-controllability are the minimal conditions required for the continuity of the minimum time function. 5.3.7

Further Necessary Conditions and Sufficient Conditions

In this section we present necessary conditions and sufficient conditions due to M. Pachter for arbitrary-interval null-

189

CONTROLLABIL ITY CONDITIONS c o n t r o l l a b i l i t y which do n o t r e q u i r e t h e assumption t h a t

C1 ( C o n i c h u l l (Convhull ( a ) ) ) i s f i n i t e l y generated. Furthermore t h e y a r e o f geometric c h a r a c t e r and y i e l d i n s i g h t i n t o t h e geometric aspects o f a r b i t r a r y - i n t e r v a l n u l l - c o n t r o l l a b i l i t y . We r e q u i r e t h e f o l l o w i n g p r e l i m i n a r y lemma and subsequent

observations. LEMMA 5.3.2

Suppose t h a t C i s a convex cone i n Rn and t h a t

S i s a subspace c o n t a i n e d i n C.

c

=

s t

Then

s'n

c.

(5.3.52)

s'

PROOF C l e a r l y S t n C E ( S + s ' ) fl C = C. To prove t h e converse we suppose t h a t c E C and c = c + c 2 where c1 E S 1 and c 2 E Then c 2 = c - cl, and -cl E S s i n c e S i s a subspace. As S E C we have -cl E C and t h e c o n v e x i t y o f C nC i m p l i e s t h a t c 2 E C. Consequently we have t h a t c E S t which i n t u r n i m p l i e s t h a t C 5 S t n C.

8.

s'

From t h i s p o i n t on we suppose t h a t S i s t h e l a r g e s t subspace contained i n C

8

Cl(Conichull(Convhull(Bs2))).

C l e a r l y we

have t h a t

s

=

cn

(-c)

( 5.3.53)

and an a l g o r i t h m f o r computing S i s a v a i l a b l e 181. Note t h a t Theorem 5.3.7 i m p l i e s t h a t (01 i s a s t r i c t subset o f S. Lemma 5.3.2 i m p l i e s t h a t C can be w r i t t e n as (5.3.54)

L I NEAR-QUADRATIC EXTENSIONS

190

fl C is a closed, convex, pointed cone. That s' tl C where is pointed follows from the fact that ($ n C) tl [-(?nC)] =

sl n c n

n (-c) = sl n c n (-c) n

(-.+I

SL n

=

sn sl

{O). Furthermore we note that (BConvhull ( a ) ) ' c s' and (BConvhull ($2)) = (C1 (Conichull (Convhull (BQ)) ) ) I

I

=

.

Before presenting our main theorem we define the following we denote by Xo the interior of notation. Given a set X c the set X relative to the subspace s', plus the set (0). We denote by {AIS) the smallest subspace invariant under A which contains the subspace S. It is well known that {AIS) = S t AS t + A n-1 S .

...

THEOREM 5.3.12 A necessary condition for arbitrary-interval null-controllability of (5.1.1) subject to (5.1.2) is that ((BConvhull(Q))')o

fl {AlSIl = (01.

( 5.3.55)

A sufficient condition for arbitrary-interval null -control 1 ability is that (BConvhull(Q))' fl {A[ S>l = t-0) which is equivalent, as 0 E BConvhull(Q) to the sufficient condition 0

E

-

Int(Convhull(BQ)t{AlS)).

(5.3.56)

cf. Theorem 5.3.4,

(5.3.57)

We first consider necessity. Let v E ((BConvhull(Q))l)o, with IlvII = 1. It follows that vTc < O for all c E C f l SL, c f O so that Lemma 5.3.1 implies that the set

PROOF

CONTROLLABILITY COND I T 1ONS A

A T = ( y ( v y = -1, y

V

E

C

n

191

$1 i s compact and convex and

n s'.

c l e a r t h a t Conichull(Av) = C

it i s

Then i t f o l l o w s because

o f t h e compactness o f Av t h a t t h e r e i s a t > 0 such t h a t v TeAT y Q 0 f o r a l l 'I, 0 Q T Q t and f o r a l l y E Av. Conseq u e n t l y by Theorem 5.3.2

t h e system (A, I,Av) i s n o t

a r b i t r a r y - i n t e r v a l n u l l - c o n t r o l l a b l e and by Theorem 5.3.5

s'),

t h i s i s t r u e a l s o o f t h e system (A, I, C n which i m p l i e s T AT t h a t t h e r e i s a t > 0 such t h a t v e y Q 0 f o r a l l T, 0 Q

'I

Q t and f o r a l l y

t o a subspace of eATTv

E+

for all

E

C

n

s'.

Now assume t h a t v belongs

9 i n v a r i a n t under AT. T

Then

2 0 so t h a t v TeAT s = 0 f o r a l l s

E

S and

>O.

Combining t h e two r e s u l t s w i t h t h e a i d o f Lemma 5.3.2 y i e l d s v TeAT c Q O f o r a l l T , 0 Q T Q t and a l l c E C

all

T

so t h a t Theorem 5.3.2

i m p l i e s t h a t t h e system i s n o t a r b i t r a r y -

i n t e r v a l nu1 l - c o n t r o l l a b l e .

Thus i t i s necessary f o r

arbitrary-interval null-controllabil ity that

T ( (BConvhull(C2))')' n ( l a r g e s t subspace i n v a r i a n t under A contained i n $) = (0). Reference [9] provides. t h e l a s t step, T v i z . t h e l a r g e s t subspace i n v a r i a n t under A c o n t a i n e d i n! 5

i s CA~SI'. Now we go on t o s u f f i c i e n c y . 0

+

(AIS)) which i s e q u i v a l e n t t o (Convhull(BC2) t ( A I S ) ) ' = (0). A s e t - t h e o r e t i c i d e n t i t y then E

Int(Convhull(BQ)

We assume t h a t

implies t h a t (Convhull(BQ))' Next we n o t e t h a t i f (5.3.58)

n

C A l S I l = CO).

( 5.3.58)

holds then S # (0).

Indeed, i f

S = (0) we would have from (5.3.58)

t h a t (Convhull(Bi2))' = (0)

192


which is equivalent to 0 E Int(Convhull(Bi2)). This then implies that 0 E Int(Convhull(R)) and R(B) = Rn so that S = Rn, which i s a contradiction, which means that (0) is a strict subset of S. As mentioned in the proof of necessity, (AISI1 is the largest subspace invariant under A T contained in $, which implies that for all v E (Convhull(BQ))' there is an i, 0 < i < n-1 and an s E S E C such that v TA j s = 0 for all s E S and j=O,l, ...,i-1 and v TA i -s > O . Also, if v t (Convhull(BQ))' then there is a y E C such that v Ty > 0. In either case, then, we have that for each v there exists a y E C and a t > 0 such that v TeAT y > 0, 0 < T < t which implies that the system ( A , I, C) is arbitrary-interval nullcontrollable. Theorem 5.3.5 then implies that the system (A, I, BQ) is arbitrary-interval null-controllable, which implies that the system (A, B, Q) is arbitrary-interval nullcontrollable. Conditions (5.3.56) and (5.3.57) are equivalent if 0 E Convhull(,BR) because in that case (Convhull(BR) t {AIS))' = (Convhull(BR))' tl {AlSl' - see Section 5.A. The following corollaries follow from the above theorem.

A necessary condition for arbitrary-interval COROLLARY 5.3.4 null-controllability o f (5.1.1) subject to (5.1.2) is that Rank(Q) = n. COROLLARY 5.3.5 Assume that 0 E Int(Convhull(R)). Then a necessary and sufficient condition for arbitrary-interval null-controllability o f (5.1.1) subject to (5.1.2) is that Rank(Q) = n.

Furthermore we note that the sufficient conditions of Theorem 5.3.12 are less stringent than those o f Theorem 5.3.9. Indeed, returning to the example of Section 5.3.5 we see that


S

=

193

linear span(

BConvhull ( Q ) = {y

E

(5.3.59)

3

R Iyl G y2, y2 > O , y3

and { A I S ) = linear span(

[ ;] 0

9

[-;1).

E

R 11 (5.3.60)

(5.3.61)

It follows that

{A~s}' = linear span( and

[

(5.3.62)

does not belong to (BConvhull(i2))'.

Therefore

condition (5.3.56) is satisfied and the system (5.3.44) subject to (5.3.45) is arbitrary-interval null-controllable. We note, however, that there is a gap between the necessary condition and the sufficient conditions o f Theorem 5.3.12.

' 0 0 A = 0 1 0 1 ,o 0

0 1 0 0 0 1 0 0

, B=14

(5.3.63)

and Q = linear span(

11

( 5.3.64)

194


we find that the sufficient conditions of Theorem 5.3.12 are not satisfied but it can be verified direct using Theorem 5.3.4 that the system (A, B, 0 ) is arbitrary-interval nullcontrol 1 able. 5.3.8

Further Special Cases

THEOREM 5 . 3 . 1 3 Assume that $ E R2. Then a necessary and sufficient condition for arbitrary-interval null-controllability of (5.1.1) subject to (5.1.2) is that (Convhull(Bi2))'

n {A~SI'

=

(01

(5.3.65)

which is equivalent, because of our assumption that Bu for some u E 0 , to 0

PROOF

E

Int(Convhull(BR)+{AlSI).

=

0

(5.3.66)

We need only prove the necessity of the condition, as sufficiency is already proved in Theorem 5.3.12. In fact all we need to prove is that all vectors in the boundary of (Convhull(Bi2))' are not in {AIS) 1 . Thus, suppose v E a(Convhull(Bi2))'. If v E {AIS)1 then since $ E R 2 either v is an eigenvector of AT or $ is invariant under A T . In the former case vTeAtc = eAtv Tc G 0 for all c E C and t > 0 so that we would not have arbitrary-interval nullcontrollability. Assume then that is invariant under A T . Then those vectors v E (Convhull(BR))' such that vTc < 0 for all c E C n 9, c f-0, are also in (AlS}' so that, following the proof of necessity in Theorem 5.3.12, there exists t > 0 such that v TeAT c GO for all c E C and T, 0 G T G t so that


195

we would not have arbitrary-interval null-controllability. Suppose that n = 3, i.e. that x E R3 . Then (5.3.65) and (5.3.66) are necessary and sufficient for arbitrary-interval null-controllability of (5.1.1) subject to COROLLARY 5.3.6

(5.1.2).

PROOF

Again we need only prove necessity. Arbitraryinterval null-controllability implies that C is not pointed so that the dimension of the largest subspace S is greater than or equal to 1. Hence the dimension of 2 is less than or equal to n - 1 = 2 and the corollary follows from Theorem 5.3.13. We remark that Corollary 5.3.6 allows us to always use Theorem 5.3.8 when n < 3. Indeed, as stated above, the dimension of S is not less than one and that of s' is not greater than 2. As every cone in R 2 is finitely generated so is 2 n C and as S is a subspace the cone C is finitely generated. 5.4

Conclusion

In the introductory section of this chapter we reviewed wellknown results on control 1 abi 1 i ty and nu1 1 -control 1 abi 1 i ty of autonomous linear systems and described and discussed the most important properties of reachable sets. Section 2 was devoted to a study of null-controllability when R and its convex hull do not contain zero in their interior. Specifically we proved Brammerls important theorem via Hajek's approach.

In Section 3 we introduced the important notion of arbitrary-

196


interval null-controllability and proved a number of preliminary necessary and sufficient conditions. Then we proved that in questions of arbitrary-interval nu1 1 -control1ability R can be replaced by the cone C = Cl(Conichull(Convhull(R))) and this led to the development of necessary conditions, such as C should be not pointed, and necessary and sufficient conditions when is a finitely generated cone. We showed by means of a sufficient condition that Pontryagin's condition of general position imp1 ies arbitrary-interval null-control lability. We also presented a number of illustrative examples. Next we proved the important result that the minimum time function is continuous in an open neighbourhood of the origin of the state space if and only if the system (A, By n), n compact, is arbitrary-interval nullcontrollable. This justifies further the introduction of the notion of arbitrary-interval null-controllabil ity. Following this we presented necessary conditions and sufficient conditions of a geometric nature which are due to M. Pachter. These conditions do not require that the cone C be finitely generated . The chapter provides a rather complete treatment of the 'controllability' of constrained, 1 inear, autonomous continuous-time, dynamic systems. 5.5 References

[ll

LEE, E.B. & MARKUS, L. Foundations of Optimal Control Theory. Wiley, New York, 1967.

[2]

BRAMMER, R.F. Controllability in Linear Autonomous Systems with Positive Control 1 ers. SIAM J . Control , 10, 1972, pp. 339-353.


197

SAPERSTONE, S.H. & YORKE, J.A. Controllability of Linear Oscillatory Systems Using Positive Controls. SIAM J. Control, 9, 1971, pp. 253-262. HiJEK, 0. A Short Proof of Brammer's Theorem. Unpublished preprint, 1975. HEYMANN, M. & STERN, R.J. Controllability of Linear Geometric ConsideraSystems with Positive Controls: tions. J. Math. Anal. Appl., to appear. PACHTER, M. & JACOBSON, D.H. Conditions for the Controllability of Constrained Linear Autonomous Systems on Time Intervals of Arbitrary Length. CSIR Special Report WISK 210, July 1976, 23 p. BOLTYANSKII, V.E. Mathematical Methods of Optimal Control, Holt, Rinehart, Winston-, New York, 1971, pp. 119-120. ECKHARDT, V. Theorems on the Dimension of Convex Sets. Linear Algebra and its Applications, 12, 1975, pp. 63-76. WONHAM, W.M. On the Matrix Riccati Equation of Stochastic Control. SIAM J. Control, 6, 1968, pp. 681-697. ROCKAFELLAR, R.T. Press, 1970.

Convex Analysis.

Princeton University

Appendix The following definitions and facts are used in Chapter 5; for further details see [lo]. The set C 5 Rn is a cone if for all c E C and all scalars a > 0 we have CIC E C. A cone C is convex if and only if C + C SC. A cone C is pointed if C l (-C)

=

(01.

A cone C is pointed if and only if there is a v that v Tc < 0 for all c E C, c f 0.

E

Rn such


198

The convex hull o f a set E R", denoted Convhull(R), i s the smallest convex set which contains 0 . The conic hull o f a s e t 52 c Rm, denoted Conichull (n), i s the smallest cone with vertex a t the origin which contains R. The closure o f a set Sl E Rm, denoted C1 ( a ), i s the smallest closed set containing n. If the set !J i s bounded, so i s Convhull(Q); i f the set n i s closed, so i s Convhull(C2). T h u s n compact implies t h a t Convhull(f2) i s compact. If the set S2 i s convex, so i s the set Conichul l(R).

Let B be a map B:Rm + Rn. W e denote by B(n) or Bn the image under B of the set R . Let T c _ R n Then T i , the p o l a r of T, i s defined as T' = Ix E R n IxTy < 0 for a l l y E TI. The set T' i s a closed 1 and convex cone, and if T i s a subspace then T' = T W e A If T i s convex then note t h a t T E TI' where T" = ( T I ) ' T' = I01 i f and only i f 0 E Int(T).

.

.

.

Let T , V c Rn. Then T' T ' n V' = (TtV)'.

n V' c (T+V)' and

if 0

E

T

n V then

6.

NEW APPROACHES TO FUNCTION MINIMIZATION

6.1

Introduction

The problem of function minimization is to find the minimizer i E R~ of a scalar function f(x),

f:Rn

+

R

1

.

(6.1.1)

Since the emergence of Fletcher and Powell's algorithm 111 in 1963 quasi-Newton methods have dominated the field of numerical algorithms for minimizing differentiable functions. Quasi-Newton methods iteratively approach the minimizer (or at least a stationary point) of f(x) via a recurrence relation of the type Xitl =

xi

- p i si g i

(6.1.2)

where p i E [O,m) is the scalar 'step length' at iteration i, Si is a symmetric n x n matrix which approximates the inverse of the second-derivative matrix fxx(x) of f(x) at the point x i' and gi E Rn is the transpose of the gradient row-vector fx(x) evaluated at the point xi. The name 'quasi-Newton' arises from the similarity of (6.1.2) to Newton's iteration formula (6.1.3) which in turn follows from minimizing the quadratic terms in the Taylor series expansion of f(x) in the neighbourhood of xi, viz. 199


200

f(x) =

foil fx(xi)(X-xi) t t(x-xi)T fxx(xi)(x-xi) t

t

(6.1.4)

higher-order terms.

Basically, then, most algorithms for function minimization are derived from a quadratic approximation of the objective function, f(x).

In this chapter we present two new approaches to function minimization which depart from quadratic approximations, or models, and which therefore exhibit characteristics and advantages not shared by conventional algorithms. We shall assume throughout that f(x) is twice-continuously differentiable in Rn, though in many instances this assumption may be weakened to once-continuous differentiability. 6.2 Homogeneous Model s In 1971 a simple observation led to a useful generalization Specifically we note that the of the quadratic model [l] quadratic function,

.

f(x) = $ ( ~ - i T) Q(x-i?)

t

;

(6.2.1)

where Q is an n x n symmetric positive semi-definite matrix and is a real scalar, may be written as f(x) = 4fx(x)(x-i) +

;

(6.2.2)

so that multiplying through by the factor 2 we obtain 2f(x) = fx(x)(x-i?)

t

2;

,

(6.2.3)

APPROACHES TO MINIMIZATION

20 1

We now g e n e r a l i z e t h i s model v i a t h e f o l l o w i n g d e f i n i t i o n .

DEFZNZT'TON 6 . 2 . 1

A function f ( x ) , f:Rn

-P

R1 i s s a i d t o be

homogeneous o f degree y i f i t s a t i s f i e s t h e e q u a t i o n y f ( x ) = fx(x)(x-2) + w where y

E

(6.2.4)

.

R

C l e a r l y t h e q u a d r a t i c f u n c t i o n (6.2.1)

i s homogeneous of

degree y = 2, b u t t h e r e a r e many o t h e r f u n c t i o n s which a r e homogeneous.

Notably t h e f u n c t i o n f ( x ) = [ ( X - ~ ) ~ Q ( X - ~p )>I ~0 ~

(6.2.5)

Quasi-Newton methods behave

i s homogeneous o f degree y = 2p.

p a r t i c u l a r l y p o o r l y when a p p l i e d t o (6.2.5)

because t h e

second d e r i v a t i v e m a t r i x a t i , f x x ( i ) , i s i d e n t i c a l l y zero whenever p

> 1.

R e l a t i o n t o Newton's Formula

6.2.1

I t i s i n s t r u c t i v e t o e x p l o r e t h e r e l a t i o n s h i p between Newton's

I f we d i f f e r e n t i a t e

formula and t h e homogeneous f u n c t i o n . (6.2.4)

w i t h r e s p e c t t o x we o b t a i n !3()0 =

1

7 fxx(x)(x-i)

+

1

Y g(x)

(6.2.6)

T where g ( x ) = f x ( x ) , which y i e l d s t h e f o l l o w i n g expression

-

-

+nv v

..

. m i.n i.m i.7 n m

t h n acciimnn

-

nt

^ I

.

+Iv\

202


Thus assuming t h a t f x x ( x ) i s i n v e r t i b l e we can l o c a t e t h e minimizer o f a homogeneous f u n c t i o n by s c a l i n g t h e 'Newton step'

by t h e f a c t o r (y-1).

I n c i d e n t a l l y a s i m i l a r r e s u l t was

obtained i n 1870 by Schroder f o r t h e c l o s e l y r e l a t e d problem o f f i n d i n g m u l t i p l e r o o t s o f a polynomial [21. The above r e l a t i o n s h i p between t h e scaled 'Newton s t e p ' and t h e homogeneous f u n c t i o n i m p l i e s , i f t h e second d e r i v a t i v e m a t r i x i s a v a i l a b l e and i s p o s i t i v e d e f i n i t e f o r a l l x

E

Rn,

t h a t t h e i t e r a t i o n formula

w i t h pi chosen so as t o minimize f(xi-pi

[fxx(xi)l-lg(x.))

1

would y i e l d an e f f e c t i v e a l g o r i t h m which would l o c a t e t h e minimum o f a homogeneous f u n c t i o n i n one step. has been borne o u t i n p r a c t i c e .

Indeed t h i s

On t h e o t h e r hand t h i s

feature i s l o s t i f f x x ( x ) i s n o t e x p l i c i t l y a v a i l a b l e

-

except on q u a d r a t i c f u n c t i o n s when quasi-Newton methods a c c u r a t e l y estimate t h e then constant second d e r i v a t i v e m a t r i x i n n steps. 6.2.2

Algorithms based upon Homogeneous Functions

The p o t e n t i a l usefulness o f t h e homogeneous f u n c t i o n (6.2.4) l i e s i n t h e f a c t t h a t t h e unknown parameters y, appear l i n e a r l y .

I f we d e f i n e

x and w

APPROACHES TO M IN IM IZATION

v A= xTg(x) TA

y

=

(6.2 .lo)

T

(6.2.11)

[g (x),f(x);-lI

aT

4

203

[iT,y,wl

(6.2.12)

we can write (6.2.4) as yTa = v

(6.2.13)

so that (n+2) linearly independent 'observations' y and the

corresponding values of v determine the vector of unknowns a. Thus the location of the minimum of a homogeneous function, its degree of homogeneity and its scaled minimum value can be determined from (n+2) data which require only the evaluation of f(x) and g(x). When seeking the minimizer of a nonhomogeneous function we specify an updating formula which generates successive approximations to the minimizer of f(x). Details of two algorithms which accomplish this via Householder's updating formula are in [l] , [3]. Good numerical results were obtained when using each algorithm.

x,

Recently Kowalik and Ramakrishnan [4] have taken a close look at the homogeneous approach and have made the important observation that the updating methods used in [l], [31 can be substantially improved upon. In fact they replace the Householder updating formula by a semi-triangular factorization which is numerically stable (in contrast to the Householder formula) provided that a pivoting strategy is used in the process of updating the factors. Their numerically stable realization performs substantially better than the original

204


homogeneous algorithms [l] , [31 and suggests that the homogeneous model will find widespread application. Two important other advantages of the model (6.2.4) are that it is not necessary to choose the 'step-size' so as to minimize f(x) along the search direction, and that the second derivative matrix fxx(x) need not be positive definite. 6.3 Differential Descent

The iteration formula (6.1.2) which is used in almost all quasi-Newton algorithms is rather simple. Specifically one determines a suitable step size pi along the fixed direction sigi

-

In this section we introduce a new approach which generalizes this iteration formula to the form xi+l

=

xi

t

Pi(Xi,t)

(6.3.1)

where f E R 1 plays the role of a 'generalized' step length. The search 'direction' pi(xi,t) is a non-linear function of the step length t. 6.3.1

Outline of the Approach

We consider the vector differential equation [51 dx a-f = -

g(x)

(6.3.2)

which is non-linear whenever f(x) is non-quadratic. The solution of (6.3.2) for different initial conditions defines a family of curves x( . ) which are normal to the contours of


205

constant function value, viz. f(x) = c.

If 2 is the unique minimizer of f(x) and if If(x)l IlxII -+ then the function

i f(x) V(x) L

-

=

-

as

(6.3.3)

f(2)

is positive definite and radially unbounded. derivative with respect to time is given by -(x) dV dt

+OD

Furthermore its

gT(x)cl(x)

(6.3.4)

It then follows from a which is negative whenever x +i. as well-known stability theorem of Liapunov that x(t) t m Consequently we wish to determine the asymptotic solution of the differential equation (6.3.2) having started at t = 0 from the initial condition +

-+

.

x ( 0 ) = xi

(6.3.5)

where xi is the best known approximation to i . Naturally we could integrate (6.3.2) from the initial condition (6.3.5) using any one of a number of integration techniques. However, for all interesting functions f(x) it turns out that (6.3.2) is a stiff differential equation and consequently most integration techniques perform satisfactorily only if small steps in t are taken. Botsaris 161 provides details of suitable numerical integration methods which take large steps while we pursue a different approach here. We approximate g(x) in the neighbourhood of xi by the

206


expression 9.

1

where

+

H.(x-xi)

(6.3.6)

1

Hi A= fxX(xi).

Equation (6.3.2)

(6.3.7)

i s t h e r e f o r e r e p l a c e d by

a dx = -

-

gi

Hix

+

Hixi,

X(0) =

X i

(6.3.8)

which has t h e s o l u t i o n x ( t ) = xi

+

(e

-tHi

(6.3.9)

-I)Hilgi.

Now i f we f i n d t h e minimum o f f ( x ( t ) ) w i t h r e s p e c t t o t (possibly a t

t

=

m)

we o b t a i n t h e new e s t i m a t e o f t h e m i n i -

m i z e r as xi+l

= xi

A c t u a l l y e q u a t i o n (6.3.10)

+ (e

-fHi

(6.3 . l o )

-I)Hflgi.

i s n o t used d i r e c t owing t o t h e Wecan, however,

f a c t t h a t i t r e q u i r e s t h a t Hi be i n v e r t i b l e .

p u t i t i n a d i f f e r e n t form by n o t i n g t h a t i f ul, t h e normalized e i g e n v e c t o r s and A1,. t h e symmetric m a t r i x Hi then

so t h a t (6.3.9)

becomes

. . ,An

...,un a r e

t h e eigenvalues of


207

-A .t

J -l u . u T gi. J j

(6.3.12)

If X j = 0 we can by L'Hospital's r u l e replace the undefined -X .t e J -1 i n equation (6.3.12) by -t. term 'j

The following lemma i s then of i n t e r e s t and importance. -A .t -A .t j: u j u i ] , w i t h e J -1 LEMMA 6.3.1 The matrix

[

-'

j=l

replaced by -t i f A = 0, i s negative d e f i n i t e f o r t j regardless of the value of X

>0

j*

PROOF -X,t

If X j

> 0 then

for t

-'

> 0,

e

-A .t

< 1 so

that

< 0. On the other hand i f A J. < 0 then f o r t A; -A .t J -xjt < O . Finally, i f X > 1 so t h a t again e j A;J we use L'Hospital's r u l e t o yield

which i s a l s o negative f o r t

> 0,

-'

> 0 , and the lemna i s proved. 6 > 0 such t h a t f ( x ( t ) ) < f ( x i )

There e x i s t s a Moreover, f o r small t , (6.3.12) i s f o r a l l t E (0,6). approximately LEMMA 6.3.2

= 0

208


= xi -

x(t) and provided t h a t Hi

(6.3.14)

tgi

i s p o s i t i v e d e f i n i t e we have t h a t f o r

large t x ( t ) = x .1

-

-1

(6.3.15)

Hi gi.

Therefore f o r small t, (6.3.12) y i e l d s a ' g r a d i e n t s t e p ' and f o r l a r g e t, a 'Newton s t e p ' .

PROOF

It i s easy t o v e r i f y t h a t

simply equal t o

-

gigi.

>

evaluated a t t = 0 i s

Consequently we have t h a t

d The e x i s t e n c e o f = f ( x ( t ) ) 6

df

0 so t h a t f o r a l l t

t h e n i m p l i e s t h a t we can f i n d a E

(0,6)

which, s i n c e t >: 0, i m p l i e s t h a t f(x(t))

-

f(xi)

Extensions of linear-quadratic control, optimization and matrix theory

Control Theory and Optimization I

Survey of matrix theory and matrix inequalities

Stochastic Processes, Optimization, and Control Theory

Control and optimization

Optimization and Optimal Control

Optimization and Control 001

Control and optimization

Cooperative Control and Optimization

Network Optimization and Control

Optimization and control of bilinear systems: Theory, algorithms, and applications

Linear Programming: Theory and extensions

Matrix Theory

Cooperative Control and Optimization (Applied Optimization)

Matrix Theory

Vector-valued optimization: Problems in control theory

Applied Optimal Control: Optimization, Estimation and Control

Applied Optimal Control: Optimization, Estimation, And Control

Matrix eigensystem routines - EISPACK guide extensions

Cooperative systems: control and optimization

Optimization and Control with Applications

Optimization and control with applications

Optimization algorithms on matrix manifolds

Optimization Algorithms on Matrix Manifolds

Optimization algorithms on matrix manifolds

Theory of vector optimization

Control and Optimization of Multiscale Process Systems

Theory of Vector Optimization

Density matrix theory and applications

Optimization Theory

Theory of Matrix Structural Analysis

Extensions of linear-quadratic control, optimization and matrix theory

Control Theory and Optimization I

Survey of matrix theory and matrix inequalities

Stochastic Processes, Optimization, and Control Theory

Control and optimization

Optimization and Optimal Control

Optimization and Control 001

Control and optimization

Cooperative Control and Optimization

Network Optimization and Control

Optimization and control of bilinear systems: Theory, algorithms, and applications

Linear Programming: Theory and extensions

Matrix Theory

Cooperative Control and Optimization (Applied Optimization)

Matrix Theory

Vector-valued optimization: Problems in control theory

Applied Optimal Control: Optimization, Estimation and Control

Applied Optimal Control: Optimization, Estimation, And Control

Matrix eigensystem routines - EISPACK guide extensions

Cooperative systems: control and optimization

Optimization and Control with Applications

Optimization and control with applications

Optimization algorithms on matrix manifolds

Optimization Algorithms on Matrix Manifolds

Optimization algorithms on matrix manifolds

Theory of vector optimization

Control and Optimization of Multiscale Process Systems

Theory of Vector Optimization

Density matrix theory and applications

Optimization Theory

Theory of Matrix Structural Analysis

Recommend Documents