Rational Matrix Equations in Stochastic Control (Lecture Notes in Control and Information Sciences)

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari 297 Springer Berlin Heidelberg NewYo...

Author: Tobias Damm

31 downloads 599 Views 2MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari

297

Springer Berlin Heidelberg NewYork Hong Kong London Milan Paris Tokyo

Tobias Damm

Rational Matrix Equations in Stochastic Control

13

Series Advisory Board A. Bensoussan · P. Fleming · M.J. Grimble · P. Kokotovic · A.B. Kurzhanski · H. Kwakernaak · J.N. Tsitsiklis

Author Dr. Tobias Damm Technische Universität Braunschweig Institut für Angewandte Mathematik 38023 Braunschweig Germany E-Mail: [email protected]

ISSN 0170-8643 ISBN 3-540-20516-0

Springer-Verlag Berlin Heidelberg New York

Damm, Tobias. Rational matrix equations in stochastic control / Tobias Damm. p. cm. -- (Lecture notes in control and information sciences, ISSN 0170-8643 ; 297) Includes bibliographical references and index. ISBN 3-540-20516-0 (acid-free paper) 1. Stochastic control theory. 2. Matrices. I. Title. II. Series. QA402.37.D36 2004 629.8 312--dc22

2003066858

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in other ways, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under German Copyright Law. Springer-Verlag is a part of Springer Science+Business Media springeronline.com © Springer-Verlag Berlin Heidelberg 2004 Printed in Germany The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Data conversion by the author. Final processing by PTP-Berlin Protago-TeX-Production GmbH, Berlin Cover-Design: design & production GmbH, Heidelberg Printed on acid-free paper 62/3020Yu - 5 4 3 2 1 0

Introduction

One often gets the impression that [the algebraic Riccati] equation in fact constitutes the bottleneck of all of linear system theory. J. C. Willems in [201]

Robust and optimal stabilization A primary goal in linear control theory is stabilization, while plant variability and parameter uncertainties are major difficulties. Clearly, a linear model can describe reality at best locally, i.e. only as long as the state of the modelled system is close to the equilibrium state of linearization. It is the task of stabilization to keep the state within a neighbourhood of the equilibrium. Nevertheless, any mathematical model can only describe reality approximately, since one always has to rely on simplifying assumptions and can never measure the parameters with absolute accuracy. Therefore, stabilization always has to take account of possibly time-varying parameter uncertainties in the linearized model. A stabilization strategy is usually called robust, if it copes with parameter uncertainties of a certain class. The problem of robust stabilization is an active topic of current research and has produced a vast amount of work over the past about forty years with several thousands of publications and numerous textbooks. An important branch of stabilization theory is concerned with optimal stabilization. Among all stabilizing controllers, one selects the one that minimizes a given cost functional. By an adequate choice of this cost functional, one tries to represent certain performance specifications of the controlled system. For instance, in linear quadratic control theory, the cost functional is a positive semidefinite quadratic form of the state and the control vector; it punishes

VI

Introduction

both large deviations of the state from the equilibrium and large values of the control input, which might be too energy-consuming or even destroy the system. A major motivation for the use of such cost functionals, however, is their mathematical tractability. They offer some means to parametrize and compare different controllers. More recently, with the emergence of H ∞ -control theory around 1980, there has been growing interest in indefinite quadratic cost functionals, which can be used to guarantee certain disturbance attenuation and robustness properties of the controlled system. Parallel to the introduction of different cost functionals, there have always been attempts to model parameter uncertainties, and to find robust optimal stabilizers with respect to these uncertainties. Of course, there are many different possibilities to model uncertainty, and again the mathematical tractability has to be taken into account. One common approach is to consider intervals of systems instead of one single nominal system. In other words, one specifies certain intervals that contain the system parameters and tries to solve the stabilization problem for a whole family of systems simultaneously. An outstanding result in this setup is the famous theorem by Kharitonov on interval polynomials [127], which has triggered off a whole line of research in this direction. In contrast to the interval approach, which treats each system in the interval with equal probability, one also might assume certain statistical properties of the parameters to be known, such that some parameter values in fact are more likely to occur than others. This leads to stochastic models like stochastic differential equations. Actually, one might argue, whether a stochastic system should be regarded as a deterministic system with parameter uncertainty or as a different nominal model itself. Linear control systems with multiplicative white noise were introduced in the works of Wonham, e.g. [211, 213]. Another very successful approach models parameter uncertainty as a disturbance system in a feedback connection with the nominal system. Important classes of parameter uncertainties can be modelled in this way (e.g. [87]). Using this setup, one can apply disturbance attenuation techniques to design robust stabilizers. This constitutes the connection between H ∞ -control and robust stabilization. It is not surprising that the different concepts of parameter uncertainty can be combined. For instance, one can consider intervals of stochastic systems, or stochastic systems with a disturbance system in a feedback connection (e.g. [163]). In the present work, we are concerned with optimal stabilization and disturbance attenuation problems for stochastic linear differential systems. These investigations were initiated by results of Hinrichsen and Pritchard in [107] and stand in a line with the above-mentioned robust stabilization problems. Our main object is the theoretical discussion and numerical solution of generalized Lyapunov and Riccati equations arising in this context. The main tools

Introduction

VII

are the theory of resolvent positive operators and Newton’s method. Let us briefly introduce these concepts. Mean-square stability and resolvent positive operators When dealing with stochastic systems, one has to make some decisions on the adequate interpretation of the model and an appropriate concept of stability. In fact, these issues are neither obvious nor completely settled in the literature. The problem is that, in order to keep the model tractable, one usually models the parameter uncertainty as white noise, which is an idealized stochastic process. The correct interpretation of this idealized process, however, depends on the true nature of the uncertainty. In our investigation, we will consider stochastic differential equations of Itô type and use the concept of mean-square stability, because this interpretation fits well in a worst-case scenario – as we will see in Chapter 1. Moreover, it leads to stability criteria that are very similar to those known from the deterministic case. While in the deterministic case stability can be judged from a standard Lyapunov equation A∗ X!+ XA!= Y!, we have to consider a generalized Lyapunov equation ∗

A X!+ XA!+

N O

i Ai∗ 0 XA0 = Y

i=1

in the stochastic case. We will interpret this generalized Lyapunov equation as a standard Lyapunov equation perturbed by some positive operator. Furthermore, we will see that the sum of a Lyapunov operator and a positive operator belongs to a special class of operators, called resolvent positive operators. Properties of resolvent positive operators are of central importance in stability and stabilization problems for stochastic linear systems. Therefore, we will spend quite some effort in the analysis of resolvent positive operators and Lyapunov operators. In particular, we will have to deal with the spectral theory of positive linear operators on a vector space ordered by a convex cone. Rational matrix equations and Newton’s method Stabilization problems for deterministic linear systems in continuous time lead to the famous algebraic Riccati equation ∗ ∗ 0 = A∗ X + XA + P0 − (S0 + XB)Q−1 0 (S0 + B X) ,

where the Hermitian matrix

0

P0 S0 M = S0∗ Q0

6

VIII

Introduction

determines the cost functional. The counterpart in stochastic control is a rational matrix equation of the form E > N N O O ∗ i∗ i i∗ i A0 XA0 − S0 + XB + A0 XB0 0 = A X!+ XA!+ P0 + i=1

> ×

Q0 +

N O i=1

i=1

E−1 >

B0i∗ XB0i

S0∗ + B ∗ X +

N O

B0i∗ XAi0

E .

i=1

By analogy, we will address this equation as a (generalized) Riccati equation. The difference between the standard and the generalized Riccati equation mirrors the difference between the standard and the generalized Lyapunov equation on a higher level. While the derivative of a standard Riccati operator is a standard Lyapunov operator, the derivative of a generalized Riccati operator is a generalized Lyapunov operator. This observation – together with the properties of resolvent positive operators – turns out to be fruitful both for a theoretical analysis of solutions to the generalized Riccati equation and for their iterative computation by Newton’s method. Outline of the book In Chapter 1, we introduce stochastic models and discuss some special properties which are relevant with respect to robustness issues. Our main goal in this chapter, however, is to motivate our notions of stability, stabilizability and detectability and to clarify their relation to the generalized Lyapunov and Riccati equation, which are in the center of our interest. Moreover, we produce a number of examples which serve as illustrations in the following chapters. In Chapter 2, we are concerned with optimal stabilization and disturbance attenuation problems for stochastic linear systems. We reformulate these problems in terms of generalized Riccati inequalities. Some results from the literature are extended and presented in a slightly more general situation. Chapter 3 is devoted to the study of resolvent positive operators. While, on the one hand, these considerations prepare the stage for the following chapters, we think, on the other hand, that the results in this chapter are of independent value. Therefore, we discuss resolvent positive operators in some more detail than is needed in our study of generalized Riccati equations. Major contributions of this chapter are results on the representation of certain operators between matrix spaces and on the numerical solution of linear equations with resolvent positive operators. Chapter 4 contains some of our main results. These are non-local convergence results for Newton’s method and modified Newton iterations applied to concave operators with resolvent positive derivatives. Moreover, we generalize a result on the use of double Newton steps. We have chosen the most general setup for our method of proof to work and formulated the result for the case of

Introduction

IX

an ordered Banach space. For illustration we have included some applications which are not related to stochastic control theory. Finally, in Chapter 5 we apply our results to solve different types of generalized Riccati equations. To this end, we introduce the notion of a dual generalized Riccati operator. The main technical problem in Chapter 5 consists in showing that the dual operator possesses certain concavity properties. Altogether, we obtain rather complete existence results for stabilizing solutions of generalized algebraic Riccati equations arising in optimal stabilization problems for regular stochastic linear systems. Concluding this chapter we resume the discussion of some of the examples presented in the first chapter and illustrate our theoretical results by numerical examples. In the appendix, we have collected some facts on Hermitian matrices and Schur-complements, which we use frequently. Mathematical background Basically, we can distinguish between five mathematical foundation pillars which our work is built on and which correspond to the partition into chapters. Of course, these mathematical fields cannot be separated strictly, and from a more detached point of view, one might just see them as topics in a larger field. But to give an orientation let us name these pillars and trace there recent origins. Firstly, we bear on the stability theory for stochastic differential equations. While the modern notion of stochastic integrals and differential equations can be traced back to the works of Kolmogorov 1931, Andronov, Vitt and Pontryagin 1934, Itô 1946, Gikhman 1955 and Stratonovich 1964 [136, 3, 117, 118, 81, 188] roughly within the years 1930–1960, the corresponding notions of stability seem to appear first in the seminal paper [126] by Kats and Krasovskij 1961 and have been worked out mainly by Khasminskij in the 1960s, whose results are collected in [130]. Other important sources are the work of Kushner 1967, collected in [143] and the collection of papers [35] edited by Curtain 1972. More recent references on the topic of stochastic differential equations are the books by Gikhman and Skorokhod 1972, Arnold 1973, Friedman 1975, Krylov 1980 and 1995, Ikeda and Watanabe 1981, Gard 1988, Karatzas and Shreve 1991, Da Prato and Zabczyk 1992, Kloeden and Platen 1995, and Oeksendal 1998, [82, 6, 73, 141, 142, 115, 80, 125, 38, 134, 156]. Secondly, we use the fundamental concepts of linear systems theory, such as linear state-space systems, feedback connection, optimal control, stabilizability, controllability, detectability and observability. These notions mainly date back to the profound work of Kalman, e.g. [119, 120, 121]. Further important sources are the textbooks by Brockett 1970, Rosenbrock 1970, Anderson and Moore 1971, Kwakernaak and Sivan 1972, Wonham 1979, Knobloch and Kwakernaak 1985, and Sontag 1998, [22, 170, 2, 144, 214, 135, 184], to name but a few. The more recent development of robust control theory has found its way e.g. into the books of Francis 1987, Ackermann 1993, Ba¸sar and Bernhard

X

Introduction

1995, Green and Limebeer 1995, Zhou, Doyle and Glover 1995, Hassibi, Sayed and Kailath 1999, Chen 2000, Dullerud and Paganini 2000, Trentelman, Stoorvogel and Hautus 2001, [70, 9, 87, 220, 96, 25, 60, 192]. We have already mentioned that the particular topic of stochastic linear control systems originates in the work of Wonham to be found e.g. in [213]. Mean-square stabilization problems for systems with state and input dependent noise have been discussed e.g. in papers by Sagirow 1970, Haussmann 1971, McLane 1971, Kleinman 1976, Willems and Willems 1976, Bismut 1976, Phillis 1983, Bernstein 1987, Bernstein and Hyland 1988, Sasagawa 1989, Tessitore 1992, Dr˘agan, Morozan and Halanay 1992, [171, 97, 150, 133, 204, 17, 164, 16, 15, 175, 190, 59]. Recent contributions to the field have been the results of El Bouhtouri and Pritchard 1993, Morozan 1995, Dr˘agan, Halanay and Stoica 1996, Hinrichsen and Pritchard 1996 and 1998, Biswas 1998, El Bouhtouri, Hinrichsen and Pritchard 1999, and Petersen, Ugrinovskii and Savkin 2000 [63, 154, 57, 106, 107, 18, 62, 163] on robust control of stochastic linear systems, as well as the results of Yong and Zhou 1999 [218] on control problems with indefinite input weight cost. Thirdly, we rely on the spectral theory of positive linear operators in ordered vector spaces, which originates with the works of Perron 1907 and Frobenius 1908, [162, 74]. A major contribution to this field was the comprehensive paper [140] by Krein and Rutman 1950, who extended the results of Perron and Frobenius to a general setting of positive operators on ordered vector spaces. These results were developed further e.g. in a number of papers by Krasnoselskij and Schaefer. Good references are the monographes by Krasnoselskij 1964, Schaefer 1971, Berman, Neumann and Stern, 1989, Krasnoselskij, Lifshits, and Sobolev 1989, Berman and Plemmons 1994, and the survey paper by Vandergraft 1968, [138, 177, 13, 139, 14, 197]. Results on resolvent positive operators, which are of particular importance for our purpose, have been obtained by Schneider 1965, Elsner 1970 and 1974, Arendt 1987, and Fischer, Hinrichsen and Son 1998 in [180, 65, 66, 5, 68]. Some of these results will be generalized slightly. Moreover, we will have to deal with special classes of operators between matrix spaces, which have attracted quite some interest during the last 30 years. The relevant references will be given later. Fourthly, we build upon earlier results on Newton’s method applied to operator equations in ordered Banach spaces. This theory originates with the work of Kantorovich 1948 in [122], which can e.g. be found in the textbook [123] by Kantorovich and Akilov 1964. Kantorovich’s results were developed further in various directions. Of major importance for us are the paper [196] of Vandergraft 1967, who made use of inverse positive operators and the notion of convexity to prove the convergence of a Newton sequence, and the paper [131] of Kleinman 1968, who was the first to prove a non-local convergence result for Newton’s method applied to a Riccati equation. Kleinman’s result was extended in a series of papers by Wonham 1968, Hewer 1971, Coppel 1974, Guo and Lancaster 1998, [212, 102, 34, 94] and others. As a main result, we will give a very general form of this non-local convergence theorem, which

Introduction

XI

applies at once to Riccati equations from deterministic and stochastic control both in continuous and discrete time. Finally, we make contributions to the vast field of algebraic Riccati equations, where we have to deal both with the existing results and the standard tools used in this area. Like the modern theory of linear control systems, the theory of matrix Riccati equations originates with the work of Kalman 1960 [119]. Since then it has grown into a very active, independent field of research. It is impossible to give complete account of all directions pursued in this field; we refer to the collection of papers [19] edited by Bittanti, Laub and Willems 1991, and the monographes, [167, 151, 145, 116] by Reid 1972, Mehrmann 1991, Lancaster and Rodman 1995, and Ionescu, Oara and Weiss 1999. The role of algebraic Riccati equations and inequalities in H ∞ -optimal control has been the topic of e.g. the paper [221] by Zhou and Khargonekar 1998, and Scherer’s thesis 1990, [178]. Furthermore, we name a few papers that are closely related to our investigations. Firstly, we have the outstanding paper by Willems 1971, and subsequently the works of Coppel 1974, Molinari 1977, Churilov 1978, Shayman 1983, and Gohberg, Lancaster and Rodman 1986, [201, 34, 152, 31, 182, 183, 84] on the existence of largest and stabilizing solutions. Monotonicity properties of largest solutions are of particular interest for us and have been discussed in papers by Wimmer 1976–1992, Ran and Vreugdenhil 1988, Freiling and Ionescu 2001 and Clements and Wimmer 2001, [207, 208, 210, 166, 71, 32]. Generalized Riccati equations of the type discussed here have been considered e.g. in the work of Wonham 1968, de Souza and Fragoso 1990, Tessitore 1992 and 1994, Freiling, Jank and AbouKandil 1996, Dr˘agan, Halanay and Stoica 1997, Fragoso, Costa and de Souza 1998, Hochhaus, and partly Reurings [212, 52, 190, 191, 72, 58, 69, 111, 168]. Acknowledgements I would like to express my thanks to all those who took part in the development of my doctoral thesis, which laid the foundations of this work. First of all, I thank Prof. D. Hinrichsen for introducing me to the topic of rational matrix equations, for his guidance, control and feedback, and for teaching me the virtue of never being satisfied. I really enjoyed to work in the research group “Regelungssysteme” at the University of Bremen, and I would like to thank all members and visitors of the group for creating such a pleasant atmosphere. In particular, I thank Fabian Wirth for his valuable comments, and Michael Karow, for many discussions on mathematics and philosophy. Moreover, I thank Peter Benner and Heike Faßbender for private lessons in numerical analysis and for thinking about my further career. My special thanks go to Prof. H. Wimmer from W¨ urzburg University, who taught me the essentials of Riccati equations and always had interesting replies to my questions. Thanks also to Hans Crauel from the TU of Ilmenau for disucssions on stochastic analysis and for pointing out some flaws to me. Furthermore, I wish to thank Prof. A. J. Pritchard from Warwick University, for reviewing my thesis.

XII

Introduction

Finally, I gratefully acknowledge the Graduiertenkolleg “Komplexe Dynamische Systeme” and the FNK at the University of Bremen for their financial support.

Contents

Introduction .!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.

V

1

Aspects of stochastic control theory . !.! . !.! . !.! . !.! . !.! .!.! .!.! .!.! .!.!.!.!.!.!.! 1.1 Stochastic integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Stochastic differential equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Itô’s formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Linear stochastic differential equations . . . . . . . . . . . . . . . . . . . . . 1.5 Stability concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Mean-square stability and robust stability . . . . . . . . . . . . . . . . . . 1.7 Stabilization of linear stochastic control systems . . . . . . . . . . . . . 1.7.1 Stabilizability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7.2 A Riccati type matrix equation . . . . . . . . . . . . . . . . . . . . . 1.8 Some remarks on stochastic detectability . . . . . . . . . . . . . . . . . . . 1.9 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.9.1 Population dynamics in a random environment . . . . . . . . 1.9.2 The inverted pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.9.3 A two-cart system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.9.4 An automobile suspension system . . . . . . . . . . . . . . . . . . . 1.9.5 A car-steering model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.9.6 Satellite dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.9.7 Further examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 5 6 7 10 19 21 22 26 28 31 31 32 33 34 36 39 41

2

Optimal stabilization of linear stochastic systems .!.!.!.!.!.!.!.!.!.! 2.1 Linear quadratic optimal stabilization . . . . . . . . . . . . . . . . . . . . . 2.2 Worst-case disturbance: A Bounded Real Lemma . . . . . . . . . . . . 2.3 Disturbance attenuation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Disturbance attenuation by static linear state feedback . 2.3.2 Systems with bounded parameter uncertainty . . . . . . . . . 2.3.3 Disturbance attenuation by dynamic output feedback . .

43 43 47 50 51 56 58

XIV

Contents

3

Linear mappings on ordered vector spaces .!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.! 3.1 Ordered Banach spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Positive and resolvent positive operators . . . . . . . . . . . . . . . . . . . . 3.2.1 Spectral properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Equivalent characterizations of resolvent positivity . . . . . 3.3 Linear mappings on the space of Hermitian matrices . . . . . . . . . 3.3.1 Representation of mappings between matrix spaces . . . . 3.3.2 Completely positive operators . . . . . . . . . . . . . . . . . . . . . . . 3.4 Lyapunov operators and resolvent positivity . . . . . . . . . . . . . . . . 3.5 Linear equations with resolvent positive operators . . . . . . . . . . . 3.5.1 Direct solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 The case of simultaneous triangularizability . . . . . . . . . . . 3.5.3 Low-rank perturbations of Lyapunov equations . . . . . . . . 3.5.4 Iterative solution with different splittings . . . . . . . . . . . . . 3.5.5 Ljusternik acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Recapitulation: Resolvent positivity, stability and detectability 3.7 Minimal representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

61 61 62 63 67 68 69 73 75 83 83 84 85 89 91 93 94 98

4

Newton’s method .! .!.! . !.! . !.! . !.! . !.! . !. . !. . !. . !. . !. . !. . !. . !. . !. . !. . !.! . !.! . !.! . !.! . !.! .!103 4.1 Concave maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.2 Resolvent positive operators and Newton’s method . . . . . . . . . . 105 4.2.1 A modified Newton iteration . . . . . . . . . . . . . . . . . . . . . . . . 110 4.3 The use of double Newton steps . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.4 Illustrative applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.4.1 L2 -sensitivity optimization of realizations . . . . . . . . . . . . . 116 4.4.2 A non-symmetric Riccati equation . . . . . . . . . . . . . . . . . . . 118 4.4.3 The standard algebraic Riccati equation . . . . . . . . . . . . . . 120

5

Solution of the Riccati equation . !. . !. . !. . !. . !. . !. . !.! . !.! . !.! . !.! .!.! .!.! .!.!.!123 5.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 5.1.1 Riccati operators and the definite constraint X!∈ dom+ R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 5.1.2 The indefinite constraint X!∈ dom± R . . . . . . . . . . . . . . . 126 5.1.3 A definiteness assumption . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.1.4 Some comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.1.5 Algebraic Riccati equations from deterministic control . 130 5.1.6 A duality transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5.1.7 A regularity transformation . . . . . . . . . . . . . . . . . . . . . . . . . 133 5.2 Analytical properties of Riccati operators . . . . . . . . . . . . . . . . . . . 134 5.2.1 The Riccati operator R . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.2.2 The dual operator G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.3 Existence and properties of stabilizing solutions . . . . . . . . . . . . . 145 5.3.1 The Riccati equation with definite constraint . . . . . . . . . 145 5.3.2 The Riccati equation with indefinite constraint . . . . . . . . 155

Contents

XV

5.4 Approximation of stabilizing solutions . . . . . . . . . . . . . . . . . . . . . . 162 5.4.1 Newton’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 5.4.2 Computation of stabilizing matrices . . . . . . . . . . . . . . . . . 164 5.4.3 A nonlinear fixed point iteration . . . . . . . . . . . . . . . . . . . . 165 5.5 Numerical examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 5.5.1 The two-cart system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 5.5.2 The automobile suspension system . . . . . . . . . . . . . . . . . . . 170 5.5.3 The car-steering problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 A

Hermitian matrices and Schur complements .!.! .!.! .!.!.!.!.!.!.!.!.!.!.!181

References .! .!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!.!185 Index .!.!.!.!.!.!.!.!.!.!.! .!.! .!.! . !.! . !.! . !.! . !.! . !. . !. . !. . !. . !. . !. . !. . !. . !. . !. . !. . !. . !. . !. . !.! . !.! . !.! . !197 Notation . !. . !. . !. . !. . !. . !. . !. . !. . !. . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !.! . !. . !. . 1! 99

1 Aspects of stochastic control theory

In the present chapter, we introduce linear stochastic control systems and the corresponding notions of stability, stabilizability and detectability. In the first part of this chapter our aim is to make the model of stochastic linear systems plausible and to give a heuristic derivation of stability criteria in terms of generalized Lyapunov equations. To this end, we have to take a look at the construction of the Itˆ o integral and Itˆ o’s formula. But – to avoid misunderstandings – it should be stressed that we do not aim at a profound derivation of these concepts. We only wish to motivate them in the context of robust control theory. For the exact arguments, we refer to the standard textbooks named in the introduction (in particular [6, 141, 142, 80, 125, 134, 156]). In the second part, basically starting with Example 1.5.8, we develop some more original ideas, which on the one hand aim at a justification of the given model in robustness analysis, and on the other hand yield a direct approach to the generalized algebraic Riccati equation, which constitutes the main topic of this work. Finally, we present a number of examples from the literature, some of which will be discussed on a numerical level in the last chapter.

1.1 Stochastic integrals Let (Ω, F , µ) be a probability space. By a k-dimensional random variable we mean a µ-measurable, integrable function ξ : Ω → Kk . The expectation of a random variable ξ is < Eξ = ξ(ω)dµ(ω) . Ω

The space of square integrable k-dimensional random variables is denoted by L2 (Ω, Kk ). A one-parameter family {ξ(t)}t∈[0,T ] (T > 0) or {ξ(t)}t∈R+ of random variables ξ(t) ∈ L2 (Ω, Kk ) is called a stochastic process. T. Damm: Rational Matrix Equations in Stochastic Control, LNCIS 297, pp. 1–42, 2004. Springer-Verlag Berlin Heidelberg 2004

2


Definition 1.1.1 A Wiener process w(t) : Ω → R, t ≥ 0 is a stochastic process with the following defining properties: (i) w(0) = 0 (ii) ∀0 < t1 < t2 < t3 < t4 : w(t1 )− w(t2 ) and w(t3 )− w(t4 ) are independent. (iii) ∀0 ≤ t1 ≤ t2 : w(t2 ) − w(t1 ) is normally distributed with zero mean and variance t2 − t1 ; that means, for all x ∈ R @ G < x 1 1 t2 L µ(w(t2 ) − w(t1 ) < x) = exp − dt . 2 t2 − t1 2π(t2 − t1 ) −∞ (iv) The paths of w(t) are almost surely continuous. The Wiener process is a standard model for the frictionless Brownian motion, i.e. the erratic motion of a microscopic particle in a liquid caused by random collisions with the molecules of the liquid. After each collision the particle obtains a different direction and different speed, independent of the direction and speed before. The increments of the Wiener process on small intervals of time can thus be interpreted as the forces acting on the particle during the collisions. The action of these forces is assumed to be completely random, with no favoured frequency. It is customary to take the increments of the Wiener process on infinitesimal intervals as a model for random perturbations, called white noise. Given a nominal differential equation ˙ x(t) = f (t, x(t)) one considers a randomly perturbed differential equation x(t) ˙ = f (t, x(t)) + f0 (t, x(t)) · ”noise” . If we attempt to substitute the noise term by the increments of the Wiener process, we are led to an integral equation of the form < t < t x(t) = f (s, x(s)) ds + f0 (s, x(s)) dw(s) . 0

0

Unfortunately, by its defining properties, the Wiener process is almost surely not differentiable at any point; moreover it is not even of bounded variation. This makes the interpretation of infinitesimal increments of the Wiener process rather problematic. In particular the second integral in the perturbed equation cannot be understood in the Riemann-Stieltjes sense. The effort to overcome this problem has lead to the so-called stochastic calculus, which has found its way into numerous monographs. We cannot discuss the definition of the stochastic integral here in detail. But there are some important concepts we cannot do without. First of all, certain measurability assumptions have to be imposed on the stochastic integrand. Together with the Wiener process w(t) we consider an increasing sequence

1.1 Stochastic integrals

3

(Ft )t∈R+ of σ-algebras Fs ⊂ Ft ⊂ F for s ≤ t, such that for all s ≤ t the random variable w(s) is measurable with respect to Ft , and Fs is independent of the random variable w(t) − w(s). A function f0 : R+ × Ω → Kk is called non-anticipating (with respect to Ft ) if for all t ≥ 0 the random variable f0 (t, ·) : Ω → Kk is Ft -measurable. Loosely speaking, this property says that for all t the random variable f0 (t, ·) does not know anything about the future behaviour of w(t). For instance the process f0 (t, ·) = w(2t) is anticipating with respect to Ft , whereas all deterministic functions of w(t) are non-anticipating. For any T ∈ R with 0 < T ≤ ∞ we define L2w = L2w ([0, T ]; L2(Ω, Kk )) to be the Hilbert space (e.g. [134]) of stochastic processes x(·) = (x(t))t∈[0,T ] that are non-anticipating with respect to (Ft )t∈[0,T ] and satisfy >< E < <x(·) . < t m O Φ(s)−1 a(s) − Ai0 ai0 (s) ds x(t) = Φ(t) x0 + 0

< +

−1

Φ(s)

0

@

N O

N O

Ai0 x(t)x(t)∗ Ai∗ 0 dt

i=1

E∗ ∗

x(t)x(t)

Ai0

dwi (t)

.

i=1

Here we make use of the fact that by Theorem 1.2.1 the second moment E<x(t) 0 : LA (X) + ΠA0 (X) < 0. ∀Y < 0 : ∃X > 0 : LA (X) + ΠA0 (X) = Y .

Proof: We first show the equivalence of (i), (ii), and (iii). As mentioned above, it follows from Theorem 1.4.3 that system (1.13) is asymptotically meansquare stable, if and only if the deterministic linear autonomous differential equation ∗ )(X) X˙ = (L∗A + ΠA 0

(1.23)

is asymptotically – and thus exponentially – stable. This holds, if and only if ∗ σ(L∗A + ΠA ) = σ(LA + ΠA0 ) ⊂ C− , 0

14


in which case (1.13) is also exponentially mean-square stable. Now we prove ‘(iv)⇒(ii)’: Let numbers c1 , c2 , c3 > 0 and a positive definite matrix X with c1 I ≤ X ≤ c2 I be given, such that LA (X) + ΠA0 (X) < −c3 I. If x(t) = x(t; x0 ) is an arbitrary solution of (1.13) then we have in analogy to the proof of Theorem 1.4.3 d(x(t)∗ Xx(t)) = (dx(t))∗ Xx(t) + x(t)X(dx(t)) + (dx(t))∗ X(dx(t)) = x(t)∗ (LA (X) + ΠA0 (X)) x(t) dt +

N O

? F i x(t)∗ Ai∗ 0 X + XA0 x(t) dwi .

i=1

In the integral form of this equation the last summand has zero expectation because Ex(t)∗ Xx(t) is bounded on any finite interval. Hence @< t G Ex(t)∗ Xx(t) − Ex∗0 Xx0 = E x(s)∗ (LA (X) + ΠA0 (X)) x(s) ds (1.24) 0

< t ≤ −c3 E x(s)∗ x(s) ds 0 < c3 t E (x(s)∗ Xx(s)) ds . ≤− c2 0 By Gronwall’s Lemma (e.g. [184]) we have E<x(t) 0 . 0

Setting P (t) = E(Φ(t)Y Φ∗ (t)) we find (again like in the proof of Theorem 1.4.3) that N O d i Ai∗ P (t) = A∗ P (t) + P (t)A + 0 P (t)A0 dt i=1

= (LA + ΠA0 )(P (t)) . Thus it follows from the linearity of the integral

< (LA + ΠA0 )(X) = =

1.5 Stability concepts ∞

0 for x O= 0, such that n

∀x ∈ R \{0} :

φ3x f (x)

+

n O

(i)

(i)

f0 (x)T φ33x f0 (x) ≤ 0 ,

(1.26)

i=1

where φ3x and φ33x denote the gradient and the Hessian of φ at x. Then the trivial solution of (1.8) is stochastically stable. If, moreover, the inequality (1.26) is strict, then the trivial solution of (1.8) is asymptotically stochastically stable. For the linear system (1.13) the quadratic function φ(x) = x∗ Xx obviously satisfies the assumptions of Theorem 1.5.6 if X satisfies condition (iv) in Theorem 1.5.3. Corollary 1.5.7 If system (1.13) is asymptotically mean-square stable, then it is asymptotically stochastically stable. The converse is not true, as the following example shows (cf. also [6]). Moreover, the example demonstrates that one can come to rather contradictory conclusions about the stability of a given system. Example 1.5.8 For different coefficient matrices A, A0 ∈ Rn×n we analyze stochastic and mean-square asymptotic stability of the linear stochastic Itô equations dx = Ax dt + A0 x dw and

(1.27)

1.5 Stability concepts

17

= 1 D dx = A + A20 x dt + A0 x dw . (1.28) 2 Note that (1.28) would be the equivalent Itˆ o equation, if we understood (1.27) in the Stratonovich sense. (i)

Stochastic stabilization of (1.27) by noise: Let n = 1 and A = a, A0 = a0 ∈ R. By Theorem 1.5.3 equation (1.27) is asymptotically mean-square stable, if and only if a + 12 a20 < 0. To analyze stochastic stability of (1.27) we apply Proposition 1.4.2 and write the fundamental solution Φ(t) explicitly as @= G 1 D a − a20 t + a0 w(t) . Φ(t) = exp 2 Since x(t; x0 ) = Φ(t)x0 it follows that (1.27) is stochastically stable, if = ε D =0, (1.29) ∀ε > 0 : lim µ sup Φ(t) > x0 →0 |x0 | t≥0 which is equivalent to µ(supt≥0 Φ(t) < ∞) = 1. Since Φ is almost surely continuous a sufficient condition for (1.29) is µ(limt→∞ Φ(t) = 0) = 1, which is equivalent to asymptotic stochastic stability. Hence (1.27) is asymptotically stochastically stable, if and only if D = 1 (1.30) µ lim (a − a20 )t + a0 w(t) = −∞ = 1 . t→∞ 2 By the strong law of large numbers (e.g. [6]) we have = D a0 w(t) µ lim =0 =1. t→∞ t

Therefore (1.30) holds true if and only if a − 12 a20 < 0. By the same arguments (1.28) is asymptotically mean square stable, if and only if a + a20 < 0 and asymptotically stochastically stable, if and only if a < 0. (ii) Stability of (1.28) independent noise: 0 6 0 1 Let n = 2 and A = aI, A0 = a0 with a, a0 ∈ R. −1 0 We have LA = 2aI and σ(ΠA0 ) = ±a20 . By Theorem 1.5.3 equation (1.27) is asymptotically mean-square stable, if and only if a + 12 a20 < 0. Since A20 = a20 I, the fundamental solution Φ(t) is (by Proposition 1.4.2) @= G 1 2D Φ(t) = exp a + a0 t + A0 w(t) . 2 The condition a + 12 a20 < 0 therefore is also necessary for stochastic asymptotic stability in this case. Similarly, (1.28) is both asymptotically mean square stable and asymptotically stochastically stable, if and only if a < 0.

18


(iii) Stability of (1.27) and (1.28) independent of noise: 0 6 01 If n = 2, A = aI, and A0 = a0 with a, a0 ∈ R then LA = 2aI and 00 σ(ΠA0 ) = 0. Hence (1.27) is asymptotically mean-square stable if and only if a < 0. Since A20 = 0 this is also necessary for stochastic stability. The same holds for (1.28). (iv) Mean-square of 6 0 stabilization 6 by noise: 0 (1.28) 1 0 0 1 , A0 = a0 Let A = with a0 ∈ R. 0 −2 −1 0 Since A (and thus LA ) is unstable, (1.27) is not mean-square stable for any a0 . Equation (1.28), however, is asymptotically mean-square stable if and only if the matrix I ⊗ (A −

a0 a0 I) + (A − I) ⊗ I + A0 ⊗ A0 2  2  2 − a20 0 0 a20  0 −1 − a20 −a20  0  =  0  −a20 −1 − a20 0 0 0 −4 − a20 a20

is stable. The eigenvalue problem for this matrix decomposes in an obvious way, and all eigenvalues are negative, if and only if 0 6 2 − a20 a20 det = −8 + 2a20 > 0 , a20 −4 − a20 i.e. |a0 | > 2. Remark 1.5.9 The previous example is both surprising and alarming. Whether the noise term acts stabilizing or destabilizing depends essentially on the interpretation. For Itˆ o equations the noise term always tends to destroy mean-square stability whereas it can be advantageous for stochastic stability; for Stratonovich equations the noise term can even enforce both stochastic and mean-square stability (see e.g. [7]). While this effect of stochastic stabilization by noise is an interesting topic in the qualitative theory of dynamical systems, we conclude that stochastic stability is not an appropriate concept for robustness analysis. Similarly the effect of the noise term does not match with our intuition, if we consider the Stratonovich interpretation. We conclude that the Itˆ o interpretation of stochastic differential equations together with the concept of mean-square stability offers an acceptable model for robustness analysis of linear systems with random parameter vibrations. In the following we will sometimes use the abbreviated term stable instead of exponentially mean-square stable.

1.6 Mean-square stability and robust stability

19

1.6 Mean-square stability and robust stability It lies in the nature of uncertainty that one can never be sure about it. If we model parameter uncertainty as white noise, interpret the formal stochastic differential equation (1.1) in the Itˆ o sense and study mean-square stability, then we have already made three rather specific assumptions, which can only be satisfied in an idealized set-up. Hence we have to check, whether our stability results are in some sense robust with respect to these assumptions. We have already seen that the notion of mean-square stability is well fit for a worst-case analysis. Let us now discuss some aspects of parameter uncertainty. If we consider a linear differential equation x˙ = Ax , as a model for some physical system, then this nominal model describes the true dynamics of the system only approximately and locally. Three important sources of error are the following. Firstly it is often impossible to determine the values of the system parameters exactly; secondly nonlinear effects have been neglected; thirdly the physical system is subject to exogenous disturbances, which might lead to changes in its dynamics. In order to cope with these problems one can allow for parameter uncertainties in the nominal system. Corresponding to the different error sources, the nature of the uncertainties can be different. If a linear time-invariant model is sufficient, but the true values of some parameters cannot be determined exactly, then one can consider a whole class of systems x˙ = (A + A0 )x,

A0 ∈ A0 ⊂ Rn×n .

The set A0 represents all possible deviations of the true system parameters from those of the nominal system. If, however, we wish to cope with nonlinear or exogenous effects, it is, in general, not sufficient to model the parameter uncertainties independent of t and x. We are rather led to a model of the form E > N O i δi (t, x)A0 x , (1.31) x˙ = A + i=1

where the Ai0 represent the uncertain parameters and the δi are unknown scalar functions. Of course, in general we have to impose some restrictions on the functions δi ; on the one hand we have to ensure that the differential equation can be solved in some sense; on the other we can not allow for arbitrarily large δi in general, if we expect (1.31) to be stable. One possibility is to assume the δi to be measurable (deterministic) functions bounded in norm, i.e. max(t,x)∈R×Rn |δi (t, x)| ≤ di for some given di ≥ 0. Then a typical question is whether (1.31) is stable for all δi satisfying this constraint.

20


Or one might – as we basically did in the previous sections – regard the δi as stochastic processes with certain statistical properties. In fact we modelled δi = w˙ i which is the formal derivative of the Wiener process, called white noise. This led us to the Itô-equation dx = Ax dt +

N O

σi Ai0 x dwi .

(1.32)

i=1

which allows for arbitrarily large deviations from the nominal system if they occur with sufficiently small probability. But on the other hand this uncertainty model requires that in the mean the nominal system is exact. We show now that if system (1.32) is exponentially mean-square stable with a given decay rate, then system (1.31) can tolerate uncertainties up to a certain bound. This was observed in [16], and we will extend the idea in Section 2.3.2 to the disturbance attenuation problem. The following simple binomial inequality is needed. n , V, W ∈ Kn×k and a, b ≥ 0 be arbitrary. Then Lemma 1.6.1 Let X ∈ H+

a2 V ∗ XV + b2 W ∗ XW ≥ c¯ V ∗ XW + c W ∗ XV

for all c ∈ C with |c| ≤ ab .

Proof: The assertion is obvious if ab = 0. Hence, assume b > 0. For X ≥ 0 and |c| ≤ ab we have G D∗ = c D @ =c |c|2 2 V +bW X V + b W + a − 2 V ∗ XV 0≤ b b b c V ∗ XW + c W ∗ XV ) , = a2 V ∗ XV + b2 W ∗ XW − (¯ which proves the assertion.

"

Proposition 1.6.2 Let system (1.32) be exponentially mean-square stable PN 2 with decay rate greater α > 0, and let 2α ≥ i=1 di for given numbers d1 , . . . , dN ≥ 0. Then (1.31) is asymptotically stable for arbitrary measurable functions δi : R × Rn → C satisfying |δi (t, x)| ≤ di σi for all t ∈ R. Proof: By Corollary 1.5.4, there exists an X > 0 such that 0 > A∗ X + XA + 2αX +

N O

i σi2 Ai∗ 0 XA0

i=1

≥ A∗ X + XA + > ≥

A+

N O

? 2 F i di X + σi2 Ai∗ 0 XA0

i=1 N O i=1

δi (t, x)Ai0

E∗

> X +X A+

N O i=1

E δi (t, x)Ai0

,

1.7 Stabilization of linear stochastic control systems

21

if |δi | ≤ di σi by Lemma 1.6.1. Hence, the quadratic real valued function x L→ x∗ Xx defines a uniform Lyapunov function for (1.31), provided |δi (t, x)| ≤ di σi for all t ∈ R and all i. " We conclude that mean-square stability in the presence of white noise parameter uncertainties guarantees a certain robustness margin with respect to bounded uncertainties. Remark 1.6.3 It is, however, an interesting and to our knowledge open question, whether the assumptions of Proposition 1.6.2 imply a certain robustness margin with respect to parameter uncertainties of e.g. coloured noise type. This question is closely related to the following. Assume that the equation dx = Ax dt + A0 x dw interpreted in the Itˆ o sense is mean-square stable with decay rate α. Can we then name conditions on the noise intensity σ, such that the equation dx = Ax dt + σA0 x dw interpreted in the Stratonovich sense is mean-square (or stochastically) stable? In view of formula (1.17) and Corollary 1.5.4 we might equivalently consider the conditions ∃X < 0 : and ∃X < 0 :

A∗ X + XA + 2X + A∗0 XA0 > 0 ,

(1.33)

@ @ G∗ G 1 1 A + σ 2 A20 X + X A + σ 2 A20 + σ 2 A∗0 XA0 > 0 . (1.34) 2 2

Clearly, for given A, A0 , condition (1.33) implies (1.34) for sufficiently small σ. But what parameters are needed to determine such a margin σ? We leave the question for further research.

1.7 Stabilization of linear stochastic control systems Let there be given a linear stochastic control systems of the general form dx(t) = Ax(t) dt +

N O i=1

Ai0 x(t) dwi (t) + Bu(t) dt +

N O

B0i u(t) dwi (t) . (1.35)

i=1

Here x(t) ∈ Kn is the state vector and u(t) ∈ Km is a control input vector at time t. For a given control u ∈ L2w ([0, ∞[; L2 (Ω, Km )) and an initial value

22


x0 ∈ Kn there exists a solution of (1.35) which we denote by x(·, x0 , u). At a later stage we also consider a disturbance input v(t) ∈ KL and an output vector z(t) ∈ Kq . In this case we write dx(t) = Ax(t) dt +

N O

Ai0 x(t) dwi (t)

i=1

+ B1 v(t) dt +

N O

i v(t) dwi (t) B10

(1.36)

i=1

+ B2 u(t) dt +

N O

i B20 u(t) dwi (t)

i=1

z(t) = Cx(t) + D1 v(t) + D2 u(t) , and use the notation x(·, x0 , v, u) for the solution of (1.36) with a given disturbance-input v ∈ L2w ([0, ∞[; L2 (Ω, KL )), a given control-input u ∈ L2w ([0, ∞[; L2 (Ω, Km )), and an initial value x0 ∈ Kn . Similarly z(·, x0 , v, u) denotes the output. 1.7.1 Stabilizability One of our tasks is to find a feedback control law of the form u(t) = F x(t) with a feedback gain matrix F ∈ Km×n such that system (1.35) is stabilized. This is a straight-forward generalization of the deterministic stabilization problem by static state feedback. In this section we wish to point at some specific features of the stochastic set up. In particular, we introduce a generalized type of Riccati equations which constitutes the central topic of this book. First of all we define stabilizability. Definition 1.7.1 We call system (1.35) open loop stabilizable if for all x0 ∈ Kn there exists a control ux0 ∈ L2w ([0, ∞[; L2 (Ω, Km )) such that x(·, x0 , ux0 ) ∈ L2w ([0, ∞[; L2 (Ω, Kn )). It is called stabilizable by static linear state-feedback if there exists a matrix F ∈ Km×n such that the closed-loop system dx = (A + BF )x dt +

N O

(Ai0 + B0i F )x dwi

(1.37)

i=1

is mean-square stable. In this case, for brevity, we also say that the quadruple (A, (Ai0 ), B, (B0i )) or (if all B0i vanish) the triple (A, (Ai0 ), B) is stabilizable. In Section 2.1 we will see that in fact open loop stabilizability implies stabilizability by static linear state-feedback. The stability of (1.37) can be determined on the basis of Theorem 1.5.3. From this, we can easily derive a necessary criterion for stabilizability, which resembles the so-called Hautus-test [99] for deterministic systems (see also e.g. [135]).


23

Lemma 1.7.2 Let (A, (Ai0 ), B, (B0i )) be stabilizable. Assume that X is an eigenvector of the operator LA + ΠA0 , corresponding to an eigenvalue λ O∈ C− . Then B ∗ X O= 0, or B0i∗ X O= 0 for some i ∈ {1, . . . , n}. Proof: By assumption, there exists an F , such that the closed-loop system (1.37) is mean-square stable. Hence, by Theorem 1.5.3, the mapping T : X L→ (A + BF )∗ X + X(A + BF ) +

N O

(Ai0 + B0i F )∗ X(Ai0 + B0i F ) ,

i=0

is stable. Now assume B ∗ X = B0i∗ X = 0 for some eigenvector X of LA + ΠA0 and all i = 1, . . . , N . Then X is also an eigenvector of T corresponding to the same eigenvalue λ. Hence Re λ < 0. " A necessary and sufficient criterion for the triple (A, (Ai0 ), B) to be stabilizable can be stated as a Lyapunov-type linear matrix inequality. Criteria in terms of linear matrix inequalities involving two unknown matrices can be found in [21, 64]. Lemma 1.7.3 Let B ∈ Kn×m . Then (A, (Ai0 ), B) is stabilizable

⇐⇒

∗ )(X) − BB ∗ < 0 . ∃X > 0 : (L∗A + ΠA 0

Proof: ‘⇒’: By Definition 1.7.1, Theorem 1.5.3 and Remark 1.5.5(ii), the triple (A, (Ai0 ), B) is stabilizable, if and only if there exist F ∈ Km×n and ∗ ∗ X > 0, such that (L∗A+BF +ΠA )(X) < 0. This implies x∗ (L∗A +ΠA )(X)x < 0 0 0 ∗ n×n be a unitary matrix, for all nonzero x ∈ Ker B . Let U = [U1 , U2 ] ∈ K ∗ with the columns of U1 spanning Ker B ∗ . If we set Y = (L∗A + ΠA )(X), then 0 for arbitrary α > 0 0 6 = D αU1∗ Y U1 αU1∗ Y U2 ∗ ∗ U = )(αX) − BB U ∗ (L∗A + ΠA , 0 αU2∗ Y U1 αU2∗ Y U2 − U2∗ BB ∗ U2 where U1∗ Y U1 < 0 and U2∗ BB ∗ U2 > 0. Choosing α sufficiently small, we have D = −U2∗ BB ∗ U2 + α U2∗ Y U2 − U2∗ Y U1 (U1∗ Y U1 )−1 U1∗ Y U2 < 0 , ∗ such that U ∗ ((L∗A + ΠA )(αX) − BB ∗ )U < 0 by Lemma A.2. 0 ‘⇐’: If (LA + ΠA0 )(X) − BB ∗ < 0 for some X > 0, then we choose ∗ F = −B ∗ X −1 to obtain (L∗A+BF + ΠA )(X) < 0. " 0

If we think of deterministic systems, then, by the previous Lemma, the condition ∃X < 0 : L∗A (X) − BB ∗ < 0

(1.38)

obviously is equivalent to the pair (−A, B) being stabilizable. Sometimes this is expressed by saying that (A, B) is anti-stabilizable. In this case, there exists

24


a feedback gain matrix F , such that all solutions of the system x˙ = (A+BF )x converge to 0 for t → −∞. Since stochastic differential equations cannot be solved backwards in time, the condition ∗ ∃X < 0 : ΠA (X) + L∗A (X) − BB ∗ < 0 0

(1.39)

does not have an analogous dynamic interpretation. Nevertheless, this condition, which we might call anti-stabilizability (and its dual version, see Remark 1.8.6) plays a role in algebraic solvability criteria for Riccati equations. Obviously, (1.38) implies (1.39). For completeness, let us recall the notions of controllability and observability. Definition 1.7.4 Let (A, B, C) ∈ Kn×n × Kn×m × Kp×n . The pair (A, B) is called controllable, if rk(B, AB, . . . , An−1 B) = n. The pair (A, C) is called observable, if (A∗ , C ∗ ) is controllable. It is well-known e.g. [135] that (±A, B) is stabilizable, if (A, B) is controllable. In particular, controllability of (A, B) implies (1.39). But controllability of (A, B) does not imply stabilizability of (A, (Ai0 ), B), as we will see in the following examples. Example 1.7.5 (i) The scalar case: Consider the scalar stochastic system dx = ax dt + bu dt + a0 x dw + b0 u dw . For f ∈ R the closed-loop system dx = (a + bf )x dt + (a0 + b0 f )x dw is stable if and only if 2(a + bf ) + (a0 + b0 f )2 < 0 (by Thm. 1.5.3). But such an f cannot exist if b0 O= 0 and the discriminant −2ab20 + 2ba0 b0 + b2 is negative. We might ask, whether it could be possible to find a non-real stabilizing f ∈ C for this system. The answer is negative, as we will learn from Lemma 1.7.7. If a real system is stabilizable by complex feedback, then it is stabilizable by real feedback as well. Thus, stabilizability in the presence of control-dependent noise is not a generic property over K. (ii) A non-stabilizable system: Now we consider a system with state-dependent noise only: dx = Ax dt + Bu dt + A0 x dw . One might ask, whether controllability of the pair (A, B), or the necto the system essary condition from Lemma 1.7.2 is sufficient for 0 6 0 be 6 11 0 stabilizable. This is not the case. Let us choose A = ,B = , 01 1

0

6


25

01 . The pair (A, B) is controllable, but the system is not stabi10 lizable. To see this, we consider the operator LA+BF + ΠA0 : H2 → H2 as in Thm. 1.5.3, with F = [f1 , f2 ] ∈ R1×2 . It has the Kronecker-product matrix representation       0001 1 0 1 0 1 1 0 0    f1 1 + f2 0  0  0 1   + 0 0 1 0 + 0 1 MF =      0 0 1 1 0 1 0 0 f1 0 1 + f2 0 0 f1 0 1 + f2 0 0 f1 1 + f2 1000   2 1 1 1   f1 2 + f2 1 1  . =  f1 1 2 + f2 1  1 f1 f1 2 + 2f2 A0 =

To simplify the problem, we consider the restriction of the operator MF 2×2 to the0three-dimensional H02 ⊂ R 6 . The symmetric matrices 6 6 0 subspace √ 00 10 01 U1 = constitute an (orthonormal) , U2 = 22 , U3 = 01 00 10 basis of this subspace. Hence, we apply the transformation   2 √0 0   ˆ F = U T MF U with U = [vec U1 , vec U2 , vec U3 ] = 1  0 √2 0  M 2 0 2 0 √   0 0 2 2 √2 √1 =  2f1 3√+ f2 2  . 1 2f1 2 + 2f2 ˆ F − I) = 2(f1 − f2 − 1)2 ≥ 0. This A short calculation yields det(M 2×1 ˆ F (and thus the operator shows that for all F ∈ R the matrix M LA+BF + ΠA0 ) has an eigenvalue λ with Re λ ≥ 1. Though we can move the eigenvalues of LA+BF as far as we wish into the left half plane, we cannot stabilize the operator LA+BF + ΠA0 by any F ∈ R2×1 . Again, it will follow from Lemma 1.7.7 below that the system is not stabilizable by any F ∈ C2×1 either. Nevertheless, the given system (A, A0 , B) satisfies the necessary stability criterion from Lemma 1.7.2. In fact, B ∗ X O= 0 for all eigenvectors of LA + ΠA0 . To see this, we consider MF with F = 0, which represents LA + ΠA0 . For X ∈ K2×2 we have B ∗ X = 0, if and only if vec X = [x1 , 0, x3 , 0]T , with x1 , x3 ∈ K. But the equation M0 vec X = λ vec X obviously requires x3 = 0 and thus also x1 = 0. Hence, X cannot be an eigenvector of LA + ΠA0 , if B ∗ X = 0. This is even stronger, than the condition from Lemma 1.7.2. It can already be guessed from these examples that the characterization of stochastic stabilizability is a difficult problem. To our knowledge there is no

26


simple classification available in the literature. Some special cases have been discussed in [204, 205, 191]. In the following we reformulate the stabilizability problem with the help of rational matrix operators. 1.7.2 A Riccati type matrix equation Stabilizability by static state-feedback is equivalent to the solvability of a Riccati type matrix equation (e.g. [191]). Here we give an ad hoc derivation of this interrelation. Later, in Section 2.1, we will see the connection to the LQ optimal control problem and open loop stabilization. By Theorem 1.5.3 system (1.37) is stable (for a given F ) if and only if there exists an X > 0 satisfying Y ≥ (A + BF )∗ X + X(A + BF ) 6∗ 0 i∗ 60 6 N 0 i O I I A0 XAi0 Ai∗ 0 XB0 + B0∗ XAi0 B0i∗ XB0i F F

(1.40)

i=1

for some Y < 0. It follows from Theorem 1.5.3 that if (1.40) admits a positive definite solution for some Y < 0 then it has a solution X > 0 for arbitrary Y < 0. In particular, we choose 0 6∗ 0 6 0 6 I I P0 S0 Y =− , with some M = M > 0. (1.41) F F S0∗ Q0 (The role of the weight matrix M will become clear in Section 2.1.) Then we can write (1.40) as P (X) + F ∗ S(X)∗ + S(X)F + F ∗ Q(X)F ≤ 0 ,

(1.42)

where P , S and Q are given by ∗

P (X) = A X + XA +

N O

i Ai∗ 0 XA0 + P0 ,

i=1

S(X) = XB +

N O

i Ai∗ 0 XB0 + S0 ,

(1.43)

i=1

Q(X) =

N O

B0i∗ XB0i + Q0 .

i=1

Since Q0 > 0 and X > 0 we have Q(X) > 0. For fixed X we minimize the left hand side of (1.42) with respect to the ordering of Hn . Lemma 1.7.6 Let 0 < Q ∈ Hm and S ∈ Kn×m . Then


min

F ∈Km×n

F ∗ S ∗ + SF + F ∗ QF = −SQ−1 S ∗ ;

27

(1.44)

the minimum is attained for F = −Q−1 S ∗ . Proof: The assertion follows from the simple inequality F ∗ S ∗ + SF + F ∗ QF = (F ∗ + SQ−1 )Q(F + Q−1 S ∗ ) − SQ−1 S ∗ ≥ −SQ−1 S ∗ ; equality takes place if and only if F = −Q−1 S ∗ .

"

Hence we have the following result. Lemma 1.7.7 For an arbitrary matrix M > 0 as in (1.41) define P , S, and Q as in (1.43). System (1.35) is stabilizable by linear static state feedback if and only if the Riccati type matrix inequality R(X) = P (X) − S(X)Q(X)−1 S(X)∗ ≤ 0

(1.45)

has a solution X > 0. In this event F = −Q(X)−1 S(X)∗ is a stabilizing feedback gain matrix. Remark 1.7.8 (i) Under the conditions of Lemma 1.7.7, we know that the real-valued function φ(x) = E (x∗ Xx) is a Lyapunov function for system (1.37). For later use we write out the derivative of φ(x(t)) with respect to t, if x is a solution of (1.35) with control u. Like in (1.24) we have 6 O 0 6∗ >0 ∗ 6E 0 6 N 0 i∗ i A X + XA XB x x A0 XAi0 Ai∗ 0 XB0 ˙ + φ(x) = E . B0i∗ XAi0 B0i∗ XB0i 0 u u B∗X i=1

(ii) Let us once more consider the Stratonovich interpretation of (1.35). It is often argued that one can always transform a Stratonovich equation into an Itˆ o equation and apply all the techniques developed for Itˆ o equations. It should, however, be noted that in the case of control-dependent noise this transformation destroys the linear dependence of the closed-loop equation on F . For if we apply a linear static state feedback control to the Stratonovich equation (1.35), then we obtain the closed-loop Stratonovich equation (1.37). The corresponding closed-loop Itˆ o equation is therefore by (1.17) E > N N = D2 D O 1 O= i A0 + B0i F Ai0 + B0i F x dwi . x dt + dx = A + BF + 2 i=1 i=1 Obviously, the dependence on F is much more complicated now, and we cannot apply any standard Riccati techniques to establish the existence of a stabilizing F . This is another practical reason for us to prefer the Itˆ o interpretation.

28


(iii) In the derivation of the Riccati inequality (1.45) we have exploited the properties of the operator LA + ΠA0 . There are other classes of linear systems, whose stability properties can be characterized by means of operators of the same form. • The discrete-time deterministic system xk+1 = Axk is asymptotically stable if and only if ∃X > 0 : A∗ XA − X < 0 (e.g. [184]). • The discrete-time stochastic system xk+1 = Axk + A0 xk wk with independent real random variables wk is mean-square stable, if and only if ∃X > 0 : A∗ XA − X + A∗0 XA0 < 0 (cf. [153]). • The deterministic delay system x(t) ˙ = Ax(t) + A1 x(t − h) is asymptotically stable for all delays h > 0 if ∃X > 0 : A∗ X + XA + X + A∗1 XA1 < 0 (cf. [129]).

1.8 Some remarks on stochastic detectability Once Theorem 1.5.3 is established, it is straightforward to formulate the problem of stabilization by feedback; the situation is similar, though technically more involved, as in the deterministic case. As for the filtering problem, the situation is more complicated. From the practical point of view, we do not have a duality of stabilizability and detectability for systems with multiplicative noise. To see the difference let us consider the deterministic system x˙ = Ax,

y = Cx

with measured output y and the dynamic observer ξ˙ = Aξ + K(y − η),

η = Cξ .

The observer is built as a copy of the system with input y, such that the error equation e˙ = (A − KC)e for e = x − ξ is stable. In this sense the detection problem for the matrix pair (A, C) is equivalent to the stabilization problem for (A∗ , C ∗ ). But if we consider a stochastic system dx = Ax dt + A0 x dw,

y = Cx ,

(1.46)

we cannot use an exact copy of the system in the observer equation, because of the parametric uncertainty. If we set e.g. dξ = Aξ dt + K(dy − dη),

η = Cξ ,

then we find the additive term A0 x dw in the error equation de = (A − KC)e dt + A0 x dw .

1.8 Some remarks on stochastic detectability

29

Hence we can only expect e to converge to zero if x is stable. This is due to the fact that we cannot reproduce the noise process in the observer. If we want to use the estimated state to stabilize the system, then we conclude that unlike in the deterministic case, we cannot separate the estimation problem from the stabilization problem. We have to construct a dynamic compensator directly instead of constructing a dynamic observer and a feedback regulator (see also [1] for uncertain systems). This approach was taken in [107]. It leads to a coupled pair of Riccati equations. While in the deterministic case one obtains one Riccati equation for the observer and one for the controller which can be solved separately, the coupling here reflects the fact that observer and controller cannot be separated. We will come back to this in Section 2.3.3. From the theoretical point of view, however, it can be useful to define detectability as a dual property of stabilizability (e.g. [58, 69]). To this end, we consider the stochastic system dx = Ax dt +

N O

Ai0 x dwi ,

dy = Cx dt +

i=1

N O

C0i x dwi ,

(1.47)

i=1

(C, C0i ∈ Kp×n ) together with the (unrealistic) observer equation dξ = Aξ dt +

N O

Ai0 ξ dwi + K(dy − dη),

dη = Cξ dt +

i=1

N O

C0i ξ dwi .

i=1

Definition 1.8.1 System (1.47) is called (mean-square) detectable if there exists an observer gain matrix K ∈ Kn×p , such that the error equation de = (A − KC)e dt +

N O (Ai0 − KC0i )e dw ,

(1.48)

i=1

is mean-square stable. For brevity, we also say that the quadruple (A, (Ai0 ), C, (C0i )) or (if all C0i vanish) the triple (A, (Ai0 ), C) is detectable. We say that the triple (A, (Ai0 ), C) is β-detectable, if CX O= 0 for any eigen∗ corresponding to an eigenvalue λ O∈ C− . vector X ≥ 0 of L∗A + ΠA 0 Remark 1.8.2 We choose the term β-detectable, since we need this property R to characterize the spectral bound β(T ) = max{Re(λ) R λ ∈ σ(T )} of an operator T (cf. Section 3.2.1). The notion of β-detectability has a simple interpretation. It says that the output of a β-detectable system can only vanish (in mean-square), if the state vector converges to zero (in mean-square). Lemma 1.8.3 Consider system (1.46). The triple (A, (Ai0 ), C) is β-detectable, if and only if for arbitrary x0 the implication

30


∀t ≥ 0 : E 0 . ∈ H+ S0∗ Q0 ∗ m×n set Let F0 = −Q−1 0 S0 and for an arbitrary F ∈ K

1.9 Examples

AF = A + BF,

Ai0,F = Ai0 + B0i F,

0 and MF =

I F

6∗

0 M

I F

31

6 .

(i) If (AF0 , MF0 ) is observable, then (AF , MF ) is observable. (ii) If (AF0 , (Ai0,F0 ), MF0 ) is detectable, then (AF , (Ai0,F ), MF ) is detectable. (iii) If (AF0 , (Ai0,F0 ), MF0 ) is β-detectable, then (AF , (Ai0,F ), MF ) is β-detectable. Proof: Assume that MF X = 0 for some X ∈ Kn×n . By inequality (1.44) with Q = Q0 and S = S0 we have MF ≥ MF0 , MF0 X = 0 and (F − F0 )X = 0. Hence also AF X = AF0 X and Ai0,F X = Ai0,F0 X for i = 1, . . . , N . From this, (i), (ii), and (iii) follow immediately. "

1.9 Examples Let us now produce a number of examples for control systems with multiplicative noise. In doing so we pursue several goals. Firstly, we wish to illustrate the notions introduced so far and make the use of models with stochastic parameter uncertainties plausible. Secondly, we motivate control problems like the disturbance attenuation problem to be introduced in the following chapter. Thirdly, we will use some of the following models as numerical examples in Chapter 5. Finally, we think that a collection of examples is useful to everyone interested in our topic. 1.9.1 Population dynamics in a random environment Let x be the size of a biological population, e.g. fish in a pond or people on a planet. The simplest model for the growth of this population is the differential equation of normal reproduction x˙ = ax . All resource restrictions and problems of crowding neglected, the growth rate is a constant number a ∈ R. Clearly, a is an average number resulting from a magnitude of random events and environmental unpredictability. Taking this into account, it is natural to replace a by a Gaussian white noise process a + wa ˙ 0 (see e.g. [193], [134]). This brings us to the equation from Example 1.5.8 (i) dx = ax dt + a0 x dw . Similarly, it is natural to introduce random parameters in a system with more than one species like e.g. the famous Volterra-Lotka model. So, let us

32


now consider a pond where two species of fish live, say carp and pike. If there were no pike, the carp would multiply exponentially. In the presence of pike one must take account of the number of carp eaten by the pike, which is jointly proportional to the number x of carp and the number y of pike. The pike, however, die out exponentially, if there are no carp, while in the presence of carp their number increases at a rate proportional to both x and y. Hence we obtain a deterministic model of the form (e.g. [8]) x˙ = kx − axy , y˙ = −ly + bxy .

(1.50)

Now let us regard the coefficients as Gaussian white noise processes. Then (see e.g. [134]) we have to replace (1.50) by a stochastic differential equation of the form dx = (kx − axy) dt + k0 x dw1 − a0 xy dw2 , dy = (−ly + bxy) dt − l0 y dw3 + b0 xy dw4 .

(1.51)

We just mention that the transition to a stochastic model after linearization can be problematic. For instance let us linearize equation (1.50) at the equilibrium (x, y) = (k/a, l/b) and set (∆x, ∆y) = (x − k/a, y − l/b). This yields the linear equation 6 60 0 6 0 d ∆x −k ∆x k − al b , (1.52) = ∆y −l + bk l dt ∆y a which describes the local dynamics around the equilibrium. But the stochastic equation (1.51) does not have any non-trivial equilibria. Hence it does not really make sense to consider stochastic coefficient in the linearized equation (1.52). 1.9.2 The inverted pendulum The inverted pendulum is one of the standard examples of linear control theory and has been described in a number of textbooks. It consists of an inverted pendulum attached to the top of a cart through a pivot. The cart is driven along a horizontal rail by a force βu(t), where the function u is regarded as the control input. The task is to stabilize the pendulum in the upper equilibrium position by choosing an appropriate control u. Let l be the length of the pendulum and assume that its mass m = 1 is concentrated at the tip. The angle between the pendulum and the vertical is denoted by φ. By M we denote the mass of the cart and by r the distance from the center of mass of the cart to some reference point. The state of the system is described by the state ˙ T , and the linearized equation of motion is given by (see vector x = [r, r, ˙ θ, θ] [184, 108]) x˙ = Ax + Bu

(1.53)

1.9 Examples

with



01 0 0 0 −mg M A= 0 0 0 0 0 m+M lM g

 0 0 , 1 0



0

33



 1  M  B=  0  . 1 − lM

Here, g denotes the acceleration due to gravity. All friction effects have been neglected. In this model it has been assumed that no external forces act on the system and that the reference coordinate system is at rest. But let us now assume that the whole structure is subject to random vertical vibrations which affect the laws of gravity in our system. In this case, it is customary (e.g. [20, 114, 124]) to model the constant g as time-varying and stochastic. In this spirit we regard g now as a Wiener process. Setting   00 0 0 0 0 −m 0 M  A0 =  0 0 0 0, 0 0 m+M lM 0 we model the randomly perturbed system by the Itˆ o equation: dx = Ax dt + A0 x dw + Bu dt . 1.9.3 A two-cart system In [194, 195, 163] a linear model of the two-mass spring system with uncertain stiffness depicted in Fig. 1.1 was considered. x1 k = k0 + ∆(t) v

m1 = 1

m2 = 1

 x1  x2   x=  x˙ 1  x˙ 2 

x2 u

1111111111111111111111111111111111111111 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 1111111111111111111111111111111111111111 Fig. 1.1. Two carts on a rail, connected by a spring

The system consists of two carts connected by a spring; there is a disturbance v acting on the left cart and a control force u driving the right cart. The control is to be chosen such that the displacements of the carts from their equilibrium position is minimized in some norm, i.e. the effects of v have to be annihilated. This is a typical disturbance attenuation problem. It is assumed that the spring constant k = k(t) has the nominal value

34


k0 = 5/4, but it can vary (e.g. due to nonlinear effects of the spring) and is considered uncertain. In a series of experiments under different conditions it was found out that at each instant of time the values approximately have a Gaussian distribution centred at k0 . The experiments also showed that k ranges over [0.5, 2]. Therefore the difference ∆(t) = k(t)−k0 was modelled as a Gaussian white noise process with intensity σ = 1/4 ensuring that |∆(t)| < 0.75 and hence k ∈ [0.5, 2] with sufficiently high probability. As pointed out before, one might argue whether this model is adequate. It is possible that the Gaussian distribution of the values k was not produced by one individual spring at different instances of time, but by different springs, with time-invariant stiffness each. Or the stiffness of each spring is in fact time-varying but not a Gaussian process. In this case the concept of bounded uncertainties might be preferable. We will come back to this point later. The state-space equations of the stochastic spring system take the general form dx = (Ax + B1 v + B2 u) dt + A0 x dw , z = Cx + Du . 1.9.4 An automobile suspension system Models with uncertain stiffness and damping parameters also occur in more realistic applications than the one presented above. A simple model for an automobile suspension system with stochastic parameter uncertainty has been discussed in [89]. The nominal system, known as the quarter vehicle, is obtained by lumping all four wheels together and considering only the different vertical movements of the car body and the axle. The tyres are modelled by a spring and the suspension system is modelled by a spring and a damper. By y1 and y2 we denote the displacements of the centres of mass of the car body and the axle, respectively, from their equilibrium under the influence of gravity. If v(t) denotes the displacement of the point of contact between tyre and road from the nominal road level, then y1 and y2 are subject to the coupled second-order differential equations [108] m1 y¨1 + c(y˙ 1 − y˙ 2 ) + k1 (y1 − y2 ) = 0 , m2 y¨2 + c(y˙ 2 − y˙ 1 ) + k1 (y2 − y1 ) + k2 y2 = k2 v .

(1.54)

Here m1 and m2 denote the masses, c is the damping coefficient of the dashpot, k1 is the stiffness of the spring in the suspension system and k2 is the stiffness of the tyre, viewed as a spring. It is pointed out in [89], however that the behaviour of a tyre is much more complicated than that of a spring. In particular, the stiffness can vary as the tyre rotates and encounters changing road conditions. These variations are partly of a periodic nature and partly stochastic. While [159] discusses a periodic excitation of the parameter k2 , [89] considers a superposition of periodic excitation and white noise. To keep

1.9 Examples

y1

y2

35

m1

k1

c

m2

k2

Fig. 1.2. Suspension system of a quarter vehicle [89]

the model tractable within our framework, we restrict ourselves to white noise parameter uncertainty only and model k2 in the form k + σ w˙ with white noise w. ˙ The unevenness v of the road is viewed as an external disturbance input. Moreover we assume that we have an active damping system. We model this by setting the left hand side of equation (1.54) equal to u(t), where u is some control input. Thus, setting x = [y1 , y2 , y˙ 1 , y˙ 2 ]T we have the linear stochastic control system     0 0 1 0 0    0 0 0 1   x dt +  0  v dt dx =   −k1 /m1  0  k1 /m1 −c/m1 c/m1  k1 /m2 −(k1 + k)/m2 c/m2 −c/m2 k/m2       0 0 00 0 0 0   0   0 0 0    x dw +  0  v dw . (1.55) +   0   1/m1  u dt +  0 0 0 0 0 −σ/m2 0 0 σ/m2 0 If we define z = [y1 , y˙ 1 ]T as the to be controlled output, then a typical disturbance attenuation problem is the following. Find a feedback control u = F x that stabilizes the system and minimizes the effect of the disturbance input v on the output z.

36


1.9.5 A car-steering model According to [1] the essential features of car-steering dynamics in a horizontal plane are described by the so-called single-track model. It is obtained by lumping the front wheels into one wheel Wf in the center line of the car; the same is done with the two rear wheels (Wr ). This is illustrated in figure 1.3. Here S is the center of gravity and v is the velocity vector. The angle β be-

v S

Wr

δf

β

δr

Wf 5r

5f

ψ x0

Fig. 1.3. Single-track model for car-steering [1]

tween v and the center line is called vehicle sideslip angle, whereas the angle ψ between the center line and some fixed direction x0 in the plane is called yaw angle. The yaw rate r = ψ˙ and the sideslip angle β are the state variables in our model. The front and rear steering angles δf and δr are the control variables (in the case of four-wheel steering). The distances between S and the front and the rear axles are denoted by ^f and ^r . Furthermore let m be the mass of the car, J the moment of inertia, and cr and cf given constants that depend e.g. on parameters like the tyre pressure and temperature. For the dynamics of this system the adhesion coefficient µ between road surface and tyre plays a crucial role. Under different conditions we have  (dry road) , 1 µ = 0.5 (wet road) ,  0.15 (ice) . We allow for different adhesion coefficients µf and µr at the front wheel and the rear wheel. If these values are fixed, we can describe the dynamics of the linearized single-track model in state space form as (1.56) x˙ = Aµ x + Bµ u . 0 0 6 6 0 6 0 6 a a b b δ β , u = f , Aµ = 11 12 , and Bµ = 11 12 with Here x = δr a21 a22 b21 b22 r 1 (cr µr + cf µf ) , mv 1 = (cr ^r µr − cf µf ^f ) − 1 , mv 2

a11 = − a12

1.9 Examples

a21 = a22 = b11 = b12 = b21 = b22 =

37

1 (cr ^r µr + cf ^f µf ) , J F 1 ? 2 − cr ^r µr + cf ^2f µf , Jv c f µf , mv c r µr , mv c f ^ f µf , J −cr ^r µr . J

As the car moves, the value of µ may change. For instance, when it starts raining on a dry road, then µ may vary rapidly between one point and another. The same may happen, if the road is wet and the temperature is close to the freezing point. In these situations it can be appropriate to model µf and µr as time-varying stochastic parameters. Let us assume that Eµf = Eµr = µ0 = 12 , and that µf − µ0 and µr − µ0 can be modelled by independent scaled standard Wiener processes σw1 (t) and σw2 (t), where σ denotes the noise intensity. Clearly, this model is only an approximation, since, for instance, it allows for negative adhesion coefficients. Nevertheless it can be useful to represent the time dependency of µ. Thus we are led to the linear stochastic control system D = (1) (2) dx = Ax dt + σ A0 x dw1 + A0 x dw2 = D (1) (2) + Bu dt + σ B0 u dw1 + B0 u dw2 , where

. A =

cr +cf cr Lr −cf Lf − 2mv 2mv22 cr Lr +cf L2f cr Lr +cf Lf − 2Jv 2J

−

and

. (1) A0

=

c

c L

1

4

0 ,

B =

.

4

cr 2mv r Lr − c2J

c r Lr mv 2 2 2 = , c r Lr c f Lf c f Lf c r Lr − Jv − Jv J J 0 cf 0 cr 6 0 0 mv (1) (2) B0 = cmv , B0 = f Lf 0 crJLr J 0

f f f − mv − mv 2

(2) A0

cr − mv

cf 2mv c f Lf 2J

6 ,

4 , 6 .

Automatic car-steering Based on (1.56) in [1] a model for automatic car-steering is discussed. The problem is the following. Our car has to follow a guiding wire in the street. This reference path is assumed to consist of circular arcs, with reference radius

38


Rref and reference curvature ρref = R1ref . For straight path segments we have ρref = 0. The displacement z of the car from the reference path is defined to be the signed distance of S to the nearest point zref on the path; if |yS | is small, then zref is uniquely determined. To measure the displacement there is a sensor S0 mounted at a distance ^0 in front of S; hence the measured displacement y is the distance from S0 to the path. Finally we define ∆ to be the angle between the tangent to the path at zref and the center line of the car. These variables are illustrated in Figure 1.4. v

S

S0

z

y

β

∆ guiding wire

Fig. 1.4. Vehicle heading and displacement y from the guiding wire (here ρref = 0) [1]

 x With the extended state variable ξ =  ∆  ∈ R4 we have the extended y linearized state space model     0 6 0 Aµ 0 ˙ξ =  0 1 0 0  ξ −  v  ρref + Bµ u , 0 v ^0 v 0 0 / 5 y= 0001 ξ. 

Like above, we obtain a stochastic differential equation, if we replace Aµ and (2) (1) (1) (2) Bµ by A + A0 w˙ 1 + A0 w˙ 2 and B + B0 w˙ 1 + B0 w˙ 2 . For simplicity, we assume that the car has to follow a straight line such that ρref = 0. In this case the trivial solution is an equilibrium. The first problem now is to stabilize this equilibrium by choosing a static linear state feedback control law u = F ξ. Note that the uncontrolled system is not asymptotically stable. Furthermore, we take into account that the guiding line might be slightly distorted. That is, now we interpret ρref as a stochastic disturbance process. A typical disturbance attenuation problem now consists in choosing F such that the influence of ρref on the output y is minimized. We will study problems of this type in Section 2.3. They lead to Riccati equations as in 1.7.7 with an

1.9 Examples

39

indefinite weight matrix M . Of course, it would be of more practical value to have a dynamic compensator of the form ˆ = AK x dξ(t) ˆ(t) dt + BK y(t) dt , u(t) = CK x ˆ(t) + DK y(t) , which does not require the knowledge of the full state ξ. In the case of disturbance-dependent noise instead of control-dependent noise this problem has been studied e.g. in [107]. It leads to a coupled pair of Riccati equations. We will mention this point later. 1.9.6 Satellite dynamics A famous example of a stochastic control system was given by Sagirow. When modelling the motion of a satellite in the upper atmosphere, one has to account for the rapid fluctuations of the atmospheric density ρ and the intensity H of the magnetic field of the earth. In deterministic models based on Newtonian mechanics these quantities occur as constant parameters. Sagirow suggests to model ρ and H in the form ρ = ρ0 (1 + δ w) ˙ and H = H0 (1 + δ w) ˙ with white ˙ noise w. We consider the pitch and the yaw motion of a satellite on a circular orbit. It is assumed that the center of gravity of the satellite always stays in the plane of the orbit. Let x, y, z denote the principal axes of inertia of the satellite; a moving frame is given by the orbital tangent (ξ), the radius vector (η) and the normal to the orbital plane (ζ). Planar pitch motion Following [171] let us first consider the pitch motion of a satellite in the upper atmosphere. We are interested in the dynamics of the pitch angle θ and introduce ˙ With the above stochastic model for the the state variables x1 = θ, x2 = θ. atmospheric density one can derive (see [171], [6] [134]) a stochastic differential equation of the form G @ G @ G @ x2 0 x1 = dt + dw ; d x2 −bx2 − sin x1 − c sin x1 −abx2 − b sin x1 here x2 = x˙ 1 and a, b, c are given constants with b, c > 0. If we linearize this equation at (x1 , x2 ) = (0, 0) we obtain the equation @ G 0 0 6@ G 6@ G x1 0 0 0 1 x1 x1 d dw . dt + = x2 x2 x2 −a −ab 2c − 1 −b

40


y

η θ ξ

x

Fig. 1.5. The plane pitch motion of a symmetrical satellite in a circular orbit [171] ζ

z

x

η=y ψ ξ

Fig. 1.6. The yaw motion of a satellite in a circular equatorial orbit [171]

Stabilization of the yaw motion by the magnetic field Now we consider the oscillations of the yaw angle ψ of a satellite in a circular equatorial orbit. Aerodynamic effects are neglected. The nominal intensity H of the magnetic field of the earth is orthogonal to the orbital plane. A magnetic bar is placed in the satellite-fixed z-axis (i) to produce a restoring torque My = −HIz sin ψ with the constant magnetic moment Iz . A second torque is due to the currents that arise when the satellite (ii) is moving in the magnetic field. It has the form My = −KH 2 ψ˙ with a constant K, depending on the thickness and the electromagnetic properties of the shell as well as the moments of inertia of the satellite. For constant H the yaw oscillations are thus described by B ψ¨ = −KH 2 ψ˙ − HIz sin ψ .

(1.57)

1.9 Examples

41

The quantities H and H 2 , however, depend on the activity of the sun. They can be modelled more correctly in the form H = H0 (1 + δ w) ˙ and H 2 = 2 H0 (1 + 2δ w). ˙ Thus (1.57) becomes a stochastic equation. For convenience we KH 2 H0 Iz ˙ Then the set α = B , β = B 0 and introduce the state variable ω = ψ. linearized stochastic model for the yaw oscillations is given by 60 6 60 6 0 0 6 0 0 ψ ψ 0 1 ψ dw . (1.58) dt + = d αδ −2βδ ω ω −α −β ω In [18] this model is extended by an active controller. It is assumed that some noise is associated with the activation of this controller. For instance, this will be the case if the controller changes the value of K. The controlled model has the form 0 6 60 6 0 0 6 60 6 0 6 0 0 ψ 0 0 ψ 0 1 ψ u dw . dw + u dt + dt + = d δu ω αδ −2βδ 1 ω −α −β ω 1.9.7 Further examples Without going into details, we mention some more examples. An electrical circuit In [195] an electric circuit consisting of a resistor, a capacitor, an inductor, and a voltage source in a serial connection is considered. The differential equations describing the capacitor voltage VC and the current i in this circuit are di 1 R = − i + (V − VC ) , dt L L 1 dVC = i, dt C

(1.59) (1.60)

where R, L and C as usual denote resistance, inductance and capacity. One can control the system by applying the voltage V . The authors claim that nearby electrical devices can induce changes in the inductance value L, such that L−1 should be modelled in the form L−1 = −1 ˙ This turns equation (1.59) into the stochastic system L−1 0 + L1 w. 6 0 R 6 0 1 6 0 60 − L0 − L10 i i L0 V dt = dt + d VC VC 0 − C1 0 6 0 1 6 60 0 R 1 − L1 − L1 i dw + L1 V dw . + VC 0 0 0 Similarly, e.g. in [173], the case of randomly perturbed capacity C has been discussed.

42


An electrically suspended gyro In [79] we find a model for an electrically suspended gyro of the form @0 dx =

6 0 6 G 2 O 0 01 u dt + x+ σi x dwi . 1 β0 i=1

According to [79], one noise term comes from the electronic circuits and the other from thermal expansion and contraction of the gap between the rotor and the electrode cavity. The task is to stabilize the system by an appropriate choice of u. Mathematical finance Another very important area of applications for stochastic differential equations is mathematical finance. A multitude of books on this topic has appeared in recent years. Here we just make the annotation that Itˆ o equations are the central model in this theory. In a few words, the situation is the following. Suppose there is a market in which m + 1 assets are traded continuously. One of the assets is the bond, whose price process P0 (t) is subject to the deterministic differential equation dP0 (t) = r(t)P0 (t) dt,

P0 (0) = p0 > 0 .

(1.61)

The other m assets are stocks whose price processes P1 (t), . . . , Pm (t) satisfy the stochastic differential equations dPi (t) = bi (t)Pi (t) dt +

m O

σij (t) dwj (t),

Pj (0) = pj > 0 .

(1.62)

j=1

The stochastic uncertainty in this model is due to the independent and partly unpredictable trading of a large number of shareholders, who drive the market price of a stock like the molecules of liquid act on a pollen. In [218] one can find a linear quadratic control problem associated to optimal portfolio selection.

2 Optimal stabilization of linear stochastic systems

Having introduced and motivated our concepts of stochastic control systems in the previous chapter we now turn to optimal and suboptimal stabilization problems. A classical problem in optimal control theory is the so-called linear quadratic (LQ-)stabilization problem. Among all controls that stabilize a given system, one determines the one that satisfies a certain quadratic nonnegative semidefinite cost-functional. This will be the topic of the first section. The same techniques used to find an LQ-optimal stabilization can also be applied to identify a worst-case perturbation of a control system. This leads to the Bounded Real Lemma, which is the topic of Section 2.2. The Bounded Real Lemma itself, is the main tool needed to tackle the disturbance attenuation problem in Section 2.3. Throughout this chapter let A, Ai0 ∈ Kn×n , C ∈ Kq×n ,

i ∈ Kn×L , B1 , B10 D1 ∈ Kq×L ,

i ∈ Kn×m , B2 , B20 D2 ∈ Kq×m .

2.1 Linear quadratic optimal stabilization According to [192], the linear quadratic regulator problem (LQ-control problem) and the Riccati equation were introduced by Kalman in [119]. The corresponding problems for stochastic differential equations were first considered by Wonham in [211, 212, 213] and subsequently e.g. in [97, 98, 164, 37, 52, 190, 69, 218]. As in Section 1.7 we consider a linear stochastic control system dx(t) = Ax(t) dt +

N O i=1

Ai0 x(t) dwi (t) + B2 u(t) dt +

N O

i B20 u(t) dwi (t) . (2.1)

i=1

If u is a given control, then the corresponding solution with initial value x(0) = x0 is denoted by x(t; x0 , u). T. Damm: Rational Matrix Equations in Stochastic Control, LNCIS 297, pp. 43–60, 2004. Springer-Verlag Berlin Heidelberg 2004

44


Furthermore, as in Section 1.7, let there be given a weight matrix 0 6 P0 S0 M = > 0. S0∗ Q0 This strictness condition will be weakened later (cf. Corollary 5.3.4), but for the moment it simplifies the presentation. We consider the following linear quadratic optimal control problem: For each given initial value x0 ∈ Kn find a control input u = ux0 such that the quadratic cost-functional 6∗ 0 6 < ∞0 x(t; x0 , u) x(t; x0 , u) J(x0 , u) = E M dt (2.2) u(t) u(t) 0 is minimized. By an adequate choice of M , one can punish large deviations of the state from the equilibrium or large values of the control input, which might be too energy-consuming or even destroy the system. But, as e.g. [135] points out, the main benefit of a positive semidefinite quadratic cost-functional is the fact, that it is mathematically easy to handle and provides some criterion to choose a controller. To formulate the solution of this problem we recall the definition of the Riccati operator from Section 1.7 and set R(X) = P (X) − S(X)Q(X)−1 S(X)∗ 0 =

6∗ 0

I

−Q(X)−1 S(X)∗

(2.3)

P (X) S(X) S(X)∗ Q(X)

60

I

−Q(X)−1 S(X)∗

6 ,

with P (X) = A∗ X + XA +

N O

i Ai∗ 0 XA0 + P0 ,

i=1

S(X) = XB2 +

N O

i Ai∗ 0 XB20 + S0 ,

i=1

Q(X) =

N O

i∗ i B20 XB20 + Q0 .

i=1

The operator R is well defined on the set R dom R = {X ∈ Hn R det Q(X) O= 0} . We also introduce R dom+ R = {X ∈ Hn R Q(X) > 0} ⊂ dom R .

(2.4)

2.1 Linear quadratic optimal stabilization

45

Clearly dom R and dom+ R are open; from Q0 > 0 it is obvious that n H+ ⊂ dom+ R. The condition X ∈ dom+ R represents a constraint on X ∈ Hn . While in Section 1.7 we have already discussed the relation between the feedback stabilization problem and the Riccati inequality, we now strengthen this result by considering optimal open-loop stabilization and the Riccati equation. Theorem 2.1.1 Let M > 0 and R be defined according to (2.3). Then the following are equivalent: (i) System (2.1) is open-loop stabilizable. (ii) System (2.1) is stabilizable by static linear state-feedback. (iii) The Riccati-type matrix equation R(X) = 0 has a positive definite solution X > 0. Moreover, if (iii) holds, then the static linear feedback control u = −Q(X)−1 S(X)∗ x stabilizes (2.1) and minimizes the cost-functional (2.2). Proof: Obviously (ii) implies (i) and by Lemma 1.7.7 we know that (iii) implies (ii). It remains to show that (i) implies (iii). Let T > 0 and define the finite-horizon cost-functional 6∗ 0 6 < T0 x(t; x0 , u) x(t; x0 , u) M JT (x0 , u) = E dt . (2.5) u(t) u(t) 0 On dom+ R we consider the differential Riccati-equation X˙ = −R(X) with boundary condition X(T ) = 0 ∈ int dom+ R; we denote the solution by XT . By time-invariance there exists a number ∆ > 0 independent of T , such that XT (t) ∈ dom+ R exists for all t ∈]T − ∆, T ]. For 0 ≤ t ≤ T < ∆ we can therefore consider a control of the form u(t) = −Q(XT (t))−1 S(XT (t))∗ x(t) + u1 (t) ,

(2.6)

where u1 ∈ L2w ([0, T ]) is arbitrary. By some standard technical calculation, which we present below, we have for all T ∈ [0, ∆[ JT (x0 , u) = x∗0 XT (0)x0 + E

< 0

T

u1 (t)∗ Q(XT (t))u1 (t) dt .

(2.7)

Since Q(X) > 0 for X ∈ dom+ R, the cost-functional is minimized if u1 ≡ 0. Obviously XT (0) > 0, since JT (x0 , u) > 0 for all x0 . Moreover, by assumption for each x0 ∈ Kn there exists a stabilizing control ux0 ∈ L2w ([0, ∞[). Hence for all T > 0

46


x∗0 XT (0)x0 =

JT (x0 , u) min R D = R ≤ J(x0 , ux0 ) < ∞ . ≤ JT x0 , ux0 R u∈L2w ([0,T ])

[0,T ]

n In particular, XT (0) ∈ int H+ exists for all T > 0. Since JT (x0 , u) increases with T , it follows that XT (t) = XT −t (0) ≤ xT (0) is uniformly bounded for t ∈]T − ∆, T ]. Thus, for each finite ∆, the solution XT can be extended to an open neighbourhood of T − ∆. We conclude that for all T > 0 the solution XT exists in fact on ] − ∞, T ]. For T 3 > T > t we have XT & (t) > XT (t), since JT (x0 , u) increases with T . Therefore XT (t) converges to some X∞ > 0 as T → ∞, where X∞ is independent of t. By continuity X∞ solves R(X) = 0, and the feedback control u = −Q(X∞ )−1 S(X∞ )∗ x stabilizes system (2.1) by Lemma 1.7.7. Moreover, if we let T = ∞ in (2.7), we see that it yields an LQ-optimal feedback control. For completeness, we verify formula (2.7), which can be found e.g. in [107]. We write x instead of x(t; x0 , u) and omit the argument t. Since XT (T ) = 0, we have for all T ∈ [0, ∆[

JT (x0 , u) =

x∗0 XT (0)x0

< +

T

0

R d R E (x∗ XT (t)x) R ds + JT (x0 , u) . dt t=s

The derivative in the integrand is given by x∗ X˙ T x = −x∗ R(XT )x plus the formula in Remark 1.7.8(i) (with XT instead of X). To this formula, we add the integrand in JT according to (2.5). Thus, we obtain the matrices P (XT ), S(XT ), and Q(XT ). Setting F (XT ) = −Q(XT )−1 S(XT )∗ we have altogether 60 6 < T 0 6∗ 0 x P (XT ) S(XT ) x dt JT (x0 , u) = x∗0 XT (0)x0 + E S(XT )∗ Q(XT ) u u 0 < −E

0

T

∗

x

0

I F (XT )

6∗ 0

P (XT ) S(XT ) S(XT )∗ Q(XT )

60

6 I x dt . F (XT )

If now u has the special form u = F (XT )x + u1 from (2.6), then everything cancels out except for those terms involving u1 . Among these the terms u1 S(XT )∗ x and their complex conjugates cancel out as well, such that finally we retain the integrand in (2.7). " We have needed the strict positivity condition M > 0 in order to guarantee that the control u = −Q(X)−1 S(X)∗ x stabilizes system (2.1). If this can be guaranteed by some other argument, then we may weaken the condition M > 0. Corollary 2.1.2 Let M ∈ Hn+m and R be defined according to (2.3). Assume that X+ ∈ dom+ R and R(X+ ) = 0. If the feedback-control u = −Q(X+ )−1 S(X+ )∗ x

2.2 Worst-case disturbance: A Bounded Real Lemma

47

stabilizes (2.1), then it minimizes the cost-functional (2.2) among all stabilizing controls. Proof: Let the control u(t) = −Q(X+ )−1 S(X+ )∗ x(t) + u1 (t) with u1 ∈ L2w ([0, ∞[) stabilize system (2.1) for some given initial state x0 . Then, like in the proof of Theorem 2.1.1, we have < ∞ ∗ J(x0 , u) = x0 X+ x0 + E u1 (t)∗ Q(X+ )u1 (t) dt ≥ x∗0 X+ x0 , 0

with equality if and only if u1 = 0.

"

The idea of the proof of Theorem 2.1.1 is the same as in the deterministic case. But apart from the fact that here we need Itˆ o‘s formula to derive (2.7) there is another important difference: For deterministic linear systems it can easily be seen from the Kalman normal form (e.g. [135]) that open-loop stabilizability implies stabilizability by static state feedback. Under the condition of state feedback stabilizability, however, one can use algebraic or iterative techniques to prove the solvability of the Riccati equation. We will develop such a general framework for a large class of operator equations in Chapters 4 and 5. On the other hand, if feedback stabilizability is not given a priori one really has to invoke an open-loop control to obtain an upper bound for the solution of the differential Riccati equation; this is necessary in order to show that the solution does not escape in finite time. The analogous situation on a more complicated level occurs in the proof of the so-called Bounded Real Lemma in [107].

2.2 Worst-case disturbance: A Bounded Real Lemma A problem opposite to that of finding an optimal stabilizing control is to determine a worst-case disturbance of a stable system. To be more precise, let us consider the system dx(t) = Ax(t) dt +

N O

Ai0 x(t) dwi (t) + B1 v(t) dt +

i=1

N O

i B10 v(t) dwi (t) (2.8)

i=1

z(t) = Cx(t) + D1 v(t) ,

(2.9)

where now we regard v ∈ L2w ([0, ∞[; L2 (Ω, KL )) as an external disturbance. Definition 2.2.1 We call system (2.8) internally stable, if the unperturbed system dx(t) = Ax(t) dt +

N O i=1

Ai0 x(t) dwi (t)

48


is asymptotically mean-square stable. We call system (2.8) externally stable, if for all perturbations v ∈ L2w ([0, ∞[; L2 (Ω, KL )) the output z is also in L2w , i.e. z ∈ L2w ([0, ∞[; L2 (Ω, Kq )). If (2.8) is externally stable, then we define the perturbation operator L : L2w ([0, ∞[; L2 (Ω, KL )) → L2w ([0, ∞[; L2 (Ω, Kq )) by Lv(·) = z(·; 0, v). We will see below that L is a bounded operator, if system (2.8) is internally i stable. In the deterministic case (when all Ai0 and B10 vanish) the perturbation operator can be represented in the frequency domain by the transfer function G(s) = C(sI − A)−1 B1 + D1 . Its norm is given by supω∈R 0, we consider a general perturbation v ∈ L2w ([0, T ]) and write x(t) = x(t; 0, v). Applying the formula from Remark 1.7.8(i) in integral form we have (with Ex(0)∗ Xx(0) = JTγ (0, v)) 6 60 6∗ 0 < T0 x(t) x(t) P (X) S(X) JTγ (0, v) = −Ex(T )∗ Xx(T ) + E dt v(t) v(t) S(X)∗ Qγ (X) 0 >0. The inequality follows from X < 0, Qγ (X) > 0, and Rγ (X) > 0 (cf. Lemma A.2). Hence (2.8) is externally stable and 0. Choosing γ sufficiently large we also have Qγ (X) > 0 and " Rγ (X) > 0 and can apply the first part of the proposition. In fact, if the assumptions of the Proposition hold for some γ > 0, then they remain valid, if we decrease γ slightly; hence, we have even 0 to prove the non-strict inequality 0 there exists an X ∈ dom+ Rγ , such that X ≤ 0 and Rγ (X) ≥ 0, then 0 and arbitrary v ∈ L2w ([0, T ]). Hence 0 and let the feedback-gain matrix F (depending on X) be defined according to (2.22). 3

(i) Rγ (X) = RγF (X) and for the derivatives at X we have RγX = (RγF )3X . i (ii) If (A + B2 F, (Ai0 + B20 F )) is stable and Rγ (X) ≥ 0, then β(T ), it suffices if C is reproducing instead of solid; see [139, Thm. 9.2]. In this case, also T (not only T ∗ ) possesses a positive eigenvector. Corresponding changes can be made in the assumptions of Proposition 3.2.6. Similarly, the assertions of Corollary 3.2.4 hold if C is only reproducing instead of solid; see [149], [23]. For the spectral theory of resolvent positive operators see also [65], [5] and [68]. The paper [197] gives a nice overview of the spectral theory of positive operators in finite-dimensional ordered vector spaces. Before we formulate the main theorem in this section, we introduce a generalized notion of detectability in analogy to Definition 1.8.1. This definition is motivated by Lemma 1.8.4 and Remark 1.8.6. Definition 3.2.8 Let X be a real Banach space, ordered by a closed, solid, normal convex cone C. Let y ∈ C∪−C and |y| = ±y, so that |y| ∈ C. Consider an operator T : X → X and assume that either T or −T is resolvent positive. The pair (T, y) is called β-detectable, if 1|y|, v2 > 0 for every eigenvector v ∈ C ∗ of T ∗ , corresponding to an eigenvalue λ with Re λ ≥ 0. If T (x)−|y| < 0 for some x ∈ C, then we call the pair (T, y) detectable.

66

3 Linear mappings on ordered vector spaces

It is easy to see that detectability implies β-detectability. The converse is not true, as we have already noted in Section 1.8. Lemma 3.2.9 Let T or −T be resolvent positive and y ∈ C. If the pair (T, y) is detectable, then it is β-detectable. Proof: If (T, y) is detectable then T (x) − y < 0 for some x ∈ C. If T ∗ v = λv for some eigenpair (λ, v) ∈ C × C ∗ \{0} satisfying 1y, v2 = 0 then ¯ v2 . 0 > 1T (x) − y, v2 = 1x, T ∗ (v)2 = λ1x, Since 1x, v2 ≥ 0, we have λ < 0.

"

The following theorem is an infinite-dimensional generalization of a result by H. Schneider and plays a central role. The proof is adapted from [180]. Theorem 3.2.10 Let X be a real Banach space ordered by a closed, solid, normal convex cone C. Suppose R : X → X to be resolvent positive and P : X → X to be positive, and set T = R + P . Then the following are equivalent: (i) (ii) (iii) (iv) (v) (vi) (vii)

T is stable, i.e. σ(T ) ⊂ C− . −T is inverse positive. ∀y ∈ int C : ∃x ∈ int C : −T (x) = y ∃x ∈ int C : −T (x) ∈ int C. ∃x ∈ C : −T (x) ∈ int C. ∃x ∈ C : y = −T (x) ? ∈ C Fand (T, y) is β-detectable. σ(R) ⊂ C− and ρ R−1 P < 1.

Proof: (i) ⇔ (ii): By Corollary 3.2.4 the operator T is resolvent positive and thus by Proposition 3.2.6 the conditions (i) and (ii) are equivalent. (ii)⇒(iii): If −T is inverse positive then −T −1 maps int C into int C, which implies (iii). The chain (iii)⇒(iv)⇒(v)⇒(vi) is trivial. (vi)⇒(i): Assume that (vi) holds for some x ∈ C, but (i) fails. Then β := β(T ) ≥ 0 and T ∗ has an eigenvector v ∈ C ∗ corresponding to β by Theorem 3.2.3. This implies 0 ≥ 1−y, v2 = 1T x, v2 = 1x, T ∗ v2 = 1x, βv2 ≥ 0 , whence 1y, v2 = 0 in contradiction to the β-detectability; hence (i) holds. It remains to prove that (vii) is equivalent to (i)–(vi). (vii)⇒(ii): Suppose (vii), then −R is inverse positive by Proposition 3.2.6 and ρ((−R)−1 P ) < 1. Applying Lemma 3.2.5 we obtain that −T = −R − P is inverse positive, i.e. (ii). (i), (ii), (iv)⇒(vii): Since β(R + P ) ≥ β(R) by Corollary 3.2.4, condition (i) implies σ(R) ⊂ C− , whence −R is inverse positive and Π := −R−1 P is

3.2 Positive and resolvent positive operators

67

positive. By (iv) there exists a positive vector x ∈ int C, such that −T x ∈ int C and since −R−1 ≥ 0 this implies R−1 T x = (I − Π) x ∈ int C. But by Theorem 3.2.3 there ?exists a vF ∈ C ∗ , such that Π ∗ v = ρ(Π)v. Therefore 0 < 1(I − Π) x, v2 = 1 − ρ(Π) 1x, v2, whence ρ(Π) < 1 because 1x, v2 > 0. " Decompositions of the form T = R + P play a role in iterative methods. We will come back to this in Section 3.5. The following definition is adapted from the theory of M -matrices (see e.g. [198, 14]). Definition 3.2.11 Let T be a resolvent positive operator. A decomposition of T into the sum T = R + P with a resolvent positive operator R and a positive operator P is called a regular splitting. The regular splitting T = R + P is called convergent if σ(R) ⊂ C− and ρ(R−1 P ) < 1. 3.2.2 Equivalent characterizations of resolvent positivity In [66] the following equivalent conditions were given for an operator in a finite dimensional space to be resolvent positive (see also [181], [13]). Theorem 3.2.12 Let X be a finite-dimensional real vector space ordered by a closed, solid, normal convex cone C. For linear operators T : X → X the following are equivalent: (i) (ii) (iii) (iv) (v)

T is resolvent positive. exp(tT ) is positive for all t ≥ 0. ∀x ∈ ∂C : ∃v ∈ ∂C ∗ : 1x, v2 = 0 and 1T x, v2 ≥ 0. x ∈ C, v ∈ C ∗ ,R 1x, v2 = 0 ⇒ 1T x, v2 ≥ 0. T ∈ cl{S − αI R S : X → X positive, α ∈ R}.

Remark 3.2.13 (i) A resolvent positive operator is also called Metzler operator e.g. in [68, 109]. Condition (ii) is often called exponential positivity or exponential nonnegativity, e.g. [13]. In [66] an operator is called quasimonotonic, if it possesses property (iii). Operators satisfying (iv) are called cross-positive in [181] and C-subtangential in [13]. An operator T is called essentially nonnegative in [13] if T +αI is positive for sufficiently large α ∈ R. (ii) Though [66] only treats the finite-dimensional case the proof carries over immediately to the situation where X is a real Banach space. But we will not make use of this generalization. (iii) From (iv) we see that the resolvent positive operators form a convex cone. This cone contains all scalar multiples of the identity operator and hence is not pointed. In the following section we determine the maximal subspace in the cone of resolvent positive operators on Hn .

68


Corollary 3.2.14 The set of resolvent positive operators R : Hn → Hn is a closed convex cone. It contains the convex cone of positive operators and all real multiples of the identity. In particular it is solid, but not pointed. Following [90], we use the term lineality space for the maximal subspace contained in the cone of resolvent positive operators. It is an interesting question, dating back to [185], whether every resolvent positive operator T can be represented as the sum T = L+P

(3.1)

of a positive operator P and an element L from the lineality space. Although in [185] an affirmative answer could be given for important classes of cones, it was shown in [90] that this representation is impossible for almost all cones in a certain categorial sense.

3.3 Linear mappings on the space of Hermitian matrices In the following we consider the ordered vector space Hn of Hermitian matrices (defined in the appendix), which plays an important role in applications. Its structure is surprisingly rich and a number of elementary questions is still unanswered. For instance, up to today it is unknown, how to classify positive operators on Hn . While in the case of the vector space Rn ordered by Rn+ one can easily check whether an operator is positive by inspecting the entries of its matrix representation, there is no analogous result for the space Hn . An early mention of this problem can be found in [180]. Several attempts have been made e.g. in [51, 28, 104, 29, 30, 56, 215, 10, 105, 146] to prove partial or related results, some of which we will recapitulate in this Section. Similarly, we do not have a representation theorem for resolvent positive operators on Hn . In particular we do not know, whether a representation of the form (3.1) is possible for resolvent positive operators T : Hn → Hn , because n the cone H+ falls into neither of the classes considered by [185] and [90]. As a partial result in this direction we show in Section 3.4 that the lineality space of Hn coincides with the set of Lyapunov operators. Hence we conjecture that every resolvent positive operator on Hn can be written as the sum of a Lyapunov operator and a positive operator. Note that for the special class of completely positive operators (see Section 3.3.2) the existence of such a representation has been proven in [147]1 . In Section 3.5 we discuss some numerical approaches to solve linear equations with resolvent positive operators, while in Section 3.6 we recapitulate the relation between resolvent positivity and stability. Finally, Section 3.7 contains some results on minimal representations of the inverse of a Sylvester operator. 1

Unaware of this, I posed the corresponding question as an open problem in [44].

3.3 Linear mappings on the space of Hermitian matrices

69

3.3.1 Representation of mappings between matrix spaces We recall some results on the representation of linear mappings between matrix spaces. Since we wish to point out some further details, that are not stated explicitly in the literature, we include the proofs. In the following we consider mappings T : Km×n → Kp×q . Notice that each mapping T : Hn → Hn can be extended to a K-linear mapping T : Kn×n → Kn×n . The extension is unique if K = C, while in the case K = R the mapping T is not uniquely determined for skew-symmetric matrices. Obviously, a linear mapping T : Km×n → Kp×q can also be regarded as a linear mapping between the vector spaces Kmn and Kpq . To make use of this observation, we recall the definition and some basic properties of the Kronecker product and the vec-operator (e.g. [113]). Definition 3.3.1 Let V = (vjk ) = (v1 , . . . , vn ) ∈ Km×n and U = (ujk ) ∈ Kp×q . Then   v1 = D   V ⊗ U = vjk U ∈ Kmp×nq and vec V =  ...  ∈ Knm . vn Lemma 3.3.2 Let U1 , U2 , V1 , V2 , and X be matrices of appropriate sizes. Then (V1 ⊗ U1 )(V2 ⊗ U2 ) = (V1 V2 ) ⊗ (U1 U2 ) and vec(U1 XV1 ) = (V1T ⊗ U1 ) vec X . (m)

In the following let ej (mn) Ejk

(m) (n) ∗ ej ek

denote the j-th canonical unit vector in Km and

= the m × n matrix with the only nonzero entry 1 in the j-th row and k-th column. By E mn we denote the m2 × n2 -block matrix = Dm,n (mn) Ejk = vec Im (vec In )∗ and with the mapping T we associate the j,k=1

mp × nq matrix = D(m,n) (mn) (Ipq ⊗ T )(E (mn) ) = T (Ejk ) . j,k=1

(3.2)

Remark 3.3.3 Here, we understand the notation (Ipq ⊗ T )(E (mn) ) as an abbreviation. If we really wanted to understand Ipq =⊗ T as Da Kroneckerm,n (mn) product, we would have to arrange the entries of Ejk in the j,k=1 = Dm,n (mn) vector vec vec Ejk and identify T with its matrix representation j,k=1

70


as an operator from K=mn → Kpq .DFinally we would have to rearrange the m,n (mn) product (Ipq ⊗ T ) vec vec Ejk ∈ Kmpnq as an mp × nq-matrix. j,k=1

The following simple identity is useful. Lemma 3.3.4 If V = (v1 , . . . , vm ) ∈ Kp×m and W = (w1 , . . . , wn ) ∈ Kq×n then = Dm,n (mn) V Ejk W ∗ = vec V (vec W )∗ . (3.3) j,k=1

Proof: By Lemma 3.3.2 we have (mn)

vec(V Ejk

Thus,

(mn)

V Ejk

¯ ⊗ V vec E (mn) W ∗) = W jk = D (nm) = w ¯1 ⊗ v1 , w ¯1 ⊗ v2 , . . . , w ¯q ⊗ vm e(k−1)m+j

W∗

=w ¯k ⊗ vj = vec(vj wk∗ ) . Dm,n = (mn) = vj wk∗ and V Ejk W ∗

j,k=1

∗

vec V (vec W ) .

=

Dm,n = vj wk∗

j,k=1

= "

Theorem 3.3.5 Let T : Km×n −→ Kp×q be linear. The matrices V1 , . . . , VN ∈ Kp×m , W1 , . . . , WN ∈ Kn×q yield a representation of T : ∀X ∈ Km×n : T (X) =

N O

Vj XWj∗

(3.4)

j=1

if and only if (Ipq ⊗ T )(E (mn) ) =

N O

vec Vj (vec Wj )∗ .

(3.5)

j=1

In particular, the minimal number of summands is ν = rk(Ipq ⊗ T )(E (mn) ), and one can choose a representation with matrices V1 , . . . , Vν ∈ Kp×m , W1 , . . . , Wν ∈ Kn×q , such that (vec Vj )∗ vec Vk = (vec Wj )∗ vec Wk = 0 for all j, k ∈ {1, . . . , ν} with j O= k. Proof: By Lemma 3.3.4 the identity (3.4) clearly implies (3.5); vice versa (3.5) P m×n mn implies T (X) = N . j=1 Vj XWj for all X = Ejk , and thus for all X ∈ K A representation with a minimal number of summands is given e.g. by a singular value decomposition of (Ipq ⊗ T )(E (mn) ), in which case also the orthogonality property holds. " Now we consider mappings between spaces of quadratic matrices.


71

Definition 3.3.6 Let T : Kn×n −→ Km×m be a linear map. Then T is called Hermitian-preserving if T (Hn ) ⊂ Hm . It is immediate to see, that the Lyapunov operator is Hermitian-preserving. The following representation result can be found in [104]; the proof is adapted from [29]. Theorem 3.3.7 For a linear map T : Kn×n −→ Km×m consider the following assertions: (i) T is Hermitian-preserving. (ii) ∀X ∈ Kn×n : T (X ∗ ) = (T (X))∗

= Dn (nn) (iii) The nm×nm matrix (Imm ⊗T )(E (nn) ) = T (Ejk ) (iv) There exist matrices V1 , . . . , VN ∈ K {−1, 1}, such that for all X ∈ Kn×n T (X) =

N O

m×n

j,k=1

is Hermitian.

and numbers ε1 , . . . , εN ∈

εj Vj XVj∗ .

(3.6)

j=1

The minimal number of summands is N = rk(Imm ⊗ T )(E (nn) ). One can choose εj = 1 for all j, if and only if (Imm ⊗ T )(E (nn) ) ≥ 0. If K = C, then all these assertions are equivalent. If K = R, then (ii), (iii) and (iv) are equivalent, and each of them implies (i). Proof: (i)⇒(ii) for K = C: For skew-Hermitian matrices S (i.e. iS Hermitian) we have on the one hand T (iS) = T (iS)∗ = −iT (S)∗ and on the other T (iS) = T ((iS)∗ ) = T (−iS ∗) = −iT (S ∗ ) which yields T (S)∗ = T (S ∗ ). As any matrix A can be decomposed in A = H + S, where H = 12 (A + A∗ ) is Hermitian and S = 12 (A − A∗ ) is skew-Hermitian, the result follows. (ii)⇒ (iii): This follows from G G @= G∗ @= @= Dn Dn Dn (nn) ∗ (nn) (nn) T (Ejk )∗ T (Ejk ) T (Ejk ) = = k,j=1 k,j=1 j,k=1 @= G @= G Dn Dn (nn) (nn) T (Ekj ) T (Ejk ) = = . k,j=1

j,k=1

(iii)⇒ (iv): By Sylvester’s inertia theorem there is a decomposition

72


= Dn (nn) T (Ejk )

j,k=1

N O

=

εL vec VL (vec VL )∗

(3.7)

L=1

for appropriate VL , where N is the rank of the matrix on the left. By equation (3.3) we can also write = Dn (nn) T (Ejk )

>

j,k=1

=

N O L=1

that means (nn)

∀Ejk

:

(nn)

T (Ejk ) =

En (nn) εL VL Ejk VL∗ j,k=1 N O L=1

(nn)

εL VL Ejk VL∗

(nn) R and thus the same holds with replaced by any X ∈ span{Ejk R j, k = 1, . . . , n} = Kn×n . Moreover it follows from Sylvester’s Theorem, that one can choose εj = 1 for all j if and only if (Imm ⊗ T )(E (nn) ) ≥ 0. (iv) ⇒ (i): For each l the matrix VL∗ XVL is Hermitian, if X is Hermitian; hence the sum is Hermitian, too. = D (nn) Ejk

(iv) ⇒ (ii): Since for each l and arbitrary X we have VL∗ XVL it follows that T (X)∗ = T (X ∗ ).

∗

= VL∗ X ∗ VL "

Remark 3.3.8 In the real case, (i) does not necessarily imply (ii). For instance, we can define a linear mapping T : R2×2 → R2×2 via T (e1 e∗1 ) = T (e2 e∗2 ) = 0,

T (e1 e∗2 ) = e1 e∗2 ,

T (e2 e∗1 ) = e1 e∗1 + e2 e∗1 .

It is easy to see that T satisfies (i) but not (ii). Since every linear mapping T : Hn → Hn can be extended to a linear mapping T : Cn×n → Cn×n such that (ii) holds, we can draw a simple conclusion. Corollary 3.3.9 Every linear mapping T : Hn → Hn has a representation of the form (3.6); i.e. there are matrices Vj ∈ Kn×n and numbers εj ∈ {−1, 1}, such that (3.6) holds for all X ∈ Hn . The trace of an extension T : Kn×n → PN Kn×n is then given by trace T = j=1 εj |trace Vj |2 . PN ¯ Proof: As T corresponds to the matrix l=1 Pi Vl ⊗ Vl , the trace formula follows from the easily verified identity, trace V ⊗ W = trace V trace W . "


73

3.3.2 Completely positive operators Let T : Kn×n → Kn×n be a Hermitian-preserving operator with the repren n ) ⊂ H+ , if all εj are sentation (3.6). Then clearly T is positive, i.e. T (H+ nonnegative. Vice versa, let us verify that for the operator T : X L→ X ∗ which 2 to itself, there are no matrices Vj ∈ Kn×n such that maps H+ T (X) =

N O

Vj XVj∗

(3.8)

j=1

for all X ∈ K2×2 . In view of Theorem 3.3.7 (iv) it suffices to observe that   1000 0 0 1 0  (I22 ⊗ T )(E (22) ) =  0 1 0 0 0001 is indefinite, which is obviously the case. But, perhaps more surprisingly, there also exist positive operators T : Hn → Hn , that do not allow a representation of the form (3.8) for all X ∈ Hn . The first example was given in [30]; it has the form   ax11 + bx22 + cx33 Ta,b,c : X = (xjk ) L→ diag  ax22 + bx33 + cx11  − X ax33 + b11 + cx22 with a = b = c = 2. We do not repeat the proof here. More generally, it was shown in [27] that Ta,b,c is positive if the following conditions are satisfied D = a ≥ 1, a + b + c ≥ 3, and 1 ≤ a ≤ 2 ⇒ bc ≥ (2 − a)2 . Further examples can be found in [189, 146]. Of course, the representation (3.8) is very convenient. It corresponds to a special type of positivity which was introduced for general C ∗ -algebras in [187]. Definition 3.3.10 An operator T : Kn×n → Km×m is called completely positive, if for any ^ ∈ N and any nonnegative definite ^n × ^n-block-matrix L L (Xjk )j,k=1 with Xjk ∈ Cn×n the matrix (T (Xjk ))j,k=1 is nonnegative definite. If T : Kn×n → Km×m is completely positive, then in particular (Imm ⊗ T )(E (nn) ) ≥ 0, since E (nn) = vec In (vec In )∗ ≥ 0. By Theorem 3.3.7 this implies that T can be written in the form (3.8). Conversely, it is immediate to see that any operator of the form (3.8) is completely positive. We will not explore the subtle distinction between positive and completely positive maps much further. But it is worth noting that our abstract results

74


in Chapter 4 only require the notion of positivity, while in our concrete applications to Riccati equations we make use of special representations in the form (3.8), i.e. we work with completely positive maps. Similarly, we will make use of complete positivity in the Sections 3.5 and 3.6. The increasing generality of the concepts of complete positivity, positivity and resolvent positivity can be seen from the following nice – though almost trivial – characterization. Lemma 3.3.11 Let T : Kn×n → Kn×n be a linear operator and consider the (n2 × n2 )-matrix X = (Inn ⊗ T )(E (nn) ). Then 2

(i) T is completely positive ⇐⇒ ∀z ∈ Kn : z ∗ X z ≥ 0. (ii) T is positive ⇐⇒ ∀x, y ∈ Kn : (x ⊗ y)∗ X (x ⊗ y) ≥ 0. (iii) T is resolvent positive ⇐⇒ ∀x, y ∈ Kn with x⊥y: (x ⊗ y)∗ X (x ⊗ y) ≥ 0. Proof: (i): We have already noted that T is completely positive, if and only if it has a representation of the form (3.8). By Theorem 3.3.7 this is equivalent to X ≥ 0. n (ii): By definition, T is positive if T (X) ≥ 0 for all X ∈ H+ . Since T is linear n it suffices to consider the extremals of H+ . Hence, T is positive if and only if y ∗ T (xx∗ )y ≥ 0 for all x, y ∈ Kn . Writing T in the form (3.6), we have y ∗ T (xx∗ )y = vec(y ∗ T (xx∗ )y) =

n O

εj vec(y ∗ V x) vec(y ∗ V x)∗

j=1

=

n O

εj (¯ x∗ ⊗ y¯∗ vec V )(¯ x∗ ⊗ y¯∗ vec V )∗

j=1

=x ¯∗ ⊗ y¯∗

n =O

D ¯ ⊗ y¯ εj (vec V )(vec V )∗ x

j=1

= (¯ x ⊗ y¯)∗ X (¯ x ⊗ y¯) .

(3.9)

Since x, y ∈ Kn were arbitrary, we can replace them by their complex conjugates to prove our assertion. (iii): By Theorem 3.2.12 the operator T is resolvent positive, if 1T (X), Y 2 ≥ 0 n with 1X, Y 2 = 0. Again it suffices to consider extremals for all X, Y ∈ H+ ∗ n . Since 1xx∗ , yy ∗ 2 = trace xx∗ yy ∗ = y ∗ xx∗ y = 0 X = xx and Y = yy ∗ of H+ if and only if x⊥y, it follows that T is resolvent positive if and only if 1T (xx∗ ), yy ∗ 2 = y ∗ T (xx∗ )y ≥ 0 for all x⊥y. Now the assertion follows from equation (3.9). " It follows immediately that, like positive and resolvent positive operators, the completely positive operators form a closed convex cone. Corollary 3.3.12 The set of completely positive operators T : Kn×n → Kn×n is a closed convex pointed solid cone.

3.4 Lyapunov operators and resolvent positivity

75

In the following sections we will discuss an important class of resolvent positive operators on Hn that are not positive, namely Lyapunov operators.

3.4 Lyapunov operators and resolvent positivity A linear operator T : Hn → Hn is said to be a (continuous-time) Lyapunov operator, if there exists an A ∈ Kn×n , such that ∀X ∈ Hn :

T (X) = A∗ X + XA .

In this case we also write T = LA . Note that LA can be written in the form (3.6) as D 1= (A + I)∗ X(A + I) − (A − I)∗ X(A − I) . LA = 2

(3.10)

(3.11)

The role of Lyapunov operators in stability theory is well-known. If for instance, we consider a homogeneous linear deterministic differential equation X˙ = A∗ X,

A, X ∈ Kn×n , X(0) = X0 ,

(3.12)

with the solution X(t), then P (t) = X(t)∗ X(t) ≥ 0 satisfies the equation P˙ = LA (P ) .

(3.13)

As an obvious conclusion, one obtains that A is stable if and only if LA is stable. This is useful, since the stability analysis of LA leads to matrix equations like in Theorem 1.5.3. Furthermore, we conclude that LA generates a positive group. For arbitrary P (0) ≥ 0 the solution P (t) of (3.13) is nonnegative for all t ∈ R. This follows from the fact that we can solve (3.13) forward and backward in time (which, in general, is not the case for stochastic differential equations). Hence, we conclude that ±LA is exponentially and thus resolvent positive. We formulate this result as a Lemma, and give another proof. Lemma 3.4.1 If T is a Lyapunov operator, then both T and −T are resolvent positive. Proof: Let A ∈ Kn×n . By Lyapunov’s Theorem (e.g. [113]), T = LA is inverse positive if σ(LA ) ⊂ C+ . Thus αI − LA = L−A+ α2 I is inverse positive for sufficiently large α ∈ R. Since −T = L−A is a Lyapunov operator, too, it is also resolvent positive. " It is not so clear that in fact Lyapunov operators are the only operators with this property. In other words, each positive linear group on Hn is associated to a differential equation of the form (3.12). This is the content of Theorem 3.4.3 below. To the best of the author’s knowledge, this result is documented in the literature only for the case, where in the assertion ’positive’ is replaced

76


by ’completely positive’, e.g. [147, 4]. Before stating the result, we verify that the situation is different, if T is a discrete-time Lyapunov operator or Stein operator. This means that there exists an A ∈ Kn×n such that ∀X ∈ Hn :

T (X) = A∗ XA − X .

(3.14)

In this case we also write T = SA . It follows e.g. from Lemma 3.2.5 or Theorem 3.2.12 (v) that SA is resolvent positive. But this is not necessarily the case for T = −SA as the following example shows. 6 0 00 and consider the family of indefinite maExample 3.4.2 Let A = 10 0 6 01 with t > 0. For all α > 0 we have αXt + SA (Xt ) = trices Xt = 61 t 0 t α−1 , which is positive for large t though Xt is always indefinite. α − 1 αt Hence αI + SA is not inverse positive for any α, and hence −SA is not resolvent positive. Theorem 3.4.3 Let T : Hn → Hn be a linear operator. The following are equivalent: (i) T is a Lyapunov operator, i.e. ∃A ∈ Kn×n : T = LA . (ii) T and −T are resolvent positive. Proof: ‘(i)⇒(ii)’ follows from Lemma 3.4.1. ‘(ii)⇒(i)’: We begin with the real case K = R. Let T and −T be resolvent positive. First we note that by Theorem 3.2.12 this is equivalent to the following criterion: = D X, Y ≥ 0 and 1X, Y 2 = 0 ⇒ 1T X, Y 2 = 0 . (3.15) If ej denotes the j-th canonical unit vector in Rn , then the set R n B := {ej eTk + ek eTj R j, k = 1, . . . , n} ⊂ H+

(3.16)

forms a basis of Hn ⊂ Rn×n . Thus we have to find an A ∈ Rn×n , such that T (X) = LA (X) for all X ∈ B. Let X = ej eTj (i.e. 2X ∈ B). To apply criterion (3.15) we characterize all matrices n Y ∈ H+ such that 1X, Y 2 = 0 .

(3.17)

Let Y ≥ 0 and 1X, Y 2 = yjj = 0. It is well known, that a diagonal entry of a positive semidefinite matrix can only vanish if both its whole row and column vanish. Hence, (3.17) is true if and only if in Y ≥ 0 the j-th row


77

and column vanish. Criterion (3.15) in turn implies that in T (X) everything vanishes except for the j-th row and column. Otherwise we could choose some Y satisfying (3.17) and 1T (X), Y 2 O= 0. For j = 1, . . . , n we thus have T (ej eTj ) = aj eTj +ej aTj with vectors a1 , . . . , an ∈ Rn . If we build the matrix A = (a1 , . . . , an ), then T (X) = AX + XAT for all X = ej eTj . In other words, we have found a unique candidate for the Lyapunov operator. It remains to show, that also for Xjk = ej eTk + ek eTj with j < k we have T (Xjk ) = AXjk + Xjk AT = aj eTk + ak eTj + ej aTj + ek aTk 

a1k .. .

···

a1j .. .

ank

···

anj

(3.18)



       a1k · · · 2ajk · · · ajj + akk · · · ank      .. .. =  . . .    a1j · · · ajj + akk · · · 2akj · · · anj      .. ..   . . Let j and k be fixed. A matrix Y satisfies condition (3.17) with X = Xjk + Xjj + Xkk ≥ 0 if in Y the j-th and k-th row and column vanish. As above we conclude from criterion (3.15), that in T (X) and hence also in T (Xjk ) everything vanishes except for the j-th and k-th row and column. Thus T (Xjk ) is of the general form T (Xjk ) = bj eTj + ej bTj + bk eTk + ek bTk , 

b1j .. .

···

b1k .. .

bnj

···

bnk

with bj , bk ∈ Rn

(3.19)



       b1j · · · 2bjj · · · bjk + bkj · · · bnj      .. .. =  . . .    b1k · · · bjk + bkj · · · 2bkk · · · bnk      .. ..   . . Now we consider matrices of the form X = xxT with x = xj ej + xk ek where xj , xk ∈ R are arbitrary real numbers. Writing X = xj xk Xjk + x2j ej eTj + x2k ek eTk , and exploiting the linearity of T we have the decomposition T (X) = xj xk T (Xjk ) + x2j T (ej eTj ) + x2k T (ek eTk )

78


= xj xk (bj eTj + ej bTj + bk eTk + ek bTk ) + x2j (aj eTj + ej aTj ) + x2k (ak eTk + ek aTk ) . Let y⊥x, for instance   y1  ..  y =  .  ∈ Rn yn

(3.20)

  yj = xk , yk = −xj , with  yL arbitrary for ^ O∈ {j, k} .

(3.21)

Then the matrix Y = yy T satisfies condition (3.17), and by (3.15) we have 1T (X), Y 2 = 0. If we write T (X) like in equation (3.20) we obtain: 1 1 1 1T (X), Y 2 = trace (T (X)Y ) = y T T (X)y 2 2 2 E > n n n n O O O O = xj xk yj bLj yL + yk bLk yL + x2j yj aLj yL + x2k yk aLk yL

0=

L=1

=

L=1

L=1

L=1

xj x3k (bjj − ajk ) + x3j xk (bkk − akj ) + x2j x2k (−bkj − bjk + ajj + akk ) O O + xj x2k yL (bLj − aLk ) + x2j xk yL (−bLk + aLj ) . L0∈{j,k}

L0∈{j,k}

The right hand side is a homogeneous polynomial in the real unknowns xj , xk , and yL for ^ O∈ {j, k}. Since these unknowns can be chosen arbitrarily, all the coefficients of the polynomial necessarily vanish, i.e. bjj = ajk , bkk = akj , bkj + bjk = ajj + akk , bLj = aLk , bLk = aLj . Inserting these data into (3.19), we see that (3.18) holds. In the complex case K = C the computations are a little bit more involved, because we have to deal with real and imaginary parts. It has to be noted that now dim Hn = n2 while in the real case the dimension was n(n + 1)/2. In particular B must be completed to a basis by R (3.22) Bi = {(ej + iek )(ej + iek )∗ R 1 ≤ j < k ≤ n} . Like in the real case we obtain a candidate for the Lyapunov operator LA with A = (a1 , . . . , an )∗ ∈ Cn×n from the relations T (ej e∗j ) = aj e∗j + ej a∗j . But this candidate is not unique in the complex case; we can add an arbitrary diagonal matrix with purely imaginary entries to A without changing (ej e∗j )A + A∗ (ej e∗j ). In particular, for arbitrarily given real numbers β1 , . . . , βn−1 ∈ R we can choose the ajj such that


Im(ajj − ann ) = βj ,

j = 1, . . . , n − 1 .

79

(3.23)

This will be needed at the end of the proof. i = iej ek − iek e∗j we have to verify that Setting Xjk = ej e∗k + ek e∗j and Xjk T (Xjk ) = AXjk + Xjk A∗ = aj e∗k + ak e∗j + ej a∗j + ek a∗k 

a1k .. .

···

a1j .. .

(3.24)



      a ¯kk · · · a ¯nk   ¯1k · · · 2 Re ajk · · · ajj + a    .. .. =  . .   a  · · · a ¯ + a · · · 2 Re a · · · a ¯ ¯ jj kk kj nj   1j   . . .. ..   ··· anj ank and i i i T (Xjk ) = AXjk + Xjk A∗ = iaj e∗k − iak e∗j + iej a∗k − iek a∗j (3.25)   ··· ia1j −ia1k   .. ..   . .    i¯ · · · −ia + i¯ a · · · ia + i¯ a · · · i¯ a a jk jk jj kk nk    1k   .. .. =  . (3.26) . .    −i¯ ajj − iakk · · · −i¯ akj + iakj · · · a ¯nj    a1j · · · −i¯   .. ..   . . ··· ianj −iank

Let j and k be fixed. Like in the real case we conclude that T (Xjk ) is of the general form T (Xjk ) = bj e∗j + ej b∗j + bk e∗k + ek b∗k , 

b1j .. .

with bj , bk ∈ Rn

 b1k  ..  .  · · · bjk + ¯bkj · · · ¯bnj    ..  . .  ¯ · · · 2 Re bkk · · · bnk    ..  . ···

    ¯b1j · · · 2 Re bjj   .. = .   ¯b1k · · · ¯bjk + bkj   ..  . ··· bnj

bnk

i ) has the same form with bj , bk replaced by some cj , ck ∈ Cn . Clearly T (Xjk Now we consider matrices of the form X = xx∗ with x = xj ej + xk ek where xj , xk ∈ C are arbitrary complex numbers. Writing

80

3 Linear mappings on ordered vector spaces i X = Re(¯ xj xk )Xjk − Im(¯ xj xk )Xjk + |xj |2 ej e∗j + |xk |2 ek e∗k ,

we have the decomposition T (X) = Re(¯ xj xk )(bj e∗j + ej b∗j + bk e∗k + ek b∗k ) − Im(¯ xj xk )(cj e∗j + ej c∗j + ck e∗k + ek c∗k ) + |xj |2 (aj e∗j + ej a∗j ) + |xk |2 (ak e∗k + ek a∗k ) .

(3.27)

Similarly as in (3.21) we choose Y = yy ∗ with    y1 ¯k ,  yj = x   y =  ...  ∈ Cn and yk = −¯ xj ,  arbitrary for ^ O∈ {j, k} , y L yn such that y⊥x. Then 1 1 1 1T (X), Y 2 = trace (T (X)Y ) = y ∗ T (X)y 2 2 E > 2 n n O O bLj y¯L + yk bLk y¯L = Re(¯ xj xk ) Re yj

0=

L=1

>

− Im(¯ xj xk ) Re yj

n O L=1

> 2

+ |xj | Re yj >

n O

aLj y¯L

L=1

cLj y¯L + yk

n O

E cLk y¯L

L=1

>

E 2

+ |xk | Re yk

L=1

n O

E aLk y¯L

L=1

O

¯k = Re(¯ xj xk ) Re |xk |2 bjj − x¯k xj bkj + x

bLj y¯L

L0∈{j,k}

E

O

2

− x¯j xk bjk + |xj | bkk − x¯j

bLk y¯L

L0∈{j,k}

>

O

¯k xj ckj + x ¯k − Im(¯ xj xk ) Re |xk |2 cjj − x

cLj y¯L

L0∈{j,k}

−x ¯j xk cjk + |xj | ckk − x ¯j 2

¯k xj akj + x ¯k + |xj | Re |xk | ajj − x > + |xk |2 Re

cLk y¯L

L0∈{j,k}

> 2

E

O

2

O

E

aLj y¯L

L0∈{j,k}

− x¯j xk ajk + |xj |2 akk + x ¯j

O L0∈{j,k}

E aLk y¯L

.


81

We distinguish between several special cases, where the variables xj , xk and yL are of the form xj = ξj zi , xk = ξk zk , yL = ηL zL with fixed complex numbers zj , zk , zL and real variables ξj , ξk , ηL . In these cases the right hand side is a homogeneous, identically vanishing polynomial in ξj , ξk , ηL . By inspecting the coefficients at different monomials we obtain relations between the entries of A, bj , bk , cj and ck . For each case considered, we provide a list of some monomials and the corresponding vanishing coefficients. (i)

xj = ξj , xk = ξk . ξj ξk3 ξj3 ξk ξj2 ξk2 ξj ξk2 ηL ξj2 ξk ηL

yL = ηL yL = iηL Re(bjj − ajk ) = 0 Re(bkk − akj ) = 0 Re(−bkj − bjk + ajj + akk ) = 0 Re(bLj − aLk ) = 0 Im(bLj − aLk ) = 0 Re(−bLk + aLj = 0 Im(−bLk + aLj ) = 0

Hence bjj , bkk and bLj , bLk for all ^ O∈ {j, k} have the required form. (ii) xj = ξj , xk = iξk . ξj ξk3 ξj3 ξk ξj2 ξk2 ξj ξk2 ηL ξj2 ξk ηL

yL = ηL yL = iηL Re(cjj ) − Im(ajk ) = 0 Re(ckk ) − Im(akj ) = 0 Im(cjk − ckj ) − Re(ajj + akk ) = 0 Re(cLj ) − Im(aLk ) = 0 Im(cLj ) − Re(aLk ) = 0 Im(cLk ) + Re(aLj ) = 0 Re(cLk ) + Im(aLj ) = 0

Hence cjj , ckk and cLj , cLk for all ^ O∈ {j, k} have the required form. (iii) xj = ξj , xk = (1 + i)ξk , yL = 0. Considering the coefficient at ξj2 ξk2 we obtain = 0 = Re − (1 − i)bkj − (1 + i)bjk + 2ajj D + (1 − i)ckj + (1 + i)cjk + 2akk = Re(−bkj − bjk + ajj + akk ) + Im(−bkj + bjk ) − Re(−ckj − cjk ) − Im(−ckj + cjk ) + Re(ajj − akk ) = Im(−bkj + bjk ) − Re(ckj + cjk ). Here we have made use of the corresponding relations in (i) and (ii). The proof would be complete, if we could show that Im(−bkj + bjk ) = Im(ajj − akk ) = − Re(ckj + cjk ) . But without any further specification of the Im ajj this need not be true. Hence, to finish the proof, let us consider the difference S = T − LA , which

82


also satisfies (3.15). Setting sjk = Im(−bkj + bjk − ajj + akk ) we know from the cases (i), (ii), and (iii) that i S(Xjk ) = sjk Xjk

and

i S(Xjk ) = sjk Xjk .

(3.28)

Since j and k were arbitrary we conclude that for all j < k there exist real numbers sjk , such that (3.28) holds. By the construction of A we have S(Xjj ) = 0 for all j = 1, . . . , n. As noted in (3.23) we can assume that Im(ajj − ann ) = Im(−bnj + bjn ) for 1 ≤ j < n, i.e. s1n = s2n = . . . = sn−1,n = 0 . Now let j < k < n be fixed again and set x = ej + iek + en and y = ej + ek − (1 − i)en , such that x∗ y = 0. Taking into account that S(Xjj ) = S(Xkk ) = S(Xnn ) = 0 and sjn = skn = 0 we find = D 0 = y ∗ S(xx∗ )y = y ∗ sjk Xjk y = 2sjk . Hence S = 0 and therefore T = LA .

"

We conclude this section with a simple observation. Proposition 3.4.4 Let A ∈ Kn×n . −1 (i) If ρ(A) < 1, then −SA is completely positive. (ii) If β(A) < 0, then −L−1 A is completely positive.

Proof: (i) Set P (X) = A∗ XA. If ρ(A) < 1 then ρ(P ) < 1 and hence −1 −SA =

∞ O

Pk .

k=0

This is a convergent series of completely positive operators. Since by Corollary 3.3.12 the set of completely positive is closed −SA is completely positive. (ii) Write −LA = R− − R+ with R± (X) =

1 (A ± I)X(A ± I)∗ . 2

−1 R− ) < 1 if σ(A) ⊂ C− . One readily verifies that R− is invertible and ρ(R+ By (i) −1 −1 −1 −1 −1 ◦ R− = S(A+I)(A−I) −L−1 −1 ◦ R− A = (I − R− R+ )

is a composition of completely positive operators and thus completely positive. " The complete positivity of −L−1 A can also be obtained from the following well-known result (e.g. [22, 113]), which we cite for completeness.

3.5 Linear equations with resolvent positive operators

83

Theorem 3.4.5 Let A ∈ Kn×n , such that σ(A) ⊂ C− . Then the inverse of the Lyapunov operator LA is given by < ∞ ∗ −1 LA (Y ) = − etA Y etA dt . 0

Other integral representations for L−1 A can be found e.g. in [100, 83].

3.5 Linear equations with resolvent positive operators One important problem we are confronted with, is to solve linear equations X, Y ∈ Hn×n ,

T (X) = Y

(3.29)

where T : Kn×n → Kn×n is Hermitian-preserving and resolvent positive. The difficulty lies in the fact that the complexity increases rapidly with n. 3.5.1 Direct solution 2 If we merely regard (3.29) as a linear equation in Kn×n ∼ = Kn then a direct solution requires O(n6 ) operations. Obviously, we can decrease the number of steps by a certain factor, if we make use of the fact that X and Y are Hermitian. This means that the lower triangle of these matrices does not contain any information independent of the upper triangle. Moreover, in the real case Hn is a proper T -invariant subspace of Rn×n with an orthogonal basis B given in (3.16). Assume that T is given in the form (3.6). We denote the restriction of T to Hn by T |Hn . The matrix representation of T |Hn with respect to the basis B can be obtained easily from the sum the Kronecker product representation

M =

N O

εj Vj ⊗ Vj ;

j=1

we just have to apply the transformation M L→ U T M U with U =

n : n : j=1 k=j

1 (ej ⊗ ek + ek ⊗ ej ) . <ej + ek
^ then we obtain the entrywise equations zkL =

N O j=1

=

N O j=1

εj

n O n O s=1 t=1

(j)

(j)

rkt wts r¯Ls

N n D O O (j) (j) wkL + εj εj rkk r¯ss

=

j=1

n O

s=L+1 t=k+1

(j)

(j)

rkt wts r¯Ls .

Hence we can compute the wkL recursively in O(N n4 ) operations from   N n n O O O 1 (j) (j) zkL − wkL = PN rkt r¯Ls wts  (3.32) εj (j) (j) ε r r ¯ ss j j=1 s=L+1 t=k+1 kk j=1 e.g. for the index pair (k, ^) increasing in lexicographic order. If X and Y are Hermitian, then it suffices to compute wkL for k ≥ ^. The equation is uniquely and universally solvable if and only if


∀k, s = 1, . . . , n :

λks =

N O j=1

(j)

(j) O= 0 . εj rkk r¯ss

85

(3.33)

From this it is easy to see that the λks are the eigenvalues of T . Of course, one can derive this result also with the help of the Kronecker product, taking into account that R1 ⊗ R2 is upper-triangular if R1 and R2 are. In the special case, where T = LA is a Lyapunov operator as defined in (3.10) we can do even better. Let us assume that A is already upper triangular. Then – in analogy to (3.32) – the solution to A∗ X + XA = Y can be computed recursively from   n n O O 1 yjk − a ¯Lj xLk + xjL aLk  . xjk = a ¯jj + akk L=j+1

L=k+1

This system can be resolved in O(n3 ) operations. Efficient algorithms have been developed in [11, 85, 95, 161]; see also [77] for an overview. The case, where T = SA is a Stein operator can be transformed to the previous case. We assume that 1 O∈ σ(A) and set B = (A − I)(A + I)−1 . Then the equation T (X) = A∗ XA − X = Y is equivalent to 1 ∗ (B X + XB) = (A∗ + I)Y (A + I) . 2 Remark 3.5.1 It is well-known (e.g. [78]) that a family of matrices is simultaneously triangularizable, if the members of this family are pairwise commutative. More generally, a family of matrices is simultaneously triangularizable, if and only if they generate a solvable Lie algebra (see [172]). In [203, 202] stochastic linear systems have been considered, whose coefficient matrices generate a solvable Lie algebra. This situation occurs e.g. in the example of the electrically suspended gyro in Section 1.9.7, where we had Aj0 = I for all j. From our point of view, however, the case of simultaneous triangularizability is degenerate. In general, it is, for instance, not reasonable to assume that the parameter matrices of the nominal system and the perturbed system commute. But, of course, we will make ample use of the fact that Lyapunov and Stein equations can be solved efficiently. The results of the present subsection can e.g. be found in [113]. For further material on simultaneous triangularization we refer to the monograph [165]. 3.5.3 Low-rank perturbations of Lyapunov equations In [169, 33] the case has been considered, where T is a low-rank perturbation of a Lyapunov operator. Below we will discuss how this case arises in our context. If a rank revealing factorization of the perturbation is available, one can reduce the computational effort significantly. Since the idea is not limited

86


to the situation of matrix equations, we describe it in more general terms. For the moment, regard T as an endomorphism of Kp , p ∈ N and consider a linear equation Tx = y .

(3.34)

Assume that T can be decomposed as T = L + P , where L is nonsingular, L−1 is known or easy to compute, and r := rk P < p. Assume further that we can factorize P as P = P1 P2 with linear operators P1 : Kr → Kp and P2 : Kp → Kr . Then the following holds (compare [169, Thm 2.2]. Lemma 3.5.2 Equation (3.34) is uniquely solvable if and only if I +P2 L−1 P1 is nonsingular. If w solves the equation (Ir + P2 L−1 P1 )w = P2 L−1 y ,

(3.35)

then x = L−1 (y − P1 w) solves (3.34). Proof: With our assumptions, equation (3.34) is equivalent to (Ip + L−1 P1 P2 )x = L−1 y . It is well-known (e.g. [186]) that σ(P2 L−1 P1 ) \ {0} = σ(L−1 P1 P2 ) \ {0} .

(3.36)

Hence T is nonsingular if and only if Ir + P2 L−1 P1 is. If w solves (3.35) and x = L−1 (y − P1 w), then T x = (L + P1 P2 )L−1 (y − P1 w) = y − P1 w + P1 (P2 L−1 y − P2 L−1 P1 w) = y − P1 w + P1 w = y , i.e. x solves (3.34).

"

Since (3.35) is an r-dimensional linear equation, it can be solved in O(r3 ) operations. If r is much smaller than p, and if the effort to invert L is small, then it is advantageous to compute x from the solution of (3.35). This way of representing the solution x is also known as the Sherman-Morrison-Woodbury formula, e.g. [86]. In the context of equation (3.29) the situation arises as follows. The operator T : Kn×n → Kn×n can be written as T = LA + P with a Lyapunov operator LA and a perturbation operator P . The inversion of LA is possible in O(n3 ) operations. If r = rk P , then the overall effort to solve (3.34) reduces to max{O(rn3 ), O(r3 )} operations. We give an example, where the rank of P is small. Example 3.5.3 Consider the stochastic control system (1.55) in Section 1.9.4. Let u = 0 and v = 0. By Theorem 1.5.3 the unperturbed and uncontrolled system is mean-square stable if and only if the equation


87

T (X) = A∗ X + XA + A∗0 XA0 = I has a negative definite solution X. Here    0 0 0 0 1 0 0 0  0 0 0 1     A= and A0 =  k1 /m1 −c/m1 c/m1  0 0 −k1 /m1 k1 /m2 −(k1 + k)/m2 −c/m2 c/m2 0 −σ

0 0 0 0

 0 0  . 0 0

If σ O= 0, then the operator P : X L→ A∗0 XA0 has rank 1, because A0 ⊗ A0 has only one non-vanishing entry. Similar situations arise, whenever one considers linear control systems, where only a few parameters are subject to multiplicative stochastic perturbations. This, however, is not the case for the perturbation ma(1) trix A0 in Section 1.9.5, which has full rank; hence also the mapping (1)∗ (1) X L→ A0 XA0 (viewed as an endomorphism of Kn×n ) has full rank. j j 2 In general, the Kn×n -endomorphism X L→ Aj∗ 0 XA0 has rank rj if rk A0 = rj . PN j Therefore the rank of the mapping P : X L→ j=1 Aj∗ 0 XA0 is bounded above PN 2 by j=1 rj . To obtain a rank-revealing factorization of P = P1 P2 one can j j j n×rj exploit factorizations Aj∗ and Aj2 ∈ Krj ×n . Then 0 = A1 A2 where A1 ∈ K P is represented by the matrix N O

j∗ A¯j∗ 0 ⊗ A0 =

j=1

N = O

A¯j1 ⊗ Aj1

D= D A¯j2 ⊗ Aj2 .

(3.37)

j=1

Hence, the columns of P1 are linear combinations of the columns of the matrices A¯j1 ⊗ Aj1 and the rows of P2 are linear combinations of the rows of the matrices A¯j2 ⊗ Aj2 . To make these ideas precise, we formulate them in algorithmic form. We restrict ourselves to the relevant case, where T is the sum of a Lyapunov operator and a completely positive operator, i.e. ∀X ∈ Kn×n :

T (X) = LA (X) + ΠA0 (X) .

(3.38)

Remember from (1.20) that LA (X) = A∗ X + XA

and

ΠA0 (X) =

N O

j Aj∗ 0 XA0

j=1

with A, Aj0 ∈ Kn×n . An algorithm For fixed n, we regard the vec-operator as a vector-space isomorphism vec : 2 2 Kn×n → Kn and denote its inverse by vec−1 : Kn → Kn×n . The Kroneckerproduct representation of an operator T : Kn×n → Kn×n is denoted by

88


kron T = vec ◦T ◦ vec−1 . This choice of notation is illustrated by the diagram T

Kn×n −→ Kn×n −1 vec ↑ ↓ vec Kn

2

kron T

−→ Kn

2

n×n Proposition 3.5.4 For A, A10 , . . . , AN , with σ(A) ∩ σ(−A∗ ) = ∅ and 0 ∈K n Y ∈ H , the following algorithm determines σ(L−1 A ΠA0 )\{0}. Π ), then the unique solution X ∈ Hn of (LA + ΠA0 )(X) = Y If 1 O∈ σ(L−1 A 0 A is computed.

1. 2.

For j = 1, . . . , N factorize Aj0 = Aj2 Aj∗ 1 , where Aj1 , Aj2 ∈ Kn×rj and rj = rk Aj0 . N 1 1 N N n2 ×r0 Set A1 = [A11 ⊗A11 , . . . , AN , 1 ⊗A1 ], A2 = [A2 ⊗A2 , . . . , A2 ⊗A2 ] ∈ K PN 2 r0 = j=1 rj . 2

2

Compute a matrix Q = [Q1 , Q2 ] ∈ Kn ×rq , Q1 ∈ Kn ×rq1 , rq1 ≤ r0 , rq ≤ 2r0 , with orthonormal columns, such that im A1 = im Q1 and im[A1 , A2 ] = im Q. 4. Compute H = Q∗1 kron(Π)Q ∈ Krq1 ×rq . 0 60 ∗6 Σ0 V1 5. Perform a singular value decomposition H = [U1 , U2 ] = 0 0 V2∗ U1 ΣV1∗ with unitary matrices U = [U1 , U2 ], V = [V1 , V2 ] and diagonal r , r = rk Σ = rk Π. Σ ∈ int H+ 6. Set P1 = (p1 , . . . , pr ) = Q1 U1 Σ and P2 = V1∗ Q∗ . −1 7. For k = 1, . . . , r compute mk = P2 vec(LA (vec−1 (pk ))) and set M = (m1 , . . . , mr ). 8. σ(L−1 A ΠA0 ) \ {0} = σ(M )\{0}. If 1 ∈ σ(M ), stop. −1 (Y )) and w = (I − M )−1 y. 9. Compute y = P2 vec(LA −1 −1 10. X = LA (Y − vec (P1 w)). P j j j j ∗ Proof: By (3.37) we have kron(Π) = N j=1 (A1 ⊗ A1 )(A2 ⊗ A2 ) . ⊥ Hence im kron Π ⊂ im A1 = im Q1 and (Ker kron(Π)) ⊂ im A2 ⊂ im Q. If Q3 completes [Q1 , Q2 ] to a unitary matrix, then   H1 H 2 0 kron(Π)[Q1 , Q2 , Q3 ] = [Q1 , Q2 , Q3 ]  0 0 0  , 0 0 0 3.

with [H1 , H2 ] = Q∗1 kron(Π)[Q1 , Q2 ] = U1 ΣV1∗ . Hence kron Π = Q1 U1 ΣV1∗ [Q1 , Q2 ]∗ = P1 P2 . Since M = P2 kron(L−1 A )P1 , the assertions follow from Lemma 3.5.2 and equation (3.36). "


89

Remark 3.5.5 The complexity of this algorithm depends mainly on the numbers n, N , r0 and r = rk Π. To see this, it is important to note that, based on the factorizations Aj0 = Aj1 Aj2 , the operator Π can be evaluated in O((r1 + . . . + rN )n2 ) ≤ O(r0 n2 ) operations. Indeed, each summand j j∗ j∗ j j 2 Aj∗ 0 XA0 = (A2 (A1 XA1 )A2 ) requires only O(rj n ) scalar multiplications. In particular, it follows that the matrix H in step 4.) can be computed in O(rq1 n2 rq ) ≤ O(2r02 n2 ) operations. The computational cost for the standard numerical procedures in each step can be estimated as follows (e.g. [54]). 4. 5. 6. 7. 8. 9. 10. Step 1. 2. 3. O N n3 n2 r0 r02 n2 rq1 n2 rq rq31 nrq r rn3 r3 n3 n3 Since rq ≤ r, the algorithm requires not more than O(N n3 + r02 n2 + rn3 ) operations. For small N and r0 , this is much faster than the O(n6 ) operations of a naive solution. 3.5.4 Iterative solution with different splittings As the dimension increases, it is sometimes useful to consider iterative methods. Again let T be given in the form (3.38) as the sum of a Lyapunov operator LA and a completely positive operator ΠA0 , and assume that T is stable. From Theorem 3.2.10 we can immediately derive an iterative strategy to solve (3.29), such that in each step we only have to solve a Lyapunov equation. Proposition 3.5.6 Let T be given by (3.38) and assume σ(T ) ⊂ C− . For an arbitrary X0 ∈ Hn define the sequence Xk+1 = L−1 A (Y − ΠA0 (Xk )) .

(3.39)

Then Xk converges to the unique solution of (3.29). Proof: By Theorem 3.2.10 we have ρ(L−1 A ΠA0 ) < 1, such that the result follows from Banach’s fixed point theorem. " The given splitting T = LA +ΠA0 , however, is only one possible representation of T which leads to an iterative scheme to solve T (X) = Y . Other splittings are advantageous, in particular to avoid the explicit solution of a Lyapunov equation in each step. To construct such splittings, we note that T has the form T (X) = λ0 V0∗ XV0 + λ1 V1∗ XV1 + λ2 V2∗ XV2 + . . . + λν Vν∗ XVν , (3.40) with numbers λ0 < 0, λ1 , . . . , λν > 0 and given matrices Vj . For instance, we may simply write LA (X) = −A∗− XA− + A∗+ XA+ ,

90


where 1 A− = √ (A − αI) , 2α

1 A+ = √ (A + αI) , 2α

α>0.

(3.41)

Thus LA + ΠA0 has the form (3.40). Or we can apply the following lemma. Lemma 3.5.7 If T is given by (3.38) and σ(T ) ⊂ C− , then the n2 ×n2 -matrix X = (Inn ⊗ T )(E (nn) ) has exactly one negative eigenvalue. Proof: Let A+ and A− be defined by (3.41). By Theorem 3.3.7 and by (3.11) we can write X = vec A+ (vec A+ )∗ − vec A− (vec A− )∗ +

N O

vec Aj0 (vec Aj0 )∗ .

j=1

Hence, by Sylvester’s inertia theorem, X has at most one negative eigenvalue. If all eigenvalues were nonnegative then T would be completely positive. But, since T is stable, we know by Theorem 3.2.10 that ∃X > 0 : T (X) < 0 .

(3.42) "

Hence X has at least one negative eigenvalue.

Theorem 3.3.7 therefore yields a representation of the form (3.40), where the λ0 < 0 < λ1 ≤ . . . ≤ λν are the non-zero eigenvalues of X . Now we multiply the equation T (X) = Y with V0−∗ from the left and V0−1 from the right, and devide by |λ0 |, such that T˜(X) = −X +

ν O

V˜j∗ X V˜j = Y˜

(3.43)

j=1

K

and Y˜ = |λ10 | V0−∗ Y V0−1 . The operator T˜ has the obvious regular splittings T˜ = R + P or T˜ = Rk + Pk with

with V˜j =

λj −1 |λ0 | Vj V0

R(X) = −X ,

P (X) =

Rk (X) = −X + V˜k∗ X V˜k ,

Pk (X) =

ν O

V˜j∗ X V˜j

j=1 ν O

V˜j∗ X V˜j .

j=1,j0=k

Here we recognize Rk as a Stein operator, while −R is just the identity operator. Since T is stable so is T˜ , such that by Theorem 3.2.10 these splittings in fact are convergent. From our experience, we favour the first splitting T = R + P , which leads to the particularly simple iteration scheme Xk+1 = P (Xk ) − Y˜ .

(3.44)


91

Note that this scheme does not require the solution of any standard Lyapunov or Stein equations. Nevertheless, one also might take advantage of the different splittings T = Rk + Pk with the Stein operators Rk . All the splittings discussed in this subsection lead to an iteration scheme Xk+1 = Π(Xk ) − Y˜ ,

(3.45)

where Π is a positive operator with ρ(Π) < 1. Unfortunately, in the relevant applications ρ(Π) often is approximately equal to 1. This, for instance, happens in the context of γ-subotimal H ∞ -problems, when γ is close to the optimal attenuation parameter. Hence, without any additional accelerations, the scheme (3.45) will turn out to be impracticable. In the following subsection we discuss a simple extrapolation method to overcome this problem in many situations. The basic idea can be found in [139, Section 15.4], where it is referred to as ’L.A. Ljusterniks approach for improving convergence’. 3.5.5 Ljusternik acceleration Consider the iteration scheme (3.45), where Π : Hn → Hn is positive, ρ(Π) < 1 and Y˜ < 0. By Theorem 3.2.3 of Krein and Rutman, we have ρ(Π) ∈ σ(Π), and there exists a nonzero matrix Xρ ≥ 0, such that Π(Xρ ) = ρ(Π)Xρ . We assume now, that for all eigenvalues of Π, different from ρ(Π) are strictly smaller than ρ(Π) in the sense that ∃κ < 1 : ∀λ ∈ σ(Π) \ {ρ(Π)} : |λ| ≤ κρ(Π)} .

(3.46)

In [139] the minimal number κ = κ(Π) satisfying this condition is called the spectral margin of Π. Let X0 = 0 and X∗ > 0 denote the solution to −X + Π(X) = Y˜ . A short computation shows that for all k ≥ 1 Xk+1 − Xk = Π(Xk − Xk−1 ) , Xk+1 − X∗ = Π(Xk − X∗ ) . Hence the iteration amounts to an application of the power method to X1 −X0 and X0 − X∗ . It is well-known that under the given assumptions on σ(Π) the Xk+1 − Xk and the Xk+1 − X∗ converge to eigenvectors of Π, corresponding to the eigenvalue ρ(Π). Hence, after a number of iterations the approximate relations Xk+1 − Xk ≈ ρ(Π)(Xk − Xk−1 ) , Xk+1 − X∗ ≈ ρ(Π)(Xk − X∗ )

(3.47) (3.48)

will hold. This may be detected by measuring the angle between Xk+1 − Xk and Xk − Xk−1 . We now apply an appropriate functional to both sides of (3.47) in order to approximate ρ(Π):

92


t1 = trace ((Xk+1 − Xk )V ) ,

t0 = trace ((Xk − Xk−1 )V ) ,

ρ(Π) ≈

t1 . t0

In principle, it suffices to choose an arbitrary positive definite matrix V > 0, e.g. V = I to guarantee that t1 , t0 O= 0. It turns out, however, that not all V > 0 lead to good results here. Experiments confirm that it makes sense to choose V as an approximation to the positive eigenvector of Π ∗ , provided this eigenvector is positive definite. Using the computed numbers t0 and t1 , we can find an approximation to X∗ from (3.48) as X∗ ≈

t1 Xk − t0 Xk+1 =: Xk+2 . t1 − t0

The iteration proceeds further with the scheme (3.45), until again for some ^ > k the angle between XL+1 −XL and XL −XL−1 is small enough and another acceleration step can be taken. The method has been tried on random examples, i.e. for operators Π(X) = PN ∗ j=1 Vj XVj , where the Vj are full random matrices with normally distributed entries, and ρ(Π) = 1. For such examples, assumption (3.46) holds generically, and therefore the results are quite convincing. On a logarithmic scale we have depicted the relative residual errors for the fixed point iteration with and without acceleration step. Moreover, we see the angles αk between two subsequent increments Xk+1 − Xk and Xk − Xk−1 . When this angle is small, an acceleration step is applied. n=450, N=45

logarithmic scale

0

standard Ljusternik angles

−5

−10

−15

0

2

4

2k

6

8

10

Since ρ(Π) = 1, the standard iteration becomes almost stationary, as the angles between the increments converge to zero. We observe two effects when the acceleration step is applied. First, the relative residual error decreases rapidly from about 10−1 to 10−8 in one step. This is – in principle – what we expected to achieve. Secondly, and maybe even more surprisingly, the speed


93

of convergence is increased significantly in the subsequent steps. This effect can be explained by noting that the error Xk+2 − X∗ (after the acceleration step) will be close to the invariant subspace of Π, which is complementary to the leading eigenvector. On this subspace, the convergence is determined by the second largest eigenvalue λ2 , which (generically for random examples) is much smaller than ρ(Π). So, the speed of convergence of this method depends on two factors, namely the speed of convergence of the angles αk and the modulus of λ2 . It is wellknown (e.g. [139, Thm. 15.4]) that the convergence of the αk is determined by the spectral margin κ(Π) = |λ2 |/ρ(Π) (≈ |λ2 |, if ρ(Π) ≈ 1). We conclude that the method works best, if ρ(Π) and |λ2 | are well seperated, i.e. if κ(Π) is small. Conversely, the method may fail, if there are other eigenvalues of nearly or exactly the same size as ρ(Π). If there is an eigenvalue λ2 O= ρ(Π) with |λ2 | = ρ(Π), then the αk and thus the directions of the Xk − Xk−1 will not converge. If the multiplicity of the eigenvalue ρ(Π) is greater than 1, or if there are eigenvalues λ2 O= ρ(Π) with |λ2 | ≈ ρ(Π), then the αk converge very slowly. In some of these cases, we may take advantage of a shifted iteration Xk+1 =

1 (Π(Xk ) + µXk − Y ) 1+µ

with some µ > 0. Obviously this iteration has the same fixed point X∗ , but 1 (Π + µI). The eigenvalues uses the shifted positive linear operator Πµ = 1+µ

of Πµ are λµ = small µ

λ+µ 1+µ ,

where λ ∈ σ(Π). If |λ − ρ/2| > ρ/2, then for sufficiently |λµ | |λ| |λ + µ| < . = ρµ ρ+µ ρ

Hence, if ρ(Π) is a simple eigenvalue R and the second largest eigenvalue λ2 is not contained in the circle {λ ∈ C R |λ − ρ/2| ≤ ρ/2}, then for appropriate µ > 0 the spectral margin of Πµ will be smaller than that of Π. The concrete choice of the shift parameter µ depends on our knowledge on the spectrum of Π. If |λ| = ρ(Π) for all λ ∈ σ(Π), then it is easy to show that the optimal choice is µ = ρ(Π). Usually we would choose e.g. µ = ρ(Π)/2 or µ = ρ(Π)/3. Obviously, the shift will not help, if there are positive real eigenvalues very close to ρ(Π). 3.5.6 Conclusions Basically, we have identified two cases, when an efficient solution of the equation (LA + ΠA0 )(X) = Y for large n is possible. Namely, the case, when PN j 2 j=1 rk A0 K n and the case, when the spectral margin of Π in (3.45) is small.

94


In the first case, a direct solution can be found relatively cheaply, while in the second an accelerated iterative method could be superior. We just mention that for large linear systems there are modern powerful iterative methods, to be found e.g. in [88]. These methods are particularly efficient, if a certain sparsity structure is given. For standard Lyapunov equations such methods have been applied in [110], but for generalized Lyapunov we are not aware of any particular results in this direction.

3.6 Recapitulation: Resolvent positivity, stability and detectability The analogy between Theorem 1.5.3 and Theorem 3.2.10 is striking. Since these results form the basis of our investigations we state them again in a unified and extended version. (1)

(N )

Theorem 3.6.1 Let A, A0 , . . . , A0 ∈ Kn×n and define LA and ΠA0 according to (1.20). With these data consider the homogeneous linear stochastic differential equation (1.13). The following are equivalent: (a) (b) (c) (d) (e) (f ) (g) (h) (i) (j) (k) (l) (m)

System (1.13) is asymptotically mean-square stable. System (1.13) is exponentially mean-square stable. σ (LA + ΠA0 ) ⊂ C− . max σ(LA + ΠA0 ) ∩ R < 0. σ (LA ) ⊂ C− and ∀τ? ∈ [0, 1] :Fdet (LA + τ ΠA0 ) O= 0. σ (LA ) ⊂ C− and ρ L−1 A ΠA0 < 1. ∀Y < 0 : ∃X > 0 : LA (X) + ΠA0 (X) = Y . ∀Y ≤ 0 with (A, Y ) observable: ∃X > 0 : LA (X) + ΠA0 (X) = Y . ∃X > 0 : LA (X) + ΠA0 (X) < 0. ∃X ≥ 0 : LA (X) + ΠA0 (X) < 0. ∃Y ≤ 0 with (A, Y ) observable: ∃X ≥ 0 : LA (X) + ΠA0 (X) ≤ Y . ∃Y ≤ 0 with (A, (Aj0 ), Y ) β-detectable: ∃X ≥ 0 : LA (X) + ΠA0 (X) ≤ Y . −1 − (LA + ΠA0 ) exists and is completely positive.

In a sense this theorem parallels [14, Theorem 2.3], which provides even fifty equivalent conditions for a Z-matrix to be an M -matrix; see also [67, 199, 155]. One might doubtlessly find further equivalent conditions for the operator LA + ΠA0 to be stable, but we restrict us now to the verification of (d), (e), (h), (k), and (m) which were not explicitly part of the Theorems 1.5.3 and 3.2.10. Proof: ‘(c) ⇐⇒ (d) ⇐⇒ (e)’: Since LA + ΠA0 is resolvent positive, we have β(LA + ΠA0 ) = max σ(LA + ΠA0 ) ∩ R by Theorem 3.2.3. Hence (c) ⇐⇒ (d). Moreover β (LA + τ ΠA0 ) increases monotonically with τ by Corollary 3.2.4. Hence β(LA + ΠA0 ) < 0

⇐⇒

∀τ ∈ [0, 1] : β (LA + τ ΠA0 ) < 0

3.6 Recapitulation: Resolvent positivity, stability and detectability

95

and hence (e). Conversely assume that (e) holds but β(LA + ΠA0 ) > 0. Then for some τ ∈]0, 1[ we have β(LA + τ ΠA0 ) = 0 and hence det(LA + τ ΠA0 ) = 0. This contradicts our assumption. ‘(h)⇒(k)’ is trivial. ‘(k)⇒(e)’ (k) implies A∗ X + XA ≤ Y , and it is well-known (e.g. [113, Thm. 2.4.7]) that this together with the observability of (A, Y ) implies X > 0 and σ(A) ⊂ C− . Suppose LA (X0 ) + τ Π(X0 ) = 0 for some τ ∈ [0, 1] and consider the convex combination Xα := αX + (1 − α)X0 . By our assumptions LA (Xα ) ≥ αY − (1 − α)τ Π(X).

(3.49)

We want to show R X0 ≤ 0 and X0 ≥ 0. Assume first X0 O≥ 0 and set α0 = max{α ∈ [ 0, 1] R Xα O> 0}. Since X1 = X > 0 we have Xα0 ≥ 0, such that α0 > 0 and LA (Xα0 ) ≤ α0 Y . This, however, implies Xα0 > 0, in contradiction to the choice of α0 . Now assume X0 O≤ 0, then the last argument can be repeated with X replaced by −X < 0 and inverted order. Thus Ker (LA + τ Π) = {0}. ‘(g)⇒(h)’ By continuity, the assertion (h) holds with X ≥ 0. But as above, one proves X > 0. ‘(f)⇒(m)’: If σ(LA ) ⊂ C− then −L−1 A is completely positive by Proposition 3.4.4. By definition ΠA0 is completely positive. Since ρ(L−1 A ΠA0 ) < 1 we can write −1 −1 LA = −(LA + ΠA0 )−1 = −(I + L−1 A ΠA 0 )

∞ O

−1 k (−L−1 A ΠA0 ) (−LA ) .

k=0

This is a convergent series of completely positive operators and hence by Corollary 3.3.12 completely positive. −1 n ‘(m)⇒(i)’: As a nonsingular positive operator, − (LA + ΠA0 ) maps int H+ to itself. "

Remark 3.6.2 Some of these assertions are taken from [176, 45, 48]. The implication ‘(l)⇒(c)’ can be found in [69] (with a lengthy proof covering about three pages). Theorem 3.6.1 transforms the stochastic mean-square stability problem for n-dimensional systems to various n2 -dimensional algebraic problems. It was already already asked in [132] whether one can reduce the dimension of the algebraic problem. In [212] the following sufficient stability-criterion has been used. Corollary 3.6.3 If σ(A) ⊂ C− and Q< Q Q Q ∞ tA∗ Q Q −1 tA QL (ΠA0 (I))Q = Q e ΠA0 (I)e dtQ A Q Q < 1, 0

(3.50)

96


then σ(LA + ΠA0 ) ⊂ C− . This condition roughly means that the noise effect on the system is not too n large. It implies that for all X ∈ H+ (not only for the eigenvectors of L−1 A ΠA 0 ) we have −1 0 :

sup

y∈D+ ,'y−x' 0 then σ(Txk+1 ) ⊂ C− by Theorem 3.2.10, which completes the proof of (i) under conditions (H1)–(H5) and (H8). To complete the proof of (i) under Assumption 4.2.1 let us suppose that Txk+1 is not stable. By Theorem 3.2.3 (ii) this is equivalent to the condition ∃v ∈ C ∗ \ {0}, β ≥ 0 : Tx∗k+1 v = βv .

(4.8)

Together with the four inequalities x ˆ ≤ xk+1 , f (ˆ x) ≥ 0, f (xk+1 ) ≤ 0 and (4.6), this implies " # x − xk+1 , βv2 = Txk+1 (ˆ x − xk+1 ), v ≥ 1−f (xk+1 ), v2 ≥ 0.(4.9) 0 ≥ 1ˆ Hence 1f (xk+1 ), v2 = 0, which by (4.3) means 1f (xk+1 ) − f (xk ), v2 = 1Txk (xk+1 − xk ), v2 . But xk+1 ≥ xˆ ∈ int D+ and D+ = D+ + C imply xk+1 ∈ int D+ , and so f is Gâteaux differentiable at xk+1 by condition (H7). From Proposition 4.1.5 (with K = X) we conclude that Txk+1 = fx3 k+1 , and Tx∗k (v) = (fx3 k+1 )∗ (v) = Tx∗k+1 (v) = βv, in contradiction with σ(Txk ) ⊂ C− . Thus Txk+1 must be stable, and this concludes our proof of (i) under Assumption 4.2.1. (ii) By (i) and the regularity of C, the xk converge in norm to some x+ ∈ x ˆ + C ⊂ D+ . Since the Tx are locally bounded on D+ , we can pass to the limit in (4.3) to obtain f (x+ ) = 0. By the first part of the proof the inequality xk+1 ≥ x ˆ holds true for all k ∈ N and all solutions xˆ ∈ D+ of the inequality f (x) ≥ 0 (we only used x ˆ ∈ D+ and not x ˆ ∈ int D+ in the first part of the proof). Therefore x+ is the largest solution of this inequality in D+ . " We say that a solution x˜ ∈ dom f of f (x) = 0 is stabilizing, if σ(Tx˜ ) ⊂ C− . The following lemma shows that there can be at most one stabilizing solution in D+ . Lemma 4.2.3 Suppose that f is D+ -concave on D with resolvent positive Tx and let y ∈ D+ , z ∈ D and f (z) ≤ f (y). If σ(Tz ) ⊂ C− then z ≥ y. In particular, under the assumptions of Theorem 4.2.2, if z ∈ D+ is a stabilizing solution of f (x) = 0 then z = x+ .

108

4 Newton’s method

Proof: By concavity Tz (y − z) ≥ f (y) − f (z) ≥ 0, whence by the stability and resolvent positivity of Tz we have y − z ≤ 0. Now suppose z ∈ D+ is a stabilizing solution of f (x) = 0, then we choose y = x+ and get z ≥ x+ . " On the other hand we have z ≤ x+ by Theorem 4.2.2. The following corollary of Theorem 4.2.2 gives a sufficient condition for the existence of a stabilizing solution. Corollary 4.2.4 If conditions (H1)–(H5) and (H8) hold then x+ in Theorem 4.2.2 is a stabilizing solution of f (x) = 0 and satisfies x+ > xˆ (whence x+ ∈ int D+ ). Proof: We already know x+ ≥ xˆ, and by concavity Tx+ (ˆ x − x+ ) ≥ f (ˆ x) − f (x+ ) = f (ˆ x) > 0 ; ? F hence σ Tx+ ⊂ C− and x+ > x ˆ follows from Theorem 3.2.10.

"

The existence of a stabilizing solution can also be proven under a detectability assumption (cf. Def. 3.2.8) instead of (H8). Corollary 4.2.5 Assume that f is concave and Gˆ ateaux-differentiable on D+ . ˆ ≤ x1 and f (ˆ x) ≥ f (x1 ). If one of the pairs ˆ, x1 ∈ int D+ satisfy x Let x (Txˆ , f (ˆ x) − f (x1 )) or (Tx1 , f (ˆ x) − f (x1 )) is β-detectable, then Tx1 is stable. x)) is βIn particular, if conditions (H1)–(H7) hold and the pair (Txˆ , f (ˆ detectable, then x+ in Theorem 4.2.2 is a stabilizing solution of f (x) = 0. Proof: By concavity, we have Tx1 (ˆ x − x1 ) ≥ f (ˆ x) − f (x1 ) ≥ 0 . We assume that Tx1 is not stable, i.e. β = β(Tx1 ) ≥ 0, and there exists an eigenvector v ∈ C ∗ , satisfying Tx∗1 v = βv. This implies 0 ≥ 1ˆ x − x1 , βv2 = 1Tx1 (ˆ x − x1 ), v2 ≥ 1f (ˆ x) − f (x1 ), v2 ≥ 0 , such that in particular 1Tx1 (ˆ x − x1 ), v2 = 1f (ˆ x) − f (x1 ), v2 = 0. Hence, (Tx1 , f (ˆ x) − f (x1 )) is not β-detectable. Moreover, like in the proof of Theorem 4.2.2 we conclude from Proposition 4.1.5 that Txˆ∗ (v) = Tx∗1 (v) = βv, which implies that (Txˆ , f (ˆ x) − f (x1 )) is not β-detectable either. It follows that each of the detectability assumptions entails the stability of Tx1 . In particular, under the assumptions (H1)–(H7) we can choose x+ from Theorem 4.2.2 in the role of x1 . " We can express this result also as follows. Corollary 4.2.6 Assume (H1)–(H4) and (H7). Then the following are equivalent:

4.2 Resolvent positive operators and Newton’s method

109

(i) ∃x+ ∈ int D+ : f (x+ ) = 0, σ(Tx+ ) ⊂ C− . (ii) ∃ˆ x, x1 ∈ int D+ : f (ˆ x) ≥ 0, (Txˆ , f (ˆ x)) is β-detectable, x1 ≥ xˆ, and f (x1 ) ≤ 0. Proof: If (i) holds, then obviously (ii) holds with x ˆ = x1 = x+ . Conversely, (ii) obviously implies (H6); moreover, by Corollary 4.2.5, (ii) im" plies (H5) with x0 = x1 . Hence, by Theorem 4.2.2, (ii) implies (i). If f is Fréchet differentiable, condition (H8) is equivalent to the existence of a stabilizing solution of f (x) = 0. Corollary 4.2.7 Assume conditions (H1)–(H5). If f is Fréchet differentiable on int D+ , then x) > 0) ⇐⇒ (∃y ∈ int D+ : f (y) = 0 and σ(fy3 ) ⊂ C− ) . (∃ˆ x ∈ D+ : f (ˆ Proof: Fréchet differentiability implies Gˆ ateaux differentiability and by Proposition 4.1.5 we have fx3 = Tx for all x ∈ int D. By Corollary 4.2.4 it only remains to prove ‘⇐’. Let y ∈ int D+ and f (y) = 0, σ(fy3 ) ⊂ C− . Then 0 O∈ σ(fy3 ) and so fy3 is an invertible bounded linear operator on X. By the inverse function theorem, f maps a small neighbourhood U ⊂ D+ of y onto a neighbourhood V of f (y) = 0. Choosing c ∈ V ∩ int C we see that there exists x ˆ ∈ U such that f (ˆ x) = c > 0. " The existence of stabilizing solutions implies quadratic convergence of the sequence (xk ), provided f is sufficiently smooth. This is e.g. a consequence of the following well known result (compare [148, 137, 36]) and also Remark 4.2.15). Proposition 4.2.8 Assume the situation of Theorem 4.2.2, and let f be Fréchet differentiable in a neighbourhood U of x+ . Moreover, assume that the Tx satisfy a Lipschitz condition 0 then σ(Txk+1 ) ⊂ C− (H1’) - (H6’) with K = −C. If we assume f (ˆ follows as in Theorem 4.2.2. This concludes the proof of (i) for the alternative assumptions. Now assume that Assumption 4.2.10 holds and suppose that Txk+1 is not stable. By Theorem 3.2.3 this is equivalent to the existence of v ∈ C ∗ \ {0} satisfying condition (4.8), which by the inequalities (4.9) implies 1f (xk+1 ), v2 = 0. Hence by (4.11) and (4.13) 1f (xk+1 ) − f (xk ), v2 = 1Rxk (xk+1 − xk ), v2 ≥ 1Txk (xk+1 − xk ), v2 ≥ 1f (xk+1 ) − f (xk ), v2 , where we obviously have equality everywhere. Since xk+1 − xk ∈ int K we ∗ can apply Proposition 4.1.5 to obtain Tx∗k (v) = f 3 xk+1 (v) = Tx∗k+1 (v) = βv, in contradiction to σ(Txk ) ⊂ C− . Thus Txk+1 must be stable. (ii) By (i) and the regularity of C, the xk converge in norm to some x+ ∈ [ˆ x, x0 ]. Since the Tx are locally bounded for x ∈ [ˆ x, x0 ], we can pass to the x, x0 ] satisfy f (x) ≥ 0. limit in (4.11) to obtain f (x+ ) = 0. Now let x ∈ [ˆ Proceeding as in the first part of the proof of (i) with x instead of xˆ, we obtain x ≤ xk+1 and so by induction x ≤ xk for all k ∈ N, whence x ≤ x+ .

112

4 Newton’s method

" Loosely speaking, the alternative in the assumption of Theorem 4.2.11 says that we can trade the smoothness condition (H7’) for the strictness condition f (ˆ x) > 0. But from the proof it is easily seen that we can substitute both these assumptions by another strictness condition. Corollary 4.2.12 Assume the hypotheses (H1’)–(H6’) with the stronger ref (x0 ) > 0 (for inquirement of strict concavity in (H2’). Moreover, let Rx−1 0 stance f (x0 ) < 0). Then the assertions (i) and (ii) of Theorem 4.2.11 hold, with strict inequalities ˆ and f (xk+1 ) < 0 for all k = 0, 1, . . .. in (i), i.e. xk+1 > xk > x Proof: The operator −Rx0 is inverse positive by (H3’) and (H5’). To start ˆ. Since x ˆ ≤ x0 by assumption, this follows an induction, we verify that x0 > x by concavity from f (ˆ x) − f (x0 ) ≤ fx3 0 (ˆ x − x0 ) which implies x − x0 ) ≤ Rx0 (ˆ x ˆ − x0 ≤ Rx−1 (f (x0 )) < 0. (f (ˆ x) − f (x0 )) ≤ −Rx−1 0 0 Now we assume that for some k ≥ 0 we have constructed x0 > x1 > . . . > xk > x ˆ, such that Txi is stable (hence −Rxi inverse positive) for 0 ≤ i ≤ k, and f (x0 ) > 0. f (xi ) < 0 for 1 ≤ i ≤ k. If k = 0 then by assumption x0 − x1 = Rx−1 0 If k ≥ 1 then Rxk (xk − xk+1 ) = f (xk ) < 0. Hence xk > xk+1 in both cases. Applying the strict concavity condition we can derive the formulae (4.12) and (4.13) in a form, where the second respectively the first inequality is strict. ˆ < xk+1 and the strict concavity Hence x ˆ < xk+1 , and f (xk+1 ) < 0. From x we infer strict inequality in (4.14) and so the stability of Txk+1 follows by Theorem 3.2.10. "

Remark 4.2.13 The main effort in the proofs of Theorem 4.2.11 and Corollary 4.2.12 (and similarly Theorem 4.2.2) is made to establish the posifor all k. Here we need the resolvent positivity of the Rx , tivity of −Rx−1 k the stability of Tx0 , and either assumption (H7’), or one of the strictness conditions. Instead of making these assumptions we could, of course, also assume the inverse positivity of −Rx for all x ∈ [ˆ x, x0 ] (compare [196], e.g. Theorem 5.4.). This property does not follow from any combination of our assumptions. In the context of Riccati equations, it is in general too restrictive (cf. Example 4.4.6). It is clear that even under the conditions of Proposition 4.2.8 we cannot expect quadratic convergence of the sequence (xk ) in Theorem 4.2.11. But we have at least linear convergence, i.e. there exists a constant 0 ≤ θ < 1 and an equivalent norm < · 0 the set N\Qθ is finite, then convergence takes place quadratically with factor κ. If, however, fx3 + is singular or almost singular, such a θ does not exist or it might be close to the machine precision. Hence for (almost) arbitrarily small θ > 0 there exists an index k with 0 and that R(tI) > 0 for sufficiently small t > 0. Thus Theorems 4.2.2 and 4.2.11 can be applied to solve the equation R(X) = 0. As a starting point we can choose X0 = tI for sufficiently large t > 0. If we apply the Newton iteration, we have to solve a linear equation of the form L−Xk (Π ∗ (X −1 )+Bˆ ∗ ) (Hk ) + Π(Hk ) + Xk Π ∗ (Xk−1 Hk Xk−1 )Xk = R(Xk ) (4.23) k

for Hk in each step to obtain the next iterate Xk+1 = Xk − Hk . Since the number of unknown scalar entries in Hk is n(n + 1)/2, a direct solution of this equation would require O(n6 ) operations. As has been pointed out in [92], and in view of Theorem 4.2.11, it might be advantageous to solve the following simplified equation for Hk L−Xk (Π ∗ (X −1 )+Bˆ ∗ ) (Hk ) = R(Xk ) . k

(4.24)

This is just a Lyapunov equation and can be solved efficiently e.g. by the Bartels-Stewart algorithm [11] in O(n3 ) operations. But obviously there is a trade-off between the rate of convergence and the complexity of the linear equations to be solved. Numerical experiments affirm that one can benefit from a combination of both methods. The idea is to use the cheaper fixed point iteration given by (4.24) until the iterates approach the solution when one can really exploit the fast quadratic convergence of Newton’s method. 4.4.2 A non-symmetric Riccati equation In a series of papers (see [93] and references therein) non-symmetric algebraic Riccati equations have been analyzed that arise in transport theory or in the Wiener-Hopf factorization of Markov chains. These equations have the general form R(X) = XCX − XD − AX + B = 0 ,

(4.25)

with (A, B, C, D) ∈ Rm×m × Rm×n × Rn×m × Rn×n . For arbitrary p, q ∈ N we regard Rp×q as an ordered vector space with the

4.4 Illustrative applications

119

p×q closed, solid, pointed convex cone R+ . The operator R thus maps the orm×n dered vector space R to itself. We are looking for the smallest nonnegative solution of the equation R(X) = 0. Like in [93] we make the following assumptions.

Assumption 4.4.3 (i) The linear operator H L→ −(AH + HD) on Rm×n is resolvent positive and stable. (ii) B ≥ 0, and ∃G > 0 : AG + GD = B. (iii) C ≥ 0, C O= 0. By definition, (i) holds if and only if I ⊗ A + DT ⊗ I is an M -matrix (Example 3.2.2). m×n rather than concave, As the condition C ≥ 0 forces R to be convex on R+ we substitute Y = −X and make the transformation f (Y ) = −R(−Y ). Now we are looking for the largest nonpositive solution of the equation f (Y ) = −Y CY − Y D − AY − B = 0,

Y ∈ Rm×n .

The derivative of f at some Y in direction H is given by fY3 (H) = −(A + Y C)H − H(D + CY ) . For Y, Z ∈ Rm×n with Y > Z or Z > Y it follows from (iii) that f (Z) − f (Y ) + fZ3 (Y − Z) = (Z − Y )C(Z − Y ) > 0 . . For Y ≤ 0 the derivative Hence f is strictly concave in direction K = −Rm×n + fY3 is resolvent positive, because H L→ −(AH + HD) is resolvent positive by 4.4.3 (i), and the mapping H L→ −(Y CH + HCY ) is positive, by Y C, CY ≤ 0. Also f03 is stable by 4.4.3 (i) and f (0) = −B ≤ 0. Hence, for any pair Yˆ ≤ Y0 = 0 the conditions (H1’) – (H5’) are fulfilled, with RY = TY = fY3 . Moreover, we have strict concavity in (H2’) and R0−1 f (0) = G > 0. If finally we assume the existence of some solution Yˆ ≤ 0 of the inequality f (Yˆ ) ≥ 0, then also (H6’) is satisfied; hence Corollary 4.2.12 can be applied to rediscover Theorem 2.1 from [93]. Corollary 4.4.4 Consider the operator R from (4.25) and let Assumption −1 ˆ ≤0 4.4.3 hold. Set X0 = 0 and Xk+1 = Xk −R3 Xk (R(Xk )) for k ≥ 0. If R(X) 3 ˆ ≥ 0, then σ(R ) ⊂ C+ and the Xk are well defined for all k. The for some X Xk ˆ and converge Xk are strictly monotonically increasing, bounded above by X, m×n to the smallest solution X∞ of (4.25) in R+ . Furthermore (by continuity) σ(R3X∞ ) ⊂ C+ ∪ iR. Remark 4.4.5 (i) In general, we can not apply Theorem 4.2.2 or 4.2.11 in ˆ < 0 or C = 0. This is because f this framework unless we assume R(X) ˜ ˜ ⊃ Rm×n \{0}. is not concave in any direction K with int K +

120

4 Newton’s method

(ii) In [93] also other fixed point iterations of the form (4.10) have been considered. In terms of the transformed operator f they amount to a splitting fY3 (H) = RY (H) + PY (H) with a resolvent positive, stable operator RY (H) = R0 (H) = −(A0 H + HD0 ), such that A0 ≥ A and D0 ≥ D. The operator PY (H) = (A0 − A − Y C)H + H(D0 − D − CY ) is then positive for all Y ≤ 0, whereas −RY = −R0 does not depend on Y , is resolvent positive and stable by assumption, hence inverse positive for all Y ∈ [Yˆ , 0]. As has been pointed out in [93] we can drop the second part of Assumption 4.4.3 (ii) in this case. This is the situation described in Remark 4.2.13. 4.4.3 The standard algebraic Riccati equation In the following chapter, we will apply our results on Newton’s method to generalized Riccati equations. Now, as an illustration we consider the technically easier case of a standard Riccati equation. In particular, we use this example to demonstrate that our results on Newton’s method do not follow from Vandergraft’s results mentioned in Remark 4.2.13. n , such that (A, Q) is stabilizable and the pair (A, P ) Let A ∈ Kn×n , P, Q ∈ H+ is (β-)detectable. Then one easily checks that the operator R : Hn → Hn defined by R(X) = A∗ X + XA + P − XQX ,

with

D = D+ = H n

satisfies Assumption 4.2.1. Hence, it follows from Theorem 4.2.2 and the Coroln laries 4.2.4 and 4.2.5 that there exists a stabilizing solution X+ ∈ int H+ , satisfying R(X+ ) = 0. If X0 ≥ X+ is any stabilizing matrix for R, i.e. σ(R3X0 ) ⊂ C− , then the Newton sequence starting at X0 converges monotonically to X+ . It is, however, not necessarily true that σ(RX ) ⊂ C− for all X ∈ [X+ , X0 ], as the following example demonstrates. 6 0 −1 6 , P = 0 and Q = I. Since A is stable, the Example 4.4.6 Let A = 0 −1 stabilizability and detectability assumptions are trivially satisfied. The Riccati operator and its derivative are given by R(X) = A∗ X + XA − X 2 , R3X (H) = (A∗ − X)H + H(A − X) . ˆ = 0 is the largest solution of the inequality R(X) ≥ 0, since Clearly X 0 6 2 −1 3 R0 is stable. Also e.g. X0 = 3I is stabilizing, whereas X = −1 1

0

4.4 Illustrative applications

ˆ but A − X = −3 7 satisfies X0 ≥ X ≥ X, 1 −2 stabilizing.

6

121

is not stable, i.e. X is not

If we abandon the detectability assumption, then R3X+ might be singular and the convergence of the Newton iteration may not be quadratic. In this case the use of a double Newton step is very advantageous. In fact, the conditions (4.16) and (4.17) are satisfied with L1 = 0, since the equation is quadratic, such that the remainder term is constant: φX (H) = −HQH for all X ∈ Hn . Hence 2(R(X1 ) − R(X0 )) = R3X0 (X1 − X0 ) + R3X1 (X1 − X0 ) , for all X0 , X1 ∈ Hn . In particular, if X1 = X+ and X+ − X0 ∈ Ker R3X+ , then X+ = X0 − 2(R3X0

−1

(R(X0 )) ,

i.e. the exact solution is obtained at once by a double Newton step (compare [94]). Finally, we mention that some of the properties of the Newton iteration can be illustrated by the simplest scalar examples. Example 4.4.7 (i) Let D = D+ = R and set f (x) = 1 − x2 . The (continuous-time algebraic Riccati-)equation f (x) = 0 has the solutions x− = −1 and x+ = 1. Since f 3 (x) = −2x, a number x ∈ D is stabilizing, if and only if x > 0. In particular, x+ is the unique stabilizing solution. The Newton sequence defined by xk+1 = (x2k + 1)/(2xk ) converges to x+ , if and only if x0 > 0, i.e. x0 is stabilizing. The sequence is monotonic, if x0 ≥ 1. Obviously, xk ≥ 1 for k ≥ 1, if x0 > 0. Hence, the sequence is guaranteed to be monotonic after the first step. But the smaller x0 ≥ 0 is chosen, the further x1 is away from x+ . (ii) Let D = R\{0}, D+ = R+ \{0} and set f (x) = −x+ 25 − x1 . The (discretetime algebraic Riccati-)equation f (x) = 0 has the solutions x− = 12 and x+ = 2. Since f 3 (x) = −1 + x12 , a number x ∈ D is stabilizing, if and only if |x| > 1. Hence x+ ∈ D+ is the unique stabilizing solution. From f 33 (x) = −2 x3 , we see that f is concave over D+ and convex over ] − ∞, 0[. But f is also D+ -concave on D, because for all x O= 0 and y > 0 we have 1 5 1 5 1 − + ( 2 − 1)(y − x) + y − + 2 x x 2 y 1 y > 0. = 2+ x y

f (x) + fx3 (y − x) − f (y) = −x +

Hence, at least for all |x0 | > 1 the Newton sequence xk+1 = converges to x+ = 2.

2xk −(5/2)x2k 1−x2k

5 Solution of the Riccati equation

In Chapter 2 we have discussed various optimal and worst-case stabilization problems for linear stochastic control systems and reformulated them in terms of rational matrix inequalities. Now we analyze these matrix inequalities and the corresponding matrix equations. To this end, we first introduce an abstract form of the Riccati operators met in the Sections 2.1 – 2.3, and the definite and the indefinite constraints mentioned in Remark 2.3.7. Recall that the LQ-stabilization problem and the Bounded Real Lemma lead to Riccati equations with definite constraints, while the disturbance attenuation problem involves an indefinite constraint. We have already noted in Remark 2.3.6 that the LQ-stabilization problem and the worst-case perturbation problem corresponding to the Bounded Real Lemma are special cases of the disturbance attenuation problem. From this point of view, it suffices to discuss the Riccati equation with indefinite constraint. Not surprisingly, however, the equation with definite constraint is easier to handle than the one with indefinite constraint. In fact, it fits exactly into the framework developed in the previous chapter, while the indefinite constraint does not match with the concavity assumptions invented in Section 4.1. Hence we discuss the definite case independently; we derive criteria for the existence of stabilizing solutions and their dependence on parameters. Motivated by results for standard Riccati equations (which we review briefly in Section 5.1.5), we apply a duality transformation to tackle the indefinite case. Thus, we obtain a dual Riccati operator, which also fits into the framework of the previous chapter. The proof that the dual operator possesses the required concavity properties constitutes the main technical obstacle in the present chapter. Using this result, we can derive criteria for the existence of stabilizing solutions and their dependence on parameters, which resemble the criteria for standard Riccati equations. Moreover, since our method is constructive, we can approximate these solutions numerically by Newton’s method or a modification of it. In fact, under fairly general detectability conditions, we even derive means to find stabilizing matrices needed to start the Newton

T. Damm: Rational Matrix Equations in Stochastic Control, LNCIS 297, pp. 123–179, 2004. Springer-Verlag Berlin Heidelberg 2004

124


iteration. These results are illustrated in Section 5.5 by numerical examples based on the applications in Section 1.9.

5.1 Preliminaries In the following we introduce a general framework for constrained Riccati equations and inequalities. This framework is aligned with the matrix equations developed in Chapter 2. From our point of view, a Riccati operator is always associated to stochastic linear control system and a certain cost functional – and vice versa. We have to distinguish between control systems with just one input vector and systems with two independent input vectors. If there is just one input the control problem lies in either minimizing or maximizing the cost functional. If there are two independent inputs, then one input is to be chosen to minimize the maximum cost achievable by the other input. As we have mentioned in Remark 2.3.7(ii) this is also called a two player game. The problems with just one input lead to Riccati equations with a definite constraint, while the problems with two inputs lead to Riccati equations with indefinite constraint. In both cases the Riccati operator has basically the same form, and it is the constraint, which tells us, whether we are solving a one-player or a two-player problem. Of course, we can always introduce a trivial inactive second player into a one-player game. In the same way, we can interpret every definite constraint as an indefinite one. To develop a general theory for Riccati equations with indefinite constraint, we will, however, have to rely on certain definiteness assumptions (Assumption 5.1.3; compare also the regularity assumption of Definition 2.3.3). Hence it follows that not every Riccati equation with definite constraint fits into this theory. Therefore we will discuss the two situations independently. 5.1.1 Riccati operators and the definite constraint X ∈ dom+ R Definition 5.1.1 Let n ∈ N. We say that an operator R, mapping some subset of Hn to Hn is a Riccati operator on Hn , if the following hold. For certain numbers s, N ∈ N and all j ∈ {1, 2, . . . , N }, there exist matrices A, Aj0 ∈ Kn×n ,

B, B0j ∈ Kn×s ,

P0 ∈ Hn ,

Q0 ∈ H s ,

S0 ∈ Kn×s . (5.1)

These data define affine linear operators P : Hn → Hn , S : Hn → Kn×s , and Q : Hn → Hs by P (X) = A∗ X + XA +

N O

j Aj∗ 0 XA0 + P0 ,

j=1

S(X) = XB +

N O j=1

j Aj∗ 0 XB0 + S0 ,

5.1 Preliminaries

Q(X) =

N O

125

B0j∗ XB0j + Q0 .

j=1

Then R is well-defined on its domain

R dom R = {X ∈ Hn R det Q(X) O= 0} ,

and for all X ∈ dom R it is given by R(X) = P (X) − S(X)Q(X)−1 S(X)∗ . Moreover, we define the target set dom+ R ⊂ dom R as R dom+ R = {X ∈ dom R R Q(X) > 0} , and call the requirement X ∈ dom+ R a definite constraint on X. Henceforth, whenever we speak of a Riccati operator R, we tacitly assume a set of data (5.1) to be given. In particular, it makes sense to speak of the control system corresponding to R, which has the form dx(t) = Ax(t) dt +

N O j=1

Aj0 x(t) dwj (t) + Bu(t) dt +

N O

B0j u(t) dwj (t) . (5.2)

j=1

Sometimes it is useful to arrange the matrices P (X), S(X), and Q(X) in a Hermitian block matrix. Then we consider the affine linear matrix operator 0 6 P (X) S(X) X L→ = Π(X) + Λ(X) + M ∈ Hn+s , S(X)∗ Q(X) where Π, Λ : Hn → Hn+s , and M ∈ Hn+s , are given by 0 6 P0 S0 M = , S0∗ Q0 0 ∗ 6 A X + XA XB Λ(X) = , B∗X 0 N 0 j∗ j6 O A0 XAj0 Aj∗ 0 XB0 Π(X) = . B0j∗ XAj0 B0j∗ XB0j j=1 We observe that Π is a completely positive operator and recall that = D R(X) = S (Π(X) + Λ(X) + M )/Q(X) for X ∈ dom R. If Λ and Π are fixed (i.e. the control system (5.2) is fixed), we also write R = RM to emphasize the dependence of the operator R on the weight matrix M .

126


5.1.2 The indefinite constraint X ∈ dom± R Let R be a Riccati operator on Hn with the given data (5.1). Assume that numbers m, ^ ∈ N are given, such that s = m + ^, and partition the given matrices (for j = 1, . . . , N ) as B = [B2 , B1 ] j j , B10 ] B0j = [B20 S0 = 0 [S20 , S10 ] 6 Q20 S30 Q0 = ∗ S30 Q10

with B2 ∈ Kn×m , B1 ∈ Kn×L , j j with B20 ∈ Kn×m , B10 ∈ Kn×L , n×m with S20 ∈ K , S10 ∈ Kn×L ,

(5.3)

with Q20 ∈ Hm , Q10 ∈ HL , S30 ∈ Km×L .

Analogously, we partition the operators S and Q in the form S(X) = [S2 (X), S1 (X)] with S2 (X) ∈ Kn×m , S1 (X) ∈ Kn×L 0 6 Q2 (X) S3 (X) with Q2 (X) ∈ Hm , Q1 (X) ∈ HL , Q(X) = ∗ S3 (X) Q1 (X) S3 (X) ∈ KL×m , such that PN j∗ j XB20 + Q20 , Q2 (X) = j=1 B20 PN j∗ j Q1 (X) = j=1 B10 XB10 + Q10 .

(5.4)

Definition 5.1.2 For Q1 and Q2 given by (5.4), we set R dom± R = {X ∈ dom R R Q2 (X) < 0 and Q1 (X) > 0} and call the requirement X ∈ dom± R an indefinite constraint on X. Henceforth, whenever we are given a Riccati-operator R on Hn and consider the set dom± R we tacitly think of a partitioning (5.3) of the given data. In particular, this partitioning specifies a stochastic control system with two inputs of the form dx(t) =

Ax(t) dt +

N O

Aj0 x(t) dwj (t)

j=1

+ B1 v(t) dt +

N O

j B10 v(t) dwj (t)

j=1

+ B2 u(t) dt +

N O

j B20 u(t) dwj (t) .

j=1

For the matrix M and the operators Λ and Π we have the partitioning

(5.5)





5.1 Preliminaries

127

P0 S20 S10 ∗ Q20 S30  , M =  S20 ∗ ∗ S10 S30 Q10   ∗ A X + XA XB2 XB1 0 0  , B2∗ X Λ(X) =  0 0 B1∗ X  j∗ j j∗ j j  Aj∗ A0 XA0 A0 XB20 N 0 XB10 O  B j∗ XAj B j∗ XB j B j∗ XB j  . Π(X) = 20 0 20 20 20 10 j=1 B j∗ XAj B j∗ XB j B j∗ XB j 10 0 10 20 10 10 By the quotient formula of Lemma A.2 we can write R as the double Schur complement = D R(X) = S Π(X) + Λ(X) + M/Q(X) = = DN = DD = S S Π(X) + Λ(X) + M/Q1 (X) S Q(X)/Q1 (X) . (5.6) 5.1.3 A definiteness assumption One of our aims is to solve the Riccati equation R(X) = 0 with the indefinite constraint X ∈ dom± R. To achieve this, we will have to impose the following definiteness assumptions. 0 Assumption 5.1.3 Let (i)

6 P0 S20 ≤ 0, (ii) Q20 < 0, and (iii) Q10 > 0. ∗ Q20 S20

5.1.4 Some comments It is easy to see that the abstract Riccati operator and the definite and indefinite constraints cover the special cases from Chapter 2. Remark 5.1.4 (i) Consider the situation of the LQ-control problem in Section 2.1 and the Riccati operator R defined in (2.3) together with the target set dom+ R defined in (2.4). Obviously, this is a Riccati operator with definite constraint in the sense of Section 5.1.1 with s = m and M > 0. Later it will be useful to observe that in this case the equation R(X) = 0 with X ∈ dom+ R can be transformed to an equation with indefinite constraint satisfying Assumption 5.1.3. To this end, we define PN j P˜ (X) = −P (−X) = A∗ X + XA + j=1 Aj∗ 0 XA0 − P0 , PN j ˜ S(X) = −S(−X) = XB + j=1 Aj∗ 0 XB0 − S0 , P N j∗ j ˜ Q(X) = −Q(−X) = j=1 B0 XB0 − Q0 ,

128


˜ = −X we have such that with X ˜ X) ˜ = P˜ (X) ˜ − S( ˜ X) ˜ Q( ˜ X) ˜ −1 S( ˜ X) ˜ ∗ = −R(X) . R( ˜ X) ˜ < 0. If we use the formal Obviously Q(X) > 0 is equivalent to Q( partition 0 6 ˜ 2 (X) ˜ S˜3 (X) ˜ Q ˜ ˜ Q(X) = ˜ ˜ ∗ ˜ ˜ S3 (X) Q1 (X) ˜ 2 (X) ˜ = Q( ˜ X) ˜ and the empty matrices S˜3 (X) ˜ and Q ˜ 1 (X) ˜ then with Q X ∈ dom+ R R ˜ = {X ˜ ∈ dom R ˜ R Q2 (X) ˜ ∈ dom± R ˜ < 0 and Q1 (X) ˜ > 0} . ⇐⇒ X Hence we have the following equivalence of the equation with definite constraint and an equation with indefinite constraint: (R(X) = 0 and X ∈ dom+ R) ˜ . ˜ X) ˜ = 0 with X ˜ = −X ∈ dom± R) ⇐⇒ (R( ˜ 20 = −Q0 Note that Assumption 5.1.3 holds for P˜0 = −P0 , S˜20 = −S0 , Q ˜ ˜ and the empty matrices S10 and Q10 . (ii) Similarly, the situation of the Bounded Real Lemma from Section 2.2 fits into both the framework of the definite and the indefinite constraint. Obviously the Riccati operator Rγ from (2.11) and the target set dom+ Rγ from (2.12) are of the type defined in Section 5.1.1 with s = ^. But we can also interpret dom+ Rγ as an indefinite constraint satisfying Assumption 5.1.3, if we set m = 0 and use the formal partition 0 6 Q2 (X) S3 (X) Qγ (X) = S3 (X)∗ Q1 (X) with Q1 (X) = Qγ (X) and the empty matrices Q2 (X) and S3 (X). Note again that Assumption 5.1.3 holds with P0 = −C ∗ C, S10 = −C ∗ D1 , Q10 = γ 2 I − D1∗ D1 , and the empty matrices S20 and Q20 . (iii) The situation of the disturbance attenuation problem in Section 2.3 is our motivating problem for a Riccati operator with indefinite constraint. It is quite easy to guess that in fact our general notation and Assumption 5.1.3 have been inspired by this application. To recover it in the general framework, we just have to set P0 = −C ∗ C , S0 = [S20 , S10 ] = −C ∗ [D2 , D1 ], 0 6 0 6 Q20 S30 −D2∗ D2 −D2∗ D1 Q0 = = . ∗ S30 Q10 −D1∗ D2 γ 2 I − D1∗ D1

0

6

5.1 Preliminaries

129

P0 S20 ≤ 0. By the regularity condition (Definition 2.3.3) ∗ S20 Q20 we have Q20 < 0. The definiteness condition Q10 > 0 is indispensible for an X < 0 with Q1 (X) = Qγ1 (X) > 0 to exist (which is what we are looking for). Hence Assumption 5.1.3 is a natural requirement in this context. (iv) Clearly, not every Riccati equation with definite constraint can be transformed to a Riccati equation with indefinite constraint satisfying Assumption 5.1.3. This situation arises e.g. in LQ-control problems with indefinite state- and input-weight cost like in [26, 218]. (v) The difference between the definite and the indefinite constraint can also be seen if we consider linear matrix inequalities. By Lemma A.2 we have Obviously

In R(X) + In Q(X) = In(Λ(X) + Π(X) + M ) . Hence, for X ∈ dom+ R we have R(X) ≥ 0 ⇐⇒ Π(X) + Λ(X) + M ≥ 0 , while X ∈ dom± R implies R(X) ≥ 0 , rk R(X) = r ⇐⇒ In(Π(X) + Λ(X) + M ) = (m + r, ^, n − r) . In other words, the constrained Riccati inequality X ∈ dom+ R, R(X) = 0 is equivalent to a linear matrix inequality in Hn+s , while X ∈ dom± R, R(X) = 0 is equivalent to a more general inertia condition in Hn+s . (vi) We recall an identity from the proof of Theorem 2.3.4. Let R be a Riccati operator, X ∈ dom R and let the given matrices be partitioned as in Section 5.1.2. Like in (2.25), (2.22), and (2.23) we set Pˆ (X) = P (X) − S1 (X)Q1 (X)−1 S1 (X)∗ , ˆ S(X) = S2 (X) − S1 (X)Q1 (X)−1 S3 (X)∗ , ˆ Q(X) = Q2 (X) − S3 (X)Q1 (X)−1 S3 (X)∗ , −1 ˆ ˆ S(X)∗ , F = F (X) = −Q(X)

PF (X) = (A∗ + F ∗ B2∗ )X + X(A + B2 F ) P i∗ ∗ i∗ i i + N i=1 (A0 + F B20 )X(A0 + B20 F ) ∗ ∗ ∗ + P0 + F S20 + S20 F + F Q20 F P i∗ ∗ i∗ i ∗ SF (X) = XB1 + N i=1 (A0 + F B20 )XB10 + S10 + F S30 . Then, in accordance with (5.6), we have 0 6∗ 0 60 6 ˆ I Pˆ (X) S(X) I R(X) = ∗ ˆ ˆ F F S(X) Q(X) = PF (X) − SF (X)Q1 (X)−1 SF (X)∗ .

(5.7)

130


In particular, the inequalities Q1 (X) > 0 and R(X) ≥ 0 together imply PF (X) ≥ 0. We will need this observation in Section 5.3.2, where we discuss solutions to the disturbance attenuation problem. We note a simple consequence of Assumption 5.1.3 concerning the evaluation of R at 0 ∈ Hn . Lemma 5.1.5 Assume the situation of Section 5.1.2 including Assumption 5.1.3. Then 0 ∈ dom± R and R(0) ≤ 0 with dim Ker R(0) = dim Ker M . Proof: Obviously 0 ∈ dom± R. If we set d = dim Ker M , then, by Lemma A.2 we can express the inertia of M in two ways: In M = In Q0 + In S(M/Q0 ) = (^, m, 0) + In R(0) = In Q10 + In S(M/Q10 ) = (^, 0, 0) + (0, n + m − d, d) . Hence In R(0) = (0, n − d, d), i.e. R(0) ≤ 0 and dim Ker R(0) = d.

"

In the following sections we show, how the results from Chapter 4 can be applied to solve equations of the form R(X) = 0. It is easy to see that the derivative of R is resolvent positive. Moreover, R is dom+ R-concave on dom R. Hence, as will be seen, we can apply the results from Chapter 4 directly in the definite case. As for the indefinite constraint, however, R is not dom± R-concave on dom R in general. Therefore, we will have to carry out a certain duality transformation of R before we can apply our results on the Newton iteration. This is the point, where Assumption 5.1.3 comes into play. To motivate the further considerations we take a look at standard Riccati equations at first. 5.1.5 Algebraic Riccati equations from deterministic control We briefly review some results on standard Riccati equations from deterministic control theory. In particular we point both at analogies and differences between the Riccati equations from deterministic and stochastic control. First of all we recall from Remark 2.3.6 that the equation R(X) = 0 reduces to the continuous-time algebraic Riccati equation (CARE) ∗ A∗ X + XA + P0 − XBQ−1 0 B X = 0

(5.8)

if Π = 0 and S0 = 0, and to the discrete-time algebraic Riccati equation (DARE) A0 ∗ XA0 − X + P0 − A0 ∗ XB0 (B0 ∗ XB0 + Q0 )

−1

B0 ∗ XA0 = 0

(5.9)

if A = − 21 I, B = 0, S0 = 0, and N = 1. We concentrate on stabilizing solutions of the CARE (5.8) and cite a result, which is central both for the analysis and the numerical solution of Riccati equations (e.g. [201, 34, 19, 145]).

5.1 Preliminaries

131

Theorem 5.1.6 Let Q0 > 0. Then the following are equivalent: (i)

The CARE (5.8) has a stabilizing solution X+ , i.e. σ(A−BQ0−1 B ∗ X+ ) ⊂ C− . In this event, X+ is the greatest solution of the inequality ∗ A∗ X + XA + P0 − XBQ−1 0 B X ≥ 0 .

If P0 ≥ 0 and (A, P0 ) is observable, then X+ > 0. (ii) The pair (A, B) is stabilizable and there exists a solution to the strict inequality ∗ A∗ X + XA + P0 − XBQ−1 0 B X > 0 .

In this event X+ is the limit of the Newton sequence defined by (P0 + Xk BQ0−1 B ∗ Xk ) , Xk+1 = −L−1 A−BQ−1 B ∗ X 0

k

∗ if σ(A − BQ−1 0 B X 0 ) ⊂ C− . (iii) The pair (A, B) is stabilizable and the Hamiltonian matrix 0 6 ∗ A −BQ−1 0 B H= −P0 −A∗

does not have any purely imaginary eigenvalues. In this event X+ = X1 X2−1 , where [X1∗ , X2∗ ]∗ spans the invariant subspace of H corresponding to the n eigenvalues with negative real part. (iv) The pair (A, B) is stabilizable and the frequency domain condition B ∗ (−iωI − A∗ )−1 P0 (iωI − A)−1 B + Q0 > 0 is valid for all ω ∈ R. Remark 5.1.7 (i) Among the criteria (ii), (iii), and (iv) only (ii) seems to offer an approach to analyze and solve the Riccati equation from stochastic control. While it is natural to apply Newton’s method to the nonlinear matrix equation R(X) = 0, a direct generalization of the Hamiltonian H or the transfer function (iωI − A)−1 B to the stochastic case is not yet available. It would, of course, be rewarding to develop such concepts. (ii) In the theorem we assume Q0 > 0. If Q0 is indefinite, the assertions do not necessarily hold. But if e.g. P0 ≤ 0 one can multiply equation (5.8) from both sides by X −1 and replace X −1 by −Y to obtain the dual equation ∗ −Y A∗ − AY − Y (−P0 )Y − BQ−1 0 B = 0 .

It is straightforward to reformulate the theorem for the dual equation. Note that the stabilizability condition on (A, B) becomes a detectability condition on (−A, P0 ), or – in the terminology of Remark 1.8.6 – an

132


anti-detectability condition on (A, P0 ). The term dual Riccati equation in this context has been used e.g. in [103]. We will apply the same duality transformation to solve the indefinite Riccati equation from stochastic control. In this case, however, the dual operator is not a Riccati operator in the sense of Definition 5.1.1. But again we will observe a duality between stabilizability and anti-detectability conditions. 5.1.6 A duality transformation Let R be a Riccati operator in the sense of Definition 5.1.1. Then we define another rational matrix operator R G : dom G = {Y R det Y O= 0, and X = −Y −1 ∈ dom R} → Hn by the relation G(Y ) = Y R(−Y −1 )Y .

(5.10)

In analogy to the deterministic case, we will call G the dual Riccati operator and the equation G(Y ) = 0 the dual Riccati equation. If X is a nonsingular solution of the Riccati equation, then, obviously, X = −Y −1 solves the dual Riccati equation. The explicit form of G is G(Y ) = Y (P (−Y −1 ))Y − Y S(−Y −1 )Q(Y −1 )S(−Y −1 )∗ Y = −Y A∗ − AY −

N O

−1 j Y Aj∗ A0 Y + Y P0 Y 0 Y

(5.11)

j=1 N D = O −1 j − B+ Y Aj∗ Y B − Y S 0 0 0 j=1

=

× Q0 −

N O

B0j∗ Y −1 B0j

N D−1 = D O B+ B0j∗ Y −1 Aj0 Y − S0 Y .

j=1

j=1

We make some simple observations. Lemma 5.1.8 Let Y ∈ dom G and X = −Y −1 . Then the following hold: R(X) = XG(−X −1 )X. @ G (>) (>) (ii) G(Y ) = 0 ⇐⇒ R(X) = 0 . (i)

(iii) If R(X) = 0, then σ(GY 3 ) = σ(RX 3 ).

5.1 Preliminaries

133

Proof: The statements (i) and (ii) are obvious. Assertion (iii) follows from GY 3 (H) = HR(X)Y + Y R(X)H + Y RX 3 (Y −1 HY −1 )Y = Y RX 3 (Y −1 HY −1 )Y ,

(5.12)

because the operator H L→ Y RX 3 (Y −1 HY −1 )Y is similar to H L→ RX 3 (H); in the notation of Kronecker products the similarity transformation is given " by Y ⊗ Y −1 . It is a legitimate question, why the equation G(Y ) = 0 should be easier to solve than R(X) = 0. Unlike in the situation of Remark 5.1.7, the dual Riccati operator G is not a Riccati operator in the sense of Definition 5.1.1. But if Assumption 5.1.3 holds, we will see that G has the concavity properties we need. To anticipate this result, we might consider the special case, where all B0j and S0 vanish and B = I. Like in the deterministic case, the concavity of R depends on the inertia of Q0 , while the concavity of G is independent of Q0 . Hence it suffices to study the Y L→ Y P0 Y , which is concave if P operator j∗ −1 j A0 Y , which we have already P0 ≤ 0, and the operator Y L→ N j=1 Y A0 Y n seen in Lemma 4.4.2 to be H+ -concave. 5.1.7 A regularity transformation In our applications, Q0 is regular. Under this assumption we can transform the Riccati operator by some congruence transformation equivalently to another Riccati operator ˜ X) ˜ ∗, ˜ = P˜ (X) ˜ − S( ˜ X) ˜ Q( ˜ X) ˜ −1 S( ˜ X) R( with ˜ = M

0

˜ P˜ (0) S(0) ˜ ∗ Q(0) ˜ S(0)

6

0 =

D1 0 0 D2

(5.13)

6 ,

(5.14)

∗ where D1 and D2 are the Sylvester normal forms of P0 − S0 Q−1 0 S0 and Q0 , respectively. ∗ ∗ ∗ To this end let U1 (P0 − S0 Q−1 0 S0 )U1 = D1 and U2 Q0 U2 = D2 with nonsingular matrices U1 and U2 . Then 6 60 60 6 0 0 ˜ S( ˜ X) ˜ U1∗ 0 P˜ (X) P (X) S(X) U1 −S0 Q−1 0 = ∗ ∗ ˜ X) ˜ ∗ Q( ˜ X) ˜ S(X)∗ Q(X) −Q−1 0 U2 S( 0 S0 U 2

with ˜ = U1 P (X)U ∗ − S0 Q−1 S(X)∗ U ∗ − U1 S(X)Q−1 S ∗ P˜ (X) 1 1 0 0 0 −1 ∗ + S0 Q−1 0 Q(X)Q0 S0

˜ +X ˜ A˜ + = A˜∗ X

N O i=1

˜ ˜i ˜ A˜i∗ 0 X A0 + P0 ,

134


˜ X) ˜ = U1 S(X)U ∗ − S0 Q−1 Q(X)U ∗ = XB ˜ + S( 2 2 0

N O

˜ ˜i A˜i∗ 0 X B0 ,

i=1

˜ X) ˜ = U2 Q(X)U ∗ = Q( 2

N O

˜B ˜i + Q ˜ i∗ X ˜0 , B 0 0

i=1

˜ = X A˜ =

˜ B A˜i0 ˜0i B P˜0

= = = =

˜0 = Q

U1 XU1∗ ∗ ∗ U1−∗ (A − BQ−1 0 S0 )U1 , U1−∗ BU2∗ , ∗ ∗ U1−∗ (Ai0 − B0 Q−1 0 S0 )U1 , −∗ i ∗ U 1 B0 U 2 , ∗ ∗ U1 (P0 − S0 Q−1 0 S0 )U1 = U2 Q0 U2∗ = D2 ,

D1 ,

as one verifies by insertion. Moreover, the transformation formula in Lemma A.2 yields ˜ X) ˜ = U1 R(X)U ∗ , R( 1 such that we have a one-to-one correspondence between the solution sets of both Riccati equations and inequalities, respectively. Hence, without loss of generality, we can assume that M has the normal form (5.14). We will, however, formulate all our results with the original data. Only in some examples we use the normal form.

5.2 Analytical properties of Riccati operators In this section we analyze the Riccati operator R and the dual Riccati operator G in view of the Assumptions 4.2.1 and 4.2.10. In particular, we show that the derivatives of R and G are resolvent positive and we characterize cases where R and G are concave. The following is obvious. Proposition 5.2.1 Let R be a Riccati operator and G its dual operator. Then R and G are analytical and their derivatives are locally bounded on dom R and dom G, respectively. 5.2.1 The Riccati operator R Let R be a Riccati operator in the sense of Definition 5.1.1. To condense the notation we sometimes write PX , SX , and QX instead of P (X), S(X), and Q(X), respectively. Moreover, for a given X in dom R, we set 0 0 6 6∗ I I −1 ∗ Π(·) SX and ΠX (·) = . AX = A − BQX ∗ ∗ −Q−1 −Q−1 X SX X SX

5.2 Analytical properties of Riccati operators

135

Theorem 5.2.2 Let X ∈ dom R, Y ∈ dom+ R. The derivative of R at X is given by

(i)

R3X (H) = LAX (H) + ΠX (H) .

(ii) (iii) (iv) (v) (vi) (vii)

In particular, R3X is the sum of a Lyapunov operator and a completely positive operator and 0 hence resolvent 6∗ 0 positive. 6 I I M R(X) = R3X (X) + −1 ∗ −1 ∗ . −QX SX −QX SX 3 R(Y ) − R(X) − RX (Y − X) −1 −1 ∗ −1 ∗ = −(SY Q−1 Y − SX QX )QY (QY SY − QX SX ). R is dom+ R-concave on dom R. n . dom+ R = dom+ R + H+ If Q0 > 0, then 0 ∈ dom+ R. If, in addition, M ≥ 0 then R(0) ≥ 0. Assume 0 ∈ dom R and R(0) ≤ 0. If (R30 , R(0)) is detectable, then ∃X > 0, ε > 0 : ∀α ∈ ]0, ε] : R(αX) < 0 .

Proof: (i) By the standard rules of calculus, we have −1 3 ∗ ∗ 3 3 R3X (H) = PX (H) − SX (H)Q−1 X SX − SX QX SX (H)

−1 ∗ 3 + SX Q−1 X QX (H)QX SX 0 6∗ = 6 D0 I I Λ(H) + Π(H) = , −1 ∗ ∗ −QX SX −Q−1 X SX

where it is easily seen that

0

LA−BQ−1 S ∗ (H) = X

(ii) This follows from R(X) −

R3X (X)

∗ −Q−1 X SX

X

0 =

0

6∗

I

Λ(H)

6

I

.

∗ −Q−1 X SX

0

6∗

I

I

6

(Λ(X) + Π(X) + M ) ∗ ∗ −Q−1 −Q−1 X SX X SX 0 6∗ 6 0 I I − . (Λ(X) + Π(X)) ∗ ∗ −Q−1 −Q−1 X SX X SX

(iii) We use (ii) to compute R(Y ) − R(X) − R3X (Y − X) = R(Y ) −

(5.15)

R3X (Y

)−

0

I

6∗

0

I

M ∗ ∗ −Q−1 −Q−1 X SX X SX 6∗ 0 60 6 I I PY SY = R(Y ) − −1 ∗ ∗ ∗ S Q −Q−1 −Q S Y Y X X X SX 0 6∗ 0 6 0 6 ∗ I I SY Q−1 Y SY SY =− ∗ ∗ −Q−1 −Q−1 SY∗ QY X SX X SX 0

−1 ∗ −1 ∗ −1 = −(SY Q−1 Y − SX QX )QY (QY SY − QX SX ) .

6

136


(iv) This follows from (iii), since the right-hand side is nonpositive definite for Y ∈ dom+ R. PN (v) By definition Q(X) = j=1 B0j∗ XB0j + Q0 . Hence Y ≥ X implies Q(Y ) ≥ Q(X). (vi) By definition 0 ∈ dom+ R ⇐⇒ Q(0) = Q0 > 0. If, in addition, M ≥ 0, ∗ then R(0) = P0 − S0 Q−1 0 S0 ≥ 0 by Lemma A.2. −1 ∗ (vii) Let F0 = −Q0 S0 . By Lemma 1.8.5 there exists an X > 0 such that D = R30 (X) + R(0) = LA+BF0 + Π(Aj +B j F0 ) (X) + R(0) < 0 . 0

0

For 1 > α > 0 we have R(αX) ≤ α(R(0) + R30 (X) + O(α)) which is negative, for sufficiently small α. " As in Chapter 4 (cf. Assumption 4.2.1), a matrix X ∈ dom R is called stabilizing (for R) if σ(R3X ) ⊂ C− and almost stabilizing, if σ(R3X ) ⊂ C− ∪iR. We call R (almost) stabilizable, if there exists an (almost) stabilizing matrix X for R. ∗ Corollary 5.2.3 If X is stabilizing for R, then F = −Q−1 X SX is a stabilizing feedback gain matrix for the stochastic control system

dx(t) = Ax(t) dt +

N O

Aj0 x(t) dwj (t) + Bu(t) dt +

j=1

N O

B0j u(t) dwj (t) .

j=1

Proof: By (5.15) R3X (H) = (A + BF )∗ X + X(A + BF ) 6 6∗ 0 j∗ N 0 j 60 O I I A0 XAj0 Ai∗ 0 XB0 + . F F B0∗ XAj0 B0j∗ XB0j i=1 Hence the stability of the closed-loop system follows from Theorem 1.5.3 and " the stability of R3X .

Remark 5.2.4 (i)

In the special case of the CARE (5.8) we have R3X = LA−BQ−1 B ∗ X 0

and X is stabilizing, if and only if σ(A − BQ0−1 B ∗ X) ⊂ C− . (ii) In the special case of the DARE (5.9) we have R3X (H) = −H + V ∗ HV where 0 6 I . V = [A0 B0 ] −1 (Q0 + B0∗ XB0 ) B0∗ XA0


137

So R3X is stabilizing, if and only if ρ(V ) < 1. We see that our concept of stabilization generalizes and unifies the concepts from continuous and discrete-time deterministic systems. O ∅. Then R is stabilizable, if and Proposition 5.2.5 Assume that dom+ R = only if (A, (Aj0 ), B, (B0j )) is stabilizable in the sense of Definition 1.7.1. Proof: If R is stabilizable, then so is (A, (Aj0 ), B, (B0j )) by Corollary 5.2.3. ˆ ∈ dom+ R. For arbitrary X ∈ dom R, set Z = X − X ˆ and Conversely let X ˆ ˆ 3 = (R)3 for all X ∈ dom R. By construction R(Z) = R(X). Obviously (R) Z X ˆ ˆ R(Z) = RM (Z) with 0 6 6 0 ˆ S(X) ˆ Pˆ0 Sˆ0 P (X) ˆ M = ˆ∗ ˆ . (5.16) = ˆ ˆ ∗ Q(X) S0 Q 0 S(X) ˆ 0 > 0 and (R) ˆ 3 does not depend on Pˆ0 , we can assume without loss Since Q Z ˆ of generality that M > 0. If (A, (Aj0 ), B, (B0j )) is stabilizable, then by Lemma 1.7.7 there exists an X0 > 0, such that ˆ 0) 0 ≥ R(X ˆ 3X (X0 ) + =R 0

0

I

ˆ 0 )∗ ˆ 0 )−1 S(X −Q(X

6∗

ˆ M

0

I

6

ˆ 0 )∗ ˆ 0 )−1 S(X −Q(X

ˆ 3X (X0 ) , >R 0 where the equation follows from Theorem 5.2.2(ii). ˆ 3 ) ⊂ C− by Theorem 3.6.1. Hence σ(R3X +Xˆ ) = σ(R X0 0

"

Finally, we draw a conclusion from Corollary A.4. ˜ let Proposition 5.2.6 For fixed Λ and Π and M ≥ M R(X) = RM (X) = P (X) − S(X)Q(X)−1 S(X)∗ ˜ −1 ˜ ˜ ˜ ˜ S(X)∗ . R(X) = RM (X) = P˜ (X) − S(X) Q(X) ˜ satisfies In Q(X) = In Q(X), ˜ ˜ If X ∈ dom R ∩ dom R then R(X) ≥ R(X) and D = D = −1 ˜ ˜ ˜ S(X)∗ . Ker R(X) − R(X) ⊂ Ker Q(X)−1 S(X)∗ − Q(X) ˜ ˜ 3 , R(X)) ˜ If, in addition, R(X) ≥ 0 and (R is β-detectable then (R3X , R(X)) X is β-detectable. Proof: The first part is a reformulation of Corollary A.4. It only remains n is an to prove the detectability assertion. To this end, assume that V ∈ H+ 3 ∗ eigenvector of (RX ) with the eigenvalue λ, and R(X)V = 0. Since R(X) ≥ ˜ ˜ R(X) ≥ 0, it follows that also R(X)V = 0. In particular

138


im V

1 2

= D ˜ = im V ⊂ Ker R(X) − R(X) D = −1 ˜ ˜ S(X)∗ . ⊂ Ker Q(X)−1 S(X)∗ − Q(X)

(5.17)

˜ 3 )∗ (V ), as the following lines show. With F = This implies (R3X )∗ (V ) = (R X −1 ∗ ˜ ∗ ˜ ˜ we have −Q(X) S(X) , F = −Q(X)−1 S(X) @0 6 0 6∗ 0 6 0 6∗ G I I I I ˜ 3X )∗ (V ) = (Λ + Π)∗ V . (5.18) − ˜ V ˜ (R3X )∗ (V ) − (R F F F F 1 1 The inclusion (5.17) implies F V = F˜ V and F V 2 = F˜ V 2 such that – indeed – the right-hand side of (5.18) vanishes. ˜ 3 )∗ with the same eigenvalue λ, and R(X)V = Thus, V is an eigenvector of (R X 3 ˜ ˜ 0. Since (RX , R(X)) is assumed to be β-detectable, it follows that λ ∈ C− . " Hence (R3X , R(X)) is β-detectable.

˜ Note that, by definition, the condition In Q(X) = In Q(X) holds, if X ∈ ˜ ˜ dom+ R ∩ dom+ R or X ∈ dom± R ∩ dom± R. 5.2.2 The dual operator G Assume the situation of Section 5.1.2. In Theorem 5.2.7 below we establish an analogue to Theorem 5.2.2 for the dual operator G. This theorem paves the way to a solution of the Riccati equation with indefinite constraint and the disturbance attenuation problem. All assertions of the theorem are valid under Assumption 5.1.3. But not all assertions of this section require all parts (i), (ii), and (iii) of Assumption 5.1.3. Hence, we do not assume any of these requirements, unless it is explicitly stated. Recall that the dual operator G to R is defined by G(Y ) = Y R(−Y −1 )Y . We define the target set

R n R X = −Y −1 ∈ dom± R} . dom+ G = {Y ∈ int H+

The operator G is more complicated than R, and the discussion below involves rather lengthy computations. A good choice of notation is essential. In addition to the conventions from Section 5.1.2 we use the partition 6 N 0 j∗ j6 O A0 XAj0 Aj∗ Π0 (X) Σ(X) 0 XB0 = Π(X) = Σ(X)∗ Π21 (X) B0j∗ XAj0 B0j∗ XB0j j=1  j∗   j j∗ j  A0 XAj0 Aj∗ N Π0 (X) Σ2 (X) Σ1 (X) 0 XB20 A0 XB10 O  B j∗ XAj B j∗ XB j B j∗ XB j  =  Σ2 (X)∗ Π2 (X) Σ3 (X)  = 20 20 0 20 20 10 ∗ ∗ j=1 B j∗ XAj B j∗ XB j B j∗ XB j Σ1 (X) Σ3 (X) Π1 (X) 10 20 10 10 10 0 0


139

with an obvious correspondence of the blocks. Furthermore, we set PY = −Y A∗ − AY − Y Π0 (Y −1 )Y + Y P0 Y , SY = B + Y Σ(Y −1 ) − Y S0 , QY = Q0 − Π21 (Y −1 ) . With these substitutions in (5.11) we can write G more compactly as ∗ G(Y ) = PY − SY Q−1 Y SY .

Finally, we define the abbreviations −1 ∗ AY = −A + SY Q−1 )) + Y (P0 − Π0 (Y −1 )) , Y (S0 − Σ(Y 0 0 6∗ 6 Y Y −1 −1 Π(Y HY ) ΠY (H) = , ∗ ∗ Q−1 Q−1 Y SY Y SY

which will allow us to write the derivative of G in a compact form as well. Theorem 5.2.7 Let Y ∈ dom G, Z ∈ dom+ G (in particular dom+ G O= ∅). (i)

(ii) (iii)

(iv) (v)

The derivative of G at Y is given by GY3 (H) = LA∗ (H) + ΠY (H) . In particular, GY3 is the sum of a Lyapunov operator and a completely positive operator and positive. 0 hence6∗resolvent 0 6 Y Y 3 G(Y ) = GY (Y ) − M . ∗ Q−1 Q−1 S∗ Y SY 0Y Y 6 P0 S20 Let Assumption 5.1.3(i) hold, i.e. ≤ 0. ∗ Q20 S20 Then G(Z) − G(Y ) − GY3 (Z − Y ) ≤ 0. In words, G is dom+ G-concave on dom G. Let Assumption 5.1.3(ii) hold, i.e. Q20 < 0. n n ⊂ int H+ . Then dom+ G = dom+ G + H+ Let Assumption 5.1.3 hold (in particular R(0) ≤ 0 by Lemma 5.1.5), and let (−R30 , R(0)) be detectable. Then G is stabilizable, and a stabilizing matrix Y > 0 can be found in the form −νX −1 where ν > 0, and X < 0 satisfies R30 (X) + R(0) < 0. In particular, if R(0) < 0, then for all Y0 > 0 there exists an ν0 > 0, 3 such that σ(GνY ) ⊂ C− for all ν ≥ ν0 . 0

Before we proceed with the technical proof, let us make some remarks. Remark 5.2.8 (i) As a consequence of Theorem 5.2.7 and Proposition 5.2.1 3 we obtain the following. Let f = G, D = dom G, D+ = dom+ G, TX = GX and let Assumption 5.1.3(i) and (ii) hold. Then the hypotheses (H1)–(H4) and (H7) of Assumption 4.2.1 are satisfied. If, in addition, Assumption 5.1.3(iii) holds and (−R30 , R(0)) is detectable, then also (H5) is satisfied.

140


(ii) Assume the regularity condition S0 = 0. Then detectability of the pair (−R30 , R(0)) is the same as anti-detectability of (A, (Aj0 ), −P0 ) in the terminology of Remark 1.8.6. Theorem 5.2.7(v) establishes an interesting relation between the stabilizability of the dual operator G and the antidetectability of the original system. . Proof: In products of the form V ∗ W V we sometimes write [ .. ] for the right factor, if it is the conjugate transpose of the left factor. (i) By the standard rules of calculus, we have PY3 (H) = −HA∗ − AH + H(P0 − Π0 (Y −1 ))Y + Y (P0 − Π0 (Y −1 ))H + Y Π0 (Y −1 HY −1 )Y SY3 (H) = H(Σ(Y −1 ) − S0 ) − Y Σ(Y −1 HY −1 ) Q3Y (H) = Π21 (Y −1 HY −1 ) . Thus −1 3 −1 3 −1 ∗ ∗ ∗ GY3 (H) = PY3 (H) − SY3 (H)Q−1 Y SY − SY QY SY (H) + SY QY QY (H)QY SY

= −AH − HA∗ + H(P0 − Π0 (Y −1 ))Y + Y (P0 − Π0 (Y −1 ))H ∗ −1 ∗ + H(S0 − Σ(Y −1 ))Q−1 HY −1 )Q−1 Y SY + Y Σ(Y Y SY −1 ∗ −1 + SY Q−1 )) H + SY Q−1 HY −1 )∗ Y Y (S0 − Σ(Y Y Σ(Y

∗ −1 + Y Π0 (Y −1 HY −1 )Y + SY Q−1 HY −1 )Q−1 Y SY Y Π21 (Y D = −1 ∗ )) + Y (P0 − Π0 (Y −1 )) H (5.19) = − A + SY Q−1 Y (S0 − Σ(Y = D ∗ −1 ))Y + H − A∗ + (S0 − Σ(Y −1 ))Q−1 Y SY + (P0 − Π0 (Y 6 0 6∗ 0 6 0 Y Y Π0 (Y −1 HY −1 ) Σ(Y −1 HY −1 ) . + ∗ ∗ Σ(Y −1 HY −1 )∗ Π21 (Y −1 HY −1 ) Q−1 Q−1 Y SY Y SY

(ii) We write (5.19) as 

∗ Y GY3 (H) = − AH − HA∗ +  H  (5.20) ∗ Q−1 S Y Y    Y Π0 (Y −1 HY −1 ) P0 − Π0 (Y −1 ) Σ(Y −1 HY −1 ) 0 S0 − Σ(Y −1 )   H  . ×  P0 − Π0 (Y −1 ) −1 −1 ∗ ∗ −1 ∗ ∗ Σ(Y HY ) S0 − Σ(Y ) Π21 (Y −1 HY −1 ) Q−1 Y SY If we insert H = Y and Π21 (Y −1 ) = Q0 − QY we obtain the formula (ii). (iii) We have to show that G is dom+ G-concave on dom G, i.e. G(Y ) − G(Z) + GY3 (Z − Y ) ≥ 0 . Letting H = Z − Y in (5.20) we obtain by a simple reordering of terms


141

∗ Y GY3 (Z − Y ) = A(Y − Z) + (Y − Z)A∗ +  Z − Y  ∗ Q−1 Y SY   Π0 (Y −1 (Z − Y )Y −1 ) P0 − Π0 (Y −1 ) Σ(Y −1 (Z − Y )Y −1 )  0 S0 − Σ(Y −1 ) P0 − Π0 (Y −1 ) × −1 −1 ∗ ∗ −1 ∗ −1 −1 Σ(Y (Z − Y )Y ) S0 − Σ(Y ) Π21 (Y (Z − Y )Y )   Y × Z  ∗ Q−1 Y SY ∗  Y = A(Y − Z) + (Y − Z)A∗ − 2Y P0 Y +  Z  QY−1 SY∗   Σ(Y −1 ZY −1 ) − S0 Π0 (Y −1 + Y −1 ZY −1 ) P0 − Π0 (Y −1 )  P0 − Π0 (Y −1 ) 0 S0 − Σ(Y −1 ) × −1 −1 ∗ −1 ∗ −1 −1 −1 −Y ) (Σ(Y ZY ) − S0 ) (S0 − Σ(Y )) Π21 (Y ZY   Y ×  Z  . ∗ Q−1 Y SY 

Hence G(Y ) − G(Z) + GY3 (Z − Y ) −1 ∗ = − AY − Y A∗ + Y (P0 − Π0 (Y −1 ))Y − SY Q−1 ))Q−1 Y (Q0 − Π21 (Y Y SY

∗ 3 + AZ + ZA∗ + Z(Π0 (Z −1 ) − P0 )Z + SZ Q−1 Z SZ + GY (Z − Y )   ∗   Y Y Π0 (Y −1 ZY −1 ) −Π0 (Y −1 ) Σ(Y −1 ZY −1 ) Π0 (Z −1 ) −Σ(Y −1 )   Z  =  Z   −Π0 (Y −1 ) −1 ∗ −1 −1 ∗ −1 ∗ ∗ Σ(Y ZY ) −Σ(Y ) Π21 (Y −1 ZY −1 ) Q Y SY Q−1 Y SY   ∗   Y Y −P0 P0 −S0 ∗ (5.21) +  Z   P0 −P0 S0   Z  + SZ Q−1 Z SZ . −1 ∗ −1 ∗ ∗ ∗ −S0 S0 −Q0 Q Y SY Q Y SY

∗ Now we factorize SZ Q−1 Z SZ in a similar fashion like the other summands. We define

˜ = Σ(·) − S0 , Σ(·) such that −1 ˜ SY = B + Y Σ(Y ) ˜ −1 ) − Y Σ(Y ˜ −1 ) in In the following computation, we replace SZ by SY + Z Σ(Z the first step and QY by QZ + Π21 (Z −1 )− Π21 (Y −1 ) = QZ + Π21 (Z −1 − Y −1 ) in the last step: −1 ∗ −1 ˜ ∗ −1 ∗ ˜ −1 )∗ Y ) Z − SY Q−1 SZ Q−1 Z Σ(Y Z SZ = SY QZ SY + SY QZ Σ(Z

142


˜ −1 )∗ Z ˜ −1 )Q−1 Σ(Z ˜ −1 )Q−1 S ∗ + Z Σ(Z + Z Σ(Z Y Z Z ˜ −1 )∗ Y − Y Σ(Y ˜ −1 )Q−1 Σ(Y ˜ −1 )Q−1 S ∗ − Z Σ(Z

(5.22) Y Z Z −1 ˜ −1 ˜ −1 −1 ∗ −1 −1 ∗ ˜ ˜ − Y Σ(Y )QZ Σ(Z ) Z + Y Σ(Y )QZ Σ(Y ) Y ∗  Y =  Z  ∗ Q−1 Y SY   ˜ −1 )Q−1 Σ(Y ˜ −1 )∗ Σ(Y p p Z ˜ −1 )∗ ˜ −1 )∗ Σ(Z ˜ −1 )Q−1 Σ(Z ˜ −1 )Q−1 Σ(Y  ×  −Σ(Z p Z Z −1 ˜ −1 ˜ −1 −1 ∗ −1 ∗ QY QZ Σ(Z ) QY QZ QY −QY QZ Σ(Y )   Y × Z  (5.23) ∗ S Q−1 Y Y ∗     ∗ ˜ −1 ) ˜ −1 ) Y −Σ(Y −Σ(Y  Q−1   ˜ −1 ) ˜ −1 ) =  Z    Σ(Z Σ(Z Z −1 ∗ −1 −1 −1 −1 Q Y SY Π21 (Z − Y ) Π21 (Z − Y )     0 0 S0 − Σ(Y −1 ) .   0 0 Σ(Z −1 ) − S0 +   ..  . ˜ −1 )∗ − S0∗ Q0 + Π21 (Z −1 − 2Y −1 ) S0∗ − Σ(Y −1 )∗ Σ(Z 

∗ Substituting this expression for SZ Q−1 Z SZ in formula (5.21) we obtain   ∗  Y Y G(Y ) − G(Z) + GY3 (Z − Y ) =  Z  Θ  Z  . ∗ ∗ Q−1 Q−1 Y SY Y SY

Here Θ = Θ1 + Θ2 + Θ3 with   Σ(Y −1 ZY −1 − Y −1 ) Π0 (Y −1 ZY −1 ) −Π0 (Y −1 )  , p Π0 (Z −1 ) Σ(Z −1 − Y −1 ) Θ1 =  −1 p p Π21 (Y ZY −1 − 2Y −1 + Z −1 )   ∗  ˜ −1 ) ˜ −1 ) Σ(Y Σ(Y   Q−1  , ˜ −1 ) ˜ −1 ) Θ2 =  −Σ(Z −Σ(Z Z −1 −1 −1 −1 Π21 (Z − Y ) Π21 (Z − Y )   −P0 P0 0 Θ3 =  P0 −P0 0  . 0 0 0 The remaining part of the proof amounts to verifying that Θ ≥ 0 for Z ∈ dom+ G. 0 6 Q2 (Z −1 ) S3 (Z −1 ) To estimate Θ2 we remember that QZ = with −1 −1 −1 S3 (Z

)

Q1 (Z

)

Q2 (Z −1 ) < 0 and Q1 (Z −1 ) > 0 for Z ∈ dom+ G. Therefore also the Schur


143

complement

D = ˆ := S QZ /Q2 (Z −1 ) = Q1 (Z −1 ) − S3 (Z −1 )∗ Q2 (Z −1 )−1 S3 (Z −1 ) Q

of QZ with respect to the left-upper block is positive definite. By the matrix inversion formula of Lemma A.2 we have 0 6 Q2 (Z −1 )−1 0 = Q−1 Z 0 0 0 0 6 6∗ −1 −1 Q2 (Z ) S3 (Z −1 ) ˆ −1 Q2 (Z −1 )−1 S3 (Z −1 ) Q + , I I which is the sum of a negative and a positive semidefinite matrix. Resubsti˜ and Π21 by their defining expressions we get tuting Σ    ∗ ˜ −1 ) ˜ −1 ) Σ(Y Σ(Y  Q−1   ˜ −1 ) ˜ −1 ) Θ2 =  −Σ(Z −Σ(Z Z −1 −1 −1 −1 Π21 (Z − Y ) Π21 (Z − Y )  ∗   Σ2 (Y −1 ) − S20 Σ1 (Y −1 ) − S10 0 6   −Σ2 (Z −1 ) + S20 −Σ1 (Z −1 ) + S10  Q2 (Z −1 )−1 0   ..    ≥   Π2 (Z −1 − Y −1 ) Σ3 (Z −1 − Y −1 )  0 0 . −1 −1 ∗ −1 −1 Σ3 (Z − Y ) Π1 (Z − Y )  ∗   Σ2 (Y −1 ) − S20 Σ2 (Y −1 ) − S20 −1  −Σ2 (Z −1 ) + S20   S20    Q2 (Z −1 )−1  −Σ2 (Z−1 ) + −1 ˜ = −1 −1  Π2 (Z − Y )   Π2 (Z − Y )  =: Θ2 . Σ3 (Z −1 − Y −1 )∗ Σ3 (Z −1 − Y −1 )∗ Hence ˜2 + Θ3 = S Θ ≥ Θ1 + Θ where



@0

G 6N ˜ + S˜ Θ1 + Θ3 Σ −1 (Z ) , − Q 2 ˜ ∗ + S˜∗ −Q2 (Z −1 ) Σ

 Σ2 (Y −1 )   −Σ2 (Z −1 )  ˜ =  Σ  Π2 (Z −1 − Y −1 )  , Σ3 (Z −1 − Y −1 )∗



 −S20  S20   S˜ =   0 , 0

and −Q2 (Z −1 ) = Π2 (Z −1 ) − Q20 > 0. We finally show that 6 0 ˜ + S˜ Σ Θ1 + Θ3 ˜ + S˜ Π2 (Z −1 ) − Q20 ≥ 0 , Σ which implies Θ ≥ 0 for Z ∈ dom+ G and therefore proves (iii). For the constant term we have

144




−P0 P0  P0 −P0 0 6  Θ3 S˜ 0 =  ∗  0 ˜ S −Q20  0 0 ∗ ∗ S20 −S20 since

0

P0 S02 S20 Q20

 0 0 −S20 0 0 S20   00 0   ≥ 0, 00 0  0 0 −Q20

6 ≤0.

i i i ) the remaining term can be written , B20 , B10 Setting Di := diag(Ai0 , Ai0 , B20 as

0

6 O N ˜ Σ Θ1 Di∗ Υ Di = ˜ ∗ Π2 (Z −1 ) Σ i=1

with

     Υ =   



  ∗ Y −1 Y −1  −Z −1   −Z −1   −1     Z − Y −1  Z  Z −1 − Y −1  Z −1 − Y −1 Z −1 − Y −1 /

Y −1 −Z −1 Z −1 − Y −1 Z −1 − Y −1



 Y −1  −Z −1   −1   Z − Y −1  Z −1 − Y −1 5

      .   

Z −1

For Z ∈ dom+ G we have Z −1 > 0, and obviously S(Υ/Z −1 ) = 0. Hence Υ ≥ 0, which completes our proof of (iii). (iv) By definition, Y ∈ dom+ G if and only if Y > 0, Q2 (−Y −1 ) < 0, and Q1 (−Y −1 ) > 0. Since Q2 (−Y −1 ) < Q20 < 0 for all Y > 0, and the mapping n Y L→ Q1 (−Y −1 ) > 0 is monotonically increasing on H+ , the assertion follows. (v) The assumption Q20 < 0, Q10 > 0 is equivalent to 0 ∈ dom± R. Since dom R is open, there exists an ε > 0, such that the closure of the ball B(0, ε) is contained in dom R. On this ball, R is analytic, and, by compactness, the second derivative is bounded. Hence there exists a number L > 0, such that for all X1 , X2 ∈ B(0, ε) 0 large enough, such that −ν −1 X ∈ B(0, ε). With Y = −X −1 formula (5.12) yields = D 3 GνY (ν −1 Y ) = Y 2R(ν −1 X) + (R3ν −1 X (−ν −1 X) Y

5.3 Existence and properties of stabilizing solutions

= = Y R(ν −1 X) + R(0) + R30 (ν −1 X) + O(ν −2 ) D + R3ν −1 X (−ν −1 X) Y = D = Y R(ν −1 X) + R(0) + O(ν −2 ) Y = D = Y (2 − ν −1 )R(0) + ν −1 (R(0) + R30 (X) + O(ν −1 )) Y .

145

The last term is strictly negative for sufficiently large ν > 0, since R(0) ≤ 0 (by Lemma 5.1.5) and R(0) + R30 (X) < 0. By Theorem 3.2.10, the inequality 3 3 (ν −1 Y ) < 0 implies σ(GνY ) ⊂ C− . GνY In particular, if R(0) < 0, then for arbitrary Y0 > 0 there exists a sufficiently small α > 0, such that R(0) + R30 (−αY0−1 ) < 0. Hence we can repeat the previous considerations with X = −αY0−1 . We find that −νX −1 = αν Y0 stabilizes G for all sufficiently large ν. "

5.3 Existence and properties of stabilizing solutions Now we are ready to apply the results from Chapter 4 to the different types of Riccati equations derived in Chapter 2. For both the definite and the indefinite case, we first clarify the relation between Riccati equations and inequalities. Then we discuss properties of the stabilizing solutions and their dependence on parameters. In particular, we consider the cases of LQ-control and the Bounded Real Lemma. Iterative methods to compute stabilizing solutions are examined in Section 5.4. The results of the present section have appeared partly in [46, 47, 48, 40]. 5.3.1 The Riccati equation with definite constraint Let R be a Riccati operator in the sense Definition 5.1.1. For brevity we sometimes write FX = −Q(X)−1 S(X)∗ , if X ∈ dom R. Our first result is an immediate consequence of Theorem 4.2.2 and Theorem 5.2.2. Theorem 5.3.1 Let R be stabilizable. Then the following implications hold. (i)

ˆ ≥0 ˆ ∈ dom+ R : R(X) ∃X =⇒

∃X+ ∈ dom+ R : R(X+ ) = 0, σ(RX+ ) ⊂ C− ∪ iR. and ∀X ∈ dom+ R with R(X) ≥ 0 : X+ ≥ X.

ˆ ∈ dom+ R : R(X) ˆ >0 (ii) ∃X ˆ ∈ dom+ R : R(X) ˆ ≥ 0, ⇐⇒ ∃X

146


ˆ is β-detectable (A + BFXˆ , (Aj0 + B0j FXˆ ), R(X)) ˆ σ(RX+ ) ⊂ C− . ⇐⇒ ∃X+ ∈ dom+ R : R(X+ ) = 0, X+ > X, Moreover, if X+ ∈ dom+ R satisfies R(X+ ) = 0, and σ(RX+ ) ⊂ C− , ˆ for all X ˆ ∈ dom+ R with R(X) ˆ > 0. then X+ > X Proof: By Proposition 5.2.1 and Theorem 5.2.2, Assumption 4.2.1 (H1)–(H5) and (H7) is satisfied with f = R, D = dom R, D+ = dom+ R and Tx = R3X . Hence, (i) follows from Theorem 4.2.2 and the continuous dependence of σ(R3X ) on X ∈ dom R, and (ii) follows from the Corollaries 4.2.5 and 4.2.7. " As an immediate consequence of Corollary 4.2.6, we have another necessary and sufficient criterion for the existence of stabilizing solutions. Corollary 5.3.2 The equation R(X) = 0 has a stabilizing solution X+ ∈ ˆ ∈ dom+ R such that X1 ≥ dom+ R, if and only if there exist matrices X1 , X 3 ˆ ˆ ˆ X, R(X1 ) ≤ 0, R(X) ≥ 0, and (RXˆ , R(X)) is β-detectable. Based on the notion of observability instead of β-detectability, we derive the following sufficient criterion for a matrix X1 to be stabilizing. ˆ ∈ dom+ R, such that X1 ≥ X, ˆ R(X) ˆ ≥ 0, Corollary 5.3.3 If X1 , X 3 ˆ R(X1 ) ≤ 0, and (A + BFX1 , R(X)) is observable, then σ(RX1 ) ⊂ C− and ˆ X1 > X. ˆ ≤ R3 (X ˆ − X1 ). Hence ˆ − X1 ) ≤ LA+BFX (X Proof: By concavity R(X) X1 1 ˆ we are in the situation of Theorem 3.6.1(k) with X = −(X − X1 ) and Y = ˆ It follows now from the equivalent statements (c) and (g) of Theorem −R(X). ˆ " 3.6.1 that σ(R3X1 ) ⊂ C− and X > 0, i.e. X1 > X. Below, we will analyze the dependence of the stabilizing solution X+ on the weight matrix 6 0 P0 S0 . (5.24) M= S0∗ Q0 First, let us note two results for the special cases corresponding to LQstabilization and the Bounded Real Lemma. In the situation of LQ-stabilization we have Q0 > 0 and M ≥ 0, and the following holds (compare also [212, 69] for the case, where all B0j vanish). Corollary 5.3.4 Assume that Q0 > 0, M ≥ 0, and R is stabilizable. Let −1 ∗ ∗ F0 = −Q−1 0 S0 and M0 = P0 − S0 Q0 S0 . Then the following hold: (i)

The equation R(X) = 0 has a greatest solution X+ ≥ 0, which is almost stabilizing, i.e. σ(R3X+ ) ⊂ C− ∪ iR.


147

(ii) If (A + BF0 , (Aj0 + B0j F0 ), M0 ) is β-detectable, then σ(R3X+ ) ⊂ C− . (iii) If (A + BF0 , M0 ) is observable then σ(R3X+ ) ⊂ C− and X+ > 0. Furthermore, in the cases (ii) and (iii), the matrix X+ is the unique nonnegative definite solution of the equation R(X) = 0, and the control u = FX+ x stabilizes system (5.2) and minimizes the cost-functional 6∗ 0 6 < ∞0 x(t; x0 , u) x(t; x0 , u) M dt . J(x0 , u) = E u(t) u(t) 0 ˆ = 0 ∈ dom+ R and R(0) ≥ 0. Proof: The assumptions guarantee that X Therefore, (i) and (ii) follow immediately from Theorem 5.3.1. Moreover, under the detectability assumption in (ii) it follows from Corollary 4.2.5, that a nonnegative definite solution of the equation R(X) = 0 is necessarily stabilizing and therefore coincides with X+ . By Theorem 5.2.2(ii) we have 0 6∗ 0 6 I I 3 M RX+ (X+ ) = − =: M+ ≤ 0 . FX+ FX+ By Lemma 1.8.7, the observability of (A + BF0 , M0 ) implies that also (A + BFX+ , M+ ) is observable. Hence (iii) follows from Theorem 3.6.1. Since we can repeat this argument with any nonnegative definite solutions of R(X) = 0 we find again that X+ is unique with this property. Moreover, if σ(R3X+ ) ⊂ C− then FX+ stabilizes the system, by Corollary 5.2.3 and minimizes the cost-functional by Corollary 2.1.2. " In the situation of the Bounded Real Lemma, we have Q0 > 0 and P0 ≤ 0, and the following holds. Corollary 5.3.5 Assume that P0 ≤ 0, Q0 > 0 and there exists a matrix ˆ ∈ dom+ R such that R(X) ˆ ≥ 0. Consider the statements X (a) (b) (c) (d) (e) (f )

LA + ΠA0 is stable, R30 is stable, ∃X ∈ dom+ R : R(X) = 0 and X ≤ 0, ∃X ∈ dom+ R : R(X) ≥ 0 and X ≤ 0, X ∈ dom+ R and R(X) ≥ 0 ⇒ X ≤ 0, X ∈ dom+ R and R(X) ≥ 0 ⇒ X < 0.

The following implications hold. (i) (a)⇒(e), (b)⇒(e), (b)⇒(c), and – trivially – (f )⇒(e)⇒(d), (c)⇒(d). (ii) If (A, (Aj0 ), P0 ) is β-detectable, then the assertions (a)–(e) are equivalent. (iii) If (A, P0 ) is observable, then the assertions (a)–(f ) are equivalent.

148


Proof: ‘(a)⇒(e)’: On dom+ R the following inequality holds by definition R(X) = P (X) − S(X)Q(X)−1 S(X)∗ ≤ P (X) = LA (X) + ΠA0 (X) + P0 (5.25) Thus R(X) ≥ 0 implies LA (X) + ΠA0 (X) ≥ −P0 whence X ≤ 0 if LA + ΠA0 is stable (by Theorem 3.2.10). ‘(b)⇒(e)’: By concavity ∗ 3 R(X) ≤ R(0) + R30 (X) = P0 − S0 Q−1 0 S0 + R0 (X),

X ∈ dom R. (5.26)

Thus, Q(0) = Q0 > 0 and R(X) ≥ 0 imply 0 ∈ dom+ R and R30 (X) ≥ −P0 whence X ≤ 0 if R30 is stable. ‘(b)⇒(c)’: This follows immediately from Theorem 5.3.1 and the implication ‘(b)⇒(e)’ just proven. Now let (A, (Aj0 ), P0 ) be β-detectable. To establish the equivalence of the assertions (a)–(e), it suffices to prove (d)⇒(a) and (d)⇒(b). ‘(d)⇒(a)’: If R(X) ≥ 0 for some X ∈ dom+ R, X ≤ 0, then in particular LA (X) + ΠA0 (X) ≥ −P0 ; hence (a) by Theorem 3.6.1. ‘(d)⇒(b)’: In view of (5.26), we only have to show that (A + BF0 , (Aj0 + B0j F0 ), M0 ) is β-detectable (with F0 , M0 from Corollary 5.3.3). This, however, follows easily from the fact that M0 X = 0 implies F0 X = 0. Now let (A, P0 ) be observable. To establish the equivalence of the assertions (a)–(f), it suffices to prove (d)⇒(a), (d)⇒(f), and (d)⇒(b). The first two implications follow (like above) from LA (X) + ΠA0 (X) ≥ −P0 and Theorem 3.6.1. The last follows (like above) from the fact that (A + BF0 , M0 ) is observable. The proof is complete. "

Remark 5.3.6 Roughly speaking, to apply Theorem 4.2.2 one needs a staˆ to the inequality bilizing matrix X0 and the existence of a solution X R(X) ≥ 0. The latter condition is trivially satisfied in the situation of ˆ = 0. In the situation of the Bounded the LQ-control problem with X Real Lemma, on the other hand, X0 = 0 must be a stabilizing matrix (at least under the detectability assumption). Hence, in either problem, one of these conditions is usually satisfied. Let us name further conditions that ensure the stabilizability of R. ˆ ∈ dom+ R ∩ Hn : Corollary 5.3.7 Assume that P0 ≤ 0, Q0 > 0 and ∃X − ˆ ≥ 0. If one of the following conditions holds, then R is stabilizable. R(X) ˆ > 0. (i) R(X) (ii) ∃X ≥ 0 : R(X) < 0. (iii) (A + BF0 , (Aj0 + B0j F0 ), M0 ) is detectable.


149

Proof: Since R(0) ≤ 0, it follows from Corollary 4.2.5 that both (i) and (ii) imply the stabilizability of R. By Theorem 5.2.2(vii), the detectability condition (iii) implies (ii). "

Parameter dependence of the stabilizing solution To analyze the dependence of the stabilizing solution X+ on the matrix M , it is convenient to introduce the following subsets of Hn+m (depending on Λ and Π but these are fixed). M+ = {M : dom+ RM O= ∅} , M0 = {M : RM is stabilizable} , M1 = {M : Λ(X) + Π(X) + M ≥ 0 is solvable in cl dom+ RM } , M2 = {M : RM (X) = 0 has a stabilizing solution in dom+ RM } . Proposition 5.3.8 (i) M0 O= ∅ ⇒ M0 ⊃ M+ (ii) Let M ∈ Hn+s , X ∈ dom RM . −M 3 3 Then −X ∈ dom R−M and (RM X ) = (R−X0 ) . In particular M0 = −M0 . n+m = int M1 . (iii) M0 O= ∅ ⇒ M2 = M2 + H+ Proof: (i): This is a reformulation of Proposition 5.2.5. (ii): If we replace M and X by −M and −X, then Q(X) and S(X) change their sign. Therefore, the product Q(X)−1 S(X)∗ is invariant, and so is the derivative of R. (iii): By the inertia formula of Lemma A.2, it is easily seen that int M1 = {M : ∃X ∈ dom+ RM such that RM (X) > 0} . Hence M2 = M+ ∩M0 ∩int M1 by Theorem 5.3.1. Obviously M+ ∩int M1 = int M1 = int M1 + Hn+s . Moreover, if M0 O= ∅ then M0 ⊃ M+ by (i). Thus M+ ∩ M0 = M+ = M+ + Hn+s , which proves M2 = int M1 = int M1 + Hn+s . "

Theorem 5.3.9 There exists a (real) analytic order-preserving function X+ : M2 → Hn such that X+ (M ) is the stabilizing solution of RM (X) = 0 for all M ∈ M2 . R Proof: Clearly D = {(M, X) ∈ Hn+m × Hn R QM (X) > 0} is non-empty and open in the real vector space Hn+m × Hn and the map G : D → Hn defined by

150


G : (M, X) L→ G(M, X) := RM (X) is (real) analytic. As a consequence, also the derivative ? F3 ∂G ∂G : (M, X) L→ (M, X) = RM X ∂X ∂X is an analytic map from D to L(Hn ). Now assume that M0 ∈ M2 and let X+ = X+ (M0 ) ∈ dom+ RM0 be the ∂G stabilizing solution of RM0 (X) = 0. Then (M0 , X0 ) ∈ D and ∂X (M0 , X0 ) = M0 3 (R )X0 is stable, in particular invertible. As a consequence of the implicit function theorem for analytic functions [55], there is an open ball B(M0 , ε0 ) in Hn+m such that for all M ∈ B(M0 , ε0 ) there exists a unique solution X(M ) ∈ Hn of RM (X) = 0 which depends analytically on M ∈ B(M0 , ε0 ). But then ? F3 ∂G (M, X(M )) = RM X(M) ∂X F3 ? is continuous (even analytic) on B(M0 , ε0 ) and since σ( RM0 X(M ) ) ⊂ C− 0 F3 ? there exists ε ∈]0, ε0 [ such that σ( RM X(M) ) ⊂ C− for all M ∈ B(M0 , ε). M L→

Hence X(M ) is a stabilizing solution of RM (X) = 0 for all M ∈ B(M0 , ε) and so B(M0 , ε) ⊂ M2 . The restriction of X+ (·) to B(M0 , ε) coincides with X(·) on B(M0 , ε). Therefore X+ (·) : M2 → Hn is analytic and by Theorem 5.3.11 below order-preserving. "

The continuity of X+ also leads to the following result for linear matrix inequalities. Corollary 5.3.10 If M ∈ M2 , then X+ (M ) is the largest solution of the linear matrix inequality Λ(X) + Π(X) + M ≥ 0 in cl dom+ R. Proof: By Remark 5.1.4(v) and Theorem 5.3.1 it is clear that X+ (M ) is the largest solution of the linear matrix inequality Λ(X) + Π(X) + M ≥ 0 ˆ ∈ cl dom+ RM be another solution of this inequality. in dom+ RM . Let X ˆ ≥ 0. Hence ˆ ∈ dom+ RM+εI and RM+εI (X) For arbitrary ε > 0 we have X ˆ By continuity also X+ (M ) ≥ X. ˆ X+ (M + εI) ≥ X. " Now we prove that X+ depends monotonically on M (cf. [207, 208, 166] for the deterministic case). ˆ ∈ dom+ RM0 Theorem 5.3.11 Let M1 ≥ M0 . If there exists a solution X to the inequality RM0 (X) ≥ 0 and RM1 is stabilizable, then there exists = D a M1 M 3 M 0 0 ˆ If (R ) , R (X) ˆ is greatest solution X+ to R (X) = 0 and X+ ≥ X. ˆ X

β-detectable, then X+ is stabilizing for RM1 . In particular, if M0 ∈ M2 , then M1 ∈ M2 and X+ (M0 ) ≤ X+ (M1 ).


151

Proof: By M1 ≥ M0 we have dom+ RM0 ⊂ dom+ RM1 and RM1 (X0 ) ≥ 0 by Remark 5.1.4(v). Thus, by Theorem 5.3.1, there exists a greatest solution X1 to RM1 (X) = 0 and X1 ≥ X0 . If the detectability assumption holds, then by Theorem 5.3.1 there exists an ˜ > 0. Again by M1 ≥ M0 and Remark ˜ ∈ dom RM0 such that RM0 (X) X M1 ˜ 5.1.4(v) we have R (X) > 0. Thus again by Theorem 5.3.1 the matrix X1 is stabilizing. " An analogous argument shows that X+ depends on M in a concave fashion. Theorem 5.3.12 Let M0 , M1 ∈ Hn+s be arbitrary, and set Mτ := (1 − τ )M0 + τ M1 for τ ∈ [0, 1]. Assume that for i = 0, 1 there exist solutions Xi ∈ dom+ RMi to RMi (X) = 0 and that RMτ0 is stabilizable for some τ0 ∈ ]0, 1[. Then there exists a greatest solution Xτ0 to RMτ0 (X) = 0 and Xτ0 ≥ (1 − τ0 )X0 + τ0 X1 . If X0 or X1 is stabilizing then so is Xτ0 . ˆ τ0 := (1 − τ0 )X0 + τ0 X1 . Obviously X ˆ τ0 ∈ dom RMτ0 and by Proof: Set X Remark 5.1.4(v) 0 ≤ (1 − τ0 ) (Π(X0 ) + Λ(X0 ) + M0 ) + τ0 (Π(X1 ) + Λ(X1 ) + M1 ) ˆ τ0 ) + Λ(X ˆ τ0 ) + Mτ0 , = Π(X (5.27) ˆ τ0 ) ≥ 0. Thus by Theorem 5.3.1 there exists a greatest soluwhence RMτ0 (X Mτ0 ˆ τ0 . tion Xτ0 to R (X) = 0 and Xτ0 ≥ X If without loss of generality X0 is stabilizing, then by Theorem 5.3.1 there ˜ τ0 := ˜ 0 ) > 0. Now we set X ˜ 0 ∈ dom RM0 such that RM0 (X exists an X Mτ0 ˜ τ0 ) + ˜ (1 − τ0 )X0 + τ0 X1 ∈ dom R and conclude as in (5.27), that Π(X ˜ Λ(Xτ0 ) + Mτ0 > 0. Thus again by Theorem 5.3.1 the greatest solution of the equation RMτ0 (X) = 0 is stabilizing. "

Behaviour on the boundary Finally we take a look at the boundary of M2 . The question arises, whether X+ (M ) explodes, as M approaches ∂M2 . The next Proposition gives a sufficient criterion to exclude such a behaviour. Proposition 5.3.13 Let (Mk )k∈N be a bounded decreasing sequence in M2 . If the pair (A, B) is controllable, then the X+ (Mk ) are bounded and converge to the greatest solution of Λ(X) + Π(X) + M ≥ 0 where M = limk→∞ Mk . This result is of particular interest in view of a non-strict version of the Bounded Real Lemma 2.2.4. Corollary 5.3.14 Assume that system (2.8) is internally stable and 0. Hence, R is stabilizable by Theorem 5.3.7, such that Mk ∈ M2 . Now, the assertion follows from Proposition 5.3.13. " To prove Proposition 5.3.13, we need two simple lemmata. Lemma 5.3.15 Let (Mk )k∈N be an unbounded increasing sequence of Hermitian matrices in Hn . Then there exists a nonzero vector e ∈ Kn such that limk→∞ 1x, Mk x2 = ∞ for all x ∈ Kn with 1x, e2 = O 0. Proof: Replacing Mk by Mk − M0 we may suppose without restriction of generality that Mk ≥ 0. By a compactness argument we find a subsequence (kj )j∈N such that the limit H = limj→∞ Mkj /<Mkj < exists. Since 0. F ? (ii) ∃X+ ∈ dom± R, X+ < 0, such that R(X+ ) = 0 and σ RX+ 3 ⊂ C− . Proof: The equivalence of (i) and (ii) under the assumption that G is stabilizable follows immediately from Corollary 4.2.7 and Lemma 5.1.8. By Lemma 5.3.23 below, the observability of (A, P0 ) together with (i) implies the stabilizability of G. Hence (b) implies the equivalence of (i) and (ii). "

Remark 5.3.20 If X+ is a stabilizing solution of Rγ (X) = 0, where Rγ is given in (2.21), then F defined in (2.22) solves the γ-suboptimal stochastic H ∞ problem for system (2.13). This follows from the fact that RγF (X) = 0 and (RγF )3X is stable by Corollary 2.3.5. Hence 0. The idea is to perturb R slightly, such that G becomes stabilizable and the strict inequality remains solvable. For ε > 0 we define Rε : X L→ R(X) − εI and its dual G ε : Y L→ G(Y ) − εY 2 . Proposition 5.3.21 Assume that for some ε > 0 there exists an X ∈ dom± R, X < 0, such that R(X) > εI. Consider the sequence (Yk ) produced by Newton’s method applied to the equation G ε (Y ) = 0 starting at Y0 = νI. If ν > 0 is chosen sufficiently large, then the Yk converge quadratically to a stabilizing solution Y+ε ∈ dom+ G of this equation. Its negative inε verse X+ = −(Y+ε )−1 ∈ dom± R is a stabilizing solution of the equation R(X) = εI; moreover, it is the largest solution of the inequality R(X) ≥ εI n . in dom± R ∩ int H− Proof: The operator G ε is well-defined on dom G and the assertions of Theorem 5.2.7 carry over to G ε . Since R(0) − εI < 0, Theorem 5.2.7(v) yields that ε 3 the operator GνI is stable for ν G 1. By assumption, the inequality G ε (Y ) > 0 is solvable in dom+ G. Thus, the result follows from Theorem 4.2.2. " ε converge to a solution of the Riccati equation. As ε → 0, the X+

Corollary 5.3.22 Assume that there exists an X ∈ dom± R, such that R(X) > D 0. Then there exists an X+ ≤ 0 such that R(X+ ) = 0 and = σ R3X+ ⊂ C− ∪ iR. Proof: For sufficiently small ε the assumptions of Proposition 5.3.21 are ε < 0 are the largest solutions of R(X) ≥ εI in satisfied. Since the X+ ε −1 n ) ∈ dom+ G, dom± R∩int H− , they increase as ε decreases. Moreover, −(X+ ε ε such that X+ ≤ 0 for all ε > 0. Hence the X+ converge to some X+ ≤ 0 as ε → 0, and the assertions hold by continuity. " If we consider the matrix X+ from Corollary 5.3.22 we can fill the gap in the proof of Theorem D by showing that the observability of (A, P0 ) implies = 5.3.19 3 X+ < 0 and σ RX+ ⊂ C− . ˆ ∈ dom± R, such that R(X) ˆ > Lemma 5.3.23 Assume that there exists an X 0. If (A, P0 ) is observable, then there exists an X ∈ dom R, X < 0, such + ± + D = that R(X+ ) = 0 and σ R3X+ ⊂ C− .

Proof: It remains to show that the matrix X+ ≤ 0 from Corollary 5.3.22 is negative definite and stabilizing. By Remark 5.1.4(vi) and the definition of PF in (2.23) it follows that (A + B2 F )∗ X+ + X+ (A + B2 F ) ≥ (C + D2 F )∗ (C + D2 F ) ,

158


whence by Lemma 1.8.7 (i) and Corollary 3.6.1 we have X+ < 0, i.e. X+ ∈ −1 dom± R. Thus also Y+ = −X+ ∈ dom+ G and G(Y+ ) = 0. Moreover Y+ is the ˆ −1 ∈ largest solution of the inequality G(Y ) ≥ 0. By our assumption Yˆ = −X dom+ G and G(Yˆ ) > 0. We conclude, that Yˆ ≤ Y+ . Since G is concave on dom+ G, we have Since GY+ 3

GY+ 3 (Yˆ − Y+ ) ≥ G(Yˆ ) − G(Y+ ) = G(Yˆ ) > 0 . F ? is resolvent positive, Theorem 3.2.10 yields σ GY+ 3 ⊂ C− .

"

Thus we have also established the second part of Theorem 5.3.19. In accordance with Theorem 5.3.1 it is natural to ask, whether we can replace the observability condition by β-detectability. At this point, it is helpful to recall the underlying disturbance attenuation problem from Section 2.3. Our aim was to construct a feedback-gain matrix F , such that the closed-loop system dx = (A + B2 F )x dt +

N O

j (Aj0 + B20 F )x dwj + B1 v dt +

j=1

N O

j B10 v dwj

j=1

z = (C + D2 F )x + D1 u is internally stable and the perturbation operator LF has norm less than γ. This is achieved by F = −(Q2 (X) − S3 (X)Q1 (X)−1 S3 (X)∗ )−1 × (S2 (X)∗ − S3 (X)Q1 (X)−1 S1 (X)∗ ) , if X = X+ is a stabilizing solution of the Riccati equation R(X) = 0. But it is slightly too much to require σ(R3X+ ) ⊂ C− , because this implies also the stability of the closed-loop system dx = (A + BF )x dt +

N O

(Aj0 + B0j F )x dwj

j=1

= (A + B2 F + B1 F )x dt +

N O

j j (Aj0 + B20 F + B10 F )x dwj .

j=1

If we merely look for an internally stabilizing feedback F , then we just need j σ(LAˆ + ΠAˆ0 ) ⊂ C− with Aˆ = A + B2 F and Aˆj0 = Aj0 + B20 F as in (2.15). This is exactly what we can guarantee under the weaker assumptions. Proposition 5.3.24 Assume that (A, (Aj0 ), P0 ) is β-detectable and there exists a solution X+ ∈ dom± R to the equation R(X) = 0. Then X+ is internally stabilizing, i.e. D = σ LAˆ + ΠAˆ0 ⊂ C− , j where Aˆ = A + B2 F , Aˆj0 = Aj0 + B20 F , and F is defined in (5.7).


Proof: By Remark 5.1.4(vi) and the definition of Aˆ and D = LAˆ + ΠAˆ0 ≥ (C + D2 F )∗ (C + D2 F ) =

159

Aˆj0 it follows that Cˆ ∗ Cˆ

ˆ (Aˆj ), C) ˆ is β-detectable by Lemma 1.8.7 with Cˆ = C + D2 F . The triple (A, 0 " (ii), such that the result follows once more from Theorem 3.6.1.

Remark 5.3.25 (i) If X+ is an internally stabilizing solution of Rγ (X) = 0, where Rγ is given in (2.21), and F is defined according to (2.22) then the perturbation operator LF corresponding to system (2.14) satisfies 0. Assume that there exists a stabilizing solution X+ ∈ int Hn ⊂ dom+ R. ˜ = By Remark 5.1.4(i) we can consider the equivalent equation −R(−X) ˜ ˜ ˜ = R(X) = 0. Obviously X > 0 satisfies R(X) = 0 if and only if X n ˜ ˜ ˜ ˜ −X ∈ int H− ⊂ dom± R satisfies R(X) = 0. By assumption, Y+ = ˜ −1 = X −1 > 0 stabilizes the dual operator G˜ and G(Y ˜ + ) = 0. Hence, −X + + by Theorem 5.3.19, Y+ is the largest positive definite solution of the ˜ + < 0 is the largest negative definite solution ˜ ) = 0 and X equation G(Y ˜ ˜ + is the smallest positive of R(X) = 0. We conclude that X+ = −X definite solution of R(X) = 0. This again proves that there can be at most one positive definite solution. Parameter dependence of the largest solution By G M we denote the operator dual to RM , where M may vary, and Λ and Π are fixed. In the indefinite case it seems more difficult to classify those M that allow a largest solution X+ ∈ dom± RM of the equation RM (X) = 0. We will restrict ourselves to some relevant cases. As before, let M have the form   P0 S20 S10 ∗ Q20 S30  . M =  S20 ∗ ∗ S10 S30 Q10 In analogy to the previous section, we introduce the following general sets: N± = {M : P0 ≤ 0, Q20 < 0, Q10 > 0} , N0 = {M ∈ N± : G M is stabilizable} , n }, N1 = {M ∈ N± : RM (X) > 0 is solvable in dom± RM ∩ int H− M M n ¯1 = {M ∈ N± : R (X) ≥ 0 is solvable in dom± R ∩ int H− } , N

160


N2 = {M ∈ N± : RM (X) = 0 has a stabilizing solution n in dom± RM ∩ int H− },

¯2 = {M ∈ N± : RM (X) = 0 has an internally stabilizing solution N n in dom± RM ∩ H− }.

These definitions are just meant as abbreviations that allow us to state the following results more concisely. Recall also the definition of A, Aj0 , and P0 in (5.31). Corollary 5.3.26 (i) N0 ⊃ int N± . ¯1 . (ii) N0 ∩ N1 = N2 ⊂ N0 ∩ N (iii) int N1 ⊂ N2 ⊂ N1 . ¯2 . (iv) If (A, (Aj0 ), P0 ) is β-detectable, then N2 ⊂ N Proof: (i), (ii), (iii), and (iv) follow from Theorem 5.2.7(v), Theorem 5.3.18, Corollary 5.3.22, and Proposition 5.3.24, respectively. Alternatively, we can derive (iv) from the Corollaries 2.3.5 and 5.3.5. " The smooth dependence of X+ on M ∈ N2 follows exactly as in the proof of Theorem 5.3.9. n Theorem 5.3.27 There exists a (real) analytic function X+ : N2 → int H− M such that X+ (M ) is the stabilizing solution of R (X) = 0 for all M ∈ N2 . It can be extended continuously to N0 ∩ N¯1 .

Finally, we analyze the monotonic dependence of X+ on M and derive some ˜ , M] ⊂ sufficient criteria for X+ to be defined on a given order interval [M n+s H . Again, the informal principle of Remark 5.3.6 applies. Roughly speakˆ ≥ 0 in ing, the existence of X+ (M ) follows from the solvability of RM (X) M M ˜ dom± R and the stabilizability of G . Existence intervals [M , M ] will now ˆ ˆ ˆ ≥ M, be specified by giving criteria that RM (X) ≥ 0 is solvable for all M ˆ ˆ ≤ M . Here we make use of the connection and G M is stabilizable for all M between the stabilizability of the dual operator G and detectability criteria for the original system. The following proposition transforms this idea into some precise statements. ˜ ∈ cl N± , such that M ≥ M ˜ , and define A, ˜ Proposition 5.3.28 Let M, M ˜ j M M ˜ ˜ ˜ A0 , and P0 in analogy to (5.31). We set R = R and R = R , with the ˜ respectively. Then the following hold: dual operators G and G, (i) (ii) (iii) (iv) (v)

˜ dom+ G ⊃ dom+ G. ˜ ˜ ∀X ∈ dom± R, X < 0: R(X) ≥ R(X). ˜ P˜0 ). If (A, P0 ) is observable, then so is (A, j ˜ (A˜j ), P˜0 ). If (A, (A0 ), P0 ) is β-detectable, then so is (A, 0 ˜ 3 , R(0)), ˜ If (±R30 , R(0)) is detectable, then so is (±R respectively. 0


161

˜ ∈ N1 , then If (A, P0 ) is observable or (−R30 , R(0)) is detectable and M ˜ , M ] ⊂ N2 . [M ˜ ∈ N1 , then [M ˜ , M ] ⊂ N¯2 . (vii) If (A, (Aj0 ), P0 ) is β-detectable and M ˜ ¯ ˜ ˜ ). (viii) If M, M ∈ N0 ∩ N1 and M ≥ M , then X+ (M ) ≥ X+ (M (ix) If (Mk ) is a monotonically increasing sequence in N2 then there exists the limit matrix X∞ = limk→∞ X+ (Mk ) ≤ 0. If, moreover, the Mk converge to M∞ ∈ cl N± , and X∞ ∈ dom± RM∞ , then RM∞ (X∞ ) = 0.

(vi)

Proof: (i): By definition, we have X ∈ dom+ G M if and only if QM 1 (X) > 0. ˜ M n ˜ , which proves the Obviously QM (X) ≥ Q (X) for all X ∈ H , if M ≥ M 1 1 assertion. ˜ as Schur-complements R(X) = S(Λ(X) + Π(X) + (ii): If we write R and R ˜

n M/Q1 (X)), we can directly apply Corollary A.4, since X ∈ dom± RM ∩int H− ˜ implies In Q(X) = In Q(X). n (iii) and (iv): We note that 0 ≥ P ≥ P˜ . Hence, if P˜ V = 0 for some V ∈ H+ , −1 −1 ∗ ∗ ˜ S˜ V . If, then also P V = 0; moreover, by Corollary A.4, also Q20 S20 V = Q 20 20 in addition, V is an eigenvector of L∗A˜ or L∗A˜ + ΠA˜0 , then it follows that V is also an eigenvector of L∗A or L∗A + ΠA0 , respectively, with the same eigenvalue β. If we assume (A, P0 ) to be observable, or (A, (Aj0 ), P0 ) to be β-detectable, then necessarily β < 0, which proves the assertions. (v) Let X < 0 or X > 0 be given such that R30 (X) + R(0) < 0. Then ˜ ≤ R(0) ≤ 0, we have x∗ R30 (X)x < 0 for all x ∈ Ker R(0). Since R(0) ˜ ˜ Ker R(0) ⊂ Ker R(0), and x ∈ Ker R(0) implies F0 x = F˜0 x by Corollary ˜ 30 (X)x < 0 for all x ∈ Ker R(0), ˜ A.4. Hence also x∗ R and a similar argument ˜ ˜ 3 (αX) + R(0) < 0 for some α > 0. as in the proof of Lemma 1.7.3 yields R 0 (vi) and (vii): From (iii) and (iv) it follows that the observability or de˜ , M ]. From (ii) we know tectability assumption holds for all elements of [M ˜ ∈N ¯1 implies [M ˜ , M ] ⊂ N1 or [M ˜ , M ] ⊂ N¯1 , respectively. ˜ ∈ N1 or M that M Hence the assertions (vi) and (vii) follow from Corollary 5.3.26. ˜ ) are the largest solu˜ ∈ N0 ∩ N¯1 then X+ (M ) and X+ (M (viii) If M, M ˜ M M ˜ implies tions of R (X) ≥ 0 and R (X) ≥ 0, respectively. Since M ≥ M ˜ M M ˜ )) ≥ R (X+ (M ˜ )) = 0 we have X+ (M ) ≥ X+ (M ˜ ). R (X+ (M (ix) By (ii) the X+ (Mk ) ≤ 0 are monotonically increasing, and hence converge. The assertion holds by continuity. "

Remark 5.3.29 For M ∈ N1 , Corollary 5.3.22 guarantees the existence of an almost stabilizing solution X+ ∈ dom± RM ∩ −Hn of the inequality RM (X) = 0. But we have not shown that X+ is the largest solution of the inequality RM (X) ≥ 0. Since we have neither found a counter-example, we leave this question open. But since X+ is obtained as the monotonic limit of stabilizing solutions of RM−εI (X) = 0, it depends monotonically on M .

162


5.4 Approximation of stabilizing solutions Since our approach in the previous section is constructive, it lends itself immediately for computational purposes. As we have shown, we can apply all the results in Chapter 4 on Newton’s method and modified Newton iterations to solve the equations R(X) = 0 and G(X) = 0. Each Newton-step consists in solving a linear equation with a resolvent positive operator, which was the topic of Section 3.5. In the following, we summarize the main facts; in particular we show, how a stabilizing matrix can be found to start the Newton iteration. Moreover, in Section 5.4.3, we present an alternative nonlinear fixed point iteration. 5.4.1 Newton’s method First, we recall the explicit form of Newton’s method and its modification to solve the equations R(X) = 0 and G(X) = 0. The equation R(X) = 0 For X ∈ dom R, let us set F (X) = −Q(X)−1 S(X)∗ . Then, by Theorem 5.2.2, the iteration R(X) = 0 takes the form Xk+1 = Xk − (R3Xk )−1 (R(Xk )) 6∗ 0 6G @0 I I = −(R3Xk )−1 M , F (Xk ) F (Xk ) where R3Xk (H) = (LAXk + ΠXk )(H) 0 = LA−BF (Xk ) (H) +

I F (Xk )

0

6∗ Π(H)

I F (Xk )

6 .

In each step, we have to evaluate F (Xk ) and compute either Xk+1 directly as the solution of 0 6∗ 0 6 I I 3 RXk (Xk+1 ) = − M (5.32) F (Xk ) F (Xk ) or the increment Xk+1 − Xk as the solution of R3Xk (Xk+1 − Xk ) = −R(Xk ) .

(5.33)

Both equations are of the type discussed in Section 3.5; by our results in Chapter 4 we know that they are solvable for all k, if X0 is stabilizing and the inequality R(X) ≥ 0 is solvable in dom+ R. The first equation corresponds to the algorithm proposed for standard Riccati

5.4 Approximation of stabilizing solutions

163

equations e.g. in [131, 174]. But it has been mentioned e.g. in [12] that the second exhibits better robustness properties in the presence of rounding errors. In Section 3.5 we have developed different strategies to solve such equations. If the dimension is small then a direct solution is possible. Otherwise, the particular cases, when (i) rk Π is small, or (ii) 0. −1 For sufficiently large ν > 0, the matrix Y0 = νX+ is contained in dom+ G and stabilizes G. 2 >0 Proof: The existence of X+ > 0 follows from Theorem 5.1.6. Since X+ 3 and R0 ≥ LA−BQ−1 S ∗ , the equality (5.36) for X = X+ implies 0

0

0 > LA−BQ−1 S ∗ (−X+ ) + R(0) ≥ R30 (−X+ ) . 0

0

Now we can apply Theorem 5.2.7(v) to finish the proof.

"

If the strict inequality R(0) < 0 holds, then a stabilizing matrix Y0 for G can e.g. be found in the form Y0 = νI. We make use of this observation to compute a stabilizing matrix for RM with M ∈ M+ . Computation of a stabilizing matrix for RM with M ∈ M+ ˆ ∈ dom+ RM . As we have seen in the proof of ProposiLet M ∈ M+ and X ˆ +X ˆ0 , tion 5.2.5, we can find a stabilizing matrix X0 for RM in the form X

5.4 Approximation of stabilizing solutions

165

ˆ 0 is a stabilizing matrix for RM with M ˆ defined in (5.16). Since the where X ˆ M stabilizability of R is independent of Pˆ0 we can replace Pˆ0 by an arbitrary ˆ > 0. Then, by matrix in Hn and assume without loss of generality that M ˆ M Lemma 1.7.7 and Proposition 5.2.5 the operator R is stabilizable if and only ˆ 0 ) < 0. This inequality is equivalent ˆ 0 > 0, such that RMˆ (X if there exists an X to ˆ

ˆ

ˆ 0 ) > 0, R−M (−X

ˆ 0 < 0, ˆ 0. M ˆ0 Sˆ0∗ Q ˆ 2. Find ν > 0 large enough, such that σ((G −M )3νI ) ⊂ C− . 3. Set Y0 = νI and compute Y1 , Y2 , . . . according to (5.35) . ˆ 4. Watch in each step, whether the iterate Yk is stabilizing for G −M . If one M Yk is not stabilizing, then R is not stabilizable. n ˆ + Yˆ −1 is stabilizing 5. If the Yk converge to some Y∞ ∈ int H+ then X0 = X + M for R . n n or converge to some Y∞ ∈ ∂H+ then RM If otherwise the Yk leave int H+ is not stabilizable. In extreme cases it is, of course, numerically hard to decide whether X∞ ∈ n ∂H+ or not; one can use a stopping criterion, and regard the system as not stabilizable if e.g. det Yk < ε for some k. 5.4.3 A nonlinear fixed point iteration Finally, we present a different approach to solve the equation R(X) = 0 with indefinite constraint X ∈ dom± R, which has its origin in the fact that there exist powerful methods for deterministic algebraic Riccati equations. ˜ of Π into positive operators Π 0 and Let there be given a splitting Π = Π 0 + Π ˜ ˜ = 0. For Π. In particular, we may think of the trivial splitting Π 0 = Π, Π M ˜ fixed Λ and variable M let R denote the Riccati operator corresponding to ˜ and M . We assume that it is easier to compute solutions of the equation Λ, Π, M ˜ = 0, because ˜ R (X) = 0 than of RM (X) = 0. In particular, this is true if Π M ˜ then R is a standard Riccati operator. For arbitrary M let X+ (M ) and

166


˜ + (M ) denote the stabilizing solutions of R(X) = 0 and R(X) ˜ X = 0, which for the moment, we assume to exist. ˜ + is Now let M be fixed and consider the equation RM (X) = 0. Since X relatively easy to compute, it makes sense to consider the nonlinear fixed point iteration ˜ + (M + Π 0 (Xk )), Xk+1 = X

X0 = 0 .

(5.38)

˜ + (M + Π 0 (X)), then If X+ is a fixed point of the mapping X L→ X ˜ M+Π 0 (X+ ) (X+ ) = R(X+ ) , 0=R

(5.39)

˜ + ) = M + Π(X+ ). Hence X+ = X+ (M ). Now let since (M + Π 0 (X+ )) + Π(X us provide sufficient criteria for the sequence defined by (5.38) to converge. In accordance with our previous notation, we write G˜M for the dual operator of ˜ M , and Y = −X −1 . R Theorem 5.4.2 Let M ∈ N± and assume the following: (i) (A, P0 ) is observable (see (5.31)) or (−R30 , R(0)) is detectable. ˆ ∈ dom± R ∩ int Hn : RM (X) ˆ > 0. (ii) ∃X − Then for all k ∈ N the Xk are well-defined by (5.38) and converge monotonically to X+ (M ). ˆ 0] the following hold. Proof: It is crucial to observe that for all X ∈ [X, (a) (b) (c)

ˆ −1 ∈ dom+ G˜M+Π 0 (X) , Yˆ = −X 0 G˜M+Π (X) (Yˆ ) > 0, 0 G˜M+Π (X) is stabilizable.

Let us verify these assertions. 0 ˆ < 0, Π ˜ 1 (X) ˆ + Q10 + (a) By definition Yˆ ∈ dom+ G˜M+Π (X) if and only if X 0 0 ˜ ˆ Π1 (X) < 0, and Π2 (X) + Q20 + Π2 (X) > 0. The first two inequalities hold by ˆ and X ˆ ∈ dom± RM . assumption, while the last follows from Π20 (X) > Π20 (X) 0 ˆ ˆ and RM (X) ˆ =R ˜ M+Π (X) (X) ˆ > 0, it (b) Since M + Π 0 (X) ≥ M + Π 0 (X) M+Π 0 (X) ˆ ˜ follows from Proposition 5.2.6 that R (X) > 0. (c) This follows from Proposition 5.3.28(vi). Proceeding with the proof of convergence, we show that the Xk are monotonˆ ically decreasing and bounded below by X. ˆ As an induction hypotheses, let us assume that By assumption, X0 = 0 > X. ˆ It suffices to for some k ≥ 0 we have constructed X0 ≥ X1 ≥ . . . ≥ Xk ≥ X. ˜ + (M + Π 0 (Xk )) is well-defined as the largest solution prove that Xk+1 = X ˜ M+Π 0 (Xk ) and satisfies Xk ≥ Xk+1 ≥ X. ˆ of R Using the observations (a), (b), and (c) with X = Xk and applying The˜ + (M + Π 0 (Xk )) < 0 in ˆ < Xk+1 = X orem 5.3.19, we find a solution X M+Π 0 (Xk ) ˜ dom± R . It remains to show that Xk+1 ≤ Xk . For k = 0 this is clear

5.5 Numerical examples

167

from Xk+1 < 0. Hence, we consider the case k ≥ 1 and take advantage of our induction hypotheses Xk−1 ≥ Xk . This implies M +Π 0 (Xk−1 ) ≥ M +Π 0(Xk ), ˜ + , we have Xk ≥ Xk+1 . whence by the monotonicity of X "

5.5 Numerical examples Let us now return to some of the examples presented in Section 1.9. In order to illustrate our results, we analyze stabilizabilty and disturbance attenuation problems for these systems and compare controllers obtained for different parameter values. We deliberately take some time to write out the corresponding Riccati equations and their dual versions explicitly and to describe how to find initial matrices for the Newton iteration. Moreover, we illustrate the benefit of double Newton steps and discuss some problems of regularization. 5.5.1 The two-cart system In Section 1.9.3 we have introduced the state-space equations dx = (Ax + B1 v + B2 u)dt + A0 x dw , z = Cx + Du as a stochastic model for a two-cart system with a random parameter. The Riccati equation of the stochastic γ-suboptimal H ∞ problem is given by Rγ (X) = A∗ X + XA + A∗0 XA0 − C ∗ C ? F + X B2 (D∗ D)−1 B2∗ − γ −2 B1 B1∗ X = 0 . For the dual operator G γ (Y ) = −Y A∗ − AY − Y A∗0 Y −1 A0 Y − Y C ∗ CY + B2 (D∗ D)−1 B2∗ − γ −2 B1 B1∗ R n with dom G γ = {Y ∈ Hn R det Y O= 0}, and dom+ G γ = int H+ we have 3

GYγ0 (H) = (−A∗ − C ∗ CY0 − A∗0 Y0−1 A0 Y0 )∗ H + H(−A∗ − C ∗ CY0 − A∗0 Y0−1 A0 Y0 ) + (Y0−1 A0 Y0 )∗ H(Y0−1 A0 Y0 ) . If (A, C) is observable, then (A, A0 , C) is anti-detectable (compare Remark 3 1.8.6), i.e. (−Rγ0 , R(0)) is detectable and G γ is stabilizable by Theorem 5.2.7. A stabilizing Y0 can e.g. be found in the form αX0−1 where X0 > 0 solves the standard Riccati equation

168


−A∗ X − XA + C ∗ C − X 2 = 0 .

(5.40)

Indeed, if X0 solves this equation, then A∗ (−X0 ) + (−X0 )A − C ∗ C < 0, and 3 we can apply Theorem 5.2.7(viii). Moreover, since GYγ0 does not depend on γ, we can choose Y0 independently of γ. A numerical example was presented in [194], and we use this example to test our theoretical results. Let     0 0 00 0 0 10  0  0 0 0 0 0 0 1    A=  −5/4 5/4 0 0  , A0 =  −1/4 1/4 0 0  , 1/4 −1/4 0 0 5/4 −5/4 0 0     0 T6 0 6 0 1000 B1 0010    0 . 0 1 0 0 = , D = , C = 2 B2T 0001 1 0000 The matrix D2 plays the role of a regularization. We will discuss this issue in later examples. With these data we found the matrix   2 0 0 −1  0 20 0   Y0 =  (5.41)  0 02 0  −1 0 0 2 to be stabilizing. For γ = 2 we could reproduce (by Newton’s method in 10 steps) the solution obtained by Ugrinovskii; by a bisection search we found the optimal attenuation value to be γ∗ ≈ 1.8293. For γ = 1.8293 the solution   4.6440 −6.4837 −4.7299 −3.3828  −6.4837 19.1359 17.4201 10.4627   X+ = Y+−1 ≈   −4.299 17.4201 18.9092 9.8687  > 0 −3.3828 10.4627 9.8687 7.0731 of Rγ (X) = 0 is obtained in 12 steps, whereas for γ = 1.8292 the 11th iterate Y11 is not stabilizing (i.e. by Theorem 5.3.18 in this case Gγ (Y ) = 0 is not n solvable in int H+ ). The question remains, whether it is adequate to model the parameter uncertainty as white noise. For comparison, we study the system x˙ = (A + δ(t)A0 )x + B1 v + B2 u , z = Cx + Du

(5.42)

with bounded parameter uncertainty δ(t). In view of Theorem 2.3.8 we wish to determine the largest value α = αmax , such that G γ (Y ) − αY = 0

5.5 Numerical examples

169

has a stabilizing solution in dom+ G γ . √ For γ = 2 we found αmax = 0.0756, whence for all |δ(t)| ≤ d =: αmax = 0.275 system (5.42) is stabilizable with attenuation level γ = 2. This bound corresponds to |∆(t)| = |k(t) − k0 | < 0.0687 =: ∆, which is rather far away from 0.75. Considering different attenuation levels γ we found (by a bisection search) the following maximal admissible uncertainty bounds ∆: 1.893 2 4 10 100 1000 ∞ γ ∆ 0 0.0687 0.1994 0.2826 0.4146 0.5590 0.8803 . ∆/k0 0 0.05 0.15 0.23 0.33 0.45 0.70 Apparently we can guarantee stability for an arbitrary time-varying parameter k : t L→ k(t) ∈ [0.5, 2] only if we abandon disturbance attenuation. But if we aim at an attenuation value of γ = 2 or γ = 4, we can still allow deviations of k(t) from k0 of about 5 or 15 percent, respectively. Double Newton steps If γ is close to the optimal attenuation value γ ≈ 1.893, then the Newton iteration is rather slow and almost linear. As we have pointed out in Section 4.3, we can benefit from a double Newton step in this situation. To see this effect, we consider the cases γ = 2 and γ ≈ 1.893 and apply Newton’s method to the equation G γ (Y ) = 0 with initial matrix Y0 from (5.41). In each step of the iteration we compute, additionally, the matrix Zk+1 = Yk − (GY3 k )−1 (G(Yk )) and the residual errors 0 and thus a stabilizing feedback-gain matrix F . Of course, this is not reasonable. Indeed, we see that Y+ is very close to singularity for large σ, and the norm of F becomes very large. Setting e.g. σ = 1000, we compute F = 107 [−0.0337, −5.4188, −0.0039, −0.7284] , which is unacceptable. Hence, we introduce the additional constraint λ1 (Y+ ) > 10−7 , where λ1 is the minimal eigenvalue of Y+ > 0, and the tolerance level 10−7 is about the square root of the machine precision. With this constraint, we find σmax ≈ 24.9370 with a corresponding stabilizing feedback-gain matrix Fmax = [−203.9290, 314.8963, −614.7729, 249.5053] .

(5.44)

To compute a guaranteed margin for robust stabilizability with respect to bounded uncertainties we consider the Riccati inequality Rσ (X) + αI > 0 (compare the Sections 1.6 and 2.3.2). With α = 4/100, we found the inequality to be solvable for σ = 52/3 = 17, ¯ 3. Hence the corresponding feedback-gain matrix F = [−154.6190, 272.6836, −547.9517, 260.2774] √ stabilizes the system robustly for |k2 − k| ≤ ασ = 52/15 = 3.4¯6. Disturbance attenuation Now let us address the disturbance attenuation problem described in Section 1.9.4. We try to choose F in such a way that the unevenness of the road v has small effect on the output y. First of all, we note that this control system is not regular in the sense of Definition 2.3.3, since D2 = 0. To apply our results, we have to regularize the system by incorporating the control u into the output z. To this end we might consider the extended output process 0 6 0 6 0 6 C 0 z ˜ ˜ ˜ ˜ , D2 = . (5.45) = Cx + D2 u with C = z˜ = 0 I u For a given feedback-matrix F ∈ R1×4 we define the perturbation operator LF : v L→ z˜ as in Section 2.3.1. We have 0, if and only ∗ if (γ 2 I + B01 XB01 )−1 > 0 and Rσ,γ (X) = A∗ X + XA + σ 2 A∗0 XA0 − C˜ ∗ C˜ − XB2 B2∗ X ∗ XB01 )−1 B1∗ X + XB1 (γ 2 I + σ 2 B01 > 0. As usually, we consider the equivalent dual inequality

172


˜ − B2 B ∗ G σ,γ (Y ) = −Y A∗ − AY − σ 2 Y A∗0 Y −1 A0 Y − Y C˜ ∗ CY 2 ∗ + B1 (γ 2 I + σ 2 B01 Y −1 B01 )−1 B1∗ > 0 .

It is easily seen that (A, C) is observable. Hence, by Lemma 5.4.1, the operator G σ,γ is stabilizable, and, again, a stabilizing matrix Y0 > 0 can e.g. be found in the form Y0 = νX0−1 , where X0 > 0 solves the standard Riccati equation (5.40). For different values of γ we try to solve the equation G σ,γ (Y ) = 0 by Newton’s method starting from Y0 . By a bisection search we determine the smallest γ, for which the sequence converges. Choosing σ = 0, we find γ > 17. With respect to our initial design problem, this does not make sense. We wanted to attenuate the effect of the unevenness of the road, not to increase it by the factor 17. What has happened? Of course, the problem lies with the regularization. Incorporating the control input u into the output z, we have necessarily increased the norm of the perturbation operator L : v L→ z˜. This effect can be diminished, if instead of (5.45), we set 0 6 0 6 0 6 z ˜ 2 = ε 0 u (5.46) ˜ +D ˜ 2 u with C˜ = C , D z˜ = = Cx 0 I u with some ε > 0. The choice of an adequate regularization, in fact, is not trivial, even in the deterministic case (compare e.g. [76]). On the one hand, we wish to choose ε as small as possible, such that L approximately describes the original input-output behaviour. On the other hand, choosing ε too small is likely to cause numerical problems. Moreover, in the present example, we find that small values of ε and γ result in high-gain feedback, i.e. an unbounded increase of the norm of F . To choose some bound, let us accept only feedback gain matrices F with 0 ⇐⇒ Q > 0 and P − SQ−1 S ∗ > 0. and if Q > 0, then M ≥ 0 ⇐⇒ P − SQ−1 S ∗ ≥ 0. (ii) Inversion formula: 6 6∗ 0 −1 6 0 0 −1 0 −P −1 S S P −1 −P −1 M = + S(M/P ) . 0 0 I I 0 6 Q2 S21 (iii) Quotient formula: If Q = then S12 Q1 N D = S(M/Q) = S S(M/Q1 ) S(Q/Q1 ) . ˜ be partitioned like M and set F = Q−1 S ∗ , (iv) Difference formula: Let M ˜ −1 S˜∗ , and ∆F = F − F˜ . Then F˜ = Q ˜ /Q − Q) ˜ = S(M/Q) − S(M ˜ /Q) ˜ + ∆F ∗ (Q−1 − Q ˜ −1 )−1 ∆F . S(M − M 6 0 U V (v) Transformation formula: Let T = have the same partitioning as 0 W M . Then S(T M T ∗ /W QW ∗ ) = U S(M/Q)U ∗ . Schur complements are used to establish the equivalence of Riccati-type matrix inequalities and linear matrix inequalities. For instance, the constrained inequality R(X) = P (X) − S(X)Q(X)−1 S(X)∗ > 0

with Q(X) > 0

is equivalent to the linear matrix inequality 6 0 P (X) S(X) >0. S(X)∗ Q(X) As a further illustrative application we note a simple consequence of Lemma A.2: Corollary A.3 If X, Y ∈ Hn are invertible and In X = In Y , then In(X − Y ) = In(Y −1 − X −1 ) . D = In particular, if X, Y > 0, then X > Y ⇐⇒ Y −1 > X −1 .

A Hermitian matrices and Schur complements

0

Proof: Consider the matrix M = By Lemma A.2 we have

6

183

X I . I Y −1

In M = In X + In(Y −1 − X −1 ) = In Y −1 + In(X − Y ) . Since In X = In Y = In Y −1 , the result follows.

"

˜ be partitioned as M with nonsingular Q ˜ and Q. AsCorollary A.4 Let M ˜ ˜ sume M ≤ M and In Q = In Q. ˜ /Q) ˜ and Then S(M/Q) ≥ S(M D = D = ˜ −1 S˜∗ . ˜ /Q) ˜ ⊂ Ker Q−1 S ∗ − Q Ker S(M/Q) − S(M ˜ then S(M/Q) > S(M ˜ /Q). ˜ If even M > M ˜ . Then, in particular, Q > Q ˜ and since Proof: We first assume M > M −1 −1 ˜ ˜ by assumption In Q = In Q, Corollary A.3 yields Q > Q . Hence, by ˜ /Q) since S(M/Q) > S(M the difference formula of Lemma A.2, we have D = ˜ 0 > 0. ˜ /Q0 − Q S M −M ˜ , then for small ε > 0, we can replace M by M + εI without Now, if M ≥ M affecting the assumptions. Hence, the first assertion holds by continuity. To prove the kernel inclusion, we have to take some more care with the perturbation argument. By the difference formula, the assertion is obvious, if ˜ In the general case, we consider the transformation Q > Q. 6 0 6 0 6 0 I 0 P − SQ−1 S ∗ 0 I −F ∗ M = . 0 I −F I 0 Q Applying e.g. a Sylvester transformations with unitary W1 , we have 0 6 Q+ 0 ∗ W1 QW1 = ∈ Hπ+ν 0 Q− with Q+ > 0 and Q− < 0. For θ > 1 we set Wθ = diag(θIπ , 1/θIν )W1 , such that 0 2 6 θ Q+ 0 Wθ QWθ∗ = > W1 QW1∗ . 0 θ−2 Q− With the inverse transformation matrix 0 0 6−1 0 6−1 6 I −F ∗ I F ∗ W1∗ I 0 T = = 0 I 0 W1∗ 0 W1 we define

184

A Hermitian matrices and Schur complements

0

0 6 6 P − SQ−1 S ∗ 0 Pθ Sθ ∗ Mθ = T , T =: 0 Wθ QWθ∗ Sθ∗ Qθ

˜ and Qθ > Q ≥ Q. ˜ Moreover, by the transformation such that Mθ ≥ M ≥ M formula, S(Mθ /Qθ ) = S(M/Q). It follows that for all θ > 1 we have the inclusion D = D = ˜ −1 S˜∗ . ˜ /Q) ˜ ⊂ Ker Q−1 S ∗ − Q Ker S(M/Q) − S(M θ θ ∗ −1 ∗ By construction limθ→1 Q−1 S , which proves the assertion. θ Sθ = Q

"

References

1. J. Ackermann. Robust Control. Springer-Verlag, 1993. 2. B. D. O. Anderson and J. B. Moore. Linear Optimal Control. Prentice Hall, Englewood Cliffs, New Jersey, 1971. 3. A. A. Andronov, A. A. Vitt, and L. S. Pontryagin. Statistische Auffassung dynamischer Systeme. Phys. Z. Sowjetunion, 6:1–24, 1934. 4. P. Ara and M. Mathieu. Local Multipliers of C ∗ -Algebras. Springer-Verlag, 2003. 5. W. Arendt. Resolvent positive operators. Proc. London Math. Soc., 54(3):321– 349, 1987. 6. L. Arnold. Stochastische Differentialgleichungen. R. Oldenburg Verlag, 1973. 7. L. Arnold, H. Crauel, and V. Wihstutz. Stabilization of linear systems by noise. SIAM J. Control Optim., 21:451–461, 1983. 8. V. I. Arnold. Gew¨ ohnliche Differentialgleichungen. Springer, 2001. 9. T. Ba¸sar and P. Bernhard. H ∞ -Optimal Control and Related Minimax Design auser, Boston, 2nd edition, 1995. Problems. A Dynamic Game Approach. Birkh¨ 10. G. P. Barker, R. D. Hill, and R. D. Haertel. On the completely positive and positive-semidefinite-preserving cones. Linear Algebra Appl., 56:221–229, 1984. 11. R. H. Bartels and G. W. Stewart. Solution of the matrix equation AX + XB = C: Algorithm 432. Comm. ACM, 15:820–826, 1972. 12. P. Benner and R. Byers. An exact line search method for solving generalized continuous-time algebraic Riccati equations. IEEE Trans. Autom. Control, 43(1):101–107, 1998. 13. A. Berman, M. Neumann, and R. J. Stern. Nonnegative Matrices in Dynamic Systems. John Wiley & Sons, New York, 1989. 14. A. Berman and R. J. Plemmons. Nonnegative Matrices in the Mathematical Sciences. Classics in Applied Mathematics. SIAM, 1994. 15. D. Bernstein and D. C. Hyland. Optimal projection equations for reducedorder modeling, estimation and control of linear systems with multiplicative noise. J. Optim. Th. & Appl., 58:387–409, 1988. 16. D. S. Bernstein. Robust static and dynamic output-feedback stabilization: Deterministic and stochastic perspectives. IEEE Trans. Autom. Control, AC32(12):1076–1084, 1987. 17. J.-M. Bismut. Linear-quadratic optimal stochastic control with random coefficients. SIAM J. Control, 14:419–444, 1976.

186

References

18. S. K. Biswas. Robust stabilization of linear systems in the presence of Gaussian perturbation of parameters. Optim. Contr. Appl. Meth., 19:271–286, 1998. 19. S. Bittanti, A. J. Laub, and J. C. Willems. The Riccati Equation. Communication and Control Engineering Series. Springer-Verlag, Berlin, Heidelberg, New York, 1991. 20. J. L. Bogdanoff and S. J. Citron. Experiments with an inverted pendulum subject to random parametric excitation. J. Acoustic Soc., 38(9):447–452, 1965. 21. S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities in Systems and Control Theory, volume 15 of Studies in Applied Mathematics. SIAM, 1994. 22. R. W. Brockett. Finite Dimensional Linear Systems. John Wiley & Sons, New York, 1970. 23. L. Burlando. Monotonicity of spectral radius for positive operators on ordered Banach spaces. Arch. Math., 56:49–57, 1991. 24. D. Carlson and R. Hill. Generalized controllability and inertia theory. Linear Algebra Appl., 15:177–187, 1976. 25. B. M. Chen. Robust and H∞ control. Communications and Control Engineering Series. Springer-Verlag, 2000. 26. S. Chen, X. Li, and X. Y. Zhou. Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim., 36(5):1685–1702, 1998. 27. S. J. Cho, S.-H. Kye, and S. G. Lee. Generalized Choi maps in threedimensional matrix algebras. Linear Algebra Appl., 171:212–224, 1992. 28. M.-D. Choi. Positive linear maps on C ∗ -algebras. Canadian J. Math., 4:520– 529, 1972. 29. M.-D. Choi. Completely positive linear maps on complex matrices. Linear Algebra Appl., 10:285–290, 1975. 30. M.-D. Choi. Positive semidefinite biquadratic forms. Linear Algebra Appl., 4:95–100, 1975. 31. A. N. Churilov. On the solution of a quadratic matrix equation. In E. L. Tonkov, editor, Nonlinear oscillations and control theory. No.2., pages 24–33. Udmurt State University, Izhevsk, 1978. 32. D. J. Clements and H. K. Wimmer. Monotonicity of the optimal cost in the discrete-time regulator problem and Schur complements. Automatica, 37:1779– 1786, 2001. 33. E. G. Collins, Jr. and A. S. Hodel. Efficient solutions of linearly coupled Lyapunov equations. SIAM J. Matrix. Anal. Appl., 18(2):291–304, 1997. 34. W. A. Coppel. Matrix quadratic equations. Bull. Austr. Math. Soc., 10:377– 401, 1974. 35. R. F. Curtain, editor. Stability of Stochastic Dynamical Systems, volume 294 of Lecture Notes in Mathematics. Springer-Verlag, 1972. 36. R. F. Curtain and A. J. Pritchard. Functional analysis in modern applied mathematics, volume 132 of Mathematics in Science and Engineering. Academic Press, London, New York, 1977. 37. G. Da Prato. Direct solution of a Riccati equation arising in stochastic control theory. Applied Mathematics and Optimization, 11:191–208, 1984. 38. G. Da Prato and J. Zabczyk. Stochastic Equations in Infinite Dimensions, volume 45 of Encyclopedia of Mathematics and Its Applications. Cambridge University Press, 1992.

References

187

39. T. Damm. Generalized Riccati equations occuring in robust stochastic control. In V. Zakharov, editor, Proceedings of the 11th IFAC Workshop, Saint Petersburg, Russia, 3-6 July 2000. Pergamon Press, 2000. 40. T. Damm. State-feedback H ∞ -control of stochastic linear systems. In Third NICONET Workshop on Numerical Software in Control Engineering, pages 45–50, Louvain-la-Neuve, Belgium, 2001. 41. T. Damm. Minimal representations of inverted Sylvester and Lyapunov operators. Linear Algebra Appl., 363:35–41, 2002. 42. T. Damm. State-feedback H ∞ -type control of linear systems with time-varying parameter uncertainty. Linear Algebra Appl., 351–352:185–210, 2002. 43. T. Damm. A car-steering problem with random adhesion coefficient. PAMM Proc. Appl. Math. Mech., 2:83–84, 2003. 44. T. Damm. Stability of linear systems and positive semigroups of symmetric matrices. In L. Benvenuti, A. D. Santis, and L. Farina, editors, Positive Systems, volume 294 of Lecture Notes in Control and Information Sciences, pages 207–214. Springer-Verlag, 2003. 45. T. Damm and D. Hinrichsen. Matrix (in)equalities for linear stochastic systems. In Proceedings of MTNS-98, Padova, Italy, 1998. Il Poligrafio. 46. T. Damm and D. Hinrichsen. On a rational matrix equation occuring in stochastic control. In Proceedings of the 5th European Control Conference 1999, Karlsruhe, pages DM–2, Karlsruhe, 1999. 47. T. Damm and D. Hinrichsen. On the parameter dependence of a class of rational matrix equations occuring in stochastic optimal control. In Proceedings of MTNS-2000, page IS19A, Perpignan, France, 2000. 48. T. Damm and D. Hinrichsen. Newton’s method for a rational matrix equation occuring in stochastic control. Linear Algebra Appl., 332–334:81–109, 2001. 49. T. Damm and D. Hinrichsen. Newton’s method for concave operators with resolvent positive derivatives in ordered Banach spaces. Linear Algebra Appl., 363:43–64, 2003. 50. B. N. Datta. Stability and inertia. Linear Algebra Appl., 302–303:563–600, 1999. 51. J. de Pillis. Linear transformations which preserve Hermitian and positive semidefinite operators. Pacific J. Math., 23(1):129–137, 1967. 52. C. E. de Souza and M. D. Fragoso. On the existence of maximal solutions for generalized algebraic Riccati equations arising in stochastic control. Systems & Control Letters, 14:233–239, 1990. 53. E. de Souza and S. Bhattacharyya. Controllability, observability and the solution of AX − XB = C. Linear Algebra Appl., 39:167–188, 1981. 54. J. W. Demmel. Applied Numerical Linear Algebra. SIAM, Philadelphia, 1997. 55. J. Dieudonné. Foundations of Modern Analysis, volume 10-I of Pure and Applied Mathematics. Academic Press, New York, 1969. 56. D. Z. Djoković. Characterization of Hermitian and skew-Hermitian maps between matrix algebras. Linear Algebra Appl., 12:165–170, 1975. 57. V. Dr˘ agan, A. Halanay, and A. Stoica. An LMI solution to a disturbance attenuation problem with state feedback for stochastic systems. Rev. Roum. Sci. Tech. Elektrotech. Energ., 41(4):513–519, 1996. 58. V. Dr˘ agan, A. Halanay, and A. Stoica. A small gain theorem for linear stochastic systems. Systems & Control Letters, 30:243–251, 1997.

188

References

59. V. Dr˘ agan, T. Morozan, and A. Halanay. Optimal stabilizing compensator for linear systems with state dependent noise. Stochastic Analysis and Application, 10:557–572, 1992. 60. F. Dullerud, G. E.; Paganini. A course in robust control theory, volume 36 of Texts in Applied Mathematics. Springer-Verlag, New York, 2000. 61. A. El Bouhtouri, D. Hinrichsen, and A. J. Pritchard. H ∞ type control for discrete-time stochastic systems. Int. J. Robust & Nonlinear Control, 9(13):923–948, 1999. 62. A. El Bouhtouri, D. Hinrichsen, and A. J. Pritchard. On the disturbance attenuation problem for a wide class of time invariant linear stochastic systems. Stochastics and Stochastics Reports, 65:255–297, 1999. 63. A. El Bouhtouri and A. J. Pritchard. A Riccati equation approach to maximizing the stability radius of a linear system by state feedback under structured stochastic Lipschitzian perturbations. Systems & Control Letters, 21:475–484, 1993. 64. L. El Ghaoui. State-feedback control of systems with multiplicative noise via linear matrix inequalities. Systems & Control Letters, 24(3):223–228, 1995. 65. L. Elsner. Monotonie und Randspektrum bei vollstetigen Operatoren. Arch. Ration. Mech. Anal., 36:356–365, 1970. 66. L. Elsner. Quasimonotonie und Ungleichungen in halbgeordneten R¨ aumen. Linear Algebra Appl., 8:249–261, 1974. 67. M. Fiedler and V. Pt` ak. On matrices with nonpositive off-diagonal elements and positive principal minors. Czech. Math., 12:382–400, 1962. 68. A. Fischer, D. Hinrichsen, and N. K. Son. Stability radii of Metzler operators. Vietnam J. Math, 26:147–163, 1998. 69. M. D. Fragoso, O. L. V. Costa, and C. E. de Souza. A new approach to linearly perturbed Riccati equations in stochastic control. Applied Mathematics and Optimization, 37:99–126, 1998. 70. B. A. Francis. A course in H∞ control theory. Number 88 in Lecture Notes in Control and Information Sciences. Springer-Verlag, Berlin-Heidelberg-New York, 1987. 71. G. Freiling and V. Ionescu. Monotonicity and convexity properties of matrix Riccati equations. IMA J. Math. Control Inf., 18(1):61–72, 2001. 72. G. Freiling, G. Jank, and H. Abou-Kandil. Generalized Riccati difference and differential equations. Linear Algebra Appl., 241-243:291–303, 1996. 73. A. Friedman. Stochastic Differential Equations and Applications. Number 28 in Probability and mathematical statistics. Academic Press, New York, 1975. ¨ 74. G. Frobenius. Uber Matrizen aus positiven Elementen. Sitzungsberichte der K¨ onigl. Preuss. Akad. Wiss, pages 471–476, 1908. 75. P. Gahinet and P. Apkarian. A linear matrix inequality approach to H∞ control. Int. J. Robust & Nonlinear Control, 4:421–448, 1994. 76. P. Gahinet and A. J. Laub. Numerically reliable computation of optimal performance in singular H∞ control. SIAM J. Control Optim., 35(5):1690–1710, 1997. 77. Z. Gajić and M. T. J. Qureshi. Lyapunov Matrix Equation in System Stability and Control, volume 195 of Mathematics in Science and Engineering. Academic Press, San Diego, California, 1995. 78. F. R. Gantmacher. Matrizentheorie. Springer, 1986. 79. Z. Y. Gao and N. U. Ahmed. Stabilizability of certain stochastic systems. Int. J. Systems Sci., 17(8):1175–1185, 1986.

References

189

80. T. C. Gard. Introduction to Stochastic Differential Equations, volume 114 of Monographs and Textbooks in Pure and Applied Mathematics. Marcel Dekker, 1988. 81. I. I. Gikhman. On the theory of differential equations of stochastic processes. I, II. Am. Math. Soc., Transl., II. Ser. 1:111–137, 139–161, 1955. 82. I. I. Gikhman and A. V. Skorokhod. Stochastic Differential Equations. Springer-Verlag, 1972. 83. S. K. Godunov. Modern Aspects of Linear Algebra, volume 175 of Translations of Mathematical Monographs. American Mathematical Society, Providence, Rhode Island, 1998. 84. I. Gohberg, P. Lancaster, and L. Rodman. On the Hermitian solutions of the symmetric algebraic Riccati equation. SIAM J. Control Optim., 24:1323–1334, 1986. 85. G. H. Golub, S. Nash, and C. F. Van Loan. A Hessenberg-Schur method for the problem AX + XB = C. IEEE Trans. Autom. Control, AC-24:909–913, 1979. 86. G. H. Golub and C. F. van Loan. Matrix Computations, volume 3 of Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press, Baltimore, Maryland, 1990. 87. M. Green and D. J. N. Limebeer. Linear Robust Control. Prentice-Hall, Englewood Cliffs, New Jersey, 1995. 88. A. Greenbaum. Iterative Methods for Solving Linear Systems, volume 17 of Frontiers in Applied Mathematics. SIAM, Philadelphia, 1997. 89. R. Griesbaum. Zur Stabilit¨ at dynamischer Systeme mit stochastischer Parametererregung. PhD thesis, Fakult¨ at f¨ ur Maschinenbau, Universit¨ at Karlsruhe, 1999. 90. P. Gritzmann, V. Klee, and B.-S. Tam. Cross-positive matrices revisited. Linear Algebra Appl., 223/224:285–305, 1995. 91. C.-H. Guo. Newton’s method for discrete algebraic Riccati equations when the closed-loop matrix has eigenvalues on the unit circle. SIAM J. Matrix. Anal. Appl., 20(2):279–294, 1998. 92. C.-H. Guo. Iterative solution of a matrix Riccati equation arising in stochastic control. Preprint, Department of Mathematics and Statistics, University of Regina, 2000. 93. C.-H. Guo. Nonsymmetric algebraic Riccati equations and Wiener-Hopf factorization of M -matrices. SIAM J. Matrix. Anal. Appl., 23(1):225–242, 2001. 94. C.-H. Guo and P. Lancaster. Analysis and modification of Newton’s method for algebraic Riccati equations. Math. Comp., 67(223):1089–1105, 1998. 95. S. J. Hammarling. Numerical solution of the stable, non-negative definite Lyapunov equation. IMA J. Numer. Anal., 2:303–323, 1982. 96. B. Hassibi, A. H. Sayed, and T. Kailath. Indefinite-quadratic estimation and control: A unified approach to H 2 and H ∞ theories, volume 15 of Studies in Applied Mathematics. SIAM, 1999. 97. U. G. Haussmann. Optimal stationary control with state and control dependent noise. SIAM J. Control, 9(2):713–739, 1971. 98. U. G. Haussmann. Stabilization of linear systems with multiplicative noise. In R. F. Curtain, editor, Stability of Stochastic Dynamical Systems, volume 294 of Lecture Notes in Mathematics, pages 125–130. Springer-Verlag, July 1972. 99. M. L. J. Hautus. Controllability and observability conditions of linear autonomous systems. Indag. Math., 31:443–448, 1969.

190

References

100. M. L. J. Hautus. Operator substitution. Linear Algebra Appl., 205-206:713– 739, 1994. 101. U. Helmke and J. B. Moore. L2 sensitivity minimization of linear system representations via gradient flows. J. Math. Syst. Estim. Control, 5(1):79–98, 1995. 102. G. A. Hewer. An iterative technique for the computation of the steady state gains for the discrete optimal regulator. IEEE Trans. Autom. Control, AC16:382–384, 1971. 103. G. A. Hewer. Existence theorems for positive semidefinite and sign indefinite stabilizing solutions of H∞ Riccati equations. SIAM J. Control Optim., 31:16– 29, 1993. 104. R. D. Hill. Linear transformations which preserve Hermitian matrices. Linear Algebra Appl., 6:257–262, 1973. 105. R. D. Hill and S. R. Waters. On the cone of positive definite matrices. Linear Algebra Appl., 90:81–88, 1987. 106. D. Hinrichsen and A. J. Pritchard. Stability radii with stochastic uncertainty and their optimization by output feedback. SIAM J. Control Optim., 34:1972– 1998, 1996. 107. D. Hinrichsen and A. J. Pritchard. Stochastic H∞ . SIAM J. Control Optim., 36(5):1504–1538, 1998. 108. D. Hinrichsen and A. J. Pritchard. Mathematical Systems Theory, chapter 1: Mathematical Models. Manuscript, Bremen, 2001. 109. D. Hinrichsen and N. K. Son. µ-analysis and robust stability of positive linear systems. Appl. Math. Comput., 8(2):253–268, 1998. 110. M. Hochbruck and G. Starke. Preconditioned Krylov subspace methods for Lyapunov matrix equations. SIAM J. Matrix. Anal. Appl., 16(1):156 – 171, 1994. see also: IPS Research Report 92–17, ETH Z¨ urich, Switzerland (1992). 111. A. Hochhaus. Existenz-, Konvergenz- und Vergleichss¨ atze f¨ ur verallgemeinerte Riccatische Matrix-Gleichungen. PhD thesis, Fakult¨ at f¨ ur Naturwissenschaften, Universit¨ at Duisburg, 2002. 112. R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, Cambridge, Massachusetts, 1985. 113. R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, Cambridge, 1991. 114. M. S. Howe. The mean square stability of an inverted pendulum subject to random parametric excitation. J. Sound Vibration, 32(3):407–421, 1974. 115. N. Ikeda and S. Watanabe. Stochastic Differential Equations and Diffusions Processes. North Holland, 1981. 116. V. Ionescu, C. Oara, and M. Weiss. Generalized Riccati Theory and Robust Control: A Popov Function Approach. John Wiley & Sons, Chichester, 1999. 117. K. Ito. On a stochastic integral equation. Proc. Japan Acad., 22(1–4):32–35, 1946. 118. K. Ito. On stochastic differential equations. Mem. Am. Math. Soc., 4, 1951. 119. R. E. Kalman. Contributions to the theory of optimal control. Bol. Soc. Matem. Mexico, 5:102–119, 1960. 120. R. E. Kalman. Mathematical description of linear dynamical systems. SIAM J. Control, 1:152–192, 1963. 121. R. E. Kalman, P. L. Falb, and M. A. Arbib. Topics in mathematical system theory. McGraw-Hill, New York, 1969.

References

191

122. L. V. Kantorovich. Functional analysis and applied mathematics. Usp. Mat. Nauk, 3(6):89–185, 1948. 123. L. V. Kantorovich and G. P. Akilov. Functional Analysis in Normed Spaces. Pergamon Press, New York, 1964. Translated by D. E. Brown, edited by A. P. Robertson. 124. J. Kao and V. Wihstutz. Stabilization of companion form systems by mean square noise. Stochastics Stochastics Rep., 49:1–25, 1994. 125. I. Karatzas and S. E. Shreve. Brownian Motion and Stochastic Calculus. 2nd ed., volume 113 of Graduate Texts in Mathematics. Springer-Verlag, New York, 1991. 126. I. Y. Kats and N. N. Krasovskij. On the stability of systems with random parameters. J. Appl. Math. Mech., 24:1225–1246, 1961. 127. V. L. Kharitonov. Asymptotic stability of an equilibrium position of a family of systems of linear differential equations. J. Diff. Eqns., 11:2086–2088, 1978. 128. V. L. Kharitonov. Distribution of the roots of the characteristic polynomial of an autonomous system. Autom. Remote Control, 42(5):589–593, 1981. 129. V. L. Kharitonov. Robust stability analysis of time delay systems: A survey. In Commande et Structure des Systèmes, pages 1–12, Nantes, 1998. Conference IFAC. 130. R. Z. Khasminskij. Stochastic Stability of Differential Equations. Sijthoff & Noordhoff, Alphen aan den Rijn, NL, 1980. 131. D. L. Kleinman. On an iterative technique for Riccati equation computations. IEEE Trans. Autom. Control, AC-13:114–115, 1968. 132. D. L. Kleinman. On the stability of linear stochastic systems. IEEE Trans. Autom. Control, AC-14:429–430, 1969. 133. D. L. Kleinman. Numerical solution of the state dependent noise problem. IEEE Trans. Autom. Control, AC-21:419–420, 1976. 134. P. E. Kloeden and E. Platen. Numerical Solution of Stochastic Differential Equations. Springer-Verlag, Berlin, 1995. 135. H.-W. Knobloch and H. Kwakernaak. Lineare Kontrolltheorie. Springer-Verlag, Berlin, 1985. ¨ die analytischen Methoden in der Wahrscheinlichkeit136. A. N. Kolmogorov. Uber srechnung. Mathematische Annalen, 104:415–458, 1931. 137. M. A. Krasnosel’skii, G. M. Vainikko, P. P. Zabreiko, Y. B. Rutitskii, and V. Y. Stetsenko. Approximate Solution of Operator Equations. Wolters-Noordhoff, Groningen, 1972. 138. M. A. Krasnosel’skij. Positive Solutions of Operator Equations. P. Noordhoff Ltd., Groningen, The Netherlands, 1964. 139. M. A. Krasnosel’skij, J. A. Lifshits, and A. V. Sobolev. Positive Linear Systems - The Method of Positive Operators, volume 5 of Sigma Series in Applied Mathematics. Heldermann Verlag, Berlin, 1989. 140. M. G. Krein and M. A. Rutman. Linear operators leaving invariant a cone in a Banach space. Amer. Math. Soc. Transl., 26:199–325, 1950. 141. N. V. Krylov. Controlled Diffusion Processes, volume 14 of Applications of Mathematics. Springer-Verlag, New York, 1980. 142. N. V. Krylov. Introduction to the Theory of Diffusion Processes, volume 142 of Translations of Mathematical Monographs. American Mathematical Society, 1995. 143. H. J. Kushner. Stochastic stability and control. Academic Press, New York, 1967.

192

References

144. H. Kwakernaak and R. Sivan. Linear Optimal Control Systems. John Wiley & Sons, New York, 1972. 145. P. Lancaster and L. Rodman. Algebraic Riccati Equations. Oxford, 1995. 146. C.-K. Li and H. J. Woerdeman. Special classes of positive and completely positive maps. Linear Algebra Appl., 255:247–258, 1997. 147. G. Lindblad. On the generators of quantum dynamical semigroups. Commun. Math. Phys., 1:219–224, 1976. 148. D. G. Luenberger. Optimization by Vector Space Methods. John Wiley & Sons, New York, 1969. 149. I. Marek. Frobenius theory of positive operators: Comparison theorems and applications. SIAM J. Appl. Math., 19:607–628, 1970. 150. P. J. McLane. Optimal stochastic control of linear systems with state- and control-dependent disturbances. IEEE Trans. Autom. Control, AC-16(6):793– 798, 1971. 151. V. Mehrmann. The Autonomous Linear Quadratic Control Problem, Theory and Numerical Solution. Number 163 in Lecture Notes in Control and Information Sciences. Springer-Verlag, Heidelberg, July 1991. 152. B. P. Molinari. The time invariant linear quadratic optimal control problem. Automatica, 13:347–357, 1977. 153. T. Morozan. Stabilization of some stochastic discrete-time control systems. Stochastic Analysis and Application, 1:89–116, 1983. 154. T. Morozan. Parametrized Riccati equations associated to input-output operators for time-varying stochastic differential equations with state-dependent noise. Technical Report 37, Institute of Mathematics of the Romanian Academy, 1995. 155. R. Nabben. A characterization of M-matrices. Preprint 95-002, SFB 343, Universit¨ at Bielefeld, 1995. 156. B. Oeksendal. Stochastic Differential Equations. Springer-Verlag, 5th edition, 1998. 157. A. Ostrowski and H. Schneider. Some theorems on the inertia of general matrices. J. Math. Anal. Appl., 4:72–84, 1962. ¨ ¨berwiegender Hauptdiagonale. die Determinanten mit u 158. A. M. Ostrowski. Uber Comm. Math. Helv., 10:69–96, 1937. 159. D. Ottl and C. R¨ ommich. Schwingungen von Autor¨ adern infolge mechanisch unrunder Reifen. Automobiltechnische Zeitschrift, 98(6):351–357, 1996. 160. D. V. Oulette. Schur complements and statistics. Linear Algebra Appl., 36:187– 295, 1981. 161. T. Penzl. Numerical solution of generalized Lyapunov equations. Adv. Comput. Math., 8(1–2):33–48, 1998. ¨ 162. O. Perron. Zur Theorie der Ubermatrizen. Math. Ann., 64:248–263, 1907. 163. I. R. Petersen, V. A. Ugrinovskii, and A. V. Savkin. Robust Control Design Using H ∞ -Methods. Springer-Verlag, London, 2000. 164. Y. A. Phillis. Optimal stabilization of stochastic systems. J. Math. Anal. Appl., 94:489–500, 1983. 165. H. Radjavi and P. Rosenthal. Simultaneous triangularization. Springer-Verlag, New York, 2000. 166. A. C. M. Ran and R. Vreugdenhil. Existence and comparison theorems for algebraic Riccati equations for continuous- and discrete-time systems. Linear Algebra Appl., 99:63–83, 1988.

References

193

167. W. T. Reid. Riccati Differential Equations. Academic Press, New York, 1972. 168. M. Reurings. Symmetric Matrix Equations. PhD thesis, Vrije Universiteit Amsterdam, 2003. 169. S. Richter, L. D. Davis, and E. G. Collins, Jr. Efficient computation of the solutions to modified Lyapunov equations. SIAM J. Matrix. Anal. Appl., 14(2):420–431, April 1993. 170. H. H. Rosenbrock. State Space and Multivariable Theory. Nelson-Wiley, 1970. 171. P. Sagirow. Stochastic Methods in the Dynamics of Satellites. Number 57 in Courses and lectures: International Centre for Mechanical Sciences. SpringerVerlag, Wien, New York, 1970. 172. A. A. Sagle and R. E. Walde. Introduction to Lie Groups and Lie Algebras, volume 51 of Pure and Applied Mathematics. Academic Press, New York– London, 1973. 173. J. C. Samuels and A. C. Eringen. On stochastic linear systems. J. Math. Phys., 38:83–103, 1959. 174. N. R. Sandell. On Newton’s method for Riccati equation solution. IEEE Trans. Autom. Control, AC-19:254–255, 1974. 175. T. Sasagawa. LP -stabilization problem for linear stochastic control systems with multiplicative noise. J. Optim. Th. & Appl., 61(3):451–471, 1989. 176. T. Sasagawa and J. L. Willems. Parametrization method for calculating exact stability bounds of stochastic linear systems with multiplicative noise. Automatica, 32(12):1741–1747, 1996. 177. H. H. Schaefer. Topological Vector Spaces. Springer-Verlag, Berlin, Heidelberg, New York, 1971. 178. C. W. Scherer. The Riccati Inequality and State-Space H∞ -Optimal Control. PhD thesis, Mathematical Institute, University of W¨ urzburg, 1990. 179. C. W. Scherer. The state feedback H∞ -problem at optimality. Automatica, 30:293–305, 1994. 180. H. Schneider. Positive operators and an inertia theorem. Numer. Math., 7:11– 17, 1965. 181. H. Schneider and M. Vidyasagar. Cross-positive matrices. SIAM J. Numer. Anal., 7(4):508–519, 1970. 182. M. A. Shayman. Geometry of the algebraic Riccati equation, Part I. SIAM J. Control Optim., 21:375–394, 1983. 183. M. A. Shayman. Geometry of the algebraic Riccati equation, Part II. SIAM J. Control Optim., 21:395–409, 1983. 184. E. D. Sontag. Mathematical Control Theory, Deterministic Finite Dimensional Systems. Springer-Verlag, New York, 2nd edition, 1998. 185. R. J. Stern and H. Wolkowitz. Exponential nonnegativity on the ice-cream cone. SIAM J. Matrix. Anal. Appl., 12:755–778, 1994. 186. G. W. Stewart and J. Sun. Matrix Perturbation Theory. Computer Science and Scientific Computing. Academic Press, Boston, 1990. 187. W. F. Stinespring. Positive functions on C ∗ -algebras. Proc. Amer. Math. Soc., 6:211–216, 1955. 188. R. L. Stratonoviˇc. A new form of representing stochastic integrals and equations. Vestnik Moskov. Univ., Ser. I 19(1):3–12, 1964. 189. W.-S. Tang. On positive linear maps between matrix algebras. Linear Algebra Appl., 79:45–51, 1986.

194

References

190. G. Tessitore. Some remarks on the Riccati equation arising in an optimal control problem with state- and control- dependent noise. SIAM J. Control Optim., 30(3):717–744, May 1992. 191. G. Tessitore. On the mean-square stabilizability of a linear stochastic differential equation. In J.-P. Zolesio, editor, Boundary Control and Variation, volume 163 of Lect. Notes Pure Appl. Math., pages 383–400, New York, June 1994. 5th Working Conference held in Sophia Antipolis, France, Marcel Dekker. 192. H. L. Trentelman, A. A. Stoorvogel, and M. L. J. Hautus. Control Theory for Linear Systems. Springer-Verlag, London, 2001. 193. M. Turelli. Random environments and stochastic calculus. Theor. Popul. Biol., 12:140–178, 1977. 194. V. A. Ugrinovskii. Robust H ∞ -control in the presence of stochastic uncertainty. Internat. J. Control, 71(2):219–237, 1998. 195. V. A. Ugrinovskii and I. R. Petersen. Absolute stabilization and minimax optimal control of uncertain systems with stochastic uncertainty. SIAM J. Control Optim., 37(4):1089–1122, 1999. 196. J. S. Vandergraft. Newton’s method for convex operators in partially ordered spaces. SIAM J. Numer. Anal., 4(3):406–432, 1967. 197. J. S. Vandergraft. Spectral properties of matrices which have invariant cones. SIAM J. Appl. Math., 16(6):1208–1222, 1968. 198. R. S. Varga. Matrix Iterative Analysis. Prentice Hall, Englewood Cliffs, New Jersey, 1962. 199. R. S. Varga. On recurring theorems on diagonal dominance. Linear Algebra Appl., 13:1–9, 1976. 200. W. J. Vetter. Vector structures and solutions of linear matrix equations. Linear Algebra Appl., 10:181–188, 1975. 201. J. C. Willems. Least squares optimal control and the algebraic Riccati equation. IEEE Trans. Autom. Control, AC-16:621–634, 1971. 202. J. L. Willems. Moment stability of linear white noise and coloured noise systems. In B. e. a. Clarkson, editor, Stoch. Probl. in Dyn., Symp. Univ. Southampton 1976, pages 67–89, 1977. 203. J. L. Willems and D. Aeyels. Moment stability of linear stochastic systems with solvable Lie algebras. IEEE Trans. Autom. Control, AC-21:285, 1976. 204. J. L. Willems and J. C. Willems. Feedback stabilizability for stochastic systems with state and control depending noise. Automatica, 12:277–283, 1976. 205. J. L. Willems and J. C. Willems. Robust stabilization of uncertain systems. SIAM J. Control Optim., 21:352–374, 1983. 206. H. K. Wimmer. Inertia theorems for matrices, controllability and linear vibration. Linear Algebra Appl., 8:337–343, 1974. 207. H. K. Wimmer. On the algebraic Riccati equation. Bull. Austral. Math. Soc., 14:457–461, 1976. 208. H. K. Wimmer. Monotonicity of maximal solutions of algebraic Riccati equations. Systems & Control Letters, 5:317–319, 1985. 209. H. K. Wimmer. Linear matrix equations, controllability and observability, and the rank of solutions. SIAM J. Matrix. Anal. Appl., 9:570–578, 1988. 210. H. K. Wimmer. Geometry of the discrete-time algebraic Riccati equation. J. Math. Syst. Estim. Control, 2(1):123–132, 1992. 211. W. M. Wonham. Optimal stationary control of a linear system with statedependent noise. SIAM J. Control, 5(3):486–500, 1967.

References

195

212. W. M. Wonham. On a matrix Riccati equation of stochastic control. SIAM J. Control, 6:681–697, 1968. 213. W. M. Wonham. Random differential equations in control theory. In A. T. Bharucha-Reid, editor, Probab. Methods Appl. Math., volume 2, pages 131–212, New York - London, 1970. Academic Press. 214. W. M. Wonham. Linear Multivariable Control: A Geometric Approach. Springer-Verlag, Heidelberg, 2nd edition, 1979. 215. S. L. Woronowicz. Positive linear maps of low dimensional matrix algebras. Rep. Math. Phys., 10:165–183, 1976. 216. V. A. Yakubovich. A frequency theorem in control theory. Siberian Mathematical Journal, 14:265–289, 1973. 217. W.-Y. Yan, J. B. Moore, and U. Helmke. Recursive algorithms for solving a class of nonlinear matrix equations with applications to certain sensitivity optimization problems. SIAM J. Control Optim., 32:1559–1576, 1994. 218. J. Yong and X. Y. Zhou. Stochastic Controls, volume 43 of Applications of Mathematics. Springer, New York, 1999. 219. G. Zames. Feedback and optimal sensitivity: Model reference transformations, multiplicative seminorms, and approximate inverses. IEEE Trans. Autom. Control, AC-26:301–320, 1981. 220. K. Zhou, J. C. Doyle, and K. Glover. Robust and Optimal Control. Prentice Hall, New Jersey, 1995. 221. K. Zhou and P. P. Khargonekar. An algebraic Riccati equation approach to H ∞ optimization. Systems & Control Letters, 11(2):85–91, 1988.

Index

Bounded Real Lemma, 50 bounded uncertainty, 19, 56

LQ-control, 43, 146, 159 Lyapunov function, 21, 27 operator, 13, 23, 63, 75, 85

completely positive operator, 73, 89, 94, 117 concavity, 104 constraint(definite/indefinite), 56, 124 controllability, 24, 151 convex cone, 61

M-matrix, 63, 119

detectability, 29, 65, 94 anti-, 30 disturbance attenuation, 33, 35, 38, 50, 158, 171, 176 duality transformation, 132

observability, 24 order interval, 61 ordering, 61

expectation, 1 extremal, 62

Newton’s method, 106, 162 double steps, 113, 169 modified, 110

perturbation operator, 48 positive operator, 13, 62

Kronecker product, 69

regular splitting, 67 regular system, 53, 133 regularization, 177 resolvent positive operator, 62 Riccati equation, 55, 145, 155 dual, 131, 170 non-symmetric, 118 standard, 120, 130, 164 inequality, 27, 53 coupled pair, 59 dual, 171 operator, 48, 134 dual, 138, 155, 167, 175

Loewner ordering, 181

Schur-complement, 54, 181

Frobenius inner product, 62, 181 Gˆ ateaux differentiability, 104 H∞ -control, 48, 51, 56 Hermitian-preserving, 71 inertia, 181 Itˆ o formula, 6 integral, 3 process, 4

198

Index

spectral abscissa, 63 radius, 63 stability, 10 mean-square, 12, 13 second moment, 12 stochastic, 11 stabilizability, 22 stabilizing matrix, 136, 164 solution, 107, 131, 145, 162 stable equation, 11 equilibrium, 10 internally/externally, 47

matrix, 63 operator, 66, 106 Stein operator, 63, 76, 85 stochastic differential equation, 5 process, 1 Stratonovich integral, 4 interpretation, 4, 7, 10, 17, 18, 21, 27 Sylvester operator, 98 white noise, 2 Wiener process, 2 Z-matrix, 63

Notation Sets, matrices and linear operators N natural numbers {0, 1, 2, . . .} R, C field of real, complex numbers K R or C Re z real part of a complex number z Im z imaginary part of a complex number z i imaginary unit or index, depending on context vector space of matrices with m rows and n columns Km×n equal to Km×1 Km ¯ A complex conjugate of a matrix (or a number) A AT transpose of a matrix A over K conjugate transpose of a matrix A over K A∗ Hn ordered Hilbert space of n × n Hermitian matrices over K n H+ cone of positive semidefinite matrices in Hn n n H− equal to −H+ n B, Bi basis of H (see Section 3.4) >, ≥,

Rational Matrix Equations in Stochastic Control (Lecture Notes in Control and Information Sciences)

Singular Control Systems (Lecture Notes in Control and Information Sciences)

Networked Control Systems (Lecture Notes in Control and Information Sciences)

Ensuring Control Accuracy (Lecture Notes in Control and Information Sciences)

Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis (Lecture Notes in Control and Information Sciences)

L2-Gain and Passivity Techniques in Nonlinear Control (Lecture Notes in Control and Information Sciences)

Recent Advances in Control and Optimization of Manufacturing Systems (Lecture Notes in Control and Information Sciences)

Topics in Time Delay Systems: Analysis, Algorithms and Control (Lecture Notes in Control and Information Sciences)

Recent Advances in Learning and Control (Lecture Notes in Control and Information Sciences)

Robustness in Identification and Control (Lecture Notes in Control and Information Sciences)

New Directions and Applications in Control Theory (Lecture Notes in Control and Information Sciences)

Advances in Communication Control Networks (Lecture Notes in Control and Information Sciences, Volume 308)

Limited Data Rate in Control Systems with Networks (Lecture Notes in Control and Information Sciences)

Nonlinear Control in the Year 2000: Volume 1 (Lecture Notes in Control and Information Sciences)

Control Theory in the Plane (Lecture Notes in Control and Information Sciences)

Finite Zeros in Discrete Time Control Systems (Lecture Notes in Control and Information Sciences)

Perspectives in Robust Control (Lecture Notes in Control and Information Sciences)

Multidisciplinary Research in Control: The Mohammed Dahleh Symposium 2002 (Lecture Notes in Control and Information Sciences)

Advances in the Control of Nonlinear Systems (Lecture Notes in Control and Information Sciences)

Nonlinear Control in the Year 2000, Volume 2 (Lecture Notes in Control and Information Sciences)

Positive Polynomials in Control (Lecture Notes in Control and Information Sciences)

Control Theory in the Plane (Lecture Notes in Control and Information Sciences)

Stochastic Theory and Control: Proceedings of a Workshop held in Lawrence, Kansas (Lecture Notes in Control and Information Sciences)

Iterative Learning Control: Convergence, Robustness and Applications (Lecture Notes in Control and Information Sciences)

Control and Estimation of Systems with Input Output Delays (Lecture Notes in Control and Information Sciences)

Feedback Control, Nonlinear Systems, and Complexity (Lecture Notes in Control and Information Sciences)

Dynamics, Bifurcations and Control (Lecture Notes in Control and Information Sciences)

Assessment and Future Directions of Nonlinear Model Predictive Control (Lecture Notes in Control and Information Sciences)

Adaptive Dual Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Chaos Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

The Confluence of Vision and Control (Lecture Notes in Control and Information Sciences)

Rational Matrix Equations in Stochastic Control (Lecture Notes in Control and Information Sciences)

Singular Control Systems (Lecture Notes in Control and Information Sciences)

Networked Control Systems (Lecture Notes in Control and Information Sciences)

Ensuring Control Accuracy (Lecture Notes in Control and Information Sciences)

Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis (Lecture Notes in Control and Information Sciences)

L2-Gain and Passivity Techniques in Nonlinear Control (Lecture Notes in Control and Information Sciences)

Recent Advances in Control and Optimization of Manufacturing Systems (Lecture Notes in Control and Information Sciences)

Topics in Time Delay Systems: Analysis, Algorithms and Control (Lecture Notes in Control and Information Sciences)

Recent Advances in Learning and Control (Lecture Notes in Control and Information Sciences)

Robustness in Identification and Control (Lecture Notes in Control and Information Sciences)

New Directions and Applications in Control Theory (Lecture Notes in Control and Information Sciences)

Advances in Communication Control Networks (Lecture Notes in Control and Information Sciences, Volume 308)

Limited Data Rate in Control Systems with Networks (Lecture Notes in Control and Information Sciences)

Nonlinear Control in the Year 2000: Volume 1 (Lecture Notes in Control and Information Sciences)

Control Theory in the Plane (Lecture Notes in Control and Information Sciences)

Finite Zeros in Discrete Time Control Systems (Lecture Notes in Control and Information Sciences)

Perspectives in Robust Control (Lecture Notes in Control and Information Sciences)

Multidisciplinary Research in Control: The Mohammed Dahleh Symposium 2002 (Lecture Notes in Control and Information Sciences)

Advances in the Control of Nonlinear Systems (Lecture Notes in Control and Information Sciences)

Nonlinear Control in the Year 2000, Volume 2 (Lecture Notes in Control and Information Sciences)

Positive Polynomials in Control (Lecture Notes in Control and Information Sciences)

Control Theory in the Plane (Lecture Notes in Control and Information Sciences)

Stochastic Theory and Control: Proceedings of a Workshop held in Lawrence, Kansas (Lecture Notes in Control and Information Sciences)

Iterative Learning Control: Convergence, Robustness and Applications (Lecture Notes in Control and Information Sciences)

Control and Estimation of Systems with Input Output Delays (Lecture Notes in Control and Information Sciences)

Feedback Control, Nonlinear Systems, and Complexity (Lecture Notes in Control and Information Sciences)

Dynamics, Bifurcations and Control (Lecture Notes in Control and Information Sciences)

Assessment and Future Directions of Nonlinear Model Predictive Control (Lecture Notes in Control and Information Sciences)

Adaptive Dual Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Chaos Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

The Confluence of Vision and Control (Lecture Notes in Control and Information Sciences)

Recommend Documents