Foundations of adaptive control

Lecture Notes in Control and Information Sciences Edited by M.Thoma and A.Wyner

160 P.V. Kokotovic (Ed.)

Foundations of Adaptive Control

Springer-Verlag Berlin Heidelberg NewYork London ParisTokyo Hong Kong Barcelona Budapest

Series Editors M. Thoma • A. Wyner Advisory Board L. D. Davisson • A. G. J. MacFarlane • H. Kwakernaak ,1. L. Massey • Ya Z. Tsypkin • A. J, Viterbi Editor Prof. Petar V. Kokotovib ECE University of California Santa Barbara CA 93106 USA

ISBN 3-540-54020-2 Springer-Verlag Berlin Heidelberg NewYork ISBN 0-387-54020-2 Springer-Verlag NewYork Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in other ways, and storage in data banks. Duplication of this publication or parts thereof is only permitted under the provisions of the German Copyright Law of September 9, 1965, in its current version, and a copyright fee must always be paid. Violations fall under the prosecution act of the German Copyright Law. © Spdnger-Verlag Berlin, Heidelberg 1991 Printed in Germany The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera ready by author Printing: Mercedes-Druck, Berlin Binding: B. Helm, Berlin 61/3020-543210 Printed on acid-free paper.

Acknowledgements While many people contributed to the organization of the 1990 Grainger Lectures and the publication of this volume, these tasks would not have been accomplished without the tireless work of Peter Sauer and Ioannis Kanellakopoulos. P e t e r $auer, Grainger Associate, was in charge of administration, finances, publicity and social programs for the Lectures. Ioannis K a n e l l a k o p o u l o s , Grainger Fellow, performed all the editorial work for the volume, including proofreading, correcting and formatting the texts. Financial support for the Lectures was provided by the Grainger Foundation. Personal attention of D a v i d Grainger, President, was extremely helpful.

Petar V. Kokotovid Grainger Professor

Preface

In the fourth decade of its development, adaptive control has reached a level at which a critical reexamination of its foundations is useful for both theoretical and practical purposes. A series of sixteen lectures, The 1990 Grainger Lectures, was dedicated to this task. The lectures were delivered at the University of Illinois, Urbana-Champaign, September 28-October 1, 1990. In the creative atmosphere of intense discussions among the lecturers, the scope and content of the lectures were expanded by the addition of new topics and most recent developments. The result of this activity are the revised and enlarged texts in this volume. As it becomes clear even after a cursory review of its contents, the volume has gone far beyond its original scope of reexamining the foundations of the field. New solutions are presented for some long-standing open problems, earlier successful approaches are unified, new problems are formulated and some of them solved. Indeed, the title of the volume could have been changed into "Recent Advances in Adaptive Control". However, as all of the new results address fundamental issues, the title F o u n d a t i o n s o f A d a p t i v e C o n t r o l is still appropriate. The two parts of the volume deal with adaptive control of linear and nonlinear systems, respectively. Part I contains unifications, reappraisals and new results in adaptive linear control. Most nonlinear problem formulations in Part II are new, and some of their solutions are likely to be further extended and generalized. Part I opens with a view of the history of the field by one of the pioneers of Model Reference Adaptive Control, Bob Narendra. He starts with a wise caveat that even for a professional historian it is hard or impossible to be objective, let alone for a deeply involved participant of many exciting events. In T h e M a t u r i n g o f A d a p t i v e C o n t r o l , the reader will find the wealth of information and freshness of views that only an eyewitness report can convey. Adaptive control has always been characterized by a great diversity of concepts and algorithms and a continuous search for their unification. As such, it has been a fertile ground for both scientists and inventors who at times speak different languages. W h a t both groups have been lacking is a common set of fundamental concepts in which to communicate the properties of their inventions and theories. A C o n c e p t u a l F r a m e w o r k f o r P a r a m e t e r A d a p t i v e C o n t r o l by Steve Morse is a major step in this direction. In the near future, his concept of tunability may become as common as the concept of controllability. Just as the proofs of stability and convergence dominated the 1970s, the issues of robustness were central throughout the 1980s. As the next three papers in this volume show, the robustness issues will continue to be among the important

VI research topics in the 1990s. Petros Ioannou, one of the leading authors on this subject, and Aniruddha Datta have given us a self-contained detailed presentation of the state-of-the-art in R o b u s t A d a p t i v e C o n t r o l : D e s i g n , A n a l y s i s a n d R o b u s t n e s s B o u n d s . What in the 1980s appeared as a collection of unrelated modifications of adaptive algorithms is now a unified methodology for preventing destabilizing effects of disturbances and unmodeled dynamics. The robustness proofs are at the same time simpler and more powerful. The next two papers present new robustness results for continuous and discrete time systems. The main message of both R o b u s t C o n t l n u o u s - T i m e A d a p t i v e C o n t r o l b y P a r a m e t e r P r o j e c t i o n by Naik, Kumar, and Ydstie, and S t a b i l i t y o f t h e D i r e c t S e l f - T u n i n g R e g u l a t o r by Ydstie, is that to achieve boundedness of all the signals in the presense of disturbances and unmodeled dynamics it is sufficient to introduce a parameter projection modification into the usual gradient update law. Although the continuous-time result by Naik, Kumar and Ydstie was inspired by the discrete-time result of Ydstie, their order is reversed in order to preserve the connection with the preceding continuous-time papers. The contribution of the Soviet Academician Tsypkiu, a pioneer of both control theory and stochastic adaptive control is at the same time historicM and innovative. In A d a p t i v e - I n v a r i a n t D i s c r e t e C o n t r o l S y s t e m s , he first reminds us that what today we call disturbance-rejeciion conditions appeared in the 1940s as selec~ive-invariance conditions. This type of invariance is achievable for disturbances generated from known models and with complete knowledge of the plant. Tsypkin then introduces the notion of adaptive-invariant systems, which have an anMogous invariance property when the parameters of the plant and disturbance generator are unknown. He restricts his presentation to the socalled interval plants, whose parameters are known to belong to given intervals. One of the leading authors in the field of stochastic adaptive control, P. R. Kumar, was invited to summarize the field and reexamine its foundations. In S t o c h a s t i c A d a p t i v e S y s t e m T h e o r y : R e c e n t A d v a n c e s a n d a R e a p praisal, he and Wei Ren have not only accomplished this task, but have also completed the solution of several long-standing open problems. They first provide a unified treatment of identification, prediction and control based on the ARMAX model and prove self-tuning and convergence properties for several adaptive schemes. They then give new general results for parallel model adaptation problems, including output-error identification and adaptive noise and echo cancelling. It is expected that these recent results will give a new impetus to both simpler and more fundamental developments in stochastic adaptive control. Part II of this volume consists of five papers dealing with adaptive control of nonlinear systems. All the authors responded to the invitation to address truly new nonlinear problems without any linear constraints imposed on the nonlinearities. (Since the unknown parameters appear linearly, imposing linear growth constraints or global Lipschitz conditions would make the problems tractable by the methods of adaptive linear control.) The first three papers assume that the full state vector is available for feedback. More realistic situations, where only an output is available for measurement, are considered in the last two papers.

VII In A d a p t i v e F e e d b a c k L i n e a r i z a t l o n of N o n l i n e a r Systems, Kokotovid, Kanellakopoulos, and Morse first survey some recent results and then design a new adaptive scheme, applicable to a much larger class of systems than before. The difficulties of the global adaptive stabilization and tracking problems are shown to increase due to two factors: level of uncertainty and nonlinear complexity. The new systematic design procedure is applicable to the highest level of uncertainty, but it limits the nonlinear complexity by the so-called pure-feedback condition. Whenever this assumption is globally satisfied, the results on adaptive stabilization and tracking are global, that is, there is no loss of globality caused by adaptation. A conceptually broader approach to adaptive control of nonlinear systems is presented in A d a p t i v e S t a b i l i z a t i o n o f N o n l i n e a r S y s t e m s by Praly, Bastin, Pomet, and :Hang. They unify and generalize most earlier results and classify them according to additional assumptions such as matching conditions (i.e., uncertainty levels) and linear growth constraints. The unification is achieved by a novel Lyapunov approach to the design of direct schemes and by generalizations of equation error filtering and regressor filtering for indirect schemes. The key assumption in this approach is that a Lyapunov-like function exists and depends on the unknown parameters in a particular way. Depending on the properties of this function, various designs are possible, including feedback linearization designs when this function is quadratic in the transformed coordinates. Possibilities of either global or only regional adaptive stabilization are also examined. Processes involving transfer of electromagnetic into mechanical energy require essentially nonlinear models. One of the most practical representatives of these processes is considered in N o n l i n e a r A d a p t i v e C o n t r o l o f I n d u c t i o n M o t o r s via E x t e n d e d M a t c h i n g by Marino, Peresada, and VMigi. Their adaptive design encompasses, as special cases, some earlier designs and patents. However, the assumption of full state measurement is a practical disadvantage which motivates the development of adaptive schemes using only output measurements. Adaptive output-feedback control of nonlinear systems is a new problem of major theoretical and practical interest. In Global A d a p t i v e O b s e r v e r s a n d O u t p u t - F e e d b a c k S t a b i l i z a t i o n for a Class o f N o n l i n e a r S y s t e m s by Marino and Tomei, this problem is addressed in two stages. First, sufficient conditions are given for the construction of global adaptive observers for singleoutput systems. The construction makes use of novel filtered transformations, that is, nonlinear changes of coordinates driven by auxiliary filters. At the second stage, an observer-based adaptive output-feedback control is designed for a somewhat narrower class of systems. A different approach to adaptive output-feedback control of nonlinear systems is presented in A d a p t i v e O u t p u t - F e e d b a c k C o n t r o l of S y s t e m s w i t h O u t p u t N o n l i n e a r i t i e s by Kanellakopoulos, Kokotovic and Morse. The main result of this paper is a nonlinear extension of a 1978 paper by Feuer and Morse. Because of its complexity, this early paper is less well known than the papers mentioned in the historical survey at the beginning of this volume. However, in

VIII contrast to some other results of adaptive linear control, its nonlinear generalization does not impose linear constraints on system nonlinearities. This brief preview suffices to show that the papers in this volume herald a decade of new breakthroughs in adaptive control research in the decade of the 1990s. With the already achieved unification of robust adaptive control, more research will undoubtedly be focused on performance. A particularly challenging task will be to develop methods to improve transients and eliminate bursting phenomena. In stochastic adaptive control, the most recent proofs of stability and convergence make it possible to address the robustness issue in a new way. In nonlinear systems, adaptive control has just made its first encouraging steps. T h e y are likely to stimulate research leading to results applicable to larger classes of systems. Rigorous analytical methods of adaptive control may also become a theoretical basis for some developments in neural networks. The intellectual depth and scientific content of adaptive control are growing at an impressive pace. Its mature vitality is as exciting as was its youthful impetuousness.

Petar V. Kokotovi6 Urbana, Illinois February 1991

Contents Part

I: A d a p t i v e

Linear

Control

1

T h e M a t u r i n g of A d a p t i v e C o n t r o l by K. S. Narendra . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

A C o n c e p t u a l F r a m e w o r k for P a r a m e t e r A d a p t i v e C o n t r o l by A. S. Morse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

37

R o b u s t A d a p t i v e C o n t r o l : Deslgn~ Analysis a n d R o b u s t n e s s B o u n d s by P. Ioarmou and A. Datta . . . . . . . . . . . . . . . . . . . . . . . . 71 Robust Continuous-Time Adaptive Control by Parameter Projection by S. M. Naik, P. 1~. Kumar, and B. E. Ydstie

.............

153

S t a b i l i t y o f t h e Direct Self-Tuning R e g u l a t o r by B. E. Ydstie . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

201

Adaptive-Invariant Discrete Control Systems by Ya. Z. Tsypkin . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

239

Stochastic Adaptive System Theory: Recent Advances and a Reappraisal by W. Ren and P. R. Kumar . . . . . . . . . . . . . . . . . . . . . . .

269

Part

II: Adaptive

Nonlinear

Control

309

A d a p t i v e F e e d b a c k L i n e a r i z a t i o n of N o n l i n e a r S y s t e m s by P. V. Kokotovi5, I. Kanellakopoulos, and A. S. Morse . . . . . . . .

311

Adaptive Stabilization of Nonlinear Systems by L. Praly, G. Bastin, J.-B. Pomet, and Z. P. Jiang . . . . . . . . . .

347

A d a p t i v e N o n l i n e a r C o n t r o l of I n d u c t l o n M o t o r s via E x t e n d e d M a t c h i n g by R. Marino, S. Peresada, and P. Valigi . . . . . . . . . . . . . . . . .

435

Global A d a p t i v e Observers a n d O u t p u t - F e e d b a c k S t a b i l i z a t i o n for a Class of N o n l i n e a r S y s t e m s by P~. Marino and P. Tomei . . . . . . . . . . . . . . . . . . . . . . . . A d a p t i v e O u t p u t - F e e d b a c k C o n t r o l of S y s t e m s with Output Nonlinearities by I. Kanellakopoulos, P. V. Kokotovi5, and A. S. Morse . . . . . . . .

455

495

Part I Adaptive Linear Control

The Maturing of Adaptive Control K. S. Narendra Department of Electrical Engineering Center for Systems Science Yale University New Haven, CT 06520, USA.

Prologue History is neither purely subjective nor purely objective. According to Edward Hallett Carr [1], the historian is engaged in a continuous process of molding his facts to his interpretation and his interpretation to his facts. The following paper is a brief history of adaptive control theory during the period 1955-1990 as interpreted by the author, who was also a participant.

1 Historical

Background

Control theory is concerned with modifying the behavior of dynamical systems so as to achieve desired goals. These goals include maintaining relevant outputs of a system around desired constant values, assuring that the outputs follow specified trajectories, or more generally ensuring that the overall system optimizes a specified performance criterion. The goal is achieved by computing a suitable control input based on the observed outputs of the system. The fundamental processes involved in controlling a dynamical system consequently include the mathematical modeling of the system', identification of the system based on experimental data, processing the outputs of the system into mathematically convenient forms, using them in turn to synthesize control inputs, and applying the latter to the system to achieve the desired behavior. In the early stages of its development, control theory was primarily concerned with linear time-invariant systems with known parameters. Around the 1940s, electrical, meehanicM and chemical engineers were designing automatic control devices in their own fields, using very similar methods but disguised under different terminologies - a situation which is prevalent even today in many of the newer areas of systems theory. GraduMly it became apparent that the concepts had a common basis and towards the end of World War II a frequency-domain theory of control that was mathematically elegant and universal in its scope came into existence. During the following decade, even as this theory was being used successfully for the systematic design of innumerable industrial feedback control

4

Narendra

systems, it became clear that a new methodology would be needed when substantial amounts of uncertainty are present in such systems. It was at this stage in the history of control theory, in the 1950s, when frequency-domain methods were well established and time-domain methods based on state representation of dynamical systems had not yet been introduced, that the field of adaptive control was born. In the early 1950s there was considerable interest in the design of autopilots for high-performance aircraft. Such aircraft operated over a wide range of speeds and altitudes (according to the standards prevailing at that time). The existing feedback theory was adequate to design an efficient controller at any one operating condition, but could not cope with problems that arose when there were rapid changes in operating conditions. One obvious solution was to store the controller parameter values for the different operating regions and switch between them as the conditions changed. But this process, referred to as gain scheduling, became infeasible when the number of possible operating regimes became very large, and it was generally acknowledged that a more sophisticated approach would be needed to cope with the problem. It also became apparent in the following years that the difficulties encountered in aircraft systems were generic in nature and that very similar problems were encountered over the entire spectrum of systems applications. As in aircraft systems, parameters also tend to vary in most practical systems - ships, cement and paper mills, distillation columns, engines, chemical processes and power systems, to name but a few. The frequency-domain methods available at this time [2,3], even when used efficiently, were inadequate to achieve satisfactory performance in the entire range over which the characteristics of the system varied. The demands of a rapidly growing technology for faster and more accurate controllers have always had a strong influence on the progress of automatic control theory and when it became apparent that the existing theory was inadequate to successfully tackle the problems arising in industry, interest gradually shifted to adaptive systems which would adjust their performance with changes in the environments. The term "adaptation" is defined in the dictionary as the modification of an organism or its parts that fits it better for the conditions of its environment. Inspired by this, in 1957, Drenick and Shahbender [4] introduced the term "adaptive system" into control theory to represent control systems that monitor their own performance and adjust their control mechanisms in the direction of improved performance. The following years witnessed considerable debate on the differences between conventional linear control systems and adaptive systems. Further, numerous feedback systems were designed which their proponents considered to possess properties peculiar to adaptive systems. Many of these were collected and presented in survey papers by Aseltine et al. [5] and Stromer [6] indicating that research on adaptive systems was flourishing even in the 1950s. An a t t e m p t was made in [5] to categorize adaptive systems into the following five classes depending upon the manner in which adaptive behavior was achieved: (i) Passive Adaptation, (ii) Input-Signal Adaptation, (iii) System Variable Adaptation, (iv) System Characteristic Adaptation, and (v) E x t r e m u m Adaptation. The first class contains those systems in which the clever design of

The Maturing of Adaptive Control

5

time-invariant controllers results in satisfactory performance over wide variations in the environment. High-gain controllers designed to cope with plant parameter variations belong to this class which, according to the present terminology, would be described as robust rather than adaptive. Class (ii) includes systems which adjust their parameters in accordance with input signal characteristics while classes (iii) and (iv) contain systems in which the control input is based on system outputs, errors or their derivatives or alternately the observed impulse response or frequency response. It is worth remembering that system representation at this time was mainly in terms of transfer functions, and distinctions were made only between inputs, outputs and parameters. Even when the parameters were adjusted on-line, the overall system was analyzed as a linear time-varying system. Adaptive systems belonging to class (v) received more attention than the others during the 1960s and attempted to seek the extremum of a performance criterion (cf. Section 2). Even though the classification of adaptive systems given above is no longer in vogue, the concepts that led to such a classification have endured for over three decades and have been rediscovered by successive generations of systems theorists in their attempts to deal with uncertainty. In the early 1960s, many sessions in the major conferences were devoted to defining adaptive control and numerous definitions were proposed [7,8,9,10]. Yet, thirty years later a universal definition of an adaptive system is still elusive. This is because adaptation is multifaceted in nature and cannot be compressed into a simple statement without vital loss of content. Itence, the many definitions proposed merely reflect the variety of personal visions encountered in the field. The seeming divergence between the different viewpoints may be partly attributed to the fact that, in the ultimate analysis, an adaptive system is merely a complex feedback system. A designer who sets out to control a system in the presence of a certain class of uncertainty and produces a controller which achieves its objective may conclude that he has designed an adaptive system. Yet, an observer unfamiliar with the process prior to the design of the controller merely observes a nonlinear control system. Hence, among the various definitions proposed, the one due to Truxal [10], that an adaptive system is one which is designed from an adaptive viewpoint, captures much of the difficulty encountered in defining adaptive systems. The fact that most adaptive systems are nonlinear systems came to be realized in the 1960s, when attempts were made to describe them using state equations. For example, if a plant is described by the vector differential equation = A(p,

+ B(p,

y = C(p,

(1)

with r(t) E IR, x(t) E IR n, y(t) e IR"~, p e IRm~ and0(t) E IR 'n~, where r,x, y are respectively the input, state and output, p is an unknown constant vector, and 9(0 a time-varying control parameter vector which is adjusted using the measured signals of the system as

= g(y, e, O ,

(2)

6

Narendra

it follows that the components 6i(t) of 6(t) (i = 1 , 2 , . . . , m 2 ) are no longer "parameters", but are state variables of the overall system described by the equations (1) and (2). Hence, any adaptive system, no matter how complex, is merely a nonlinear feedback system involving estimation and control, and the design of adaptive systems consequently corresponds to the design of special classes of nonlinear systems [11]. During the past three decades great advances have been made in adaptive control theory and adaptive controllers have been used in numerous industrial problems. While we are still very far from designing autonomous systems which will take care of themselves, a considerable amount of insight has been gained regarding the nature of the new problems and the concepts and theoretical questions associated with them. During the early years some of the research efforts led to dead ends and new starts had to be made. There were also periods when the field appeared to be in the doldrums, and these were invariably followed by breakthroughs which led to a resurgence of interest in the field. The main objective of this paper is to trace some of these major developments, particularly those related to continuous-time adaptive systems, and to provide the reader with a sense of the excitement and frustration that have alternately marked the growth of the field.

2 Developments

in t h e e a r l y 1960s

The early days of adaptive control coincided with one of the most active periods in the history of control theory. The state representation of dynamical systems was introduced at this time and the relationship between frequency- and timedomain methods was a subject of frequent discussion. The maximum principle of Pontryagin [12] which had been introduced in the 1950s and which had opened the way for a systematic study of optimal trajectories in the presence of state and control constraints was gradually being absorbed into the mainstream of control theory. The contributions of Lurie, Popov, Kalman, Yakubovich and others stimulated a great deal of interest in the absolute stability problem and more generally in Lyapunov's methods for establishing the stability or instability of nonlinear systems. This was also the period when advances in stochastic estimation were accelerated by the introduction of the Kalman-Bucy filter, when the linear optimal regulator problem with a quadratic performance index was resolved and when Bellman's dynamic programming method was being increasingly used for sequential planning and optimal resource allocation. All these had a profound impact on researchers in the area of adaptive control and on their understanding of adaptive control processes. Even at this early stage, following the work of Feldbaum [13], it was generally realized that two philosophically different approaches exist for the control of plants with unknown parameters. In the first approach [Fig. la] referred to as indirect control (or explicit identification) the unknown plant parameters are estimated on-line and the controller parameters are adjusted based on these estimates. In the second approach, referred to as direct control [Fig. lb] (or

The Maturing of Adaptive Control

f

7

1P

ec

I

1

/

(a)

•

I

e~

r

I

ik

Ident.~

e¢

i (b)

i

Fig. 1. (a) Direct adaptive control, (b) Indirect adaptive control.

implicit identification) the plant parameters are not estimated but the control parameters are directly adjusted to improve a performance index. A paper by Kalman in 1958 [14] combining recursive estimation with controller design was to a large extent responsible for indirect control being used almost exclusively in the late 1950s. This is reflected in the first book on adaptive control by Mishkin and Braun [15] published in the United States in 1961. Soon after this, however, direct methods also became popular for adaptive control. 2.1 M R A S a n d S T R As mentioned in Section 1, the aim of control is to keep the relevant outputs of a given dynamical plant within prescribed limits. If yp denotes the output of the plant and Ym the desired output, the objective is to keep the error el = yp - Ym between desired values. If Ym is a constant, the problem is one of regulation, and when Ym is a function of time the problem is referred to as tracking. When the

8

Narendra

characteristics of the plant are unknown, both regulation and tracking can be viewed as adaptive control problems. In practice, adaptive control problems can arise in a variety of ways. For instance, in a dynamical plant, the input and output may be the only accessible variables and a parameter ~ in the plant may have to be adjusted to minimize some functional of the error el -- yp - Ym. Alternately, a feedforward a n d / o r feedback controller may be used to control the unknown plant and the parameters of the controller may have to be adjusted based on the information obtained as the system is in operation. For mathematical tractability, the plant is generally assumed to be linear and time-invariant with unknown parameters. In such a case whether indirect or direct control is used depends on whether or not the parameters of the plant are estimated explicitly. Much of the research that has been carried out in adaptive control is concerned with two classes of systems called Model Reference Adaptive Systems (MRAS) and Self Tuning Regulators (STR). While the former evolved from deterministic servo problems, the latter arose in the context of stochastic regulation problems. In spite of their different origins they are closely related as is evident from the discussion that follows. If an indirect approach is to be used, the unknown plant parameters are estimated, using a model of the plant, before a control input is chosen. If the problem is one of tracking, the desired output Ym can be expressed as the output of a reference model (ref. sections 3 and 5). Adaptive systems that make explicit use of such models for identification or control purposes are called MRAS [16]. The Self Tuning Regulator, which is based on the indirect approach, consists of a parameter estimator, a linear controller and a mechanism for computing the controller parameters from those of the plant parameter estimates. Since any estimation scheme can be combined with a suitable control scheme, various possibilities exist for designing such a controller. It is clear that STRs can also be designed based on a direct method in which the regulator parameters are directly updated. In spite of the seeming similarity of MRAS and STR, historically the former has been used mainly in the analysis of continuous-time systems, while the latter has found application in discrete-time systems. Since our interest in this paper is primarily in continuous-time adaptive systems, the emphasis in the following sections will be on MRAS rather than STR. For the sake of historical continuity, we first deal in this section with parameter perturbation methods as well as methods based on sensitivity models which represented the two principal approaches to adaptive control in the 1960s. All subsequent sections deal with problems which arise in MRAS. 2.2 P a r a m e t e r

Perturbation

Method

E x t r e m u m adaptation, mentioned in Section 1 was perhaps the most popular m e t h o d for adaptive control in the 1960s. It had considerable appeal to researchers because of its simplicity and applicability to nonlinear plants and since it did not require explicit identification of plant parameters. In 1951, Draper and Li [17] had demonstrated that the performance of an internal combustion engine

The Maturing of Adaptive Control

9

could be optimized using a parameter perturbation method involving perturbation, correlation and adjustment. Assuming that a single parameter value has to be chosen, the parameter is perturbed around a nominal value, the parameter variations are correlated with changes in the performance index and used in turn to adjust the parameter in the direction of optimum performance. Numerous papers [18,19,20,21] appeared on this subject and detailed analyses were carried out regarding the amplitude and frequency of the perturbations, the time over which the correlation is to be carried out and the amount by which the parameter is to be adjusted. The analysis in [21], for example, revealed that sophisticated perturbation methods involving differential equations with multiple time-scales would be needed for a precise analysis of the behavior of even a second-order system with a single adjustable parameter. The problem became significantly more complex when several parameters had to be adjusted simultaneously. To determine the gradient of a performance index in parameter space, the frequencies of perturbation of the different parameters had to be separated and this in turn resulted in slow convergence. The inability to use the method successfully in practical situations where multiple parameters had to be adjusted, along with the difficulty encountered in determining conditions for their stability, resulted in the rapid demise of parameter perturbation methods in the mid 1960s. 2.3 S e n s i t i v i t y M o d e l s The parameter perturbation method is a direct method for estimating the gradient of a performance index. An alternate gradient approach, based on significantly more prior information concerning the plant, used sensitivity models for the generation of the desired partial derivatives. Given a system with an inputoutput pair {u, y} and a parameter 8, the partial derivative ~ of the output y with respect to/? can be computed if the structure of the system is known. For example, if a system is described by the second-order differential equation ~)+ 0y + y = u, the partial derivative "~e = z satisfies the differential equation + 0k + z = - 9 -

(3)

By constructing a model described by equation (3) with - ~ as its input the desired partial derivative ~ can be generated. Similarly, using a model corresponding to each adjustable parameter, the gradient of y with respect to ~ can be determined and in turn used to adjust the parameter vector to improve the performance [22,23,24]. The method using sensitivity models gained great popularity in the 1960s and was extensively used in industrial problems for the optimal adjustment of parameters. Since substantially more information is assumed in this method as compared to the parameter perturbation method, the convergence of the parameters was significantly faster than in the latter case. In many adaptive situations the structure of plant may not be known (i.e., a differential equation of the form (3) may not be available). In such cases an identification model of the plant has to be constructed and used in the sensitivity model. The output

10

Narendra

of the sensitivity model then corresponds to an estimated rather than a true gradient of the system. 2.4 M . I . T . R u l e The first MRAS was proposed in 1958 by Whitaker et al. [25] and contained a reference model whose output represented the desired output of the plant. The parameters ~ of the controller were adjusted based on the error e between the output of the reference model and that of the plant. The adaptive law for the adjustment of 0 was based on the fact that if the parameters ~ changed much slower than the system variables, then a reasonable approach was to change 0 along the negative gradient of e2: da 0e ~ - / = -Te~-~.

(4)

However, since the plant parameters were unknown, the components of the sensitivity vector 0¢ could not be obtained using the sensitivity models described earlier. Instead, they were replaced by some available signals according to a "rule" which came to be known as the M.I.T. rule.

3 Adaptive

Control

of Simple

Systems

(late 1960s)

3.1 S t a b l e A d a p t i v e S y s t e m s In the gradient methods described thus far for the study of adaptive systems, the emphasis was on their performance. The control parameters were adjusted along the gradient of a performance index. Once the design of the controller was completed, the stability of the overall system was analyzed and conditions were established for local stability. Since adaptive systems are nonlinear nonautonomous systems, the derivation of sufficient conditions for even local stability was not possible in many cases. Since stability is a fundamental requirement for the satisfactory operation of any system, it was suggested in 1963 by Grayson [26] that a reversal of the procedure adopted earlier would be more efficient. He argued that adaptive systerns should be designed to be globally stable for all values of a parameter 7 belonging to a set S 7. The optimization of the system could then be carried out by choosing the optimum value ")'opt E S- t. The above idea can be mathematically s t a t e d a s follows. Let the differential equation governing an adaptive system be

p, 0, t]

(5)

y = h[z,p,O,t], where x(t) and y(t) are the state and output of the system respectively, p is a constant but unknown parameter vector, and O(t) is a control parameter vector.

The Maturing of Adaptive Control

11

Let ym(t) be a desired trajectory. The choice of designer. Let an adaptive law of the form

O(t) is

at the discretion of the

= g[y, Ym, 7, 5]

(6)

exist, such that the nonlinear system described by (5)-(6) is globally stable for all values of the parameter ~/ in some set S~. Then, as stated earlier, the performance of the system can be improved according to some criterion by the choice of 7. The above procedure is very similar to that adopted in the design of optimal linear time-invariant control systems, where the control parameters are chosen subject to the constraint that the poles of the overall system lie in the open left-half of the complex plane. The suggestion that adaptive control systems should be designed from a stability viewpoint was enthusiastically received by the adaptive control community, particularly since by this time the difficulties associated with establishing the stability of gradient-based schemes was well recognized. Numerous contributions were made to the design of stable adaptive systems, the most notable of which are due to Shackcloth and Sutchart [27], Parks [28] Monopoli [29], Philipson [30], and Winsor and Roy [31]. The strict positive real condition, which subsequently became an important design tool, made its first adaptive appearance in [27,28]. It should be borne in mind that in the 1960s many of the subtle questions of stability were not yet well understood. Hence, while the contributions of the early authors were undoubtedly seminal in nature, some of them contained technical flaws which were eventually eliminated in more rigorous analyses [28,32,33]. The 1966 paper by Parks [28], which is of great historical significance in adaptive control, clearly demonstrated using a specific example that gradient methods of adjusting the parameters in an adaptive system can result in instability. At the same time it was also shown that the system could be made globally stable by using a design procedure based on Lyapunov's method. This clear demonstration that gradient-based adaptive systems could be unstable, eventually tolled the death knell of such systems and witnessed a gradual shift on the part of researchers to the design of adaptive systems based on stability methods. 3.2 L y a p u n o v Design o f Simple A d a p t i v e S y s t e m s The concepts used in the design of stable adaptive systems are best illustrated by considering plants which are described by first-order differential equations with unknown parameters. Following this, it can be readily shown that the same concepts carry over to higher-order systems when the state vectors of the plants are accessible. More general cases, where plants of order greater than one have to be controlled using only input and output information, are treated later in Section 5. For a detailed treatment of these topics the reader is referred to [11]. Let a plant with input-output pair {u(.), xp(-)} be described by the scalar differential equation ~p = ap~p(t) + kpU(~), (7)

12

Narendra

where ap and kp are constants and Zp, Up : I~ + --* ]1%. Let the control input u(t) be of the form u(t) = OzAt ) + kr(t) , (8) where 0 and k are adjustable parameters. It is assumed that the plant parameters ap and kp are unknown, but that the sign of kp is known. The aim of adaptive control is to adjust the control parameters O(t) and k(t) so that the output Xp(t) follows asymptotically the output Zm(1) of a reference model described by

~m(t)

=

amZrn(t) -F kmr(t) ,

am < O,

(9)

where r(t) is a known bounded piecewise-continuous function of time. Defining

e(t) ~= Z p ( t ) - ~rn(t), ¢(t) ~ O ( t ) - 0", and ¢(t) ~ k ( t ) - k*, where O* = (am - ap)/kp and k* = km/kp, the differential equation describing e(t) can be expressed as

~(t) = ame(t) + kpCxp(t) + kp¢(t)r(t).

(10)

The realization that the analysis is best carried out by considering the error equations, in which only the parametric errors ¢ and ¢ and the output error e(t) feature, was slow in coming. The importance of the error equation is that the adaptive control problem can be stated as the stability problem of the equilibrium state of a set of error differential equations. For instance, if 0(t) and k(t) are to be adjusted adaptively, the problem can be posed as follows: Determine the adaptive laws

= 0 = fl(e, zp,r)

(11)

so that the equilibrium state of the third-order system described by (10)-(11) is uniformly stable. Choosing as a Lyapunov candidate V ( e , ¢ , ¢ ) = 71 [e2 + ikpl(¢2 + ¢2)] and evaluating the time derivative of V along the solutions of equations (10) and (11) leads to

v(e, ¢, ¢)

=

2 + kpC

p +

-- Ikpl[¢¢ +

(12)

Since ¢ and ¢ are parameter errors which can never be known, it follows that and ¢ should be chosen to cancel the terms in ¢ and ¢ if V is to be non-positive for all e, ¢ and ¢. This is accomplished by choosing =

-sgn(kp)

p

(13)

= -sgn(kp)er, where sgn(kp) = 1 if kp > 0 and sgn(kp) = - 1 if kp < 0. Since V ( e , ¢ , ¢ ) is indeed a Lyapunov function of the system, it follows that the system is stable. Since V = ame 2 < 0, e E £2. Furthermore, since by equation (10) ~ is bounded, we have limt.-.co e(t) = O.

The Maturing of Adaptive Control

13

The proof given above for a first-order system forms the essence of the proofs given for more general cases. For example, let an nth-order plant be described by the vector differential equation

kp = Apzp + bmu,

(14)

whereAp E I~nxn, bm E ]Rn,xp :II~+--~]R n, andu:]P~+ ~ I R . Let bm b e a known vector and Ap an unknown matrix. Let a reference model be described by the differential equation

~m

=

Amxm + bmr,

(15)

where Am is an asymptotically stable matrix and r a specified scalar input which is bounded and piecewise-continuous. The objective, as in the scalar case, is to determine the input to the plant so that limt...~ e(t) = 0 where e(t) = • p(t)

-

To assure the existence of a solution, we assume that a vector k* exists so that if u = k*Tzp(t) + r, both xp(t) and zm(t) satisfy identical differential equations. Since Am is asymptotically stable it follows that a symmetric positive definite matrix P exists such that

ATp+PAm=-Q,

Q=QT>0.

(16)

Using V(e,¢) = eWpe + ¢T¢ as a candidate for a Lyapunov function, (where k(t) - k* = ¢), it can be shown, using the same approach as in the scalar case, that the adaptive law

k(t) "- - e T (i)Pbmxp

(17)

yields V(e,¢) = - e T Q e < O, thereby assuring global stability. The procedure outlined above was also used for the identification of dynamical systems [11] and later for both indirect control as well as in combined direct-indirect methods, for the adaptive control of systems whose state variables are accessible. 4 Adaptive

Observers

(early 1970s)

In linear systems theory it is well known that a controllable linear time-invariant system can be stabilized by state feedback. This formed the theoretical basis for the adaptive control methods discussed in the previous section. In more realistic cases the entire state vector cannot be measured and adaptation has to be carried out using only input-output measurements. For linear systems with known parameters, the use of a Luenberger observer to estimate the state variables of the system from input-output measurements had been established by this time. This in turn motivated researchers in the adaptive control field to attempt to estimate the state variables of linear systems from similar data even when the system parameters were unknown. Since the estimation of the state variables using an observer requires knowledge of the system parameters, and estimation of the latter requires knowledge of the state variables, it soon became evident that the problem was substantially more complex in the adaptive

14

Narendra

context. An adaptive observer was defined as one which simultaneously estimates the parameters and state variables of a linear time-invariant system from inputoutput measurements. Around the late 1960s, the field of adaptive control was not particularly popular in the United States. Most of the work on continuous time adaptive observers in the country was being carried out at two universities - the University of Connecticut under the direction of Lindorff and Yale University under the direction of Narendra. There was considerable interaction between these two groups and the University of Massachusetts, where Monopoli headed the control activity. During this period it became evident that the structure of the adaptive observer, as well as the adaptive laws, would have to be chosen judiciously if the adaptive observer was to be stable. At the same time, no one was confident that even the existence of such a stable adaptive observer could be established. In the author's opinion, the period 1972-74 constituted one of the important stages in the development of continuous-time adaptive control. The mid 1970s witnessed a renaissance in adaptive control. In Europe, Tsypkin [34] made fundamental contributions to the field and showed that many of the methods proposed for learning and adaptive control could be described in a common framework as special types of recursive equations. Identification methods were extensively investigated and different estimation methods were made the basis of design procedures. /~strSm and Wittenmark introduced the self-tuning idea in a seminal paper [35], and studied the asymptotic properties of the self-tuning regulator. On the practical side, minicomputers were used to implement adaptive concepts. In the United States major strides were made in the development of adaptive observers and a comprehensive survey on the subject by Narendra and Kudva [32,33] appeared in 1974. In view of the impact of adaptive observers on later developments in adaptive systems, a brief description of the nature of the problems as well as the solutions proposed are included in this section. The first solution to the problem was given by Carroll and Lindorff [36]. Using the Kalman-Yakubovich Lemma, a similar adaptive observer was later proposed by Kudva and Narendra [37] (Fig. 2). Various canonical representations of unknown plants were suggested by Luders and Narendra [38,39], which eventually led to a substantially simplified non-minimal representation of adaptive observers. 4.1 M i n i m a l R e a l i z a t i o n o f A d a p t i v e O b s e r v e r s Any single-input single-output linear time-invariant plant which is observable can be parametrized as y

= Xpl

where a T = [ a l , a 2 , . . . , an] and b T = [bl, b2 . . . . , bn] are unknown, and -4 is a known matrix [11]. The objective of the adaptive observer is to estimate the vectors a and b as well as the state vector xp(t) from the input-output data. A

The Maturing of Adaptive Control

15

more convenient parametrization for the design of the adaptive observer has the form zp = [-al-4]~p + gyp + bu (18) yp = hTXp -- Xl , where ~ is a known constant vector which makes [ - a l A ] = K a stable matrix, and g and b are unknown vectors in lRn which have to be estimated. From the earlier work on simple adaptive systems (Section 3) it was clear that 2n-independent signals would be needed to estimate the 2n unknown parameters. However, since only two signals u and y are accessible, the required signals had to be generated as part of the adaptive process. The observer was chosen to have the form Xp = K~:p + 9(t)yp + [~(t)u + vl(t) + v2(t), (19) where vl(t) and v2 (t) are vectors in IR'~ which are at the discretion of the designer and 5:r,(t), 9(t) and b(t) represent the estimates of the state vector and unknown parameter vectors respectively. If e(t) = 2p(t) - zp(t) and el(t) = 9p(t) - yp(t), the error equation (subtracting (18) from (19)) has the form

= K e + Cyp + Cu + vl + v2.

(20)

In [37] the vectors vl(t) and v2(t) are generated as the state variables of two identical controllable linear time-invariant systems with u(t) and y(t) as inputs respectively. The adaptive laws for adjusting ~(t) and b(t) are then derived using the signals vl(t) and v2(t) and it is demonstrated that a Lyapunov function exists which assures the stability of the system. The structure of the adaptive observer is shown in Fig. 2.

Iobservr Fig. 2. The minimal adaptive observer.

×1

16

Narendra

4.2 N o n m i n i m a l R e a l i z a t i o n o f t h e A d a p t i v e O b s e r v e r The adaptive observer in Fig. 2 is unwieldy because of the signals vl and vs. The question was raised, after the papers [36] and [37] appeared, as to whether it would be possible to eliminate these signals entirely by choosing a different parametrization of the adaptive observer. Several different parametrizations [38, 39] were partially successful and eventually led to a final structure based on a nonminimal representation of the model. This structure, suggested by Luders and Narendra [39] is shown in Fig. 3. Using the input u and output y and two identical controllable systems of dimension n, the signals 91 and w2 are first generated, and a linear combination of these signals (using 2n parameters) is chosen as the input to a stable first-order system. The output of the latter was used as the estimate of the output of the plant. The output error el together with ~1 and ~2 determined the adaptive laws for the estimation of the unknown parameters. In the late 1970s, Kreisselmeier [40] made numerous contributions to adaptive observers. He proposed different structures for such observers and derived conditions under which the parameters would converge arbitrarily fast. One of the significant improvements suggested by him led to an integral adaptive law which improved convergence rates substantially. From a theoretical standpoint, the integral laws enabled the procedures developed for continuous-time systems to be related to those generated using discrete-time systems.

A

Yo

Pig. 3. The non-minimal adaptive observer.

4.3 P e r s i s t e n t E x c i t a t i o n It was realized even in the early 1970s that the output error in an adaptive system could tend to zero without the parameters converging to their true values. The convergence of the parameter estimates was known to depend upon the properties of certain signals in the system which in turn depend on the nature of the reference input. The property of the signals which results in the parameter

The Maturing of Adaptive Control

17

errors tending to zero is referred to as persistent excitation and is consequently a central concept in adaptive control. The concept arose in the 1960s in the context of system identification of discrete systems, and was coined to express the qualitative notion that all the modes of the plant are excited. In 1966, •strSm and Bohlin [41] gave a formal statement relating persistent excitation to the convergence of the identification model. Several equivalent definitions, related in one form or another to the existence of a solution to a set of linear equations, appeared soon after. As will be shown in the following sections, persistent excitation also plays a central role in the convergence of adaptive control parameters, as well as in the robustness of such systems in the presence of bounded disturbances. In the late 1970s, numerous workers, including Sondhi and Mitra [42], Morgan and Narendra [43,44], Yuan and Wonham [45] and Anderson [46], addressed the question of persistent excitation in continuous-time systems. While all these papers contain essentially similar results, we describe here those contained in [43] and [44], which pertain to two classes of differential equations and which give rise to different definitions of persistent excitation. These are discussed briefly here since they are relevant for some of the discussions in later sections. The error equations in the adaptive observers discussed in Section 3 and many identification problems have the form

cT(t)~(]~) = C1(~) ,

(21)

where ¢(t), u(t) E IR", u(t) is a known input and ¢(t) is the parameter error vector. If the adaptive law ¢(t) = -el (t)u(t) (22) is used, we have

= --u(t)uT(t)¢(t).

(23)

In [43], necessary and sufficient conditions were derived for the uniform asymptotic stability of the equilibrium state of the differential equation (23) (i.e., the exponential convergence of the parameter error vector to zero). These conditions are also termed the conditions for the persistent excitation of the input vector u. One of several equivalent conditions for the persistent excitation of u is that it satisfy the inequality

jft+To u(~')uT(v) dv >_ aI,

Vt >_to

(24)

for some positive constants to, To and c~. The second differential equation, analyzed in [44], has the form =

_u(t)b T

x

(2s)

where A + A T < 0, (A,b) is controllable and u : IR+ --* IR2 is a bounded piecewise-continuous vector. T h e necessary and sufficient conditions for the

18

Narendra

asymptotic stability of the equilibrium state of (25) is that positive constants T0,/~0, and el exist such that for some t2 E [t,t+To]

1 [*~+~°uT(r)Wdr > el V t > t o

(26)

for every vector W E lR". In the context of the differential equation (25), persistent excitation of u would be defined in terms of the inequality (26). The constant el in (26) is referred to as the degree of persistent excitation. In continuous-time adaptive systems both equations (23) and (25) find application, and persistent excitation in both cases implies the exponential convergence of the parameter errors to zero. For a long time it had been realized that if the reference input contains n distinct frequencies, the vector w(t) E IR ~'~ used in the identification of an LTI plant would be persistently exciting, and hence the parameters would converge to their true values. A general statement of this condition was derived in 1983 by Boyd and Sastry [47] using the concept of spectral lines.

5 Breakthroughs

in the

late

1970s

The results described in Section 3 revealed that linear time-invariant syster~s whose state vectors are accessible can be controlled in a stable adaptive fashion even when the system parameters are unknown. When it was realized that stable adaptive observers could be designed, which estimate the parameters as well as the state variables of a linear time-invariant system, there was renewed interest in adaptive control in the mid 1970s. Based on well-known results in linear systems theory when the parameters of the system are known, it was felt that the estimates obtained by adaptive observers could be used to control systems with unknown parameters in a stable fashion. In fact, for a short period in 1974, it was even believed that the solution to the adaptive observer problem had also provided the solution to the control problem. It was only during the following months that it was realized that the control problem was more involved. In the adaptive observer, the assumption had been made that the plant is stable and that the input and o u t p u t to the observed plant are uniformly bounded. But this is no longer valid in the control problem where it is precisely the boundedness of these signals that has to be established. In fact, it was during one of the sessions at the IFAC World Congress held at Boston in 1975 that it became evident to the leading research workers in the field of adaptive control that the stability problem of adaptive control systems had not been resolved. The next five years witnessed tremendous research activity and culminated in the resolution of the problem in the ideal case in 1980. This result represents the most important development in adaptive systems theory in three decades.

The Maturing of Adaptive Control

19

The Adaptive Control Problem. invariant differential equation

A plant P is represented by a linear time-

~p = Apzp + up =

bpu

(27)

where u : IR + ~ IR is the input yp : ]R+ --* ]R is the output and Zp : 1R+ --* IR" is the state of the plant. The triple {h T, Ap, bp} is assumed to be controllable and observable. T h e elements of hp,Ap and bp are assumed to be unknown. Using only input and output information, the objective is to determine a control function u, using a differentiator-free controller, which stabilizes the overall system. This is referred to as the adaptive regulation problem. If ym(t) is a uniformly bounded desired output signal that is specified, the problem of tracking is defined as the determination of a control input u, using a differentiator-free controller, such that limt-.oo lyp(t) -- ym(t)l = 0. As described earlier, in model reference adaptive control (MRAC) the desired output ym(t) is specified as the output of a reference model whose input r(t) is a uniformly bounded piecewise-continuous function of time. It is worth pointing out that in the initial stages, around 1976, even precise statements of the regulation and tracking problems were not available. In fact, as stated above, it was not clear that a solution existed for either of the two problems. It was only four years later, with perfect hindsight, that a precise problem statement along with a solution was available. The adaptive control problem can also be stated in the frequency domain as follows: A linear time-invariant plant is described by a transfer function Wp(s) where KpZp(S) wp(s)= Rp(s) ' (28) where Zp(s) and Rp(s) are monic polynomials in s of degrees m ( < n - 1) and n respectively and Kp is a constant. The coefficients of the polynomials as well as Kp are assumed to be unknown. A reference model with an input-output pair {r, Ym) which is linear and time-invariant with an asymptotically stable transfer function Wm(s),where

Wm(s) -

KmZm(s) -~m (8) '

(29)

is specified. T h e objective is then to determine, as mentioned earlier, a control input u to the unknown plant such that

lim t~oo

[yp(t)

- ym(t)] = 0.

(30)

Once again, the prior information regarding the plant as well as the transfer function Wm(s) that would be sufficient to solve the problem was by no means clear in the early stages. As numerous workers attempted the solution of the problem, the requirements became more evident. It is also worth pointing out that, over a decade after the adaptive control problem for the ideal case was

20

Narendra

solved in 1980, numerous workers in the field are concerned at present with questions of existence of solutions for other adaptive control problems. During the period 1975-1978 it was generally realized by those working on the adaptive control of continuous-time systems, that the control problem for the case when the relative degree n* (number of poles - number of zeros) of the plant is unity, is substantially simpler than that for the case n" > 2. For such systems, the reference model could be chosen to be strictly positive real and the powerful results of stability theory developed in the 1960s could be used to prove global stability. T h e A d a p t i v e C o n t r o l P r o b l e m f o r n* = x. All the idealized adaptive control problems attempted in the 1970s can be conveniently divided into two parts. These are generally referred to as (i) the algebraic part and (ii) the analytic part. The algebraic part is concerned with the existence of a solution to the adaptive control problem. It consists in demonstrating that a control structure with constant parameters 0*(i = 1, 2 , . . . ) exists such that the plant together with the controller has a transfer function identical to that of the reference model. Once such a controller is shown to exist, the analytic part consists in determining stable adaptive laws for the adjustment of the control parameters 0i, i = 1 , 2 , . . . , so that they evolve to the desired (but unknown) constant values 0~. Even though work on adaptive systems had been in progress for several years, it was only in 1975 that it was realized that the algebraic and analytic parts of the problem could be treated separately. The structure of a controller for the adaptive control problem defined in this section is shown in Fig. 4. The input to the plant u as well as the output y are processed through identical (n - 1)-dimensional stable systems described by the controllable pair [.4, ~]. The output of the former is the ( n - 1)-vector wl (t), while the output of the latter is the (n - 1)-vector w2(t). Together with the reference input r(t) and the output yp(t), these constitute the 2n signals fed back to the plant. The control input to the plant is then expressed as a linear combination of the above signals as follows: =

= k(t)r(t) + oT(t)wl(t)+ oT(t)~2(t) + Oo(t)yp(t) = [k(t), oT(t), oT(t), O0(t)]w(t) T(t) = It(t),

(31)

up(t)].

The algebraic part consisted in showing that constant values of the parameters exist such that the transfer function of the plant together with a controller having the structure given by (31) and these parameters would have a transfer function identical to that of the reference model. Such a controller parameter vector is denoted by 0". Expressing the parameter error vector as ¢(t) = O(t) - 0", the o u t p u t error o(t) = yp(t) - ym(t) can be expressed in terms of the parameter errors as Wm (s)¢T(t)w(t) = el (t). (32)

The Maturing of Adaptive Control

21

From equation (32) it is clear that when the parameter error is zero, the output error will also tend to zero. The analytic part of the problem was to determine adaptive laws for adjusting 0(t) (or equivalently ¢(t)) so that lim,._.oo el (t) = 0. Several research workers working independently showed that the following prior information concerning the plant was sufficient to assure the existence of 0* as well as to develop stable adaptive control laws. The prior information is stated as a set of four assumptions: A s s u m p t i o n s (I) (i) The sign of the high-frequency gain kp of the plant, (ii) an upper bound on the order of the plant, and (iii) the relative degree of Wp(s) are known, and (iv) Zp(s) is a Hurwitz polynomial in s. Assumptions (I), under which this idealized problem could be solved, became important in adaptive control and formed the starting point for many investigations in the 1980s. Returning to equation (31), it was shown by Narendra and Valavani [48] that if the transfer function Wm(S) is strictly positive real, the adaptive laws

O(t) = -sgn(ko)el(t)w(t )

(33)

would result in all the signals in the adaptive system being bounded and the output error e,(t) tending to zero asymptotically. T h e A d a p t i v e C o n t r o l P r o b l e m for n* ~_ 2. Following the solution of the adaptive control problem for the case n* = 1, there was world-wide interest in solving the problem for the general case when n* :> 2. A relative degree of the plant greater than or equal to two implies that the reference model transfer function Wm(s) cannot be positive real, since the latter must have a relative degree greater than or equal to that of the plant. This in turn implies that in the error equation (32) Win(s) is not strictly positive real so that the same procedure used in the case when n* = 1 can no longer be adopted. During this period it almost seemed impossible to solve the problem. The major breakthrough came from a suggestion made by Monopoli [49] that an augmented error (rather than the true error between plant and model outputs) should be used to generate the adaptive laws. The process involved the addition of auxiliary signals to the error equation (32) to reduce it to a mathematically tractable form. As acknowledged by Monopoli to the author, this idea was strongly influenced by the earlier work on adaptive observers in the early 1970s. Defining cT(t)Wm(s)w(t) -- Wm(s)¢T(t)w(t) -" e2(t), (34) it follows that

el(t) + e2(t) = gl(t) = cT(t)Wm(s)w(t) : c T ( t ) ( ( t ) ,

(35)

22

Narendra

r{t)

y~{t}

, lI kmZm(sI Flm(s) I

~(0 +

] k~ZpCs) [ R~,Is)

%{t)

I

q ~2 (t}

F i g . 4. T h e a d a p t i v e controller for n* = 1.

where e2(t) is referred to as the auxiliary error and ¢1(t) is the augmented error. From equation (35) it follows that if e2(t) is added to the true error el(t), then the resulting augmented error ~t(t) is related to the parameter error vector ¢(t) by the equation eT(~)~(~) : E l ( t ) ,

(36)

which has been extensively studied in the context of adaptive systems. It was therefore concluded that adaptation should be carried out using the augmented error ¢1(t) and the vector signal ~(t) rather than the true error signal el(t) and the signal w(t) as in the case when n* = 1. This represented a significant step since the generation of the adaptive law had finally been separated from the control process itself and the signals associated with it. Following the methods used in adaptive observers the natural tendency was to choose the adaptive law as

d(t) = d(t) = --cl (t)¢(t) = --~(t)¢T (t)¢(t) ,

(37)

which assures the boundedness of the parameter vector. However, it was soon realized that this would not assure the boundedness of all the signals in the system. The period 1978-1979 was a period of intense activity in the adaptive control area, with different groups working in different parts of the world trying frantically to determine the modifications needed in the adaptive law (33) to assure the global stability of the adaptive system. As it has happened so often in the history of science, the breakthrough was made independently by several

The Maturing of Adaptive Control

23

groups simultaneously. The first wave of solutions, by Morse [50] and Narendra et al. [51] for continuous-time systems, and by Goodwin, Ramadge and Caines [52] and Narendra and Lin [53] for discrete-time systems, appeared in the June 1980 issue of the IEEE Transactions on Automatic Control, followed by solutions by other authors in later issues. An early unified treatment of these results was given by Egardt [54]. The problem of adaptively controlling a linear time-invariant plant which satisfied the four assumptions given in (I) had been solved. The key to the solutions was the use of a normalization factor in the adaptive control law which assumed the form = 1

(3s)

This ensured that ¢ E £2. The boundedness of the parameter vector together with this condition on the rate of change of the control parameter vector was sufficient to demonstrate that the signals of the adaptive system would be bounded and that the output error el(t) would tend to zero asymptotically. Since many of the results derived in the 1980s used arguments similar to those used in the ideal case, the various steps used in the proof of stability are outlined here. (i) Using the adaptive law (38) it follows that the control parameters are bounded. This in turn implies that the state variables of the adaptive system can grow at most exponentially. (ii) From the fact that ¢ e /22 it is shown that the augmented error el(t) is related to ¢(t) as ~l(t) =/~(t)x/1 + (w(t)¢(t) where/~ e £2. (iii). Assuming that the signals grow in an unbounded fashion, the fact that ¢ E £2 implies that IIw2]l,I1¢11,Ilcall and lYpl grow at the same rate, and that Hw2]I grows at a different rate than Hwll. This leads to a contradiction, so that it is concluded that all signals in the feedback loop are bounded and that limt_.~o el(t) = limt--.oo el(t) = 0. (iv) Finally, it follows that if w(t) is persistently exciting, the parameter vector 0(t) converges to the desired value 0*. For detailed stability arguments based on growth rate of signals the reader is referred to the paper by Narendra, Annaswamy and Singh [55]. 6 Diversity

in the

1980s

The solution of the adaptive control problem in the idealized case described in the previous section provided a tremendous impetus to the field and attracted many researchers from different areas of systems theory. During the first half of the last decade, four major directions of research emerged. The first dealt with situations where the ideal assumptions (I) are satisfied but adaptation has to be carried out in the presence of different types of perturbations. The second direction of research was towards relaxing the assumptions (I) and determining the least restrictive conditions under which regulation and tracking are possible. Extension of results derived for single-input single-output systems to multivariable

24

Narendra

problems was a third direction in which research evolved, and a fourth direction was aimed at determining the conditions for assuring stochastic stability of adaptive systems when the disturbances are random in nature. Since most of the results in stochastic adaptive control apply to discrete-time systems and this paper deals mainly with continuous-time adaptive systems, only the developments in the first three directions are discussed here. Further, during the second half of the decade, the field continued to expand rapidly and many new areas of research came into existence, so that it becomes difficult to track precisely the different advances in the field. In this section we consequently attempt to provide merely a comprehensive review of the principal developments during the 1980s. 6.1 R o b u s t A d a p t i v e C o n t r o l In the statement of the idealized adaptive control problem treated in Section 5, the plant parameters were assumed to be constant and the system was assumed to be disturbance-free. However, in practice, these assumptions are rarely met. No plant is truly time invariant, finite-dimensional, linear, and noise-free. The question therefore naturally arose as to whether the idealized adaptive laws would perform satisfactorily even when external disturbances and parameter variations are present and imperfect models of the plant are used. The realization that small external and internal perturbations can result in unbounded signals in the system eventually led many researchers to what is termed robust adaptive control or the search for techniques for achieving bounded response even in the presence of different disturbances. To understand some of the theoretical difficulties encountered in robust adaptive control problems, consider the Lyapunov function V(e, ¢) used to prove uniform stability in the ideal case. The time-derivative V(e,¢) of V along the trajectories of the system is invariably negative semi-definite. When disturbances and/or unmodeled dynamics are present, V(e, ¢) is indefinite and V is no longer a Lyapunov function guaranteeing stability. The objective in such cases is to modify the adaptive laws so that V is negative semidefinite outside a compact region containing the origin in the state space, so that the response of the system is bounded. B o u n d e d D i s t u r b a n c e s . For the case of arbitrary bounded disturbances, Egardt [54], Peterson and Narendra [56] and Samson [57] suggested the use of a dead-zone in the adaptive law to assure bounded solutions. This modification implies that adaptation is stopped when the error is smaller than a value determined by the prior information concerning the disturbance. Assuming that 0*, the desired control parameter vector, lies inside a sphere S, Kreisselmeier and Narendra [58] demonstrated that the ideal adaptive law, modified only at the boundary of S would result in all the signals in the adaptive system being bounded. This was the precursor, in the continuous case, of the projection algorithms that are commonly used at present in both discrete- and continuous-time systems.

The Maturing of Adaptive Control

25

The two modifications to the adaptive law suggested above require additional information concerning the disturbance or the plant. In 1983, Ioannou and Kokotovic [59] proposed a modified law which does not require such additional information to assure robustness. In place of the adaptive law 8 = - e l w used in the ideal case, the modified law has the form ~=-etw-

nO.

(39)

As mentioned earlier, one of the major problems in the analysis of adaptive systems using the stability approach is that the time derivative V of the Lyapunov function is semidefinite even in the ideal case. The introduction of the term - a 0 in (39) makes V negative outside a compact region containing the origin in the state space, thereby assuring the boundedness of all the signals. In 1987, Narendra and Annaswamy [60] proposed a somewhat different adaptive law based on the same principles, while avoiding some of the shortcomings of (39). This adaptive law has the form = --el~

-- l e l l 0 ,

(40)

The ~-modification law and the lel I-modification law given in [59] and [60] have been studied extensively and are often used in applications to assure robustness in the presence of bounded external perturbations. In contrast to the above methods, which attempt to assure robustness by modifying the adaptive laws, efforts were also made to use the concept of persistent excitation to ensure boundedness of all the signals in the system. A significant result in this direction given by Narendra and Annaswamy [61] states that, if the reference input has a sufficiently large degree of persistent excitation relative to the magnitude of the disturbance, then all the signals in the adaptive system will be bounded. H y b r i d A d a p t i v e Control. All the adaptive laws described so far are continuous in time. Using the same controller structure, it can be shown that the parameter vector 9(t) can also be adjusted using a hybrid algorithm while assuring global stability in the ideal case and robustness in the presence of disturbances. In such a case, 0(0 is held constant over intervals of length T and updated at the end of the interval using the information collected. Several such hybrid modifications were given by Narendra, Khalifa and Annaswamy in [62]. A typical adaptive law has the form t+T

O(t + T ) = O(t) -

Jt

7el(r)w(r)dr ,

(41)

so that 8 is updated along the average gradient over the interval. In extensive simulation studies, hybrid adaptation was found to be preferable to continuous adaptation, as far as robustness is concerned.

26

Narendra

T i m e - V a r y l n g S y s t e m s . One of the compelling reasons for considering adaptive control is to compensate for time variations in plant parameters. While much of the research reported thus far has dealt with plants with constant parameters, interest in adaptive control in the presence of plant parameter variations has existed from the very beginning. In the early stages, most of the researchers made rather restrictive assumptions on the nature of the parameter variations. Anderson and Johnstone [63] demonstrated that if the reference input is persistently exciting and the parameter variations are sufficiently small, the signals in the system will be bounded provided the initial errors are small. Later papers by Chen and Caines [64], Hill et al. [65], and Martin-Sanchez [66], considered exponential decay of parameters to constant values, finite number of jump variations, small perturbations around constant values or a combination of these. Since time variations of plant parameters may violate the assumptions (I) instantaneously, further assumptions were made by many authors to achieve robustness. In [67,68,69,70], globally bounded solutions are established assuming that the plants vary sufficiently slowly. Towards the end of the 1980s, several workers became interested in adaptive systems in which the parameters vary rapidly [72,71]. Tsakalis and Ioannou [72] suggested a modification in the controller structure, by which, at least in theory, the control objective can be met exactly. This important development has enabled the authors to study the behavior of time-varying systems with arbitrarily varying parameters. U n m o d e l e d P l a n t D y n a m i c s . From the results presented in Section 4 it is clear that 2n controller parameters are needed to control an nth-order plant. This poses a major practical limitation since, even for plants of moderate complexity, this requires a large number of adjustable parameters. Further, from a theoretical point of view, the assumption that the plant order is known is rarely met. Hence, in the opinion of many researchers, adaptively controlling an unknown plant using a reduced-order controller was the most important problem in adaptive control in the mid 1980s. The output ~/of the unmodeled part of the plant can be considered as a statedependent disturbance on the plant. Hence, it cannot be assumed to be bounded a priori and a modified adaptive law is needed, which does not require this a priori information. Ioannou and Kokotovic [59] showed that such an update law is the ~-modification (39). The restrictive case when the true plant together with a constant controller is SPR was analyzed by Narendra and Annaswamy [73]. They showed that the degree of persistent excitation of the modeled part should be larger than the magnitude of the output of the unmodeled part to assure the boundedness of all the signals in the system. Important contributions to the problem were made by Praly [74], who employed a normalization technique, and Kreisselmeier and Anderson [75], who used a variable dead-zone to implement the adaptive law. Along with the development of modified adaptive laws, instability mechanisms of ideal adaptive laws were analyzed./~strSm [76] used averaging, previ-

The Maturing of Adaptive Control

27

ously employed in the ODE method of Ljung [77], to give an explanation of the instability examples of l ~ h r s et al. [78]. Applying the method of averaging, Kokotovic et al. [79,80], Fu et al. [81], and Kosut et al. [82], analyzed adaptive control when unmodeled dynamics are present. Kokotovic and co-workers showed that the frequency content of the excitation of certain signals determines a sharp stability-instability boundary for slow adaptation. Sufficient conditions for the boundedness of signals were also derived in [82] based on the frequency range of the dominant excitation.

6.2 Relaxation of A s s u m p t i o n s Many practical adaptive controllers were found to perform satisfactorily even when the ideal Assumptions (I) given in Section 5 were not valid. This eventually led to a search for the least restrictive assumptions under which a solution can be obtained for the adaptive control problem. A major step in this direction was the result of Nussbaum [83] that, even when the sign of A is unknown, the differential equations ~1 = azl + A.f(xl,x2), = g(xl,

a> 0

(42)

can be made to have solutions for which limt-oo Xl(t) = 0 with xz(t) bounded, by the proper choice of the functions f and g. This had an immediate impact. Mudgett and Morse [84] demonstrated soon after that the sign of the highfrequency gain is not needed to stabilize the adaptive system. When Willems and Byrnes [85] considered an nth-order plant with relative degree one and zeros in the left half plane, and demonstrated that it could be stabilized by an adaptive high-gain feedback, it became clear that at least in special cases the bound on the order of the system may not be needed. The search for weaker assumptions was on. Numerous contributions were made during the following years and many partial results were obtained. For example in [86], Morse showed that a 3-dimensional high-gain adaptive controller can stabilize any minimum phase process of any finite dimension provided that the relative degree is less than three and later showed [87] that, if in addition the high-frequency gain is known a priori, a 3-dimensional controller could stabilize a process of relative degree _< 3 without using an augmented error. Indirect control methods rely on the estimation of the plant parameters to adjust the control parameters. Researchers in this area are interested in an approach that would eliminate the requirement that the zeros of the plant should lie in the open left half of the complex plane. An important contribution to relaxing the assumptions for stable adaptive control was made by Kreisselmeier and Smith [88], who showed that a plant with unknown parameters can be regulated in a stable fashion, assuming that only the order of the plant is known. A major advance in the search for weaker assumptions on the plant to ensure stable adaptation was made in 1985 by Martensson, who showed in [89] that the knowledge of the order £ of an LTI stabilizing compensator is sufficient to design

28

Narendra

a stabilizing controller. While the practicability of such a controller is dubious, its importance lies in the demonstration that stable controllers can be designed in theory with very little information concerning the plant. Martensson's contribution has encouraged a number of researchers to investigate adaptive controllers which, while based on weak assumptions, are also practically feasible. Active research based on switching functions is currently in progress in this area. While a complete answer to the question of least restrictive assumptions is not available at present, it appears that stable adaptive control may be possible even when all the assumptions in (I) are violated. 6.3 M u l t i v a r i a b l e A d a p t i v e C o n t r o l Attempts to extend stable adaptive control laws derived for SISO systems to multi-input multi-output systems (MIMO) were also initiated in the early 1980s. Since large uncertainty is generally associated with complex multivariable systems, adaptive control methodologies are even more relevant in such cases. While the adaptive control of MIM0 systems leads to questions which are similar to the single variable case, the solutions to these questions are substantially more involved. In particular the issue of parametrization of the adaptive controller becomes an important problem. The four assumptions corresponding to those in (I), which are sufficient to ensure the existence of a stable adaptive controller in the MIMO case are: Assumptions

(II)

(i) The high-frequency gain matrix Kp of the plant must satisfy the matrix equation FKp+KTpF=Qo, F = F w > 0 , (43) where Q0 is a symmetric sign definite matrix for some positive definite symmetric m a t r i x / ' , (ii) the right Hermite normal form Hp(s) of the plant transfer matrix Wp(s) is known, (iii) an upper bound v on the observability index of Wp(s) is known, and (iv) the zeros of Wp(s) lie in C - . As in the scalar case, assumptions (i) and (iv) are used in the analytic part of the adaptive control problem while (ii) and (iii) are needed for the algebraic part. Several adaptive control algorithms have been suggested in the literature which assure the stability of the overall system when Assumptions (II) are satisfied [90,91,92]. In [92] it is shown that the assumption (ii) concerning the Hermite normal form of Wp(s) has the practical implication that only those multivariable systems Wp(s) whose Hermite normal forms (or whose Hermite normal forms after compensation, i.e., Wp(s)We(8))are diagonal can be controlled in an adaptive fashion. The same questions concerning robustness in the presence of different types of perturbations of single variable questions also arise in the multivariable case.

The Maturing of Adaptive Control

29

All the methods discussed in the earlier sections can be extended to multivariable systems with minor modifications. 6.4 R e c e n t D e v e l o p m e n t s While the main thrust of the research in adaptive systems during the greater part of the 1980s was in the three directions described earlier in this section, many new areas of research emerged during the latter part of the decade. For the sake of completeness we merely list some of these below. Prompted by developments in nonlinear control theory dealing with feedback linearizable systems [93], the adaptive control of special classes of nonlinear systems with unknown parameters has been attempted recently [94,95]. The techniques used in these cases are naturally strongly motivated by the results for linear systems discussed in this paper. Another recent trend in adaptive control circles is towards the use of concepts based on variable structure switching controllers [96,97,98]. The proofs of stability using these methods follow closely along the lines indicated in Section 4 for continuous adaptive laws. While extensive simulation results have indicated that these methods result in substantially improved performance, the variable structure approach raises theoretical questions which need further investigation. Direct and indirect methods were briefly mentioned in Section 2 as the two principal methods used in adaptive control. Both of them have been used widely in robust adaptive systems. Recently it has been demonstrated that the two methods can be combined [99] to benefit from their individual advantages. Even as the research effort in the new areas gains momentum, work continues with increased vigor in the more established areas. Attempts to determine the least stringent conditions under which stable adaptive control is possible, the search for improved methods for controlling time-varying systems, the study of phenomena such as bursting which are peculiar to robust adaptive systems, as well as the application of adaptive control principles to practical systems, are all as active as ever. However, no single area dominates the field. The author is confident that the confluence of the different ideas that are being currently explored will lead to significant developments in the area of adaptive control in the 1990s.

7 Conclusion Having spent its fledgling years in the 1960s in relative peace and quiet among gradient methods, and its youth in the 1970s exploring the mysteries of stable nonlinear systems, a mature adaptive control theory entered the turbulent 1980s with unbridled confidence. Ten years later, on the threshold of a new decade, a more experienced and wiser field views the prospects for the future with cautious optimism. The introduction of new concepts and tools has significantly widened the field and radically changed its structure and boundaries. Towards the end of the last

30

Narendra

decade the field has also witnessed the proliferation of research areas, with the addition of adaptive nonlinear control, variable structure adaptive control, adaptive control using integrated methods, and switching adaptive systems. This, in turn, has made communication between experts in different areas no longer as easy as it was in the past. With this ever-increasing diversity in the field, the need for conceptual frameworks for viewing adaptive control problems and formulating the right questions is assuming greater importance. The burgeoning of new areas, while making the field more attractive to the researcher, has not been an unmixed blessing. Many important questions in the older areas have been left behind, only partially answered. While robustness has been studied extensively, we do not currently have a convenient method of quantifying it. The importance of persistent excitation for the convergence of parameters as well as for the robustness of the adaptive system is well realized, but few prescriptive methods for choosing the reference input or the filters in the controller currently exist. This implies that the choice of parameters in the adaptive system is a matter of trial and error without theoretical support. Since all the stability proofs are asymptotic in character, very little is known at present about the transient behavior of adaptive systems. In the analysis of systems with time-varying parameters, few elegant results exist which are comparable to those derived in the 1960s for linear time-invariant systems. Finally, it is good to remember that while the trend away from gradient methods to stability-based methods was strongly motivated by the desire for faster response, all the proofs of stability that have been derived in the last two decades are based on the time derivative ~(t) of the control parameters being sufficiently small. In light of this, the methods discarded in the 1960s no longer appear unattractive and may even be worthy of further examination. The revolutionary changes in computer technology in the last decade have made the practical implementation of adaptive algorithms economically feasible, and there is a growing demand for industrial adaptive controllers which meet increased speed, accuracy and robustness requirements. In fact, in many areas, the scope of the application is to a large extent limited by the existing theory. During the 1970s and the 1980s, the emphasis of theoretical research was on ensuring the boundedness of all the signals in adaptive systems rather than on performance. Hence, many of the results obtained are not directly applicable to problems in technology. The trend that started in the 1970s, which shifted the emphasis from performance to stability in adaptive systems, appears to have reached the zenith. With our current knowledge of the stability of adaptive systems, it is perhaps time that the trend be reversed to some extent and greater emphasis placed in the future on performance. Thirty-five years ago, the biological implications of the term "adaptive" lent the field of adaptive control an aura which was responsible for the great interest evinced in it. The same reasons are also responsible for the enormous attraction the field enjoys even today. During the intervening years a variety of terms borrowed from biology and psychology, such as learning, pattern recognition, self-organization, artificial intelligence, and intelligent control have found their way into the systems literature. These terms were motivated by the desire to

The Maturing of Adaptive Control

31

incorporate, in artificial systems, the ability of living systems to cope with large uncertainties in their environments. T h e various terms have developed into major independent disciplines with wide followings. Although the sources of the problems and the terminologies in these areas are different from those of adaptive control, m a n y of the concepts used and the theoretical difficulties encountered are quite similar. It is therefore safe to say that advances in adaptive control theory in the 1990s will have a significant impact on all the methodologies dealing with systems containing large uncertainties.

Acknowledgments The author would like to thank Prof. A. M. Annaswamy and Dr. R. M. Wheeler, Jr., for interesting discussions. Some of the sections in this paper are summaries of chapters in the book "Stable Adaptive Systems" by K. S. Narendra and A. M. Annaswamy, published by Prentice-Hall in 1989.

References 1. E. H. Carr, What is History?, Penguin Books, 1981. 2. J. G. Truxal, Automatic Feedback System Synthesis, McGraw-Hill, New York, NY, 1955. 3. B. C. Kuo, Automatic Control Systems, Prentice-Hall, Englewood Cliffs, N J, 1987. 4. R. F. Drenick and R. A. Shahbender, "Adaptive servomechanisms," AIEE Transactions, vol. 76, pp. 286-292, Nov. 1957. 5. J. A. Aseltine, A. R. Mancini, and G. W. Sarture, "A survey of adaptive control systems," IRE Trans. Aut. Control, vol. 3, pp. 102-108, Dec. 1958. 6. P. R. Stromer, "Adaptive or self-optimizing control systems - a bibliography," IRE Trans. Aut. Control, vol. 4, pp. 65-68, May 1959. 7. M. Margolis and G. T. Leondes, "A parameter tracking servo for adaptive control systems," IRE Trans. Aut. Control, vol. 4, pp. 100-111, Nov. 1959. 8. L. A. Zadeh, "On the definition of adaptivity, ~ Proceedings o/the IEEE, vol. 51, pp. 569-570, March 1963. 9. R. Bellman, Adaptive Control Processes - A Guided Tour, Princeton University Press, Princeton, N J, 1961. 10. J. G. Truxal, "Adaptive Control," Proc. ~nd 1FAG World Congress, Basle, Switzerland, 1963. 11. K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems, Prentice-Hall, Englewood Cliffs, N J, 1989. 12. L. S. Pontryagin et al., The Mathematical Theory of Optimal Processes, Interscience, New York, NY, 1962. 13. A. A. Feldbaum, Optimal Control Systems, Academic Press, New York, NY, 1965. 14. R. E. Kalman, "Design of a self-optimizing control system," Transactions of ASME, vol. 80, pp. 468-478, 1958. 15. E. Mishkin and L. Braun, Adaptive Control Systems, McGraw-Hill, New York, NY, 1961.

32

Narendra

16. I. D. Landau, Adaptive Control: The Model Reference Approach, Marcel Dekker, New York, NY, 1979. 17. C. S. Draper and Y. T. Li, "Principles of optimalizing control systems and an application to the internal combustion engine," ASME Publication, Sep. 1951. 18. V. W. Eveleigh, Adaptive Control and Optimization Techniques, McGraw-Hill, New York, NY, 1967. 19. R. J McGrath, V. Rajaraman, and V. C. Rideout, "A parameter perturbation adaptive control system," IRE Trans. Aut. Control, vol. 6, pp. 154-161, May 1961. 20. J. L. Douce, K. C. Ng, and M. M. Gup~a, "Dynamics of the parameter perturbation processes," Proceedings of the IEE, vol. 113, no. 6, pp. 1077-1083, June 1966. 21. R. E. Kronaner and P. G. Drew, "Design of the feedback loop in parameterperturbation adaptive contzols," in Theory ol Self-Adaptive Control Systems, Proc. ~nd IFAC Syrup. (Teddington, UK, 1965), P. H. Hammond, ed., Plenum Press, New York, NY, 1966. 22. K. S. Narendra and L. E. McBride, Jr., "Multiparameter self-optimizing systems using correlation techniques," IRE Trans. Aut. Control, vol. 9, pp. 31-38, Jan. 1964. 23. P. V. Kokotovic, "Method of sensitivity points in the investigation and optimization of linear control systems," Automation and Remote Control, vol. 25, pp. 1512-1518, 1964. 24. J. B. Cruz, Jr., ed., Systems Sensitivity Analysis, Dowden, Hutchinson and Ross, Stroudsburg, PA, 1973. 25. H. P. Whitaker, J. Yamron, and A. Kezer, "Design of model-reference adaptive control systems for aircraft," Report R-164, Instrumentation Laboratory, MIT, Cambridge, MA, 1958. 26. L. P. Grayson, "Design via Lyapunov's second method," Proc. 4th Joint Amer. Control Con]., 1963. 27. B. Shackcloth and R. L. Butchart, "Synthesis of model reference adaptive systems by Lyapunov's second method," in Theory of Self-Adaptive Control Systems, Proc. ~nd 1FAC Symp. (Teddington, UK, 1965), P. H. Hammond, ed., Plenum Press,. New York, NY, 1966. 28. P. C. Parks, "Lyapunov redesign of model reference adaptive control systems," IEEE Trans. Aut. Control, vol. AC-11, pp. 367.-367, 1966. 29. R. V. Monopoli, "Lyapunov's method for adaptive control system design," IEEE Trans. Aut. Control, vol. AC-12, pp. 334-335, 1967. 30. P. H. Phillipson, "Design methods for model reference adaptive systems," Proc. Inst. Mech. Engineers, vol. 183, pp. 695-700, 1969. 31. C. A. Winsor and R. J. Roy, "Design of model reference adaptive control systems by Lyapunov's second method," 1EEE Trans. Aut. Control, vol. AC-13, p. 204, April 1968. 32. K. S. Narendra and P. Kudva, "Stable adaptive schemes for system identification and control - part I" 1EEE Trans. Systems, Man and Cybernetics, vol. SMC-4, pp. 541-551, Nov. 1974. 33. K. S. Narendra and P. Kudva, "Stable adaptive schemes for system identification and control - part II" IEEE Trans. Systems, Man and Cybernetics, vol. SMC-4, pp. 552-560, Nov. 1974. 34. Ya. Z. Tsypkin, Adaptation and Learning in Automatic Systems, Academic Press, New York, NY, 1971.

The Maturing of Adaptive Control

33

35. K. J. AstrSm and B. Wittenmark, "On self-tuning regulators, ~ A u t o m a t i c a , vol. 9, pp. 185-199, 1973. 36. 1%. Carroll and D. P. Lindorff, "An adaptive observer for single-input singleoutput linear systems, z IEEE Trans. Aut. Control, vol. AC-18,pp. 428-435, Oct. 1973. 37. P. Kudva and K. S. Narendra, "Synthesis of an adaptive observer using Lya. punov's direct method," Int. J. Control, vol. 18, pp. 1201-1210, 1973. 38. G. Luders and K. S. Narendra, "An adaptive observer and identifier for a linear system," IEEE Trans. Aut. Control, vol. AC-18, pp. 496-499, Oct. 1973. 39. G. Luders and K. S. Narendra, "A new canonical form for an adaptive observer," IEEE Trans. Aut. Control, vol. AC-19, pp. 117-119, April 1974. 40. G. Kreisselmeier, "Adaptive observers with exponential rate of convergence," IEEE Trans. Aut. Control, vol. AC-22, pp. 2-8, Jan. 1977. 41. K. J. ,~strSm and T. Bohlin, "Numerical identification of linear dynamic systems from normal operating records," in Theory o] Self-Adaptive Control Systems, Proe. 2nd IFAC Syrnp. (Teddington, UK, 1965), P. H. Hammond, ed., Plenum Press, New York, NY, 1966. 42. M. M. Sondhi and D. Mitra, "New results on the performance of a well-known class of adaptive filters," Proceedings of the IEEE, vol. 64, pp. 1583-1597, Nov. 1976. 43. A. P. Morgan and K. S. Naxendra, "On the uniform asymptotic stability of certain linear non-autonomous differential equations," SIAM J. Control Optimiz., vol. 15, pp. 5-24~ Jan. 1977. 44. A. P. Morgan and K. S. Narendra, "On the stability of nonautonomous differential equations ~ = [A + B(t)]x with skew-symmetric matrix B(t)" S l A M J. Control Optimiz., vol. 15, pp. 163-176, Jan. 1977. 45. J. S. C. Yuan and W. M. Wonham, "Probing signals for model reference identification," IEEE Trans. Aut. Control, vol. AC-22, pp. 530-538, Aug. 1977. 46. B. D. O. Anderson, "Exponential stability of linear equations arising in adaptive identification," IEEE Trans. Aut. Control, vol. AC-22, pp. 83-88, Jan. 1977. 47. S. P. Boyd and S. S. Sastry, "On parameter convergence in adaptive control," Syst. Control Left., vol. 2, pp. 311-319, 1983. 48. K. S. Narendra and L. S. Valavaui, "Stable adaptive controller design-direct control," IEEE Trans. Aut. Control, vol. AC-23, pp. 570-583, Aug. 1978. 49. R. V. Monopoli, "Model reference adaptive control with an augmented error signal," IEEE Trans. Aut. Control, vol. AC-19, pp. 474-484, 1974. 50. A. S. Morse, "Global stability of parameter adaptive control systems," IEEE Trans. Aut. Control, vol. AC-25, pp. 433-439, June 1980. 51. K. S. Narendra, Y. H. Lin, and L. S. Valavani, "Stable adaptive controller design -part II: proof of stability," IEEE Trans. Aut. Control, vol. AC-25, pp. 440-448, June 1980. 52. G. C. Goodwin, P. J. Ramadge, and P. E. Caines, "Discrete time multivariable adaptive control," IEEE Trans. Aut. Control, vol. AC-25, pp. 449-456, June 1980. 53. K. S. Narendra and Y. H. Lin, "Stable discrete adaptive control," IEEE Trans. Aut. Control, vol. AC-25, pp. 456-461, June 1980. 54. B. Egardt, Stability of Adaptive Controllers, Springer-Verlag, Berlin, 1979. 55. K. S. Narendra, A. M. Annaswamy, and It. P. Singh, "A general approach to the stability analysis of adaptive systems," Int. J. Control, vol. 41, pp. 193-216, 1985. 56. B. B. Peterson and K. S. Narendra, "Bounded error adaptive control," IEEE Trans. Aut. Control, vol. AC-27, pp. 1161-1168, Dec. 1982.

34

Nazendra

57. C. Samson, "Stability analysis of adaptively controlled systems subject to bounded disturbances," Automatiea, vol. 19, pp. 81-86, 1983. 58. G. Kreisselmeier and K. S, Nitrendra, =Stable model reference adaptive control in the presence of bounded disturbances," ]EEE Trans. Aut. Control, vol. AC-27, pp. 1169-1175, Dec. 1982. 59. P. A. Ioannou and P.V. Kokotovic, Adaptive Systems with Reduced Models, Springer-Verlag, New York, NY, 1983. 60. K. S. Narendra and A. M. Annaswamy, "A new adaptive law for robust adaptive control without persistent excitation," IEEE Trans. Aut. Control, vol. AC-32, pp. 134-145, Feb. 1987. 61. K. S. Narendra and A. M. Annaswamy, "Robust adaptive control in the presence of bounded disturbances," IEEE Trans. Aut. Control, vol. AC-31, pp. 306-315, April 1986. 62. K. S. Narendra, I. H. Khalifa, and A. M. Annaswamy, "Error models for stable hybrid adaptive systems-part II," I E E E Trans. Aut. Control, vol. AC-30, pp. 339-347, April 1985. 63. B. D. O. Anderson and R. M. Johnstone, "Adaptive systems and time-varying plants," Int. J. Control,vol. 37, pp. 367-377, Feb. 1983. 64. H. F. Chen and P. E. Calnes, "On the adaptive control of stochastic systems with random parameters," Proc. $3rd 1 E E E Conf. Dec. Control, 1984. 65. D. J. Hill, G. C. Goodwin, and X. Xianya, "Stochastic adaptive control for exponentially convergent time-varying systems," Proc. ~3rd I E E E Conf. Dec. Control, 1984. 66. J. M. Martin-Sanchez, "Adaptive control of time-variant processes," Proe. 1985 Amer. Control Conf. 67. G. Kreisseimeier, "Adaptive control of a class of slowly time-varying plants," Syst. Control Lett., yah 8, Dec. 1986. 68. T. H. Lee and K. S. Narendra, "Stable direct adaptive control of time-varying discrete-time systems," Technical Report 8720, Center for Systems Science, Yale University, New Haven, CT, 1987. 69. K. S. Tsakalis and P.A. Ioannou, "Adaptive control of linear time-varying plants," Automatics, vol. 23, pp. 459-468, July 1987. 70. R. H. Middleton and G. C. Goodwin, "Adaptive control of time-varying linear systems," IEEE Trans. Ant. Control, vol. AC-33, pp. 150-155, Feb. 1988. 71. A. M. Annaswamy and K. S. Narendra, "Adaptive control of a first-order plant with a time-varying parameter," Proc. 1990 Amer. Control Conj. 72. K. S. Tsakalis and P. A. Ioannou, "Adaptive control of linear time-varying plants: a new model reference controller structure," IEEE Trans. Aut. Control, vol. 34, pp. 1038-1046, Oct. 1989. 73. K. S. Narendra and A. M. Annaswamy, in Adaptive and Learning Systems, K. S. Narendra, ed., Plenum Press, New York, NY, 1985. 74. L. Praly, "Robustness of model reference adaptive control," Proc. 3rd Yale Workshop on Applications of Adaptive Systems Theory, Yale University, New Haven, CT, 1983. 75. G. Kreisselmeier and B. D. O. Anderson, "Robust model reference adaptive control," 1EEE Trans. Aut. Control, vol. AC-31, pp. 127-133, 1986. 76. K. J. )kstrSm, "Analysis of Rohrs counterexamples to adaptive control," Proe. ~2nd IEEE Conf. Dee. Control, 1983. 77. L. Ljung, "Analysis of recursive stochastic algorithms," 1BEE Trans. Ant. Control, vol. AC-22, pp. 551-575, Aug. 1977.

The Maturing of Adaptive Control

35

78. C. E. Rohrs, L. VMavani, M. Athans, and G. Stein, "Robustness of adaptive control algorithms in the presence of unmodeled dynamics," Proc. 21st IEEE Conf. Dec. Control, 1982. 79. B. Riedle and P. V. Kokotovic, "A stability-instability boundary for disturbancefree slow adaptation and unmodeled dynamics," Proc. g3rd IEEE Conf. Dec. Control, 1984. 80. P. V. Kokotovic, B. Riedle, and L. Praly, "On a stability criterion for continuous slow adaptation," Syst. Control Left., vol. 6, pp. 7-14, June 1985. 81. L. C. Fu, M. Bodson, and S. S. Sastry, "New stability theorems for averaging and their applications to the convergence anMysis of adaptive identification and control schemes," Proc. 2~th IEEE Conf. Dec. Control, 1985. 82. R. L. Kosut, B. D. O. Anderson, and I. M. Y. Mareels, "Stability theory of adaptive systems: methods of averaging and persistent excitation," Proe. ~Jth IEEE Conf. Dec. Control, 1985. 83. R. D. Nussbaum, "Some remarks on a conjecture in parameter adaptive control," Syst. Control Lett., vol. 3, pp. 243-246, 1983. 84. D. R. Mudgett and A. S. Morse, "Adaptive stabilization of linear systems with unknown high frequency gains," IEEE Trans. Aut. Control, vol. AC-30, pp. 549554, June 1985. 85. J. C. Willems and C. J. Byrnes, "Global adaptive stabilization in the absence of information on the sign of the high frequency gain," Proc. INRIA Conf. on Analysis and Optimization of Systems, Lecture Notes in Control and Information Sciences, vol. 62, Springer-Verlag, Berlin, 1984, pp. 49-57. 86. A. S. Morse, "A three dimensional universal controller for the adaptive stabilization of any strictly proper minimum-phase system with relative degree not exceeding two," IEEE Trans. Aut. Control, vol. AC-30, pp. 1188-1191, 1985. 87. A. S. Morse, "High gain feedback algorithms for adaptive stabilization," Proc. 5th Yale Workshop on Applications of Adaptive Systems Theory, Yale University, New Haven, CT, 1987. 88. G. Kreisselmeier and M. C. Smith, "Stable adaptive regulation of arbitrary nthorder plants," IEEE Trans. Aut. Control, vol. AC-31, pp. 299-305, 1986. 89. B. Martensson, "The order of any stabilizing regulator is sufficient for adaptive stabilization," Syst. Control Lett., vol. 6, pp. 87-91, July 1985. 90. G. C. Goodwin and R. S. Long, "Generalization of results on multivariable adaptive control," IEEE Trans. Aut. Control, vol. AC-25, pp. 449-456, 1980. 91. H. Elliott and W. A. Wolovich, "Parameter adaptive control of linear multivariable systems," IEEE Trans. Aut. Control, vol. AC-27, pp. 340-352, 1982. 92. R. P. Singh and K. S. Naxendra, "Prior information in the design of multivariable adaptive controllers," IEEE Trans. Aut. Control, vol. AC-29, pp. 1108-1111, Dec. 1984. 93. A. Isidorl, Nonlinear Control Systems, 2nd ed., Springer-Verlag, Berlin, 1989. 94. J.-B. Pomet and L. Praly, "Adaptive nonlinear regulation: equation error from the Lyapunov equation," Proc. ~8th IEEE Conf. Dec. Control, 1989. 95. I. Kanellakopoulos, P.V. Kokotovic, and R. H. Middleton, "Observer-based adaptive control of nonlinear systems under matching conditions," Proc. 1990 Amer. Control Conf. 96. L. Hsu, "Variable structure model-reference adaptive control (VS-MRAC) using only input and output measurements: part II," Proc. ~7th IEEE Conf. Dec. Control, 1988.

36

Narendra

97. L. C. Fu, "A new robust model refeience a~laptive control using variable structure adaptation for plants with relative degree two," Proc. 1990 Amer. Control Con]. 98. K. S. Narendra and J. D. Boskovic, "A combined direct, indirect and variable structure method for robust control," IEEE Trans. Aut. Control, to appeax. 99. M. A. Duarte and K. S. Narendra, ~A new approach to model reference adaptive control," Int. J. Adopt. Control Sig. Process., vol. 3, pp. 53-73, 1989.

A Conceptual Framework for Parameter A d a p t i v e Control* A. S. Morse Department of Electrical Engineering Yale University

P. O. Box 1968 New Haven, CT, 06520-1968,USA.

A b s t r a c t . A conceptual framework is described in which a parameter adaptive control system is taken to be the feedback interconnection of a process •p and a parameterized controller •c(k) whose parameter vector k is adjusted by a tuner ~ T . The framework is general enough to encompass Mmost all parameterized controllers proposed in the literature for stabilizing linear process models. Emphasis is placed on the importance to adaptation of one of ~C'S outputs called a tuning error eT, which is the main signal driving E T . For the closed-loop parameterized system E(/;) consisting of ~p and Ec(/r), definitions and characterizations are given of the concepts of weak tunability and tunability of E(b) on a subset £ of the parameter space 7~ in which k takes values. It proves to be necessary to know a subset ~ on which E(k) is weakly tunable in order to be able to construct a tuner ~T which adaptively stabilizes E(/~). For a large class of linear multivaxiable process models, a properly designed certainty equivalence controller results in a tunable closed-loop parameterized system. The importance of this result to both the analysis and synthesis of parameter adaptive control systems is discussed. It is demonstrated by means of examples how the connection between certainty equivalence and tunability, together with the concept of tunability itself, can be used to markedly simplify the stability analysis of adaptive control systems. A new family of indirect parameterized controllers is described which have capabilities comparable to those of the well-known direct parametefized controllers which for a long time have served as basic building blocks for adaptive control systems of all types. The concept of implicit tuning is formalized and its potential importance to adaptive control is briefly discussed.

Introduction In a series of recent papers [1,2], a conceptual framework is introduced for defining, discussing and anMyzing parameter adaptive control systems of all types. The idea is to think of an adaptive control system as the feedback interconnection of a process ,UP and a "parameterized controller" E c ( k ) whose parameter vector k is adjusted by a "tuner" ~7T. A parameterized controller is a parameterdependent dynamicM system whose outputs include not only a control signal u c * This research was supported by the National Science Foundation under grant ECS9012551.

38

Morse

which in closed loop serves as the feedback control to the process, but also a suitably defined ~tuning error" eT which during adaptation drives the tuner ,UT. Depending on the application, eT might be an identification error, an augmented error, a process output, or something similar. In this context, a tuner is any algorithm driven by eT which adjusts k. N o matter h o w eT and ,UT are defined, the purpose of `UT is always the same: to tune k to m a k e eT "small". T h e aim of this paper, which is an abbreviated version of [1] and [2],is to discuss some of the implications of thinking of an adaptive control system from this point-of-view.

Section 1 briefly summarizes the salient features of a parameter adaptive control system viewed in the aforementioned way. In Section 2 attention is focused on properties of the subsystem ,U(k) consisting of `up in feedback with `uc(k). The concept of "tunability" is defined and discussed. Under mild conditions, tunability of `U on a known subset C of the space in which k takes values, proves to be necessary for adaptive stability. The problem of choosing `uc so that `U has this property is addressed in Section 3 for the case when ,Uc is "identifierbased". By an identifier-based paxameterized controller ,Uc is meant a system consisting of an "identifier" ,~x and an 'internal regulator" or "certainty equivalence controller" ~'R- ~'R is chosen off-line to endow a suitable defined "design model" ~UD with desirable features (e.g., internal stability) for each fixed value of the design model's parameter vector PI- The design model is presumed to have a transfer matrix which is "close" to that of the process model, for some unknown value of PI- The coefficient matrices defining L'D are related to those of `UI by an appropriately defined, parameter-dependent, linear transformation. By making use of this transformation, it is possible to prove that if the internal regulator stabilizes `UD for some set of possible design model parameter values ~, then the closed-loop system ,U consisting of ,Up, ,Ux and ,UR is tunable on C-this is the Certainty-Equivalence Stabilization Theorem. For the case when ,Up is minimum phase, to achieve tunability on C, it turns out to bc enough to require ,UR only to "output-stabilize" ~'D on ~. This result, called the Certainty Equivalence Output Stabilization Theorem, is the key technical link needed to explain how to achieve with indirect control those results achieved previously with direct adaptive control. The preceding theorems, which can each be proved using very elementary arguments, have many useful consequences. Since in their proofs there is no presumption of process model matching, the theorems can be applied without having to talk about "true process model parameter values", an idea which loses much of its meaning as soon as the process under consideration is assumed to have unmodeled dynamics. Since the theorems are parameterization independent, it is possible to use them to analyze directly and indirectly parameterized adaptive controllers in essentially the same way. Since the theorems enable one to sidestep much of the extraneous, detailed structure of error modcls in studying system behavior, it is possible to dramatically simplify the stability analyses of many adaptive control systems. An example which demonstrates this is given at the end of Section 2. To make things explicit, examples are given in Section 3, which show how

A Conceptual Framework

39

to apply the preceding ideas to SISO parameterized design models of both the direct and indirect control types. To output stabilize the latter it is necessary to restrict its parametric structure; what results is a parameterized design model which, together with its output stabilizing internal regulator, provides a new family of indirect, parameterized controllers, with capabilities comparable to those of the well-known direct parameterised controllers which for a long time have served as basic building blocks for adaptive control systems of all types [3-8]. Section 4 formalizes the concept of implicit tuning and explains how to analyze implicitly tuned systems using the ideas developed in Section 3. By implicit tuning is meant the adjustment of an identification parameter vector kx and an implicitly defined internal regulator parameter vector kR to make both a tuning error (e.g., an identification error) and a suitably defined "design" error simultaneously small. Implicit tuning algorithms can be either of the "one level" or "two level" type. We present examples of each and briefly discuss some of their possible generalizations. In the sequel, prime denotes transpose, IBn is the real linear space of ncomponent column vectors z, and Ilzll is the norm x/-~~.

1 Framework Classically [3-8], parameter adaptive control systems have been defined and discussed in terms of error models in which parameters typically enter linearly. It turns out t h a t for many adaptive algorithms (e.g., [9-13]) error models play no essential role and parameters enter nonlinearly. Hence, a new conceptual framework in needed to describe both the classical and more recent adaptive structures. To construct such a framework it is useful to view a parameter adaptive control system as the feedback interconnection of a process ,Up and a parameterized controller E c ( k ) whose parameter vector k is adjusted by a tuner ,UT. The process is a dynamical system with control input u E IR'~ , disturbance input w E IW'w and measured output y E IRn ' . The parameterized controller is a dynamical system ,Uc(k) depending on a control parameter k which takes values in some parameter space 7~ C IK"'. The inputs to 57c are the process open-loop control u, the process output y, and possibly a reference input r. The outputs generated by 27c are a tuning error ew which drives ,UT, a control signal uc which becomes the feedback control to the process when u is set equal to uc, and a vector of supplementary tuning data d consisting of known functions of r , y and the parameterized controller's state. The tuner is an algorithm ~[Tw(k0) initialized by k0 (i.e., k(0) = k0) with inputs d and ew and output k, k(t) being the "tuned-value" of k at time t; ,Uw(k0) is presumed to be well-defined for each

k0 e~,. The function of •W is to adjust k to make ew "small" in some sense. Although the specifics of ~T may vary greatly from algorithm to algorithm, in most instances tuning is carried out in one of two fundamentally different ways, depending on whether 7~ is countable or not. For the countable case (e.g., see

40

Morse

TUNER

zT w

1

I PARAMETERIZED CONTROLLER

PROCESS

=

Zp

~:c(k)

t. Fig. 1. Parameter adaptive control system.

[10,13]), tuning is achieved by sequentially stepping k through T~ along a predetermined path, using on-line (i.e., real-time) data to decide only when to switch k from one value along the path to the next. In contrast, for the uncountable case (e.g., see [3-8]) the path in :P along which k is adjusted is not determined off-line but instead is computed in real time from "gradient-like" data. The main advantage of countable search algorithms over gradient-like procedures appears to be their broader applicability. On the other hand, when applicable, gradientlike algorithms are likely to exhibit far superior performance, but so far this has not been clearly demonstrated. Since a tuner 2~W is an algorithm driven by ew, ~T will typically possess certain "rest" or "equlibrium" values of k at which tuning ceases if eT ---- 0. To be more precise, let us agree to say that in open-loop, a tuner ,UT is at equilibrium value Po E 7~ at time t0 > 0, if k(to) = Po and if for the input eT(t) = O, t >_ to, lc(t) remains fixed at P0 for t > to. In the sequel we assume that ,UT(k0) is stationary to the extent that its possible equilibrium values at to are independent of to and k0 E •; and we define the tuner's equilibrium set ET to be the set of all such values in •. One fairly general algorithm for a tuner might be a dynamical system of the form

= l(k) + k(0) = k0,

2(k, d)eT (1)

where the 7ri(.) are nonlinear functions and k0 E IR"~. In this case 7~ can be

A Conceptual Framework

41

taken as lR '~" and ~:T = {P : ~rl (p) = 0}. Alternatively, ,UT might be a switching algorithm of the form [13]

k(O =

ko, t e [t0,tl) h(i), t e It. t~+l), i ___ 1,

where h is a function from the positive integers to IR"', ko E image {h}, to = 0, and for i > 0 •

t

t, = [ mm,>,,_~ { t : f~ HeT[ldr = i!}, if this set is nonempty, [ oo, otherwise. In this case :P is the image of h (which is countable), and ET = :P. The ideas which follow apply to both types of algorithms. Thus, unless otherwise stated, no special assumptions will be made about ,UT, other than that it be stationary and possess a nonempty equlibrium set. In this paper we assume w to be zero, since the presence or absence of disturbances is not important for the points we want to make. We take the process model ,Up = (Cp, Ap, Bp) to be a member of some known class Cp ofstabilizable, detectable, time invariant, continuous-time linear systems; i.e., ~:p = A p x p

+

Bpu

y = Cp.Tp .

(2)

As a parameterized controller 27c(k), we shall consider a system of the form

kc = Ac(k)zc + Bc(k)y + Bv(k)r + B,(k)u ~c = F c ( k ) x c + Gc(k)v + G.(k)~ e T ---- C c ( / c ) a ~ c +

Dc(k)y,

(3)

where Ac(.), Bc(.), .... , Dc(-) are matrix-valued functions on :P. An equation for d is not needed for the discussion which follows. 2 Tunability The closed-loop parameterized system E(k) determined by (2), (3) and the feedback connection " = "c, (4) can be concisely denoted by the equations = ACk)z + BCk)r eT = C ( ~ ) x ,

(5)

where z = [z~,z~]'. Here A(.),B(.), and C(.) are defied in the obvious way using (2)-(4). In this setting, the following question arises: W h a t must be true of E in order for there to exist a tuning algorithm 2YT for which the closed-loop

42

Morse

adaptive system, consisting of ~TT and 57, is "stable"? In the sequel we provide some answers to this question. With £ any fixed, nonempty subset of ,U's parameter space 7~, let us agree to call (5) weakly tunable on E, if for each fixed p • g and each bounded, piecewisecontinuous exogenous input r : [0, oo) ---* IRnr, every possible system trajectory for which k(t) = p and ew(t) = 0, t • [0,oo), is bounded on [0, c~). Call 57 tunable on L" if for each p • t;, x goes to zero as t ~ oo along each trajectory on which k(t) = p and both ew and r equal zero, t • [0, c~). R e m a r k 1: It is easy to verify that 57 is weakly tunable on E just in case, for each p E £, the matrix pair (C(p), A(p)) is weakly detectable ~ and the matrix pair obtained by restricting C(p) and A(p) to the controllable space of (A(p), B(p)) is detectable. Similarly, (C(p), A(p)) is tunable on £ if and only if (C(p), A(p)) is detectable for each p • g. Thus, tunability of 57 on E implies weak tunability of ,U on S and is equivalent to weak tunability of ~U on £ whenever 2~ is controllable on £.O Our aim here is to briefly explain why weak tunability is necessary for adaptive stabilization. To be specific, call ET(.) an unbiased stabilizer of S if for each initialization k0 E 7~, each bounded, piecewise-continuous input r : [0,c~) --~ llq.n" and each initial state x(0), the state response x of Z ( k ) , with k tuned by 2~w(k0), is bounded on [0, c~). Suppose 57T is a candidate tuner for 27(k). The definition of weak tunability implies that if ,U(k) is not weakly tunable on ,Uw's equilibrium set £T, then for some exogenous input r, initial state x(0), and parameter value P0 E ~W, the untuned system 57(po) will admit an unbounded state response x(t) along which ew(t) = 0. If the same input r is applied to 57(k), with k tuned by -UT(P0), then clearly k(t) = Po, t >_ O, and ew(t) = 0, t > 0. Therefore ~ ( k ) will have exactly the same unbounded response to r that S(p0) has. We are led to the following T h e o r e m 1: A necessary condition for the tuner S T ( ' ) to be an unbiased stabilizer of the tuned system S ( k ) , is that S ( k ) be weakly tunable on the equilibrium

set CT o/ST(.). Clearly, weak tunability on ~ r is a fundamental property that any parameter adaptive control system of the aforementioned general type must have if stability is to be assured. 3 An interesting problem then, is to determine what's required of 2 A matrix pair (C, A) is weakly detectable if for each vector x for which CeAtx is bounded on [0, co), it follows that eAtx is bounded on [0, co) as well. (C, A) is detectable if for each eigenvalue-eigenvector pair ()~,x), .~ has a negative real part whenever Cz = 0. Detectability implies weak detectability but the converse is not necessarily true. For algorithms utilizing persistently exciting probing signals, weak tunability on ET may well be more than is required for stability.

A Conceptual Framework

43

a process model Z:p and its parameterized controller ZTc for the resulting closedloop system 27 to be weakly tunable or tunable on some given subset g C P . This problem is discussed further in Sect. 3. In the sequel we give some examples of tunable and untunable systems. E x a m p l e 1: Suppose for £7p, we take the one-dimensional system it = a y + O u ,

(6)

with a and g unknown constants satisfying a > 0 and g ~ 0. To stabilize this system, consider using a control law of the form

u = fu,

(~)

where, if we had our d'rnthers, we'd choose .f so that a + 9 f ' - - - 1 since this would stabilize (6); but since a and g are unknown, we might instead try to choose f i n accordance with the "certainty-equivalence principle" (of. Sect. 3} so that ~ q- ~ f ' = --1, (8) where ~ and ~ are estimates of a and g respectively. However, since standard identification algorithms may cause ~ to pass through zero, to avoid the possibility of 'division by zero', in place of (8) consider tuning f with the "gradient" adjustment law f = ----2 o]

= --geD,

(9)

where eD is the "design error" eD -- a + ~ } + I

(10)

(cf. Sect. 4). Finally, to construct estimates ~ and ~, observe from (6) that

y = (a+ 1 ) ~ + g ~ + e , where e = e - t ( y ( O )

- (a +

1)9(0)), and

b+ u = u u+fi=u.

(11)

Thus, to generate ~ and fi, it makes sense to use an algorithm driven by the "identification error" el -- (~+ 1).~+ ~ f i - y, (12) since this results in the familiar error equation e, = 9(a - a) + ~ ( ~ - g) - e.

(13)

If a standard identification algorithm is used, identification ceases when ex = 0, in which case ~ and ~ become constant. Viewing this algorithm together with

44

Morse

(9) and (10) as a tuner ,UT with tuning error input eT = ei and tuned parameter k = [~, ~, ~ ' , ,Uw's equilibrium set will be ET = { [Pl,P~,l~]' : PI(P2 + P l P a + 1) = 0, [Pl,P~,P3]' • IRa } • The overall parameterized controller ,Uc is thus ~= -,9+y ~=-~+u eT =

+

(14) +

y

BC ~ f y ~

and ,U(k) is the closed-loop parameterized system, described by (6), (14) and the feedback connection u = uc. (15) Observe that the point [0, a, 0]~ • F.T. It is easy to verify that for k fixed at this value, Z admits the unbounded soultion y - e ~t, ~ "- e~/(1-l-a), ~ = O, e T " - O, so ~U is not weakly tunable on ~T.O E x a m p l e 2: In E x a m p ~ 1, 57 is not tunable on ~T because ET contains points [~, ~, j~t at which ~ + ~ f > 0. It is possible to eliminate this problem and to achieve stability, if sign(g) is assumed known, by using in place of (9) the adjustment law f = --(sign(g))eD, together with the tuning equations 4 ,k a

= - - e D - - ,~e T

~----- - - f e D

-- ~eT •

In this case ~T'S equilibrium set is precisely those points [ f f , ~ , ~ E 1Ra for which (8) holds. It is straightforward to check that for k fixed at any point [Pl,P2, P3]' • £T, y(t) = pl(fi(0)--p3Tg(0))e -~ along any solution [y(t), ~(t), fi(t)]' to (6,14,15) for which ew(t ) ~ 0. Since this, (14), and (15) imply that any such solution is bounded, Z7 is now tunable on ST.O E x a m p l e 3: Take P = IR and let 57 be any closed-loop parameterized system with C ( k ) = [1,0] and

Since (C(.), A(.)) is detectable on :P, by Remark 1 ,U must be tunable on each nonempty subset £ C 7). In spite of this, observe that no matter how k is tuned, ,U(k) can have an unbounded state response (e.g., with r = 0 and x(0) = [1, 0]~, z ( t ) = et[1, 0]'), so adaptive stabilization is impossible.O 4 Motivation for these tuning equations stems from (13) and the observation that eD can be written as eD = ( ~ - a ) + f ( ~ - g ) - b g ( f ' - f ) [14]; this is an example of "single level implicit tuning" (cf. Sect. 4).

A Conceptual Framework

45

The preceding example shows that tunability of 27 on a known subset £ C IRn ' , is not sufficient to insure that there is a tuner ~'w which can adaptively stabilize ,U(k). However, using the ideas from [10], it can be shown that if 27 is tunable on £ and, in addition £ contains a countable subset £" containing a parameter value p0 for which L'(p0) is internally stable, then without knowing P0 it is possible to construct a switching algorithm 27T, with gw = £*, which is an unbiased stabilizer of,U(k). Thus for the case when P = IR"P , to achieve adaptive stability with some tuner 27T, it's enough to design ,Uc so that (C(.), A(.)) is detectable on a known countable subset £* C IR'*" containing a point P0 which stabilizes A(po). T h e following example shows that this is very easy to do without assuming very much about ,Up. E x a m p l e 4: For a fixed integer n > 0, define the parameter vector p = [Pl, Pu, • .. ,P2n+t]', and parameterized polynomials/~(p, s) = s'* + p n s " - I + . . . p2s + Pl, a ( p , s ) = p2n+ls n + p2ns n-1 + . . . p n + 2 s + P,~+I. Choose 7(s) to be any monic stable polynomial of degree n and let •I(P) - ( A l ( p ) , b i ( p ) , c l ( p ) , d l ( p ) ) and

= (A2(p), b2(p), c2(p), d2(p)) be n-dimensional realizations of

8)/V(s)

and 7 ( s ) / 3 ( p , s) respectively. Define E c ( k ) to be the cascade interconnection of ,Us(k) with 271(k) as shown in Fig. 2.

U

=

Zl(k)

Zp

:

Y

Z2(k)

Fig. 2. Parameterized system •(k).

Fix k at any value p E IR2n+1. Observe that along any loop-response for which eT is identically zero, both u and y must go to zero since 271(P) is stable and ,U2(p) has a stable proper inverse. From this and the standing assumption

46

Morse

that ,Up is stabilizable and detectable it follows that for each p E IR2"+x, ,U(p) must be detectable through eT; s in other words, ~7 is tunable on 7~ = IR 2"+I . Let E* be a countable, dense subset of T*; clearly, 27 is also tunable on E*. Moreover, if in the linear space I~l×nP (~ ~npXnp (~ i~npxl, ,Up is sufficiently close to a system which is stabilizable, detectable and of McMillan degree not exceeding n, then there must exist a vector P0 E E* for which A(p0) is stable. Thus E* and ~7 will have what's required for adaptive stabilization as long as ,Up is close enough to a stabilizable, detectable system with McMillan degree no greater than n.O Before concluding this section, we illustrate by means of an example how the concept of tunability can be used in the analysis of an overall adaptive control system. Although in the example which follows, the tuner is of a very simple form, the reasoning can be extended, without major modification, to handle the most general gradient-like tuning algorithms which have been considered so far in the literature [1]. E x a m p l e 5: Consider the widely studied situation in which the tuning error eT in (5), is a scalar-valued identification error which can be reexpressed in the form eT = z~c(k - Po) + e, (16) where P0 is a fixed but unknown vector in 7) = IRnP and e is a linear combination of decaying exponentials (of. Remark 2, Sect. 3). Assume that A, B, and C are continuously differentiable on ~o, that `u(k) is tunable on 79, and that ,UT is a "gradient-like" tuner of the form k=--ZC~T,

(17)

where ~W is the "normalized" tuning error

eT

=

eT/(1 +

IIx¢ll =) =

eT -IIx¢ll=eT.

(18)

Because the overall parameter adaptive control system consisting of (5), (17) and (18) is a dynamical system with state (x, k) and locally Lipschitz vector field, for each initial state (x0, k0) there must exist an interval [0, T ) of maximal length on which a unique solution (x, k) existsfi Along such a solution, (16)-(18) can be used to derive the expressions

d (Ilk - p0112)/a

-- - 2 ( ( 1 + Ilxcll2)

-

_ --(1 + 211xCII2)

+

5 It is interesting to note that this will no longer be true if 271 and 272 are interchanged, unless ~Tp is restricted to be minimum phase. o Since there is no global existence theory for differential equations whose vector fields are only locally Lipschitz, it cannot logically be concluded that T = oo, unless (for example) it can first be established that (z, k) is bounded wherever it exists (cf.

[15]).

A Conceptual Framework

47

Rearranging terms and integrating one obtains

Ilk -

po[I2 +

(1 + 2llzcll2)~,~dt _< co,

where c 0 - Ilk(0) - p01l 2 + f o e2d " Since c0 < oo, it follows that on [0, T) k is bounded and ~T and ]z are square integrable; moreover, the boundedness of k together with (16) and (18) imply that ~T is bounded as well. Up to here the analysis is straightforward and completely standard. To get beyond this point and show that x is bounded has in the past been a challenging matter requiring intricate, lengthy arguments. It is exactly here where the concept of tunability proves its worth. To show that z is bounded, we need two technical facts:

1. Because of tunability, there exists a matrix H(p) which stabilizes A(p) + H(p)C(p) for each fixed p E P ; moreover, because C and A are continuously differentiable, H can be constructed (using a Riccati equation) to be continuously differentiable as well (cf. [1]). 2. The matrix A.(t) = A(k(t)) + H(k(t))(C(t) + [0, k(t)')]) is exponentially stable on [0,T); i.e., the state transmission matrix ~(t, r) of A.(t) satisfies Ilk(t, )11 ___cl 0< < t < T , where $ and cl are positive constants depending on H and co. This result, which depends only on A + HC being a continuously differentiable stability matrix, k being bounded and k being square integrable, is well-known and can easily be proved using a standard argument along the lines of [16,17,1]. Armed with these technical facts, a proof that z is bounded is at hand: All one has to do is to use the expression ~W = (C(k) + [0, k'])x, which is a consequence of (5), (17) and (18), to rewrite (5) as ~ = A.(t)x - H(k(t))'~w. Since A, is exponentially stable and H and ~T are bounded, x can be viewed as the output of an exponentially stable linear system with bounded input -HEw. Therefore x must be bounded wherever it exists. From this, the boundedness of k and the hypothesis that [0,T) is the existence interval of maximal length it follows that T = ¢x~ and thus that both x and k must exist and be bounded on [0, c¢). The reader may wish to compare the simplicity of this reasoning, with that used previously in [4,6,7,8] and elsewhere.O 3 Identifier-Based

Parameterized

Controllers

For the purposes of this paper, a parameterized controller ,Uc(k) of the "identitier-based" type consists of two subsystems, one an "identifier" 2Yi(ki) depending on an identification parameter vector ki, the other an 'Snternal regulator" ,Ua(k) depending on a control parameter vector k. Typically k = kl; in some cases however, k = [k~, k~]' where ka is a vector of additional internal regulator parameters (cf. Sect. 4). Depending on how ,~x and ,Ua are defined, ,Uc may be called either a "direct" or "indirect" controller.

48

Morse

Although not always explicitly stated, the starting point for the development of an identifier-based parameterized controller is typically the selection of an identifier parameter space :P! C ]RnP~ and a design model,UD(Pl) upon which the design of 2Yc is to be based. ~D(PI) is chosen to have the same control input and measured output value spaces as the unknown process model L'p to be controlled, and also so that the process model transfer matrix from u to y can be presumed to be "close" in some sense to the transfer matrix TD(s,pI) of 2~D(pI) for some unknown P I E ~i. 7 The reduced transfer matrices of ED(pi) as PI ranges over ~I, are the transfer matrices which ,UD(pI) matches. It is of considerable practical importance to develop adaptive stabilizers for design models whose classes of matchable transfer matrices are as "large" as possible.

L

rI

ZR(k)

I I

Y

ZI0~i)

f

I

I I

T

Fig. 3. Closed-loop system ,U using an identifier-based controller ,Uc.

In most applications, ,UD can be chosen to be of the form (CD(pI), AD + DD(PI)CD(PI), BD(pI)), where B D , . . . , DD are matrix-valued functions on 7~i and AD is a constant stability matrix. The stability of AD implies that 2YD is detectable on PI. On the other hand, stabilizability of ,UD on 7~i generally cannot be guaranteed unless special constraints are placed on ~! [16,17] or ~UD is nonlinearly parameterized [18]. This has important implications which are well known [3,5,6,7,8] and which will be discussed briefly in the sequel. 7 By exploiting the concept of a design model, it is possible to discuss adaptive control algorithms without being too specific about Sp ; this is of course consistent with the idea that a process model can never be known exactly.

A Conceptual Framework

49

Because of the particular structure of ,~D(PI), it is a routine m a t t e r to construct a matrix-valued function EI(pI) and constant matrices A h BI, and DI, with AI stable, so that the estimation equations

EI(w)AI = ADEx(r ) EI(PI)BI = BD(pX) EI(/~)DI = DD(PI)

Ple

7~I

(19)

Ple

:PI

(20)

hold. s These matrices together with the matrix Ci(PI) -~ CD(PI)E(pl) determine an identifier S,i(ki) of the form xI -'- AxzI +

DIy +

BI u

eI ----"CI(kI)gl -- y

~D : EI(kl)xI,

(21) (22)

(23)

where zi is the identifier's state, e] is an identification error, and zD is a "surrogate" design model state which is often used in constructing the feedback control to the process. 9 The estimation equations imply that if kl were held fixed at Pl and u and y were replaced by design system input up and output YD respectively, then 2YI would be a state estimator for the design system kD ~- (AD + DD(PI)CD(PI))ZD + BD(Pl)UD uo = c D ( r a ) - o ,

(24)

in that ZD would be given by ~D(t) + eAD'(ZD(0) -- ~D(0)). E x a m p l e 6: I n d i r e c t C o n t r o l S I S O D e s i g n M o d e l - Pick n > 0 and choose (c, A, b) to be any n-dimensional, controllable, observable, SISO system with A stable. For each pair pl,P2 E lRn, define PI = [P~,p~]', AD = A, BD(PI) = P2, CD(PI) = c and DD(PI) = Pl. Then with 79I = IR ~", 2YD is the n-dimensional design model described by the equations

= (A + p : ) Z D + p2uo YD -~" CXD.

(25)

s These matrices can always be chosen so that EI depends linearly on the columns of BD and DD and also so that each eigenvalue of AI is an eigenvalue of AD. One method for doing this can easily be obtained by generalizing the construction in Example 6. 9 Note that each identifier (21)-(23) uniquely determines a parameterized design model (q(PI), AI + DICI(PI), BI(pI)), called 27i's natural design model, for which (19) and (20) hold with EI = I. Although this design model's class of matchable transfer matrices is exactly the same as that of the design model which originally determined EI, the former can often have a state space dimension which is significantly larger than that of the latter.

50

Morse

This is one form of the parameterized SISO design model commonly used for indirect control. As defined, 27D is observable on 7~i and stabilizable at those values ofpi E 79I for which the transfer function of Z:D has no unstable pole-zero cancellations. In addition, ,UD can match any transfer function with McMillan degree not exceeding n. Define identifier matrices Ax = block diag {A, A},/3I = [0', b¢]', DI = [b', 0']'. It is easy to check that estimation equations (19) hold with

=

(26)

where S(p) = [M1Rp, M2Rp,..., M , Rp]', R is the inverse of the controllability matrix of (A, b), Mi is the transpose of the observability matrix of (e~, A) and ei is the i-th unit vector in IBn. A simple calculation shows that cS(p) = p ' Q ' ,

(27)

where Q is the parameter transformation matrix Q = NoR and N is the observability matrix of (c, A). Thus, for this design model, CI(PI) = [P~Q', p~Q'], which is linear in pI- E x a m p l e 7: D i r e c t C o n t r o l S I S O D e s i g n M o d e l - Let n and (c,A,b) be as in Example 6. Pick m > 0 and choose (A., b.) to be any single-input, mdimensional, controllable pair with A. stable. For each Pl,P~ E IBn and P3 E ! ! IBm , define Pl = [P~,P~,p3] ,BD(Pl) ----- [p~,b~,]',CD(pi) : [01×r~,P3], DD(Pl) :

[p~, Oa×m]', and A

0

AD= [b.cA.]" With :PI = IB2n+,~, ~UD(px) is the (n + m)-dimensional parameterized design model XD1 = AZD1 -b PlP3I x D2 q-p2uD ~D2 -" b, czD1 -I- A.XD2 -I- b.uD

(28)

I x

YD----P3 D2, which, as will be shown in the sequel, is appropriate for direct control. As defined, ,UD is detectable on 79I and stabilizable at those values of px E 79i for which the transfer function of 27D has no unstable pole-zero cancellations. The class 7" of transfer functions which 57D can match can easily be shown to contain all transfer functions of relative degree n* and McMillan degree not exceeding n, provided n ° < n and the characteristic polynomial of A. has a real factor of degree n ' . Thus, by picking m < n and choosing A. to have o n l y real eigenvalues, 7" will contain all strictly proper transfer functions with relative degree not exceeding m and McMillan degree not exceeding n. If m _< n + 1, 7- can also be shown to contain all strictly proper transfer functions of McMillan degree equal to m.

A Conceptual Framework

51

It is not difficult to verify that as an identifier 27i(kl) for this design model, one can use a system described by the equations

zx = A z l + by "22 -" A z 2 + bu

~. = A , x , + b,u fiI1 = A . H 1 + b.z~ [-I2 = A , H 2 + b,z~

(29)

~D2 = z. + H1Qkl + H2Qk2 el = k~(z. + H1Qkl + H2Qk~) - y, !

!

where S(.) and Q are as in Example 6, and ]~I = []¢1,]C2,k~]'. For a more motivated development of this type of identifier see [3]. Identifier equations (29) can of course be written in the form of (21)-(23), with a:i a column vector, using the definition a:x = [z~, z~, z.,' nl," ha,"..., h~,~]' where hj is the j - t h column of H ---- [H1,H2]; calculation of matrices A I , B I , D I and EI(pI) for which (19) holds is a straightforward, but tedious task. For example, EI(PI) is of the form EI(pI) =block diag {[S(pl), S(p2)], T(Ia ) } where T(pl) = [I, -e~ Qpl I, . . . , t - e ~ n Q p l I , - e ~ Q p 2 I , . . . , - e nQl~I]mx(2n+l)ra and el is the i-th unit vector in IR". The linear dependence of Ei(pi) on [P~,//2]' and CD(PI) on P3 imply that CI(pI), as defined in (20), is a bilinear function of [P~,/12]' and/~. R e m a r k 2: Although not central to the purpose of this paper, at this point it nevertheless seems logical to digress briefly and give an alternative formula for ex, which is of particular importance to the analysis of all identifier-based parameter adaptive control systems. For this, fix P0 E PI and let 270 = (Co,Ao, Bo) be an n0-dimensional, observable, stabilizable realization of the reduced transfer matrix of 27D(P0)- Let 120 be an open connected set of observable systems in lK"~xn° @ IR'~°xn° @ I~ '~°x"" which contains £70. The expression for ei given by the following theorem is the key relationship upon which the analyses of most tuning algorithms are based. I d e n t i f i c a t i o n E r r o r T h e o r e m : There exists an integer n > O, and analytic f u n c t i o n s A :I20 --*lRnXn,B : J2o ~ ] R n x n ' , c : 120 ~ l ~ n ' × n , a n d D :120 --* ~:~nU×n" with the following properties:

(i) spec (A(27)) C spec (AI)U {uncontrollable spectrum of £70}, VZ' E /20. (ii) C(Zo)eA(:~°)'[B(Zo),D(Eo)] = O, t >_ 0 (iii) g Ep no. the,, for each =p(O) C e ,rid each pieeewi, econtinuous input u: [0, oo) ---, IR"', el can be written as el = eMM 4" ~ 4- ~ ,

(30)

52

Morse

where eMM ----(CI(]¢I) - Ci(P0))=i e -- C(~'p)(eA(2p)t(Mp~vp(0) -4- Ui:vi(0))

-- f t C(~p)eA(2p)(t_r)(B(.~p)u(r ) + D(~p)y(7-))dr, and Mp and MI are constant matrices not depending on po.

A number of expressions for ei similar to (30) have been derived previously [19,20,21] and are by now reasonably well known. Equation (30) shows that if ,Up E K20,1° ei can be written as the sum of three terms, the first (eMM), called the "mismatch error", is due to parameter discrepancy, the second (e) accounts for nonzero initial process model and identifier states and the third (6) appears because of unmodeled dynamics. In view of property (i), e must decay to zero exponentially fast so long as ,Up E /-2o. Assuming a known upper bound on the real part of the uncontrollable spectrum of 2Y0, property (i) and the expression for ~ clearly make it possible to generate a '%ounding" signal fl (cf. [19]), using u and y, for which Ila(t)ll _< el(~P) "~ e2(2YP)IIfl(t)II, V ~ p E 1"~0;here c1(-) and e2(.) are positive-valued analytic functions on f20 which, because of property (iii), satisfy ei(2Y0) = 0, i = 1,2. Such bounding signals have been used to construct "dynamically normahzed" tuning errors in an effort to develop tuning algorithms capable of handling unmodeled dynamics [20,21]. The implications of (30) have not yet been fully exploited. The other subsystem of ,Uc(k), called an internal regulator 2~R(k), is often simply a state feedback law of the form UC ----FR(kl)~D -~- V R ( k I ) r

where FR(pI) and GR(pI) are matrix-valued functions designed to endow the closed-loop design model (CD(pI),AD + DD(pI)CD(pI) + BD(pI)FR(pI), BR(PI)GR(pI)) with some desired properties (e.g., internal stability) for each PI E :PI- More generally, ,UR(k) might be an nR-dimensional dynamic compensator of the form uc = CR(k)~R + GR(k)r + FR(k)~D + MR(k)y ~R = AR(k)~R + BR(k)r + DR(k)~D + NR(k)y,

(31)

l I I where k = [ki, kR] and kR is a vector of internal regulator parameters which takes values in some parameter space PR C nTt",a. In this case, internal regulator

lo Since ,Up will typically lie in many such subsets, each with a different value of po, there is nothing especially unique about the particular value ofpo appearing in (30), in relationship to ,Up.

A Conceptual Framework

53

matrices Crt, Grt . . . . . NR are chosen to give the closed.loop design model,UDd(p), consisting of the design model (24) in feedback with

UD : CR(p)zR -t- GR(p)r q- FR(p)ZD -I- Mrt(p)YD XR = AFt(p)zR + Brt(p)r + Dp.(p)zD + NR(p)YD,

(32)

desired properties for each p E ~ = PI x :PR- In the sequel it is assumed, without loss of generality, that (CR(p), Aa(p)) is a detectable pair for each p e 7~. In many situations, the tuning error eT is chosen to be the same as the identification error ei. There are nevertheless cases (to be discussed briefly later) in which it is useful to define eT in an alternative way. For the present, however, we assume eW = e l

(33)

and view an identifier-based parameterized controller ,Uc(k) to be as depicted in Fig. 3. The heuristic idea of designing an internal regulator to control a design model, and then using the regulator in conjunction with an identifier to control the process is sometimes called the "certainty equivalence principle." The resulting internal regulator 27R is accordingly called a certainty-equivalence control. Thus, an internal regulator is the same as a certainty equivalence controller; in the sequel the two terms are used interchangeably. 11 Our aim is to make clear, in algorithmically independent terms, one of the fundamental implications of certainty equivalence control. For this, set xo¢, = [zk, xb]' and define the closed-loop design model 27D°,(p) = (CD¢I(P), ADd(P), BD,,(p)) so that the equations

XD¢, = ADc,(P)XD¢I "k BD¢,(p)r = CD,,(p)xDo,

(34)

model the feedback interconnection of design model (24) with (32). ,UR is said to stabilize the design model ,Up at p if ,UD,t is internally stable at p; i.e., if AD,I(P) is a stability matrix at p. Let ~ be a nonempty subset of 7). A certainty equivalence controller £:R(k) is called a stabilizing controller on E if for each p = [p~,pk]' E £, ,UR(p) stabilizes design model 27D at p. We can now state the

I f 27R(k) is a certainty equivalence, stabilizing controller on ~, then 27(k) is tunable on E.

Certainty Equivalence Stabilization Theorem:

11 The definition of 27R in (31) might appear more in accordance with the intuitive idea of a certainty equivalence controner, if the terms involving ~D were not present. We've elected to include these terms since they actually do appear in the defining equations for many certainty equivalence controllers.

54

Morse

-]

L

~ R(p)

l

• o

I I

Pig. 4. Closed-loop design model ED,I.

The theorem makes clear one of the central implications of certainty equivalence control: If ~Urt is chosen to stabilize design model `UD on £ (cf. Fig. 4), and is then used as a component subsystem of ,Uc, the overall parameterized system ,U (cf. Fig. 3), consisting of 2Yc in feedback with 5Yp, is tunable on E. For this to be true it is not necessary for the process model ,Up to have a transfer matrix in the class of transfer matrices matchable by LYD. In this context the notion of "true process model parameters" is neither necessary nor relevant. It is usually not possible to make use of the preceding theorem to achieve tunability on all of Pl since 7~r typically contains points at which ~D is not stabilizable. Various ideas proposed in the literature might be used to deal with this problem. One is to use a tuning algorithm which restricts ki to a subset 75i of ~I on which loss of stabilizability does not occur [16,17]; since this in effect reduces ,UD'S parameter space to ~I, ~UD'S class of matchable transfer matrices is correspondingly reduced, typically by a large amount. Another idea [8,22] is to only stabilize 2Yr~ at points outside of some region T~ C :Pl which contains the points at which ,Up loses stabilizability; since tunability cannot usually be guaranteed at points inside of 7~, in order to avoid instability, some form of persistent excitation must be used to make certain that ki does not get stuck in ~ . A third approach might be to use a specially parameterized design model which does not lose stabilizability at any point in its parameter space [18]; this unfortunately leads to difficult tuner design problems for which there are at present no satisfactory solutions. A fourth approach, for the most part unexplored in the literature, is to make use of implicit tuning. This will be discussed briefly in Sect. 4. For the case when ,Up is "minimum phase", the problem we've been discussing can often be avoided all together. To understand why, we need a few more ideas. Recall that a constant, linear multivariable system ( C , A , B ) is m i n i m u m phase if its transfer matrix is left invertible and its transmission zeros 12 [23] are 12 For a system (C, A, B) with left-invertible transfer matrix T(s) = C ( s I - A) -I B, a complex number X is a transmission zero if there exists a nonzero vector [z', ul] ' such that C x --- 0 and ( h i - A)x + B u = O.

A Conceptual Framework

55

all stable; as defined, minimum phase systems are necessarily detectable. Call a matrix pair ( e , A) (or a system (C, A, B)) output stabilized if Ce At ---, 0 as t --* 00.13 27rt is said to output stabilize the design model 27D at p if 27Dc~ is output stabilized at p. Let g be a nonempty subset of :P. A certainty equivalence controller ~ R ( k ) is called an output stabilizing controller on £ if for each p -[p~, p~]' E E, ER(p) output stabilizes the design model ,Up at p. T h e following theorem provides the key technical link needed to explain how to achieve with indirect control that previously achieved in [4,6,7,8] and elsewhere with direct adaptive control.

C e r t a i n t y E q u i v a l e n c e O u t p u t S t a b i l i z a t i o n T h e o r e m : If 27p is minimum phase and 22R(k) is a certainty equivalence, output stabilizing controller on £, then E ( k ) is ~unable on E.

R e m a r k 3: The certainty equivalence output stabilization theorem does not require ~Up to have a transfer matrix which can be matched by 27D(pI) for some value of PZ E :PI- It is interesting to note, however, that in the case when exact matching is possible, to achieve tunability it is not necessary for the design model to be stabilizable, even at the "true design model parameter value" at which matching takes place! For example, the process model ~ = - 4 y T u is matched by the direct control design model ZD1 ---- --2ZD1 -b PlP3XD2 -I- p2UD, X D 2 - " --XD1 -- XD2 "b UD, YD : P3XD2, at p = [Pl,P2,P3Y -- [--6, 3, 1]' and for this value of p, the design model has an uncontrollable eigenvalue at s = 1. Since for obvious reasons a process model ,Up must always be assumed to be at least stabilizable and detectable, this example shows that a design model cannot necessarily be viewed merely as a process model with variable parameters.

R e m a r k 4: Both of the preceding theorems apply to the case when eT = ei. In order to facilitate the tuning of k it is sometimes helpful to use an alternative definition of ew. One possibility is ew ----[e l, e~t]' where e a = Cia(k)zR and C m ( k ) is some matrix-valued function of k; since detectability through el clearly implies detectability through eT, the preceding theorems apply to this case as well. Yet another possible definition of eT, useful in the case when AR is a constant stabihty matrix, DR = 0, and eT and eR are of the same size, is eT -- ex+eR; the preceding theorems are also applicable in this case since the stability of AR and the assumption DR = 0 insure that detectability through ei implies detectability through ew. This definition of ew finds application in model reference adaptive control as the following example illustrates. 13 Note that (C, A) is output stabilized if and only if each of its unstable modes is unobservable. On the other hand, (C, A) is detectable if and only if each of its unobservable modes is stable. It follows that (C, A) is both output stabilized and detectable if and only if A has no unstable modes; i.e., if and only if A is a stability matrix.

56

Morse

E x R m p l e 7 ( c o n t ' d ) : Observe that the control up = --CZD1 canses the o u t p u t YD of design model (28) to satisfy YD -- p3eA°tZDl(0) • Thus, the internal regulator matrix F a - I - c , 0] o u t p u t stabilizes `UD on IR 2'~+'~. Therefore, by the Certainty Equivalence O u t p u t Stabilization Theorem, if ,Up is minimum phase, the closed-loop parameterized system consisting of ,Up, identifier (29), internal regulator u -- --CXD1 - - - k ~ Q ' z 1 - k~Q'z~ (cf. (27)) and tuning error eT -- el is tunable on IR2'*+m. The simple dependence of this control law on ki is the principal virtue of direct control. T h e challenge in coming up with an overall adaptive control system based on this direct control design model, lies in the synthesis of an appropriate tuner since CI(kI) (i.e., the equation for ew) is nonlinear in ki. The tuning problem simplifies significantly if, for fixed n* < m, the characteristic polynomial of A. is chosen to have a real monic factor a . of degree n*, and P3 is constrained to be of the form P3 = p0c., P0 E IR, where c. is defined by the formula c . ( s I - A . ) - l b . = l / a . . In this case ,F,D([p~,p~,poc.]') has 2n ÷ 1 parameters and a class of matchable transfer functions containing only those of relative degree n* with McMillan degree not exceeding n. Moreover, e W "-- koC,(~, ÷ H 1 Q k l ÷ H 2 Q k 2 ) - y which is a form for which by now appropriate tuning algorithms are well known (e.g., see [17,19]). 14 To convert the preceding into a model reference algorithm, we need a reference model ETr which we define to be

XR ---~ A . z R ÷ b . r ,

(35)

and a modified control law which we take to be u a -- --CZD1 ÷ pRr. This control law o u t p u t stabilizes `UD on IR 2n+2 and results in the closed-loop transfer function POPR/a. from r to YD. Thus as before, the closed-loop parameterized system consisting of the minimum phase process model (2), the internal regulator u = --C~D1 + kR r -- -kll Q' Zl -- k'~W ~ ' Z2 + kRr (cf. (27)), the identifier (29) with modified identification error ei = k0c.(z, + Hi/el + H2t2) - y

(36)

and tuning error eT = el, is tunable on IR 2"+2. T o achieve model following it is necessary to change the definition of eT. To motivate the change, note from (35) that the ~racking error y - Yr can be written as Y -- Yr -- --(el ÷ eS) -~- ]¢0C*~ ,

(37)

where eR is the referenceparameter error ea -" (1- kOkR)Yr and ~ : z, + H I kl + H 2 ] g 2 -- ZRkR; moreover if k is differentiablethen by a simple calculation,

= A,~ ÷ HI]el ÷/-/~:2 ÷ ZRkR.

(38)

14 Things can be simplified somewhat by redefining kl and k2 to be Qk, and Qk2 respectively; for then, u = - k ~ z l - k~z2 and eT = koc,(z, + H l k l + H2k2) - y.

A Conceptual Framework

57

Equations (37) and (38) clearly show that if eT is defined to be eT -" [ei, eR]' then a zero tracking error will be attained asymptotically, along any system trajectory on which k becomes constant and eT goes to zero. Equations (37) and (38) also imply that this will still be true if eT is alternatively defined to be simply e1+eR. In this case, eT proves to be precisely the same as the "augmented error" originally introduced by Monopoli [24]. As noted in Remark 4, neither of these definitions of eT effect the tunability of the overall parameterized system E(k).O E x a m p l e 6 ( c o n t ' d ) : Simple examples show that it is impossible to output stabilize on lR 2n the indirect control design model ,UD of Example 6. One way to circumvent this problem is to modify the parameterization of ,Up. The price paid for this, however, is a significant decrease in the size of the class of transfer functions which the modified design model can match. The modified parameterization is constructed as follows: with c and A as before, choose a nonnegative integer m < n and a monic, stable polynomial a(s) of degree m; define vectors bl and b2 so that e ( s I - A ) - l b l = 1//3(s) and c ( s I - A ) - X b 2 = ot(s)/fl(s) respectively, where j3(s) is the characteristic polynomial of A. In place of (25), define the '~multiplicatively parameterized" indirect control design model SD(PZ) by the equations J:o = (A + poPlC)ZD -1- (b2 -'1"-Tp2)uD (39) YD = p O e Z D ,

where

T = [bx,Abt,... ,Am-%1],

(40)

t I ! Pl E IRa, P2 E IR a and Pt = [P0,Px,P2] • As defined, ,Up is detectable on :PI = IRn+m+l. However, unlike the previous indirect control design model which can match any transfer function of McMillan degree not exceeding n, the modified design model (39) can only match those transfer functions of McMillan degree with relative degrees exactly equal to n - m. This can be deduced using the identities

cAJ-lbi=O, j < n - m , cA'~-'~-tb, - 1, i E {1,2}

i6{1,2} (41)

which hold because the transfer functions 1//3(s) and a(s)/3(s) used to define bl and b2 respectively, both have relative degrees no smaller than n - m. What's especially important about the design model (39) is that it can be output stabilized on :PI by a continuously differentiable feedback function. P r o p o s i t i o n 1: Let T(s) be any monie, stable polynomial of degree n - m . f(PI)

-- --c~(A

+ pOPlC),

With (42)

the control law UD = f(pI)ZD output stabilizes the indirect control design model (39) on T'i.

58

Morse

Proposition 1 gives an explicit formula for a feedback function .f which output stabilizes the multiplicatively parameterized design model (39) for each PI E ~I. The n components of f are each polynomials in the n + 1 components of [P0, idl]'. It can easily be shown that ff f is defined by (42), then the closed-loop design model characteristic polynomial satisfies det ( s I - A - pople - (b2 + T1~)f(pi)) = a D ( s , p , ) 7 ( s ) ,

p E 79I,

(43)

where aD(s,pi) is the m-th degree, monic numerator polynomial of the transfer function c ( s I - A ) - l ( b 2 + T p 2 ) . It should be noted, however, that merely decreeing that f should satisfy (43) does not guarantee that f will satisfy (42) or that the closed-loop design model 27D¢~ will be output stabilized. In other words, the specification that f(pI) should assign to A + PoP1 + (b~ + T p s ) f ( p l ) the closedloop characteristic polynomial OtD(s,pI)7(s) does not pin down f enough for the purposes of indirect adaptive control. Proposition 1 implies that to achieve tunability on 79x, it's enough to choose f on T~I in accordance with (42) so that (43) holds, even though for some Pz E :PI,~D(S,pI) may be an unstable polynomial. By recognizing that Po~D(s,pi) is the numerator of the transfer function of open-loop, indirect control design model (39), it can be seen that what's being done here is a kind of "adaptive pole-zero cancellation" similar to what's often implicitly done with algorithms of the direct control type. To construct an indirect parameterized controller for adaptive model following, one can proceed from this point in essentially the same way as in the direct control case of Example 7. First, define an identifier for the multiplicatively pnrameterized design model (39) by modifying the identifier for the design model (25) in the obvious way; that is, by replacing c and ks in the identifier for (25) by koc and b9 + Tkg. respectively, where ks is an m-vector. What result are the equations ~1 = A z l + by 42 = Az2 + bu

= k0(klO'z + + = S(k )z + S(b2 + Tk2)z2,

-

y

(44)

where b, Q, and S(.) are as defined previously. Next, set 7 = a . and u -- f(kI)XD -I- k R r ,

(45)

where f is as defined in (42), ki = [k0, kl, k~]' e IR"+'~+1, krt is an additional internal regulator parameter to be tuned, and r is the input to the reference model (35). In view of the Certainty Equivalence Output Stabilization Theorem, the closed-loop parameterized system consisting of the SISO minimum phase process model ~ ' p , the parameterized controller (44),(45) and the tuning error eT = ei,

A Conceptual Framework

59

is tunable on IKn+m+1. To develop tracking error equations analogous to (37) and (38), firstuse (17),(35),and (44) to write Y -- Yr ~- --(ei -{- eR) -~- k0(eXD -- ]gRC, XR),

(46)

where, as before, eR = (1 -- kokR)Yr. Assume that k = [k{, kR]' is differentiable. Using the estimation equations (19) together with the definition of u in (45), one gets

~D = (A + kokle + (b2 + Tk2)f(ki)) "XD +(b2 + Tk2)kRr + S(kl)Zl + S(k2)z2 - koklex .

(47)

To proceed, we need the following result. L e m m a 1: Let T(s ) be as in Proposition 1 and write (~,A, b) for the unique con. troUable, observable realization of 1/7(s) with (A, b) in control canonical form.

If f(pI) satisfies (42) and C

e(A + PoP1c) M(pI) =

c(A + p0plc)"-"-I then for all Px E ]Rn+m+l, c = eM(pl)

M(pl)(b2 + Tp2) = M(pI)(A + poptc + (b2 + Tp~)f(pl)) = fi~M(pt).

(48) (49)

(50)

Now set 7(s) = a.(s), a.(s) being the characteristicpolynomial of A., and suppose without loss of generality,that (c.,A., b.) is in control canonical form; thus (~,A, b) = (c.,A., b.). Hence from (46) and (48), Y -- Yr : --(el + ez) + k0c,~,

(51)

where ~ = M ( k i ) ~ o - kRZR. Moreover from (35),(47),(49) .and (50) = A.~ % M(kI)~D % M(ki)(S(kl)Zl % S(k2)z2 - kokxex).

(52)

The imphcation of (51) and (52) is clear: If k is tuned in such a way t h a t k, ei and eR each tend to zero as t ---* oo, then the tracking y - Yr will tend to zero as well. Thus, an appropriate definition for ew would be eT = [eI, eR] I, just as in Example 7. It is interesting to note however, that unlike Example 7, zeroing the scalar tuning signal ew -- eI @ eR will not necessarily produce a zero tracking error because of the presence of ei in (52).0

60

Morse

4 Implicit Tuning As noted in Sect. 3, one of the things to be done in constructing a certainty equivalence controller for a given design model 2YD, is to select an internal regulator 2YR which causes the closed-loop design model 27Dd to have certain desirable properties. This task, which might be accomplished using established synthesis techniques from linear system theory (e.g., pole placement, model following, decoupling theory, linear quadratic optimal control, etc.) can often be reduced to finding a solution pR to some design equation of the form

(PI,PR) = 0,

(53)

where Pt E :Pl C 11~n'x is a vector of design model parameters, P a E Prt C lR2,a is a vector whose elements are elements of the coefficient matrices defining 2Ya and A is a "design" function mapping 11~"" ~IK",R into IR'*,R; (53) might arise from a Riccati equation, a pole placement equation or something similar. If (53) is solved explicitly for p a as a function of pt, then the resulting identifier-based controller's parameter space P is simply Pi and the only parameters to be tuned are the elements of kx. If, on the other hand, pl~ is taken to be a vector of additional regulator parameters, then the corresponding controller's parameter space :P is Px x :Prt and the parameters to be tuned are both the elements of k1 and those of kR; in this case the role of 2YT is to adjust the parameter vector k = [k~, k~]' to make both the tuning error ew and the design error eD = A(k,, ka)

(54)

simultaneously small. An identifier-based controller whose parameters are defined and adjusted in this manner is said to be implicitly tuned. 15 Implicit tuning algorithms can be classified as being of either the "one level" or "two level" type depending on which errors kl is being adjusted to make small. In the two level case, ki is adjusted to make just the identification error ex small whereas in the one level case k1 is tuned to make both the identification error e1 and the design error eD small. Two level tuning is thus more in line with the traditional idea of identifier-based adaptive control than is one level tuning and thus is perhaps more intuitively appealing. In addition, two level tuning allows one to use conventional identification algorithms to adjust kx whereas single level tuning does not. 16 In spite of this there are compelling reasons for also considering the concept of single level tuning. At the end of this section we provide an example in support of this view. The general idea of implicit tuning has been around in one form or another for a long time. The need for dynamically tuned regulator parameters is alluded to in [3]. Some of the reparameterization techniques developed in [26] can be 15 A more general concept of implicit tuning would allow the design function A(.) in (54) to also be a function of controller state z C. [21] is The design of a single level implicit tuner is a nonstandard identification problem for which there is apparently almost no literature even for the interesting, highly structured case in which C is linear in k and A is bilinear in ki and kR.

A Conceptual Framework

61

construed as a form of implicit tuning. T w o level implicit tuning is proposed in [27] in order to avoid some of the parametric nonlinearities which arise in connection with the multiplicative indirect control design model of Example 6. Single level implicit tuning is studied in [14] in a model reference context in an effort to achieve improved system performance. While there are no doubt many other examples of implicit tuning in the literature, the potential value of the concept does not appear to be widely appreciated. The aim of this section is to explain and illustrate by means of informal discussion and examples how the idea of implicit tuning fits into the framework of this paper and why it is likely to prove of importance to parameter adaptive control. In the sequel we briefly sketch how the concept of tunability can be used to analyze an implicitly tuned adaptive system. For this, assume that one has chosen a design model Z:D, a design function A and an internal regulator ~'R SO that for each value of the parameter p = [p~,idR]'in the set E =

:Z

(pl,pR) = 0},

(55)

the closed-loop design model 2YDd depicted in Fig. 4, is at least internally stable. Then, because of the Certainty Equivalence Stabilization Theorem, the closedloop parameterized system 57 shown in Fig. 3 and described by = A(k)~ + B(k)r e~ = c(k)x

(56) (57)

must be tunable on E. For purposes of analysis define the "extended tuning error" =

[ c(k) 1 [cE(k)J

(58)

where Cv.(k)z is a column representation of the elements of the outer product A(ki, kR)z'; in view of (54)

IICE(k)ll =

(59)

where npR is the size of PR- It is known (cf. the Tunability Lemma of [1]) that tunability of 27 on g implies tunability on P = PI x PR of the system described by (56)-(57) and the extended tuning error (58). Because of this and Proposition 1 of [1], we can state the following: O b s e r v a t i o n 1: There exists a matrix [H(p), HE(p)], depending on p as smoothly as C, A and A do, for which A(p) + U(p)C(p) + HE(p)CE(p) is a stability matrix for each p E 7). In the light of the preceding it is not difficult to see how to modify the analysis at the end of Sect. 1 in order to establish houndedness of x for an implicitly tuned system in which both eD and a normalized tuning error eT are simultaneously small. The following examples, which are of interest in their own right, serve to illustrate how this can be done.

Morse

62

E x a m p l e 8: T w o level implicit t u n i n g - Assume that 7~ = IRn'z (B IItn'a , that the closed-loop parameterized system (56)-(57) is tunable on E, that C and A are continuously differentiable and that eT can be reexpressed in the form e T = Wt(kI -- q) + e,

(60)

where w = Lz, L is a constant matrix, q is a constant vector, and c is a linear combination of decaying exponentials (ef. Remark 2, Sect. 3). Assume, in addition, that z~(pI,pa) has continuously differentiable first partials z~p~ = 0A/0pt and ,~pa = a A / 0 p a and that for each p = Lv~,p~]' E ~ , the smallest eigenvalue of the matrix A~aApR is no smaller than some positive number p; i.e., A~aApR >_PI,

(61)

Vp E 7~.

This last assumption is of course restrictive. Consider the simple two level implicit tuning algorithm defined by the equations

t~ = - - ~ T ~R = --A;a(~I, kR)eD ,

(62)

where ~T is the normalized tuning error

~T = e T / 0 + II~Jl2 + II~P~(kI, kR)~ll2) •

(63)

Let [0, T) denote the maximal interval of existence of the dynamical system defined by (54), (55), (56), (62), and (63), Using these equations it is easy to see that dt

Ilkl -- qll2 "I" lleDll 2

2 ( ( i + llwl] 2 -~-llAp,~[] 2) ~2 _ GeT)

- 2 (ebzapx~T + l[~p~eD[l 2) _< - -

I

((i + 2[I~I? + 211a~ll 2) ~ - ~ )

< _ ! ((~ + 2[I~I? + llap,~ll ~) ~ - ~ ) P

-.[kvl[ ~ •

Thus, by rearranging terms and integrating,

1 IIki -

qll 2 + llevll 2 +

)

(1 + 2[Iwll 2 + IlAp, wl[ 2) ~? + .llcDII 2 dr _< co,

where co = (~llkx(0)- qll2 + IleD(0)ll 2 + f ~ ~2dr). This shows that k[ and eD are bounded on [0,T) and that 6T, W~T, ]¢I, Apx~T and eD are in L2(T), the space of square integrable functions on [0, T). Moreover, since the determinant of Apa is a Jacobian for (53), assumption (61) and the implicit function theorem make it possible to express kR as a continuous function of ki and eD; since the latter are

A Conceptual Framework

63

bounded on [0, T) the former must also be. Note in addition that eT must be bounded on [0, T) because of (60) and (63). The utility of Observation 1 becomes apparent as we turn to the problem of establishing the boundedness of z on [0,T). As in Sect. 2, what we need to do first is to show that the matrix A . ( O -- A ( k ( t ) ) + H ( k ( t ) ) ( C ( k ( t ) ) + ~'(t)L) is exponentially stable on [0, T) where [H, He] is as described in Observation 1 and ~ is the £9(T) function ~ = - ( I + A~ /tp,)W~T. This can be done by first noting that A. can be rewritten as A, = A,, - HECE

(64)

,

where A** = A + H C + HBCE + H f ' L . Because of the pointwise stability of A + H C + H E C B noted in Observation 1, and the fact that ~ and k are in £ 2 ( T ) , it is easy to see that A** is exponentially stable on [0, T) (cf. [1,16,17]). Since (59) and eD E £2(T) imply t h a t H~CB 6 L:2(T), the same reasoning can be used again to conclude that A. as expressed in (64) is exponentially stable on [0, T). From this point on, the logic is exactly as in Sect. 2: T h a t is, using the expression ew = ( C ( k ) + t~'L)z, which is a consequence of (62) and (63), the differential equation for z in (56) can be rewritten as z = A . x + B r - HgT. From this and the boundedness of r and ew it follows that z is bounded on [0, T); and this together with the boundedness of k and the definition of T imply that T = oo and thus that (z, k) is bounded on [0, oo).O E x a m p l e 9: S i n g l e level i m p l i c i t t u n i n g - Let 2YD denote the two parameter indirect control design model

"}'plUD ZD2 : ZD1 -- (Pl "b p2)ZD2 + UD ZD1 -~----ZD2

(65)

YD ~ ZD2

defined on the parameter space PI = IR2. Note that ,UD'S transfer function is (s + p , ) / ( s 2 + (Pl + p2)s + 1) and that a pole-zero cancellation occurs whenever PiP2 - 1 = O. Define Pl = [Pl,P2]' and f(pI) = (Pl +P2 - 1 ) [ - p l , p , ( p l +P2) - 1]. A straightforward calculation shows t h a t the state feedback law UD = ( 1 ~ ( p i p 2 1))f(PI)ZD assigns to (65) the closed-loop characteristic polynomial s 2 + s + 1 for every value of PI 6 IR2 for which PiP2 - 1 does not vanish. This implies that with UD redefined to be UD = p 3 f ( P l ) Z D ,

(66)

the closed-loop characteristic polynomial of (65) will be s 2 + s + 1 along the algebraic surface in IR3 on which PZ(Plt~ - 1) - 1 = 0. While it is tempting to define A to be p3(plP2 - 1) - 1, this leads to a difficult tuner design problem. In order to avoid this difficulty, we shall "lift" the problem to ]1%4 by defining

-p2

+

'

(67)

64

Morse

where PR = [Pa, P41'- Since pa = 1~(pip2 - 1) for every value of p = [p~,P~t]' in the set = {[pl,pld' : A ( ~ , ~ ) = 0 } , (68) the feedback law (66) stabilizes design model (65) at each point p E £. Let E](ki) be an identifier for the design model (25), as described in Example 6. Then el can be written as el ---- w 2 ( 1 -- k l -- ks) q- Wakl -- y ,

(139)

where [Wl, w2, wa, w4]' = (block diag [Q, Q])' ~I, zl is the identifier's state, and Q is as in Example 6. In accordance with certainty equivalence, define the internal regulator state feedback law =

k3f(ki) o,

(70)

where XD is the identifier's surrogate state defined by (23). Let er = C(k)x

= A(k)x

(71)

denote the closed-loop parameterized system ,U(k) consisting of any fixed SISO model ~Up, ,UI, the tuning error eT = ei and the certainty equivalence controller (70). Since the closed-loop design model ,UD,,(p) described by (65)-(66) is stable for each p E E, by the Certainty Equivalence Stabilization Theorem, E(k) is tunable on g. Now assume that ,Up has a transfer function of the form (s + q l ) / ( s 2 + (q~ + q2)s + 1) where q = [ql, q2]¢ is in the set 7~- = {PI : PiP2 - 1 > 0, Pl < 0, P2 < 0}. Note t h a t ,Up is nonminimum phase and that 7Z- is an open set bounded by the algebraic surface on which the design model (65) loses controllability. In view of (69), the definition of eW and Remark 2 in Sect. 3, this assumption enables us to write ew in the familiar form

where e is a linear combination of decaying exponentials. The assumed structure of ,Up's transfer function also allows us to rewrite the design error eD

= A(kx, k~)

(73)

as

-k4

k2

-

q2

]c4

q4

(74) '

where

and [qa, q4] = [0,-1](Q~1) '. Note that QP is positive definite for each q E ~ - . This proves to be a consequence of 7~- being convex, a requirement which at present appears to preclude the generalization of these particular ideas.

A Conceptual Framework

65

The structure of eT in (72) and eD in (74) provides plenty of motivation for using a single level implicit tuner of the form

]CR " - - - e D ,

where ~---- [ w 3 - w 2 , - w 2 r , / ~ @ eD -----[k3eD~, b4eD2]j and eT is the normalized tuning error eT

~T

--

1 + II, ll

"

Note that, unlike Example 8, ki is being adjusted here to make both eT and eD small. We leave it to the interested reader to verify that on the overall system's maximal interval of existence [0, T), k, e D and ew are bounded and eD, eT and k are in /:~(T); moreover, just as in Example 8, the equation for z in (71) can be rewritten as ~ = A . ( t ) z - H~T where H is as in Observation 1, A° is the exponentially stable matrix A , ( t ) = A ( k ( t ) ) + H ( k ( t ) ) ( V ( k ( ~ ) ) + ~'L), = ]ci + kR Q eD and L is the constant matrix for which Lx = ~. All this of course implies, just as before, that the entire system state (z,k) exists and is bounded on [0, c~). As noted previously, the successful development of a stabilizing tuner for this example relies on q being in a convex subset, namely 7~-, whose boundary is determined by the algebraic surface on which the underlying design model loses controllability. Examination of this surface reveals that there is another subset, 7~+ -- {Pl : PiP2 - 1 > 0,pl > 0,p2 > 0}, with the same properties. It is easy to show that, by simply changing the signs of the diagonal terms in the square matrix in (67) which defines A, one can obtain a stabilizing adaptive control for the case when q E T~+. And to handle the case when q might be in either 7~+ or T~- one can use hysteresis switching, as is demonstrated in [28]. From the preceding it is clear that it is the detailed geometry of the algebraic surface on which an indirect control design model loses controllability, which is of importance to adaptive control. This example thus shows that if we cart learn how to implicitly tune on each of the open subsets of the parameter space bounded by this surface, whether or not they are convex, then we will finally have a reasonably complete solution, not relying on persistent excitation, to the long standing adaptive pole placement problem for SISO linear systems.(> R e m a r k 5: In the preceding examples, the problem of making both eT and eD simultaneously small is approached by exploiting the specific algebraic forms of both eT and eD viewed as functions of ki and kR. An alternative and highly innovative approach to this problem, which does not make use of the algebraic structure of eD, has been proposed in [29] for discrete-time systems. The idea can be explained as follows. Assume that 27D is the discrete-time version of the indirect control SISO design model of Example 6 and that 2Yi is its associated discrete-time identifier. Using an interesting idea due to Elliot [30], it is possible to construct a discrete-time internal regulator SI~ of the same form as (31), a design function A(pi,prt) and a row vector cC(~,pR) depending linearly on PR, with the following properties:

66

Morse

1. For any SISO process model ,Up, the closed-loop parameterized system consisting of ,Up, ,UI, ,UI~ and the tuning error eT = ei, is tunable on ¢ =

:

= 0}.

2. With UD defined by the discrete time analog of (32) and T(s,pI,l~) the ! I closed-loop design model transfer function from r to cc(p~,pu)[ZD, zrt ]I ,

T(s,p~,prt)=O .'. ,. [P'] Assume that standard gradient-like tuning algorithms, driven by normalized versions of tuning error eT and "control error" ec = ec(kl, kR)[~D, x~t]', are used to adjust k! and krt respectively. Since eT and ec depend linearly on kl and kR respectively, in the absence of unmodeled dynamics these algorithms will cause ew and ec to tend to zero. Property 2 above insures that as this occurs, [kx, .k~]' will approach ~: (or equivalently eD will approach zero), provided [~D, ~R]' is "sufficiently rich". The central contribution of [29] has been to show that [ ~ , ~t]' can be made to have this property by setting r equal to a suitably defined persistently exciting probing signal which is internally generated by an algorithm driven by the norm of eD! With the algorithm of [29] viewed from this perspective, it is not difficult to think of alternative algorithms, of the same type, for solving the problem under consideration. For example, since with the Elliot parameterization A also turns out to be linear in Pa, one might consider adjusting ka with a gradient-like tuning algorithm driven by eD rather than e¢. To carry out a stability analysis in this case, it would have to be shown that r is persistingly exciting enough to insure that ki is eventually driven into a region of the parameter space for which the Jacobian matrix Apa(ki , kR) of A remains nonsingular. For if this can be accomplished, and it is likely that it can be, then it would be easy to show that such an overall tuner would cause both ew and eD to go to zero, which because of property 1 above, is all that is required for internal stability.O Concluding

Remarks

The purpose of this paper has been to discuss some of the reasons why it is especially useful to think of a parameter adaptive control system as the feedback interconnection of a process 2~p and a parameterized controller ,U(k) whose parameter vector k is adjusted by a tuner `UT- The configuration is general enough to describe many different kinds of adaptive systems including those of the model reference, self-tuning and high-gain feedback types. While error models are not used in this setting, special emphasis is placed on the importance of a tuning error in the characterization of an adaptive system. This leads naturally to the concept of tunability - an idea of particular utility to adaptive control, as explained in Section 2. The central conclusion to be drawn from the discussion in Section 3 is that, subject to mild conditions, certainty equivalence implies tunability. Although the section's stated results hold for a large class of continuous and discrete-time

A Conceptual Framework

67

multivariable adaptive control systems, as presented they are not as general as they might be. For example, there are a large number of alternative identifier structures which could be substituted for (21)-(23) without changing the validity of the two theorems which follow. Among these are the identifier implicitly used in [3], identifiers in which ei is the output of a stationary linear system with a positive real transfer matrix whose input is of the form Ci(h~)a:i - y, and identifiers of the "equation error" [31] type in which zI is generated by an equation of the form r.I = (AI + D i C i ( k i ) z l + Bxu. In addition, there is good reason to believe that these results can be extended to slowly time-varying linear systems and even to some restricted classes of nonlinear systems, using, for example, ideas along the lines of [32]. In other words, the observation that certainty equivalence implies tunability, is probably as universal as the validity of the idea of certainty equivalence itself. The aims of Section 4 are to call attention to the concept of implicit tuning and to explain how the ideas of Section 3 might be used in the analysis of an implicitly tuned system. The section's main message is that many meaningful adaptive control problems can be reduced to nonstandard, but highly structured identification/tuner design problems in which the concepts of implicit tuning, certainty equivalence and tunability play prominent roles. Examples 8 and 9 describe two different implicit tuning methodologies. Each appears to have potential for dealing with the loss of controllability/stabilizability problem mentioned earlier, without having to resort to ad hoc methods. Clearly what's needed are new tuning or identification algorithms capable of handling the somewhat unusual combination of error equations which arise from implicit parameteriza. tions. In most cases the tuning error equation will be linear in kl; to achieve this may sometimes require a reparameterization along the lines of [26]. Usually A will be a continuous algebraic or polynomial function with the property that, for almost every value of pl, the design equation A(phpR) -- 0 has a unique solution PR; moreover, A will typically be of a form for which there is at least one well-known recursive algorithm for computing PR whenever it exists. With this much known, it would seem that significant advances in this direction in adaptive control are inevitable.

References 1. A.S. Morse, "Towards a unified theory of parameter adaptive control-tunability," IEEE Trans. Aut. Control, voL AC-35, pp. 1002-1012, Sept. 1990. 2. A. S. Morse, "Towards a unified theory of parameter adaptive control-Part 2: certainty equivalence and implicit tuning," IEEE Trans. Aut. Control, to appear. 3. A. Feuer and A. S. Morse, "Adaptive control of single-input single-output linear systems," IEEE Trans. Aut. Control, vol. AC-23, pp. 557-569, Aug. 1978. 4. A. S. Morse, "Global stability of parameter-adaptive control systems," IEEE Trans. Aut. Control, vol. AC-25, pp. 433-439, June 1980. 5. G.C. Goodwill and K. S. Sin, Adaptive Filtering Prediction and Control, PrenticeHall, Inc., Englewood Cliffs, NJ, 1984. 6. K. J. Astrom and B. Wittenmark, Adaptive Control, Addison-Wesley Publishing Co., 1989.

68

Morse

7. K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems, Prentice-Hall, Inc., Englewood Cliffs, N J, 1989. 8. S. S. Sastry and M. Bodson, Adaptive Control: Stability, Convergence and Robustness, Prentice-Hall, Inc., Englewood Cliffs, N J, 1989. 9. R. D. Nussbaum, "Some remarks on a conjecture in parameter adaptive control," Syst. Control Lett.,vol. 3, pp. 243-246, 1983. 10. B. Martensson, "The order of any stabilizing regulator is sumcient a priori information for adaptive stabilizing," Sgst. Control Left., vol. 6, pp. 87-91, 1985. 11. A. S. Morse, "A three dimensional universal controller for the adaptive stabilization of any strictly proper minimum-phase system with relative degree not exceeding two," IEEE Trans. Aut. Control, vol. AC-30, pp. 1188-1191, Dec. 1985. 12. J. C. Willems and C. I. Byrnes, =Global adaptive stabilization in the absence of information on the sign of the high-frequency gain,~ Lecture Notes in Control and Information Sciences, Proc. 6th Int. Con]. on Analysis and Optimization of Systems, Nice, June 1984, pp. 49-57. 13. D. E. Mi]Jer, Adaptive Control of Uncertain Systems, Doctoral Thesis, University of Toronto, Oct. 1989. 14. M. A. Duarte and K. S. Natendra, "Combined direct and indirect approach to adaptive control," Center for Systems Science, Yale University, Report No. 8711, Sept. 1987. 15. J. K. Hale, Ordinary Di~erential Equations, Wiley-Interscience, 1969. 16. G. Kreisselmeier, "An approach to stable indirect adaptive control," Automatica, vol. 21, pp. 425-431, July 1985. 17. K. H. Middieton, G. C. Goodwin, D. J. Hill and D. Q. Mayne, "Design Issues in Adaptive Control," IEEE Trans. Aut. Control, vol. 33, pp. 50-58, Jan. 1988. 18. F. Pait and A. S. Morse, "A smoothly parameterized family ofstabilizable, observable lineax systems containing realizations of all transfer functions of McMillan degree not exceeding n," IEEE Trans. Aut. Control, to appear. 19. P. Ioannou and J. Sun, "Theory and design of robust direct and indirect adaptivecontrol schemes," Int. J. Control, vol. 47, pp. 775-813, 1988. 20. B. Egaxdt, Stability of Adaptive Controllers, Springer-Verlag, Lecture Notes in Control and Information Sciences, vol. 20, 1979. 21. L. Praly, "Global stability of direct adaptive control schemes with respect to a group topology," Adaptive and Learning Systems, Plenum Press, 1986, pp. 10091014. 22. B. D. O. Anderson and R. M. Johnstone,"Global adaptive pole positioning," IEEE Trans. Aut. Control, vol. AC-30, pp. 11-22, Jan. 1985. 23. A. S. Morse, "Structural invariants of linear multivadable systems," SIAM J. Control, vo]. 11, pp. 446-465, Aug. 1973. 24. R. V. Monopoli, "Model reference adaptive control with an augmented error signal," IEEE Trans. Aut. Control, vol. AC-19, pp. 474-484, Oct. 1974. 25. L. Praly, =Towards a globally stable direct adaptive control scheme for not necessarily minimum phase systems," IEEE Trans. Aut. Control, vol. AC-29, pp. 946949, Oct. 1984. 26. S. Dasgupta and B. D. O. Anderson, "Physically based parametezizations for the design of adaptive controllers," Dept. of Systems Eng. Report, Australian National University, Canberra, 1986. 27. A. S. Morse, "Indirect adaptive control of processes satisfying the classical assumptions of direct adaptive control," Proc. 1988 Amer. Control Conf., Atlanta, GA.

A Conceptual Framework

69

28. A. S. Morse, D. Q. Mayne and G. C. Goodwin, "Identifier-based, switchedparameter algorithms for the adaptive stabilization ofllnear systems, ~ Prac. Sixth Yale Workshop on Adaptive and Learning Systems, New Haven, Aug. 1990. 29. G. Kreisselmeier and M. C. Smith, "St&hie adaptive resulation of arbitrary nthorder plants, ~ IEEE Trans. Aut. Control, vol. AC-31, pp. 299-305, April 1986. 30. H. Elliot, "Direct adaptive pole placement with application to nonminimum phase systems, = IEEE Trans. Aut. Control, vol. AC-27, pp. 720-722, June 1982. 31. C. R. Johnson, Jr., Lectures on Adaptive Parameter Estimation, Prentice-Hall, Inc., 1987. 32. G. Bastin and M. R. Gevers, "Stable adaptive observers for nonlinear time-varying systems, ~ IEEE Trans. Aut. Control, vol. AC-33, pp. 650-658, July 1988. 33. A. S. Morse and W. M. Wonham, "Status of non-interacting control," IEEE Trans. Aut. Control, vol. AC-16, pp. 568-581, Dec. 1971.

Robust Adaptive Control: Design, Analysis and Robustness Bounds* Petros Ioannou and Aniruddha Datta Department of Electrical Engineering-Systems University of Southern California Los Angeles, CA 90089-0781, USA.

A b s t r a c t . Despite the publication of numerous papers and several books dealing with the design, analysis and robustness properties of adaptive control, the theory of adaptive control may appear too technical and sometimes confusing to some people due to the so many different approaches, modifications and stability tools employed by various researchers in the field. The effort of this paper is to alleviate some of this confusion by presenting a procedure for designing and analyzing adaptive schemes. This procedure not only unifies most of the modifications used for stability and robustness but also clarifies why different approaches lead to the same result. The general framework that w e developed can be used not only to analyze and unify most of the adaptive control schemes proposed in the literature but also to generate new ones. In addition, the ana]ytical tools used allow the calculation of robustness margins and error bounds in a straightforward manner.

1 Introduction The p r o b l e m of designing stable adaptive controllers for a linear time-invariant system with no modeling errors was resolved as far back as 1980 [1-5]. Subsequently it was d e m o n s t r a t e d [5-7] t h a t in the presence of modeling errors and disturbances, an adaptive controller designed for the ideal situation, i.e., no modefing error or disturbances, could exhibit instability. Since then, for almost a decade, a considerable amount of effort has been directed towards the development of so-called "robust adaptive control schemes", i.e., adaptive control schemes which can retain certain stability properties in the presence of a wide class of modeling errors [8--19]. Due to these efforts, the 1980s have witnessed the development of a large number of robustness results in adaptive control. These results employ different robustness modifications and different methods of analysis and a p p e a r to be substantially different. However, in [13] it was shown t h a t there does exist an underlying theory which could be used to unify within a single framework some of the robust adaptive control schemes presented in the literature. Furthermore, the results in [13] revealed t h a t the key to a unified analysis of robust adaptive * This work is supported in part by the National Science Foundation under Grant DMC-8452002 and in part by General Motors Foundation.

72

Ioannou and Datta

control schemes lies in first analyzing the properties of the robust adaptive law independent of the controller structure to be used and then combining the two of them together using the Certainty Equivalence Approach [20]. As far as the analysis of the adaptive laws is concerned, the ones derived in [13] are all based on gradient error minimization techniques. Indeed, such adaptive laws have the useful property that the stability analysis in the ideal case, i.e., in the absence of unmodeled dynamics and disturbances, extends, after some modifications, to the case where unmodeled dynamics and disturbances are present. Thus, it is not surprising that most of the robustness results to be found in the adaptive control literature use gradient or Newton's techniques for the derivation of the adaptive law. Nevertheless, it is well known that some of the first stability results in continuous-time adaptive control [1,2,3] were established using a positivity and Lyapunov-type synthesis approach which led to the emergence of the celebrated Strictly Positive Real (SPK) condition in adaptive control. The SPIt condition was further exploited in [15] for the local analysis of adaptive systems using averaging techniques. In this paper we show that robust adaptive laws can be generated using not only gradient and Newton's techniques, but also by using the positivity and Lyapunov-type synthesis approach of [2] in a unified manner. One principal ingredient of this unification is the definition of the estimation error (known as prediction error in discrete-time algorithms) and the normalized estimation error, which clarifies the role of the artificial "augmented error" and "auxiliary" signal used in [2]. The robust adaptive laws developed in this paper can be combined, using the Certainty Equivalence Approach, with robust controller structures to yield robust adaptive control schemes. In particular, we consider the design and analysis of adaptive control schemes obtained from model reference, pole placement and linear quadratic (LQ) control structures, which are some of the most popular control structures in the adaptive control literature. The paper is organized as follows: Section 2 contains some mathematical preliminaries. In Section 3 we develop the theory for the design and analysis of adaptive laws. Section 4 contains a discussion of commonly used adaptive controller structures and their associated robustness properties. In particular, we consider and analyze a model reference, a pole placement and a linear quadratic controller structure. In Section 5 we design and analyze robust model reference, pole placement and linear quadratic adaptive controllers by combining the appropriate controller structures of the previous section with any of the robust adaptive laws of Section 3. We illustrate the design procedure and analysis using simple examples. In Section 6, we summarize our conclusions and outline the directions for future research.

2 M a t h e m a t i c a l Preliminaries In this section, we give some definitions and lemmas which will be used in the subsequent sections.

Robust Adaptive Control

73

D e f i n i t i o n 2.1 For any signal x : [0, oo) ~ z to the interval [0, t] and is defined as zt(r)=

JR", xt denotes the truncation of

{x(0r)otherwise.ifr n* for any arbitrary constant c~ > O. Furthermore, for large a

IIAx(,,,~)ll~_ 0. T h e normalizing effect of ns is more transparent when we express (3.13) in a state-space form and solve for the quasisteady state of •, i.e., = A c e + b¢(~bTff-- en~ -- ,7) • = hoe,

(3.14)

Robust Adaptive Control

85

where (he, A¢, b¢) is a minimal state representation of W ( s ) L ( s ) - hc(sI Ac)-tbc. Setting ~ -- 0 and solving for the quasi-steady-state response e.. of e we obtain - h c A c l b c ( ~ T ~ - tl) Clss 1 - hcA~lbcn~ 1 - hcA~lbcn 2 " Since W ( s ) L ( s ) is SPIt, hcA~lb¢ < 0 and therefore ess is equal to the normalized quasi-steady-state response elu of the estimation error ex. Due to n., e88 can only be driven unbounded by the parameter error ¢. In [2] Q is called the "augmented error" and en~ the "auxiliary signal" and they are developed using different considerations. Treating ex, e as the estimation and normalized estimation errors helps us to understand the separation of the identification and control parts and allows us to unify the various approaches based on other techniques such as gradient and least-squares methods. It should be noted that ¢ = 0 - 0" in (3.12)-(3.14) is an unknown signal vector and therefore the second equalities in (3.12) and (3.13) cannot be used to generate el, e. T h e signals el, e are generated from the first equalities in (3.12) and (3.13), whereas the second equalities are used for analysis only. The adaptive law for generating the parameter estimates 0(t) is developed by considering the Lyapunov-like function eWpce cw/'-l~b v(¢,~) = -'5-- + ----Y-- ' where F - - r T >

0 and Pc - p T

> 0. T h e matrix Pc is chosen as the one

which satisfies the algebraic equations P c A c + A T pc = _qqW _ uL (3.15) Pcbc = hc for some vector q, L = L T > 0 and a small constant u > 0 and whose existence is guaranteed by the S P K property of W ( s ) L ( s ) = h c ( s I - A c ) - l b c [2,10]. Using (3.15), the time derivative II of V along the solution of (3.14) is given by = --

--

e n s -- e~ + ~bTF-I~ •

Choosing = ~ = -r~(

- rw(t)0,

(3.16)

where w(t) > 0 is a signal to be designed, referred to as the leakage, we have ~'(~b,e) < - ~ eP

T

Le

--

~

~n s2 - -

e~

_w(t)~bT0

Using --~eTLe < - - ~ H 2, where ~ = 2Ao/llhcll 2 and )~o = VAmin(L)/4 > 0, we have

¢(¢, e)
0, it can be shown that

~(¢, ¢,e) --< -~o11~11= ~="~ 2 +

e¢~ + ep*¢T~ + ~1'71= +

+ L0*l ~ T r - l ~ , 7

where m 2 is as defined in (3.17). For (3.21)

= ¢ = - r e ~ sgnCP*) - r w ( O o , we have

2 2

? - -~°llell2

e 2-s

WlCp- wCTalP*l + 1~12 m2 "

(3.22)

The leakage terms w l , w >__ 0 are to be chosen so t h a t for V >__ V0 and some

constant Vo > 0, I¢" < 0, which guarantees that e, ¢, ¢ E Loo. A straightforward choice for w(0 is w(t) = or, where or > 0 is a constant referred to as the or-modification first used in [8] to improve robustness of adaptive schemes with respect to bounded disturbances and unmodeled dynamics. As indicated in [8], the or-modification achieves robustness at the expense of destroying the ideal properties of the schemes, i.e., in the absence of plant uncertainties or > 0 acts as a disturbance in the adaptive law which leads to non-zero estimation errors at steady state. This drawback of the or-modification motivated two new choices for w(t), the switching-or [9] and e-modification [10]. With the switching-or, w(t) = ors, where as > 0, when ]]011 is larger than some constant M0 > 110"[I, and a~ = 0 when I]0H < M0. As shown in [9], with this choice of w the ideal properties of the adaptive laws are maintained at the expense of knowing an upper bound for the unknown [10*H. The e-modification is given by w(f) = ]ern]vo and was first introduced in [10] with the rationale that in the absence of modeling errors w(0 will go to zero with the estimation error and therefore the ideal properties of the adaptive law will be guaranteed. Apparently this was not the case unless some persistently exciting conditions are used as shown in [10]. Simulations, however, indicate that with the e-modification the convergence properties of the adaptive law are improved [10]. The following theorem describes the stability properties of the adaptive law (3.21) for different choices of the leakage terms w(f) and wl(t).

Ioannou a n d D a t t a

88

T h e o r e m 3.1 The adaptive law (3.21) guarantees the following properties: (a) Fixed-a [8]: For w(t) = ~r, wx(t) = ~ , where o, ~1 > 0 are constants, we /lave ( 0 ,, o, p c Loo, 5 0 ,, ,n,, O, i' c s ( ~ + ~ + (b) Switching-~, [9]: For w(t) = o,, Wz(t) = ¢r1., where

~).

0 )m ([10[[ \Mo _ 1

er. =

ifllOll _< Mo ~ro if Mo < IlOll < 2Mo

~o

0)o

~x, =

- 1 ao

if llOll > 2Mo

ao

i f m l < Ipl ~ 2ml iflp[ > 2 m l ,

with the design constants no, a0, > 0, Mo > IIO*ll, ml > Ip*l, and m any finite positive integer, we have (i) ,, O, p e L ~ (ii) ,, , n , , O, ~ e S ( ~ ) .

(c) e - m o d i f i c a t i o n [10]: For w(t) = lemlvo, w z ( t ) = l e m l ~ l , wh~r~ vo, vz > 0 are design constants, we have (i) e, O, p e Loo (ii) e, en., O, k e S(vo + Vl + m-m~-~ ). (d) Ideal Case: For y = 0 and w = wz = 0 or w = cr,, wl = #1., we have (i) e, O, p C Loo (ii) e, en., O, b E L:.

The properties of the adaptive law (3.16) follow directly from those of Theorem 3.1 by setting p = p* = 1 and w l = 0.

Proof. (a) F o r w = ~, Wz = o l , we have -~xCp

- ~ ¢ T o __. --~a ( ¢ 2 _ I¢11P*I) -- ~ ( ~ T ¢ _ I1¢11110*ll) --

2--

Hence, (3.22) becomes

f" -< -:~°llel12

e2n~

2

,,12,,, - £211¢112+ ~,~1 P. 2 ÷ 2"11°'112 + m2' '72 ' 2

or

(3.23)

1]2

(3.24)

r~ V0 and some V0 > 0, V < 0 and therefore e, 0, p E Loo. From (3.27) we can show that e, en, E S(v0 + Vl + ~-~), which together with Ip[2 '< _ c¢2m 2 , ]0[ 2 < c : m ~ , - ~ = 2(8 + n~), imply t h a t b, 0 e S(v0 + Vl + ~-~). (d) For r} = 0, (3.22) becomes 2

f" ----~°llell2

E

2 n s

2

,.¢T01P*I- wlCp.

(3.28)

For w = a,, W 1 = als , or for w = wl = 0, we have w~)To ~ 0 and w l C p >_ O. Hence, P" _< 0 and V, e, 0, p E Loo. Since V is a nonincreasing function of time which is bounded from below, limt__.oo V(t) = Voo exists. Therefore, from (3.28) we have

A°HeH2 + T i.e., e, ¢ns, c, ~ ,

"~-wcT0]P*[ + ~/)I¢P d r < V(O) - Voo < ~ ,

~

I~12 _< e~2r~ 2,

e L2. Since for w = Wl = 0 2

]0

< c ~ . ~ ~ , m 2 = 2 (/~ + n ~ ) ,

it follows that p, 0 E L2. For w = ~rs, Wl *" O'ls, we have from (b) that _

which implies that ~, 0 E L2.

,

< c (lem[ 2 + O'sCT0) , 13

The use of the parametric model (3.11) and the S P R condition to derive robust adaptive laws whose properties are given by T h e o r e m 3.1 has not been done before, and that constitutes one of the new results given in the paper. In Sect. 5, we show how such adaptive laws can be combined with control laws to form robust adaptive control schemes.

Robust Adaptive Control

91

3.2 Gradient M e t h o d This method is based on the development of an algebraic error equation and the minimization of a certain cost function J(O, t) w.r.t, the estimated parameter 0 for each time t by using the steepest descent method as shown below: Since 0* is constant, we can rewrite (3.3) in the form Z -" 0 * T ¢ -']- 7 ,

where ¢ =

W(s)[w], 7} =

W(s)[y0].

The estimate £ of z at time t is given by = 0T~,

and the estimation error by e I ~-- z - z : 0T~-- z. The normalized estimation error e is constructed as el 7712

~~'oTt,--Z ~2 I

where m 2 = 1 + n !2 and ns is the normalizing signal designed so that (A1)

- - , - - y( E L o o . m

m

The adaptive law for updating 0 is derived by minimizing various cost functions of 0 w.r.t. 0. We start by considering the instantaneous cost

J(O,t) = '2m----'~22+ 2 0 T 0 - (0T¢--2rn2 z)2 + 2 0 T 0 ,

(3.29)

where w(t) >>0 is a design function which acts as a weighting. Using the gradient method we obtain = -FVJ(0), w h e r e / " = F T > 0 is a scaling matrix, and VJ(O) is the gradient vector of J(O,t) w.r.t. 0, i.e., V J(0) = e( + wO and

= - F e ( - FwO, 0(0) = 00.

(3.30)

The adaptive law (3.30) has the same form as (3.16) except that e and are defined in a different way. Similarly, we can derive an adaptive law for the bilinear model (3.4) by rewriting it in the form

Z = f (o'T¢ + zl) + where zl = W(s)[zo].

92

Ioannou and Datta

Then, the estimate ~ of z, the estimation error ~t and the normalized estimation error e can be constructed as follows: = p o T ( + pzl e1 = ~ , - - z ' - p * O T ( - l - ( p - - p * ) ~ + p * z

l-z,

~= oT(-f'z 1

£'-- z _ p*OT( + ( p - - p*)~ "4- p* z l -- z

m2

m2

where m s = 1 + n~ and ns is chosen so t h a t (AI')

(

,

m

~ m

,

r/

ELoo,

m

and p, 0 are as defined before. T h e cost function to be minimized w.r.t, p, 0 is given by J ( p , O , t ) = e2m~ 2 +-~P"

=

+ 2 Ip*lOwO

+ -2-

2m 2

+ 70wWoip, I,

where w, wl > 0 are design functions which act as weightings. Using the gradient method, we have b = -'rVJ(p)

= -Te~

-

7wig,

= - F a V J ( O ) = - / ~ x p * e ( - Ip*lFlWO, Since F1 is arbitrary, we can assume t h a t /"1 = ~

7 > 0 F1 = F~r > 0. for some other arbitrary

matrix F = F T > 0 and use it to get rid of the unknown p a r a m e t e r p* in the adaptive law for O, i.e., = - r e < sgn(p*) - t w o

(3.31)

= - T e ~ -- 7 w l p . T h e stability properties of (3.31) for various choices of w, wl are given by the following theorem. Theorem

3.2 The adaptive law (3.31) guarantees the following properties:

(a) Fixed-a: F o r w ( t ) = a, w l ( t ) = al, w h e r e a l , a > 0 are constants, we have

(i)~,en.,O,p,O,~L~o (ii)¢,~..,O,~S(a+al+~). (b) Switching-a: For w ( t ) = as, w l ( t ) = als, where as, a~s are as defined in T h e o r e m 3.1(b), we have

(i) ~, ..., o, p, d,/. e Loo (it) e, e.., O, ~ e s ( ~ ) . (c) e-modification: F o r w ( t ) = Icml~0, w l ( t ) = Icml~x, and for some constants vo, vl > O, we have

Robust Adaptive Control

93

(d) Ideal Case: For ~ = 0 and w = wx = O, or w(t) = a, and w~ = axe, we have

(i) e, en,, O, p, O, ]~ E n ~ (ii) e, en,, O, ]~ E n2. /n

If, in addition,

E L ~ , then •, •na, O, ]~ ~ 0 as t -+ oo.

17~ ~ D 2 ~ rt~ ~ D'~

Proof. We consider the positive definite function

v ( ¢ , ¢) = ¢2 + llP, lCWr_bl, '

(3.32)

which has the same form as that given by (3.20) with Pc = 0. The time derivative of V along the solution of (3.31) is given by

V" < --¢T 0 is a design constant, we have (i) e, ens, O, 0 • Loo (ii) e, ens, 0 • S(Vo + m~ ).

96

Ioannou and Datta

(d) Ideal Case: For ~ = O and w = 0 or w = ~,, we have (i) e, ens, O, O e L ~ (ii) e, en,, O e L=. If, in addition, --' ~ m ¢n' - E Loo, then e, en,, 0 --+ 0 as t -+ oo. m m Proof. (a) For w = a, we have

_ ~ T e < _~ (~T~ _ I1¢1l lle*ll)

Hence, (3.44) becomes

CTR~

'(i

tt:)=

Thus, as in the proof of Theorem 3.1(a), we can show that 3 Vo > 0 such that for V > V0, V < 0, which implies that V and hence ¢, 0, e, ens E Loo. Since R E L ~ , from (3.39) it follows that 0 E L ~ . Now integrating on both sides of (3.45), we have Vt, T > 0:

1 f- -t + T

_

7/2

_

+ ~t+T = V(t) - V(t + T)

~lle*ll=d,

le-'('-')~-dr]ds+e,I [f, J ft+T

+~

.It

s

2

m2

where we used the fact that m]~ E L ~ . Interchanging the order of integration in the second integral on the right hand side and using the boundedness of V, we obtain that X / T Y ~ , V~[[¢[] e ,q(~r + $-~). Now from (3.39) we have

d [¢TR¢] = 2¢TR ~ + cTR¢ a7 = -2¢,TR { t I R e + ,.] + r ~ o } + ¢~ [-ZR + ¢¢r] m=j ¢ = -2~TR_F'[R~ + r + o'O] + cT - f i r + m2 j

=--2~TR/~I R e -

Jof'~-ZO-~) ~l(r)((r) m2(r ) _ +a0]

~ l ~"

(3.46)

RobustAdaptiveControl

97

Also, ~bT( -- 17

ms

¢w¢ ==~ ~ m = - m

,7 m

2~(¢w¢)2 + 25

=I~ 6 2 m 2 < _

(3.47)

where we used the identity (a + b) 2 _2¢TRER¢ 4- 2¢TR/"

/ ' _~(,_,)~(r)¢(r)

--21rq~WR F o -- fl~W R ¢ 4-

mS(r ) dr

-

dt

e2m2

72

2

m ~"

Integrating both sides of the above inequality from t to t 4- T and making use of the fact that CTR¢[ t >_O, we obtain an inequality involving CWR~b[t+T. Using this inequality in (3.42) we obtain

f/(t + T)
0. Since R, ~ • Loo, ~ • S(a + ~), ~ S(tr + ---~), it follows that 0 E S ( a + m--~). This completes the proof of (a). (b) For w(t) = o'., we have

•

~ . ¢ Te = ~,(llall ~ - o ' T o ) > ~,llell(llOll - Mo + Mo - Ila*ll).

Since ~.(llOll - Mo) >_ 0 and Mo > [1O*ll, it follows that ~=~T0 >_ ~.llell(Mo -IIO*ll) _> o,

i.e,~ CTO '~dlOII __ 0, p - 1 may grow without bound; therefore, P may become arbitrarily small and slow down adaptation in some directions. This so-called covariance wind-up problem can be prevented by using various modifications which prevent P(t) from going to zero. One such modification is the so-called covariance resetting defined as follows:

=

p¢¢Tp TT~2

~

P('+) = Po = poI

(3.61)

where tr is the time for which Amin(P(t)) _< Pi and P0 _> Pl > 0 are some design scalars. Due to (3.61), P(t) >_piI Vt >_O.

102

Ioannou and Datta

When/~ > 0 in (3.59) the problem of P(t) becoming arbitrarily small in some directions does no longer exist, and (3.59) is usually referred to as the least squares with forgetting factor. In this case, however, P(t) may grow without bound due to tiP > 0 and to the fact that ( P ( ( T p ) / m 2 is only positive semidefinite. One way to avoid this complication is to modify (3.59) as p¢¢Wp /5 = tiP - ~-~ if IP(t)I < Ro (3.62) 0 otherwise, where P ( 0 ) = P0 = p T > 0, IP01 < R0 and Ro is a c o n s t a n t that serves as an upper bound for P. This modification guarantees that P E Loo and the adaptive law (3.58) with P given by (3.62) is referred to as the modified least-squares with forgetting factor. With either modification, the adaptive law (3.59) with P given by (3.61) or (3.62) behaves as a gradient algorithm with a time-varying gain, and is no longer the least-squares algorithm that we developed by setting V J(0) = 0. Despite its deficiency, the pure least-squares algorithm (3.58)-(3.59) has the unique property of guaranteeing parameter convergence to constant values in the case of ~/= 0 and w = 0 as described by the following theorem. T h e o r e m 3.4 The adaptive law (3.58) with ~ = O, w = O, and P given by (3.59% guarantees that

(i) O, e, ens, P E Loo, (ii) e, en,, ~ e L2, and (iii) limt_.oo O(t) = O, where 0 is a constant vector.

Proof. From (3.59) we have that t5 < 0, i.e., P(t) < Po. Since P(t) is nonincreasing and bounded from below (i.e., P = p r > 0 Vt > 0), it has a limit, i,e.,

lim P(t) = P ,

~--*Oo

where P = p T _> 0 is a constant matrix. Let us now consider ~t ( p - l ¢ ) = _ p - 1 / S p - l ¢ Since for ~/= 0, • = G

+ p-1 d =

e~.

= m'~' we have d (p-1¢) = 0 dt

i.e., P - l ( t ) ¢ ( t ) = Po1¢(0) and ¢?(tI = P(t)PoX¢(O). Therefore, limt-~oo ¢(t) = P P o l ¢ ( 0 ) and limt-~oo 0(t) = / s p ~ . ¢ ( 0 ) + 0 " = 0. Since P E Loo, it follows that ¢,0 E Loo, which together with -~-m E Loo imply that e, em, •ns E Loo. Let us now choose the function cTp-lcb cT(t)P0-" 1¢(0 ) v(¢) =

2

=

2

Robust Adaptive Control

103

Then, the time derivative r~ of V along the solution of (3.58) is given by ~r = _e(bW~ = _e2m~ __P - l ( t ) > P o l I .

(3.64)

104

Ioannou and Datta

In (3.62), IP(t)[ < Ro Vt > 0. Using the identity ~ p - 1 = have

_ p - l p p - 1 , we

go and switch adaptation off when em is small, i.e.,]~m I < go. In view of (3.70), we have o

=-F((e+g),

g----

> go

-e if[em[ go

(3.72)

171

- e if ]eml go(t)>_ (~), 0=

~ > 0 being arbitrary, 0

(3.76)

otherwise.

In view of (3.76), we have o = -r

[f

]

e - ~ c - T ) ~ ( t , ~-)¢(,-)d,- + g ,

(3.77)

where

f0 ,

= -

0 if II(e(t,r)m),ll~ e-a('-T)~(t,r)¢(r)dr otherwise.

> g0(t)

(3.78)

In order to avoid any implementation problems [27] which may arise due to the discontinuity in (3.77)-(3.78), the dead-zone function is made continuous as follows:

O= -F [fo' e-'('-r)c(t,r),(r)dr + g]

(3.79)

108

Ioannou and Datta

0

g=

if II(~(t, ~)m),lh~ > 2~](t)

if g0(t) < [[(e(t, r)rn),[[~ < 2g0(t)

,

fo e-~('-~)e(t,

(3.80)

if [lee(t, v)m)dl~ < go(t).

T)~(T)dT

The following theorem gives the stability properties of the adaptive laws (3.72), (3.73), (3.79), (3.80). T h e o r e m 3.6 (a) The adaptive laws (3.72), (3.79), (3.80)for the linearparamelric model guarantee: (i) e, ens, 0 ~ L ~ ,

(it) ,, ~ns e s(go + ~ ), (iii) 0 E L2 N L1, and (iv.) Flm,....,~ O = #. (b) The adaptive law (3.73) for the bilinear parametric model guarantees that: (i) e, en,, O, b e L ~ , (it) e, en, e S(g~ + m~ ), (iii) /J, 0 e L= n L1, and (iv) limt.-.~ O = 0, limt__,~ p = p. Here, 0 and ~ denote a constant vector and scalar respedively. Proof. We start with (3.72). We consider the function V = cTF-I¢

2 whose time derivative II along the solution of (3.72) is given by

? = _¢V;(~ + a) = --(~m 2 + ,)(~ + g)Now

')

(em + go)2 - (gO - m ~. (em + ( era2 + rl)(e + g) =

(era

--

( g0) 2 q- go + ~

(era

0

--

(3.81)

> 0 if em < go) > 0 if e m >

~go

go

if ]em[ < go.

(3.82) Hence, (era 2 + rl)(e + g) > 0 Vt and 1/ < 0, which implies that V, 0 E L ~ and ~f(em2-t - ~/)(e + g) E L2. Furthermore, 8 e L~o implies that ¢, e, enB E L ~ . From the adaptive law (3.72) we have ~T0 =

m 2 (e+g)2m2.

(3.83)

109

Robust Ad~ptive Control But

(era+go) ~ if era < -go

(~ + g)~m ~ = (~m + gin) ~ =

(era - go) ~ if ~m > - g o if I~ml ___go.

0

which, together with (3.82), implies that

0 _< (~ + g)2m2 _< (~m 2 + ~)(~ + g) ~

(~ + g)m e L~.

Hence, from (3.83) and ~ E L~o we have that/) E L2. Equation (3.81) can also be written as I

I

_< _~2~ + i~ml~~r n + I~lgo + I~_lgo, by using Ig] _ 0, the trajectory 0(0 , if it exists, will satisfy g(O(t)) = 0 Vt >_ 0. A scaled version of (3.93) can be obtained by using the change of coordinates F½~ = 0,

112

Ioannou and Datta

-vJ(a,)f...

~

~t(~,)

Fig. 3. Gradient projection.

where F ] is a positive definite matrix that satisfies/" = / ~ is a constant scaling matrix, i.e., 0 = Pr(/9) = - F ]

1

T F~and F = FT > 0

V g V g T ~1 ] I - F-~ V g T F V g l ~] F ~ V J .

(3.94)

The minimization problem (3.89) can now be extended to minimize J(/9) subject to g(/9) < O. The solution to (3.95) is

(3.95)

given as follows: -VJ

if/9 E CO , or if/9 E 6(C) and - v T j V g 0 is a small time delay in the feedback. The transfer function is y(s) = P ( s ) = 1 u(s) s + e- T ' - I

For low frequencies e - T ° -- 1 is O ( T ) . Therefore, for small T the nominal plant can be taken to be

P0(,) =

1 -

8

(4.36)

With Po(s) = 7, i however, P(s) cannot be expressed in the form (4.11) with a stable additive perturbation Aa(s ) or in the form (4.12) with a stable multiplicative perturbation Zlm(S). W e can express P(s), however, in the form (4.13) as follows: 1 P(s) =

where N o ( s ) -- • +x 1

s+ No(s) + s e -T° - 1 = D o ( s ) + A 2 ( s ) ' --.Jff - s+A s+A

Do(s) -

'

with A > 0 is a coprime factorization of

Po(s) = ~ and A1(s ) = 0, As(s) = e-Y#_~ ,+x are proper stable factor perturba-

tions. []

126

Ioannou and Datta

u

±

T

I

y(t),

I x-1 I I

=

Fig. 5. System for Example 4.2.2.

4.3 R o b u s t n e s s Analysis: K n o w n P a r A m e t e r s In this subsection, we consider the general feedback system of Fig. 4 subject to the plant perturbations discussed in the last subsection. Our goal is to derive conditions which, if satisfied by the perturbations, will guarantee closed-loop stability. The launching point for any robustness analysis is the stability of the nominal (i.e., unperturbed) plant-controller pair. Accordingly, we assume that in the absence of any perturbations each of the nine entries of H(s) is stable. Using this assumption, we can obtain the following robustness results. T h e o r e m 4.1 Let the nominal plant Po(s) be subjected to an additive per-

turbation Aa(s). Then, the closed-loop system of Fig. 4 with Po(s) replaced by Po(s) + A~(s) is Lp-stable provided that II[I + CIC2Po]-lClC;Aallir < 1 .a

(4.37)

Proof. The perturbed plant-controller configuration for this case is shown in Fig. 6. We first note that in the absence of As, we have a stable nominal closed loop consisting of only rational transfer function matrices. As a result, it follows that [24,29] the nominal feedback system is Lp-stable Vp E [1, oo]. Thus, the stability of the perturbed closed-loop system will be established once it is shown that r, dl, d2 E L~ implies that e2 E Lp. From Fig. 6, using (4.8), it follows that

e2 -- [I q- C1C2Po]-lClr - [I 4- C1C2Po]-Ic1C2d, q- [I q- C1C2Po]-X d~ - [ I + C1C Po]-iC1C2z . 2 .

z In this paper, H" lit, denotes the operator norm induced by the Lp norms on the input and output signals

Robust Adaptive Control

127

Y$2

d, ÷ *

|

÷

tl

y

Fig. 6. Perturbed plant-controller configuration (~dditive perturbation).

Truncating the signals, taking Lp norms on both sides and making use of Minkowski's Inequality [24], it follows that

I1~,11, ~ I1([z + cic~Po]-lc,,'),ll, +

+

I1([ + CxC, Po]-'d,),[I,

+

I1([1 + c,c=po]-*cic=a,),[l, I[([I

÷

c,c, Po]-lc, c,a,*~),ll~ •

Now from the Lp-stability of the nominal H(s), it follows that each of the operators [ I + C1C2Po]-ICx, [I+CIC2Po]-ICIC~, [I+C1C2Po]-x have finite induced Lp norms. Now, assuming that Aa is causal, we obtain

c1O=Po]-lCl(J=Ilip]ldl,llp + [l[x + c,c=Po]-I I]~pIId=,llp ÷ II[I + ClC~.Pol-'C,C=Ao[l~p ll~2,11p •

tle2,1I, _< I[[I +

ClC=Pol-lCliJiplit, lip +

11[I +

It now follows that if (4.37) holds, then

Ile2,11p ~ k, llr,[Ip ÷ k211d1,11p+ kalld2,11p for some kl, k2, lea > 0. Thus, r, dl, d2 6 Lp =~ e2 6 Lp. This completes the proof.

[]

A few remarks concerning Theorem 4.1 are now in order:

R e m a r k 4.1. It is well known that [[.l[.4[29]of any lineartime-invariantoperator provides an upper bound on its induced Lp norm Vp 6 [i,co]. Thus, if (4.37) holds with [[.[[ipreplaced by ][-[[.4,then the closed loop is guaranteed to be stable Vp 6 [1,co]. However, H" H.4 is difficultto compute and also the condition (4.37),with []-]llpreplaced by [[.H~, cannot be easilyverifiedin the frequency domain. []

128

Ioannou and Datta

R e m a r k 4.2. ff we choose p -- 2 in Theorem 4.1, then we obtain a condition which can be easily verified in the frequency domain, since (4.37) now becomes

II[_r+ CxC' P0]- Gc2 a(8)lloo < 1. This is precisely the popular "loop-gain-leas-than-one" stability result in robust control [32]. [] R e m a r k 4.3. In an effort to obtain a stability condition which can be readily checked in the frequency domain, we are now restricted to concluding only L2 stability. However, if (4.37) holds with I1" [lip replaced by the operator norm induced by I1('),11~ for some ~ > 0, then it can be shown as in [24] that the perturbed system is also Loo-stable, i.e., L2 stability with exponential weighting implies Leo stability. T h e o r e m 4.2 Let the nominal plant Po(s) be subjected to a muitiplicative per-

turbation Am(8). Then, the closed-loop system in Fig. 4 with Po(s) replaced by [I + Am(s)]P0(s) is np-stable provided that H[I + PoClC2]-XPoCxC~Amllip < 1.

(4.38)

Proof. The proof is very similar to that of Theorem 4.1 and is therefore omitted. [] R e m a r k 4.4. In many controller structures C2 --- I. Then, S ~ ( I + PoC1)-I is called the Sensitivity, while T ~- (I+PoC1)-lPoC1 is called the Complementary Sensitivity. Clearly S and T are related by S + T - I.

(4.39)

Now from (4.38), it is clear that the smaller T is, 4 the larger the robustness margin. In other words, a small T enhances robustness. On the other hand, from the (1,1) entry of H(s), it is clear that for good tracking performance S must be made small. Moreover, it is clear that a small S will lead to better output disturbance rejection. Thus, a smaller S enhances performance. Since S and T are constrained by the relationship (4.39), it follows that in any control design, there is a fundamental tradeoff between performance and robustness. H2 and Hoo optimal control [33] have emerged as powerful tools for achieving such a tradeoff. Now tracking high frequency signals is not very meaningful. Thus, for tracking purposes, S should be made small only in the low-frequency range. On the other hand, at high frequencies the plant is very poorly known so that it is desireable to have T small at high frequencies. These simple observations give the control engineer a qualitative idea of how to achieve such a tradeoff. Of course, the actual design may involve H~/Hoo controller synthesis techniques. [] 4 Here the size of T is measured by its H~

norm.

Robust Adaptive Control

129

T h e o r e m 4.3 £et the nominal plant Po(s) have a right coprime factorization (No(s), Do(s)) and let (Aa(s),A~(s)) be a stable factor perturbation of Po(s),

so that P(s) = [No(s) + Al(s)l[Do(s) + A2(s)] -1

(4.40)

represents the perturbed plant. Then, the feedback system of Fig. ~ with Pc(s) replaced by P(s) of (~.40) is Lp-stable provided that

[-[Do + GC No]-lClC2, [D0 + ClC Nol

zh

ip

Proof. The proof is essentially the same as that of Theorem 4.1 and is therefore omitted.

[]

Remarks 4.1, 4.2 and 4.3 apply to Theorems 4.2 and 4.3 also. 4.4 C o n t r o l S t r u c t u r e s f o r A d a p t i v e C o n t r o l In this subsection, we describe three control structures which have become very popular in the adaptive control literature. T h e y are the model reference control structure, the pole placement control structure and the linear quadratic control structure. The controllers themselves are designed on the assumption that there are no disturbances (i.e., dl = d2 = 0) or unmodeled dynamics. However, each of these control structures is shown to be a special case of the General Feedback System of Fig. 4 so that the robustness results of the last subsection apply to each one of them. We begin with the model reference control structure. M o d e l R e f e r e n c e C o n t r o l S t r u c t u r e . In model reference control, the objective is to stabihze the plant and to force its output y to track the output ym(t) of a reference model given by:

Zm(S) ~ks)

ym(8) ~-- W m ( 8 ) r ( 8 ) -: km'~'-7"7_~r(s)

(4.42)

for any bounded piecewise continuous reference signal r(t), where k m > 0 is a constant and Zm(S), Rm(s) are monic Hurwitz polynomials of degree mr, nr respectively. In order to achieve such an objective the following assumptions [2] are made about the modeled part of the plant transfer function

Zo(s) :

P0( ) = kp no(,)

(M1) Zo(s) is a monic Hurwitz polynomial of degree m < n - 1, (M2) Ro(s) is a monic polynomial of degree n, (M3) the relative degree of Win(s) is nr - mr = n - m, and nr _< n.

(4.43)

130

Ioannou and Datta

The MRC law is given by U = e 1 "a(s) ~U

y * a ( s ) 0"3Y + Co"r , + 2 A(s)

(4.44)

where a(s) = Is "-~, s"-3, . . . , 1]T, A(s) = Ao(s)Zm(S), Ao(s) is a monic IIurwitz polynomial of degree n - mr - 1 and 0~, i = 1,2,3 and c~ are the constant controller parameters to be determined so that the control objective is achieved for the modeled part of the plant Po(s). It can be shown [2] that there exist 0~, 0~, 0~, c0 such that the control objective is achieved for the nominal plant Po(s). Now with c;A(s) C , ( s ) = A(s) - O~ra(s) ' C~(s) =

O;r a(s) + O;A(s) c~A(s) '

it can be easily shown that the general feedback system of Fig. 4 reduces to the model reference control structure of (4.44). We use the following simple example to illustrate the design analysis and robustness of a model reference control scheme. E x a m p l e 4.4.1. Consider the plant 1 Y = s + a (1 + ZSm(S))U,

(4.45)

where Am(s) is a multiplicative plant uncertainty which is stable, and a is the parameter of the modeled part of the plant. The control objective is to choose u such that all the signals in the closedloop plant are bounded and y is as close as possible to the output Ym of the reference model b,~ Ym = S + a'm [r], (4.46) where bin, am > 0 are known, for any bounded reference input signal r(t). I f a is known, the MRC law (4.44) yields u = O~y + b,~r

(4.47)

with O~ = a - am, and this choice of u guarantees that the control objective is met for Am(s) = 0. When Am(s) ¢ 0, we have [s

+ am

-

0~Zlm(s)]y = bin(1 + Am(s))r,

so that the closed-loop characteristic equation is

(s + am) -- 0azamC,) = 0, or

am(S)

- 0 . 1 - 0~ m (s +- am)

Robust Adaptive Control

131

Since 0 ~ is a stable transfer function, it follows from the small gain theorem that a sufficient condition for the closed-loop system to be stable is that Am(s ) satisfy = s -I- a m

co

s -t- a m

I1~

< 1

(4.48) '

which agrees with the robustness condition (4.38). Furthermore, the tracking error

A er ~-

y

--

Yrn satisfies

b.~

(s + am + 0~)

a

,,o~r~.l

er - - (8 -I- am) (S + % = = 0~Z~rn---($))- - r n ' ° / t ' J "

O The same example can tor perturbations without also involves the transfer so that a - am is as small objective.

be analyzed in the presence of additive and stable facany significant changes. Condition (4.48) for stability function of the reference model. One can choose am as possible depending on the flexibility of the control

Pole Placement Control Structure. In pole placement control,the objective isto stabilizethe plant Po(s)by assigning the poles of the closed-loop plant to the roots of some desired polynomial. This stabilization objective can be extended to tackle the tracking problem for a special class of reference input signals by using the so-called internal model principle. Suppose the signal ym(t) to be tracked satisfies:

O(p)ym(t) -~ O,

(4.49) ,x

where Q(p) is a fixed known monic polynomial of the differential operator p = d ( . ) of degree q0 with non-repeated roots on the jw axis. In pole placement control, we seek to synthesize the control input u(t) which places the closed-loop poles of the system at the roots of a given Hurwitz polynomial A*(s) of degree 2n + q0 - 1 and guarantees that the output y(t) tracks ym(t). Q(s) is called the in~ernai model of ym(t). In order to achieve the pole placement control objective, we make the following assumptions about the modeled part of the plant transfer function Po(s) =

Zo(s)/Ro(S): (P1) Ro(s) = s n + Ro(s, 07) is a monic polynomial of degree n, (P2) Zo(s) = 20(s,0~) is a polynomial of degree < n - 1, and (P3) Q(s)Ro(s) and Zo(S) are coprime.

Here 0F, 0~ are vectors whose elements are the coefficients of ~0(s), spectively. The pole placement control law is given by [10,20] u = [Al(s) - P(s)Q(s)],l-~_,[u] + L(s),l'~_,(ym - y), Ill(S) /.tits )

Zo(s)

re-

(4.50)

132

Ioannou and Datta

where L(s), P(s) are polynomials of degree qc + n - 1 and n - 1 respectively, obtained by solving the Bezout equation

P(s)Q(s)Ro(s) + L(s)Zo(S) = A*(s),

(4.51)

where q0 is the degree of Q(s), Al(s) is a Hurwitz polynomial of degree n + q0 - 1, and - A*(s) is the desired closed-loop stable polynomial of degree 2n + q0 - 1. -

-

The coprimeness of Q(s)Ro(s) and Zo(s) guarantees that (4.51) has a unique solution [30]. With Cl(s) = L(s)/P(s)Q(s), C~(s) = 1 it can be easily shown that the general feedback system of Fig. 4 reduces to the pole placement control structure. From (4.50),

P(s)Q(s)u = L(s)(ym - y) P(s)Q(s)Zo(s)u = L(s)Zo(s)(ym - y). Using Zo(s)u = Ro(s)y, we obtain

[P(s)Q(s)Ro(8) + L(s)Zo(s)]v = L(s)Zo(s)ym. Using (4.51)

Y=

L(s)Zo(s) [A*(s) - P(s)Q(s)Ro(s)] Yr,, A*(s) Ym= a*(8)

i.e.,

[P(8)R0(,)] [Q(,)Vm] • Y - Ym -- - [ A*(8) J Since Q(s)ym = 0, it follows that the tracking error er = y - Ym will converge to zero exponentially with time. Since no cancellation of unstable polynomials took place, the closed-loop plant is also internally stable. In order to implement (4.50)-(4.51), however, we need to know R0(s) and Zo(s), i.e., the parameters 0~, 0~ of the plant. Let us now use a simple example to illustrate the design, analysis and robustness of a pole placement control scheme. E x a m p l e 4.4.2. Consider the plant

v=

b

(1 + Am(S))u,

(4.52)

where Am(s) is a multiplicative plant uncertainty which is stable, and a, b are the parameters of the modeled part of the plant. The control objective is to choose u such that the poles of the closed-loop system are placed at the roots of a given Hurwitz polynomial, say A* (s) = (s +

Robust Adaptive Control

133

1)2, and the output y(t) tracks a unit step input u,(t). Thus, here ym(t) -- u,(t) and Q(s) = s. The control law

Ax u = TTE[u]

lls + 12, + ~LYm

- Y],

where 11, 12 satisfy

s(s + a) + (lls + 12)b = (s + 1) 2 ,

i.e.,

2-a It=--T-,

1 12 = -~ ,

and A1 > 0 is a design constant, can be shown to meet the control objective when Am(s) = 0. When Am(s) ¢ 0, the closed-loop plant is given by

b(1 + Am(s))(ils + t2) Y - [(s + 1) 2 + bAm(s)(lls "~i2)] [Ym]" Therefore, for (s + 1) 2

~o

< 1,

(4.53)

the closed-loop plant is stable. Furthermore, the tracking error er -" y - Ym is given by er

s+a 1 s+a [(s + 1) 2 + bAm(s)qls + 12)1s[ymj = [(s + 1) 2 + bAm(S)q,s + :2)1 [0]'

i.e., if (4.53) is satisfied er(t) converges to zero exponentially fast.

O

The same result can be established in the presence of additive and stable factor perturbations. L i n e a r Q u a d r a t i c C o n t r o l S t r u c t u r e . In this case the objective is to minimize a particular cost criterion while stabilizing the plant and forcing its output y(t) to track the reference signal ym(t), which satisfies Q(p)ym(0 - 0 (where A

p = ~ ( . ) ) for some monic polynomial Q(p) of degree q0 with no repeated roots on the jw axis. To obtain a solution to this problem, we make the following assumptions about the modeled part of the plant transfer function Po(s) = Zo(s)/Ro(s): (L1) Ro(s) = s n + flRo(s,O~) and Zo(s) = 2o(s,O;) are coprime polynomials of degree rn and n respectively, with m < n, and (L2) Zo(s) and Q(s) do not have any common zeros on the jw axis. Here, 0~ and 0~ are vectors with the coefficients of/~0(s), Zo(s) respectively. The tracking problem is first transformed into a regulator problem as follows. In Fig. 4, let C2(s) = 1. Then - e 0 = y - Ym, which can be rewritten as

Zo(s)

--co -" - ~ - - ~ u - - Ym :=~ --R0(s)e0 "- Zo(s)u - Ro(s)ym.

Ioannou and Datta

134

Filtering each side of the above equation by Q(s)/Ql(s), where Ql(s) is an arbitrary monic Hurwitz polynomial of degree q0, and using the fact that Q(p)ym(t) - 0, we obtain eo = P~(s)fi, (4.54) -

where

P~(8) = z0(8) QI(s) ~ = Q(~) u Ro(O Q ( O ' Q-~ " The LQ controller is designed by minimizing the cost criterion

J(~) =

(4.55)

[eg(O + ~ ~(01 d~,

where ~ > 0 is the control weighting. Let A, b, C be a minimal state-space realization of Pg(s) in the observable canonical form. Then,

----

,

',o

--

02,

(4.56)

where 0~, 0~ are the coefficient vectors of [sn -{- Ro(s,O~)]Q(s) - s '~+q° and Zo( s, O~)Ql ( s ) respectively, and C = [10--- 0].

(4.57)

From standard LQR theory for the infinite interval problem [34], it follows that the optimal control fi is given by

f~- -Ge,

G-

1---bwp, )~

(4.58)

where e is the state vector corresponding to the realization (A,b, C) and P satisfies the Algebraic Ricatti Equation

AT p -t- P A -}- c T

c -

.1----PbbTp = O .

(4.59)

When the state vector e is not available, it is replaced in (4.58 / by its estimate obtained from the following state observer:

= A~ + bf, + k(C~ + e0),

(4.60)

where k is chosen so that A + kC is a stable matrix. Using standard LQG theory [34], it is possible to obtain an optimal choice for k which involves the solution of a dual Ricatti equation. However, any k that makes A + k C a stable matrix is an acceptable choice. Thus, k is chosen to satisfy A + kC = A*,

(4.61)

Robust Adaptive Control

135

where A* is a stable matrix. Without loss of generality, we assume t h a t A* has the following form

[,

A=

-a*

[ I"+~°-1

I I

(4.62)

0

so that (4.61) is guaranteed to have a solution for k. The control u(~) is thus given by 1 Qx(s)

= Q---~.,

~ =-a~,

a = _~Tp,

(4.63)

where ~ is obtained from (4.60) and P is the unique nonnegative definite solution of (4.59). With

Cl(S)

-~G(sz-

A - kC + bG)-lk,

C2(s) = 1,

it can be easily shown that the general feedback system of Fig. 4 reduces to the linear quadratic control structure. Let us now use the same example as in the pole placement case to illustrate the design, analysis and robustness of an LQ control scheme. E x a m p l e 4.4.3. Consider the plant

bl

Y -- S + a (1 + Am(S))u'

(4.64)

where Am(S) is a multiplicative plant uncertainty which is stable, and a, bl are the parameters of the modeled part of the plant. The control objective is to choose u to stabilize the plant in a way so as to minimize a particular performance index while making the output y ( t ) track a unit step input u , ( t ) . Thus, here ym(t) = u,(t) and Q ( s ) = s. Choosing Qx(s) = s + q, q > 0 being arbitrary we can convert the tracking problem for (4.64) into an equivalent regulation problem for

co-

(8--~-q)bl s(s+a)

[l_[_Z~m(8)] ~

(4.65)

where z~ eo - - Y m - - Y

= ~[u].

s+q

Equation (4.65) can be realized in the state-space form as = Ae -{- b(~ J¢- Z~m(S)~) e 0 "- - C e

,

(4.66)

136

Ioannou and Datta

where

A=

[-olo]

, b=

blq

'

The LQ control law

f~ = - G e

G =

1--~-bwp

'

~Ul

(4.67)

'

where P is the unique non negative definite solution of the Algebraic l:ticatti Equation (ARE)

A T p + P A + c T c -- 1-.~-PbbWp= 0,

(4.68)

can be shown to meet the control objective of minimizing the cost criterion fo[e0~(t) + Atofi2(t)]dt in the ideal case, i.e., for Am(s ) = 0. Since the state vector e is not available, it is replaced in (4.67) by its estimate obtained from the state observer

= A~ + bfi + k(C~ + e0),

(4.69)

where k = [a - a t , - a 2 ] T and a l , ag. > 0 are chosen such that s 2 + a l s + as is a Hdrwitz polynomial. To summarize, the control input u for (4.64) is given by u =

s+q

[a],

G = 1-LbTp,

fi=-G~

S

'

(4.70)

'~W

where ~ is obtained from (4.69) and P is the unique nonnegative definite solution of (4.68). Equation (4.70) along with (4.68), (4.69) guarantee that the LQ control objective is met when Am(s) = 0. When Am(s) # 0, we analyze the closed-loop system as follows: From (4.69), we have = [sl - (A + kC)] -1 [ba + ke0].

Substituting this expression in (4.70) we obtain a = - G [sI - (A + kC)] -~ [ba + keo], or

[1 + G[sI - (A + kC)]-lb] ~ = - G [sI - (A + kC)] -1 keo. Combining the above expression with (4.65), we obtain [1 + G ( s I - (A + kC))-lb]eo -

(s + q)bx [1 + Am(s)]V[sI - (A + k C ) ] - l k e 0 , s ( s + a)

or

{s(s + a)[1 + a ( s I - (A + kC))-Ib] - (s + q)blG[SI - (A + k C ) ] - l k --(s T q)blAm(s)G[sI - (A + k C ) ] - l k } e0 = 0. (4.71)

Robust Adaptive Control

137

Using the small gain theorem as in the earlier examples, we conclude that the closed-loop system is stable provided that

Am(s) s(s -I- a)[1 -I- G(sI

-

-

(A "-}-kC))-lb] - (s + q)blG[sI - (A + kC)]-lk < 1,

oo

(4.72)

which agrees with the robustness condition (4.38). Also, from (4.71) and the A

stability of the closed loop it is clear that the tracking error er -" y - Ym --e0 --+ 0 exponentially fast. I:] The same result can be established in the presence of additive and stable factor perturbations.

5 Robust

Adaptive

Control

Schemes

In Sect. 4.4, we have considered the design of three particular robust controller structures. However, when the parameters of the modeled part of the plant are unknown, the control input u cannot be calculated and therefore none of these controller structures can be implemented. A natural approazh to follow in the unknown parameter case is to use the same control law as in the known parameter case, replacing the unknown controller parameters with their estimates obtained by using the identification techniques of Sect. 3. This approach is called Certainty Equivalence and has been widely used in the design of adaptive control schemes. The way the controller parameters are estimated led to two different classes of adaptive control schemes, called direct adaptive control and indirect adaptive control, which are treated in the following subsections. 5.1 D i r e c t R o b u s t A d a p t i v e C o n t r o l The appropriate control input u that will force the unknown plant to meet the control objective is obtained by using the following steps:

Step 1: We derive the control law that can meet the control objective when the plant parameters are unknown. This step demonstrates that there is sufficient structural flexibility in the closed-loop plant which allows us to meet the control objective. Step 2: We use the same control law as in Step 1 but with the controller parameters repla~ed by their estimates generated by an adaptive law. The adaptive law is designed by first parametrizing the plant in terms of the unknown desired controller parameters and then using any of the methods given in Sect. 3 to derive the adaptive law. Step 3: We analyze the adaptive control scheme formed in Step 2 and show that it meets the control objective.

138

Ioannou and Datta

The adaptive control scheme developed as in Step 2 is referred to as direct adaptive control for the simple reason that the controller parameters are estimated directly without any exphcit information about the plant parameters. The derivation of the adaptive law in Step 2 assumes that the plant equation can be parametrized in terms of the unknown controller parameter vector 0", i.e., the unknown plant parameters are expressed in terms of 0" and no longer appear in the plant equation. In order to eliminate the unknown plant parameters and express the plant equation in terms of 0* and in the form of the parametric models considered in Sect. 3, we have to cancel the unknown zeros of the modeled part of the plant by filtering. This cancellation restricts the modeled part of the plant to be minimum-phase, i.e., to have stable zeros. As a result, the only class of direct adaptive control for which a complete stability analysis exists, is the one based on the model reference approach [2,9,10,11,13,16,20], which requires the modeled part of the plant to have stable zeros. We illustrate the design and analysis of model reference adaptive control (MRAC) using a simple example. E x a m p l e 5.1.1. Consider the plant 1 Y = s + a (1 + Am(s))u'

(5.1)

where a is an unknown constant and Am(s) is a multiplicative plant uncertainty which is assumed to be analytic in Re[s] > 0. The control objective is to choose the input u(t) such that all the signals in the closed-loop are bounded and the output y tracks the output of the reference model

ym = - - [ d ,

(5.2)

s -4- a m

where am > 0, bm are known parameters, as closely as possible for any bounded reference input signal r(t). We design the MRAC scheme by following Steps 1-3. Step 1: As we have shown in Sect.4.4 the control law * U=8oY+bmr,

00* - - a - a m ,

(5.3)

meets the control objective exactly when Am(s ) -- 0. Furthermore, when Z~m(S) ¢ 0, the control law (5.3) guarantees that for all Am(S) that satisfy

] 0 ; A m f__(6) $ "1- am vo

.

.

](a--am)Am(s)l . . . S 3t- u m

vo

< 1

'

(5.4)

we have signal boundedness, and the tracking error er = y - Ym satisfies er = (s + am) (* + "m -- e

Am(s))

m,

Step 2: When a is unknown (5.3) cannot be implemented. Therefore, instead of (5.3) we propose the control law = oo(t)y + bm ,

(5.5)

Robust Adaptive Control

139

where Oo(t) is the estimate of 8~ at time t. The adaptive law for generating 00 is developed as follows: W e substitute a = 0~ + am in the plant equation to obtain (s + a= + O;)[V] = (1 + Am(s))[u]. Filtering by 1/(s + am), we have =

1

+

s + am

(5.6)

where z = - y + ,+--~=[ul, w = y, t/0 = --Am(s)tu], which is in the form of the linear parametric model (3.3). The adaptive law for estimating e* can be developed by using any one of the methods given in Sect. 3. Let us consider the Lyapunov method and the gradient method. (i) Lyapnnov Method. Since ~ 1 is SPR, we choose L(s) = 1 and use the results of Sect. 3.1 to obtain the adaptive law t~o=-Te~-7cr,0o, 1 , = ~ [ 0 o ¢

S + am

~=w=y 1

(5.7)

- e-~l - z = ~ [ ¢ o ¢

s + am

- e . ~ - 71,

where ~bo = 0o - 0~ and t / = ~/o = --Am(8)[ll] (ii) Gradient Method. Using the results of Sect.3.2, we have g=-?e¢-~,e0,

¢=

1

s + am

0o~ - z ¢0~ - ~ e= m= = m~" , 0

Iv]

1 S+amAm(S)[ul'

(5.8)

where m = = cr + fln~ for some a, fl > 0. The normalizing signal n~ is generated a S n s2 "-- 7Tt 8 , where ri~ = -,~om, + lul 2 + Ivl 2 , m . ( o ) = o

(5.9)

for some am > 60 > 0 and such that Am(8 -- 60/2 ) has stable poles. Hence, the first aSsumption that Am(s ) haS to satisfy in the adaptive case is (A1) Am(s) is analytic in Re[s] > - 6 0 / 2 for some known 60 > 0. Let us now check whether ns generated from (5.9) has the property ~ , ~ E Loo in (5.7), (5.8). We first consider (5.7), where ff = y and ~ = -Am(s)[u]. We have # = - a m y + (1 + Am(S))U - - ( a - am)y. Using Lemma 2.2, we have [Y(t)l 2c, we have

_

m

-I-c,

where 72(r) = 2 ~1¢0"12 + 2ce2n~, i.e., 7 • S(~-~), which by (5.12) implies that 7 • S(A2) • Therefore, from (5.11) we have

m ~ cAlra-t- c ( ~3. -}- A2) ra-[- c ( ~ e-6"('-r)72(r)m2(r)dr ) + c. Hence, for

c (z~l + z~+-~) < 1,

(5.14)

we have

(f ' e_6O(,_T)72(r)m2(r)dr)½ or

m 2 0. Since 7(7) E S(A~), it follows from Lemma 2.7 that m E Loo. Therefore, y = ~ and )/axe bounded, which implies that u and therefore all the signals in the closed loop are bounded. The tracking error er = y - Ym is given by er -- s a t-1a"-'-~[¢oy] _ ) / = e + S ~ a m [end] + S ~ a m [)/] -- r/,

which implies that 9 ~l m allf I levi _< I~(t)l + I,7(t)l + II(¢n,),ll2 + 11'7,112 •

Hence,

[ ler(t)l=d7 ___cza~t + c.

(5.16)

The conditions for stabihty that Am(S) has to satisfy when (5.7) is used as the adaptive law are summarized below (C1) Am(S) is analytic in l~[s] _> --~0/2 for some known 60 > 0. (C2) A 1 -"

A m ( 8 ) 'l~° A 2 " - I [ A m ( s ) l [ ~ , z53 =

8 + am oo' II(s + p)zam(s)ll~ are finite. (C3) c ( A x + A 2 + ~ 3 )

SAm(S) ~o and

S-l-am oo

< 1, w h e r e c e l R + a n d c ~ c a n b e a x b i t r a r i l y l a r g e .

(c4) zal < ~_0 for some c e ~t +. c

Condition (C2) will be satisfied if Am (s) is assumed to be strictly proper. The arbitrary constants denoted by c in (C1) to (C4) can be calculated by following the steps given above in a similar way as in [18], where such explicit calculations are performed for a gradient algorithm. In a similar way we can proceed with the analysis of the MRAC scheme, where (5.8) is used as the adaptive law. In this case, it can be shown that the conditions on Am(s) for stability are given by (C1 t) Am(8 ) is analytic in Re[s] > - ~ o / 2 for some known/50 > 0. (C2') A I = (C3') A 2 :

~

I

< c for somec > 0. ($ "~-am)

< c6o for some c > 0.

Robust Adaptive Control

143

It is clear that (CI')-(C3') allow Am($ ) to be proper, in contrast to (CI)(C4) for (5.7), where Am(s) is required to be strictly proper. [] Comparing the stability condition (5.4) in the known parameter case (i.e., the stability condition for robust nonadaptive control) with conditions (C1')-(C3') or (C1)-(C4) one may conclude that the conditions in the adaptive case are more restrictive. While this conclusion m a y be true in some cases, it is not true in others. In particular, it is not true in the case where in robust non-adaptive control Am(s ) is largely due to parametric uncertainty, as shown by the following example. E x a m p l e 5.1.2. 1 [,1 s+l-e for some constant e. In robust nonadaptive control we express (5.17) as y=

(5.17)

1 y = - - ~ [ 1 + ,am(S)], where Am(s ) = ,-VT'~-~,~i.e., in (5.1) a -- 1. Then, condition (5.4) becomes

( s + T - .- -eT~ Tb am )

OD

< 1,

which requires e to be small, i.e., for am ---- 0.1 and 1 - ~ > 0 we require ~ < 0.1. In adaptive control, however, (5.17) is modeled as 1 s-{-a

i.e., Am(S ) = 0, which satisfies (CV)-(C3') for all e.

[]

The above example together with our analysis for Example 5.1.1 demonstrate that robust adaptive control can handle a larger class of parametric uncertainties than robust nonadaptive control but a smaller class of non-parametric uncertainties. 5.2 I n d i r e c t R o b u s t A d a p t i v e C o n t r o l In Sect. 5.1, the control of the unknown plant was carried out by directly estimating the desired controller parameters. An alternative method is to estimate the plant parameters on-line and use them to calculate the controller parameters at each time t. The scheme derived with this method is commonly known as indirect adaptive control because the evaluation of the controller parameters is achieved indirectly by using the estimated plant model. The main steps used in the design and analysis of indirect adaptive control are the following:

Step 1: We derive the control law that can be used to meet the control objective as if the plant parameters were known.

144

Ioannou and Dz~tta

Step 2: We propose the same control law as in Step 1 but with the controller parameters calculated at each time t from the estimated plant parameters generated by an adaptive law. Step 3: We analyze the adaptive control scheme formed in Step 2 and show that it meets the control objective. In the adaptive control literature, the most frequently encountered indirect schemes are of two types. They are (i) the adaptive pole placement control (APPC) scheme [10,12,!3,14,16,20 ] for which the control objective in Step 1 is pole placement and (ii) the adaptive LQ control (ALQC) scheme [38,39,40,41] for which the control law in Step 1 is obtained by minimizing the LQ cost as discussed in Sect. 4.4. We consider these schemes one by one. A d a p t i v e P o l e P l a c e m e n t C o n t r o l . We illustrate the design and analysis of an adaptive pole placement control scheme using a simple example. E x a m p l e 5.2.1. Consider the plant b y = --4--~ (1 + ~ m ( s ) ) u ,

(5.18)

where a, b are unknown constants. The control objective is to choose u such that the poles of the closed-loop system are placed at the roots of a given Hurwitz polynomial, say A*(s) = (s + 1) 2, and the output y(t) tracks a unit step input us(t). Thus, here ym(t) = Us(t) and Q(s) -" s. Step 1: As we have already shown in Sect. 4.4, the control law

s ÷ 12 + 11 ~-----7 [ym - y], s+

u = ~[u] where

2-a

tl = - y - ,

1

12 = ~ ,

(5.19)

(~.20)

and ~1 > 0 is a design constant, can be used to meet the control objective exactly when Am(S ) = 0. Furthermore, if/kin(8 ) ~ 0, then for all Am (s) satisfying

bz:~m(S)(llS "{- 12) [ ~¥T)~ I

< 1,

(5.21)

OO

the closed-loop plant is stable and the tracking error er = y - Ym is given by s+a s+a e~ = - [ ( s + 1) 2 + b~m(S)(Zls + Z~)]"[ym] = [(s + 1) ~ + b~m(S)(Zls + t~)] [0], i.e., if (5.21) is satisfied, er(t) converges to zero exponentially fast. Step 2: Since a, b are unknown, (5.20) cannot be used to calculate the controller parameters 11, 12. Instead of (5.19)-(5.20) we use

(

) 1

~, - ~ [ u l . + ~ + h(t), + i~(t) --~-~[Ym -- Y],

(5.22)

Robust Adaptive Control

145

where 11, [2 are calculated as

t~=2-a

, i2 = ~ 1,

(5.23)

where ~, b are the estimates of a, b generated by an adaptive law as follows: From (5.18) we have (s -b a)y --- b. --~ bAm(s)[u ] . (5.24) Filtering both sides of (5.24) by 1/(s + A), with ~ > 0, and rearranging term,s we obtain Z = 0*W~ -~- r], (5.25) where z=

,

i=

1[_:] 0"Io T

s ~

'

b 8-~- AmIB]LtLj,[ \F 1

which is in the form of the linear parametric model used with the gradient and least-squares methods. The adaptive law for estimating 0* is then given by

o T ~ -- Z

(5.26)

(~T~ - -

m2

m2

'

where m 2 = 1 + n~ and n~ = m, is generated from yh, = - 6 0 m , + u 2 + y2,

m.(0) = 0.

As shown in Sect. 3.2, the adaptive law (5.26) has the following properties (i) e, 0, 0 e Loo (ii) e, en,, 0 e S(m-~). These properties, however, do not guarantee that [1, [2 are bounded, since (5.26) may generate estimates b which are arbitrarily close to zero. If the sign of b and a lower bound bo > 0 for Ib] are known, then (5.26) can be modified so that Ib(t)[ _> b0 Vt > 0 without altering the properties (i) and (ii) [14,11]. 1 Step 3: Let 5: A_ _~.i[x]" Since A and A1 are both design constants, we can choose 2 = 2x. Then, using the estimation error ern 2 = oW( -- z and the control law (5.22), we can write

= -i~a-

(i2 -

ita)~+

.Om+ i~'~ 2

.~ = - ~ m 2 - a.O + ha, ~a+f~ r 1 where f/m = i ,+xl tyro] iS bounded due to [1, [2, Ym E Leo. Defining xt = f/, z2 = ~, we have

Ioannou and Datt~

146

where x = [zl, Z2] T and

For each fixed t, d e t ( s I - A ( t ) ) = s ~ + 2s + 1, and

Provided A _> 60, using Lenuna 2.2, we can show that m= < $

i o,

IIA(,)I

c > O,

~,~in~

A(t) is exponentially

IlbZ~m(')ll

= z~02 ,

Lemma 2.1 it follows that for A0 < c a n d stable. Hence, from L e m m a 2

some

2.3 we have

6

tlx, ltg _< c II( m ),11 +c for some 6 > 0. Now u and y can be expressed as

Since 9m, A • Loo, we have

(lly, lh) + (ll=,llg) 2 _< c (llx, ll~) ~

c

Therefore, m S = 1 +n~ < c

(It(

em2)t

+ ~.

,:)'

+ c,

i.e., m 2 0. Such a condition is needed to ensure that the solution P(t) of the ARE (5.32) with A, b replaced by .4(t), b(t) exists and is uniformly bounded. If the sign of bl and a lower bound b0 > 0 for [bx[ are

148

Ioannou and Datta

known, then (5.26) can be modified to guarantee that Ibl(t)l > b0 v t _> o and a(t) is strictly hounded away from q W > 0, without altering the properties (i) and (ii)[14,11]. This will guarantee that (.4, b) is uniformly controllable Yt > O. Step 3: The adaptively controlled closed-loop plant can be written as: s + A1) [y] + ¢2 (s + (s),)(s + q)+ A1) [~1 e = A¢(t)e + k(Ss~ ++A)(Sa~s++a~A1) - ¢ 1 (s + A)(s

-ks2 blsAm(s) + trls +

[u]

(5.34)

a2

s + x1)[u] + ¢2 (s + (s~)(~ + q)+ A1) [fi] e0 = - c ~ + (Ss2++~)(s~ls++~1)~2 -¢1 (s + ~)(~

blSZlm(S) s 2 + ~ls + ~2

[u]

(s + a)(s + q)M'(s)

't/.

(5.35)

(s + a)(s + q)F'(s)

s:' +,61s 2 +,62s +/~:3 t.a, + r ' , s 3 + f~ls 2 + ~2s + f13 [y] bl(s + q ) F * ( s ) A m ( s ) r ,

(5.36) (5.37) (5.38)

a = -V(O~

y= ym--eo,

where

- Ae(t) ~- A(t) - b(t)G(t), - c = [10], -- k = [~i-~I - a 2 ] T,

- s2 + iris + a2 is an arbitrary monic polynomial of degree 2 with all its roots in Re[s] < - ~ 2 ' - ( s + Xx) is a Hurwitz polynomial with X1 > 60, - Wbi(s), Wei(s),i - 1, 2, are stable strictly proper transfer function matrices obtained using Lemma 2.5, - M*(s) and f*(s) solve the Diophantine equation

M'(s)s(s + a) + F'(s)bl(8 + q) = s 3 + ~ls ~ + Z2s + ~3, where s 3 + ills 2 + fl2s + f13 is an arbitrary monic polynomial of degree 3 having all roots in Re[s] < - ~ 2 " For each fixed t, it follows from standard LQR Theory [34] that the matrix At(t) is stable. Furthermore,

Robust Adaptive Control

149

Provided ~ > $0, using Lemma 2.2, we can show that

,,

1(

~-~ -< X Ilbl/t.(,)ll

= / t 0 ~,

i.e., Loo. Furthermore, since the eigenvalues of Ae(t) are continuous functions of the parameter estimates 0(t) and O(t) E L ~ (i.e., 0 E K, Kcompact C IR2 ), it follows that 3ers > 0 such that Re{)q(Ae(t))} _< -ere Vt ~ 0. Hence, applying Lemma 2.1, it follows that for /t o < c and some c > 0, At(t) is exponentially stable. Using Lemma 2.5 and the relationship (s + q)fi = su, we obtain

s

, =

¢

-¢1 (~ + ,x)(, +

+ ~)(. + ,h)[a]

+wo3(,) {(wb3(,)[¢TI) Since c T ~ __ ~m2 .~. T], we can write

s [~,] + (s + a)Cs + ~ 1 ) ( ¢2 s

(s + q) 2+ [~] _ _ r S 17] + a)Cs + a l ) 7 - 4 - - ~ t'm

-w03(,) Using the above expression in (5.34)-(5.35) and making use of (5.36)-(5.3S), it is clear that the stability analysis for this example can be completed by essentially duplicating the steps involved in the analysis of Example 5.2.1. The condition for stability is that

zl0-~

1

60

IlbxZ3m(s)ll~ < c

for some c > 0 which can be calculated if an estimate of the rate of exponential convergence of the transition matrix of A¢(t) can be obtained. As in Example 5.2.1, the use of a dead-zone instead of a leakage will make the bound f o r / t o independent of the transition matrix of Az(t) and hence easier to calculate. The following expression for the tracking error

fo

* le"('r)12dr < C/to2-L + c

can also be obtained in a similar manner as in Example 5.1.1.

[]

An important drawback of indirect adaptive control is what is called the "sta~ bilizability problem": the control law is designed based on the estimated plant, which has to be controllable and observable at each time t. The adaptive laws cannot guarantee such a property alone without additional modifications. One

150

Ioannou and Datta

possible modification is to use projection and constrain the estimated parameters to be in a convex set which contains the unknown parameters and is such that for every member of the set the stabihzability condition required for the calculation of the control law is satisfied [14,42]. While such modification seems feasible in theory, in practice the development of such convex sets for higher order plants is complicated and awkward. Other approaches based on persistence of excitation are used in [43,44] to handle the stabilizability problem. 6 Conclusions In this paper we have given a unified treatment of the design, analysis and robustness of most of the continuous-time robust adaptive control schemes that are currently available in the adaptive control literature. The methodology presented can also be used to synthesize new robust adaptive control schemes by simply combining an adaptive law, developed using the procedure of Sect. 3, with a robust controller structure. Thus, this paper is part of an ongoing effort to develop a systematic and unified theory for the design and analysis of robust adaptive control systems so that the latter cease to be looked upon as a mere collection of assorted tricks and algorithms. The theory of robust adaptive control is still far from complete. A satisfactory theory for the quantitative analysis and transient performance of robust adaptive control systems is yet to be developed. The results presented in this paper are applicable to LTI plants. Their extension to linear time-varying plants is attempted with considerable success in [45,46,47,48,491. References 1. A. S. Morse, "Global stability of parameter adaptive control systems," IEEE Trans. Aut. Control, vol. AC-25, pp. 433-439, June 1980. 2. K. S. Naxendra, Y. H. Lin, and L. S. Valavani, "Stable adaptive controller design -part II: proof of stability," IEEE Trans. Aut. Control, vol. AC-25, pp. 440-448, June 1980. 3. I. D. Landau, Adaptive Control: The Model Reference Approach, Marcel Dekker, New York, NY, 1979. 4. G. C. Goodwin, P. J. Ramadge, and P. E. Gaines, "Discrete time multivariable adaptive control," IEEE Trans. Aut. Control, vol. AC-25, pp. 449-456, June 1980. 5. B. Egardt, Stability of Adaptive Controllers, Springer-Verlag, Berlin, 1979. 6. P. A. Ioannou and P.V. Kokotovic, Adaptive Systems with Reduced Models, Springer-Verlag, New York, NY, 1983. 7. C. E. Rohrs, L. Valavani, M. Athans, and G. Stein, "Robustness of adaptive control algorithms in the presence of unmodehd dynamics," Proc. ~lst IEEE Con]. Dec. Control, 1982. 8. P. A. Iormnou and P. V. Kokotovic, "Instability analysis and improvement of robustness of adaptive control," Automatica, vol. 20, pp. 583-594, Sept. 1984. 9. P. A. Ioannou and K. S. Tsakalis, "A robust direct adaptive controller,~ IEEE Trans. Aut. Control, vol. AG-31, pp. 1033-1043, Nov. 1986.

Robust Adaptive Control

151

10. K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems, Prentice-HMl, Englewood Cliffs, N J, 1989. 11. L. Praly, "Robust model reference adaptive controllers-part 1: stability analysis, = Proc. ~3rd IEEE Conf. Dec. Control, 1984. 12. G. Kreisselmeier, "A robust indirect adaptive control approach," Int..7. Control, vol. 43, pp. 161-175, 1986. 13. P. Ioannou and J. Sun, "Theory and design of robust direct and indirect adaptive control schemes," _Int. J. Control, vol. 47, pp. 775-813, 1988. 14. R. H. Middleton, G. C. Goodwin, D. J. Hill, and D Q. Mayne, "Design issues in adaptive control," IEEE Trans. Aut. Control, vol. AC-33, pp. 50-58, Jan. 1988. 15. B. D. O. Anderson, R.R. Bitmead, C.R. Johnson, Jr., P.V. Kokotovic, R. L. Kosut, I. M. Y. Mareels, L. Praly, and B. D. Riedle, Stability of Adaptive Systems: Passivity and Averaging Analysis, MIT Press, Cambridge, MA, 1986. 16. S. S. Sastry and M. Bodson, Adaptive Control: Stability, Convergence and Robustness, Prentice-Hall, Englewood Cliffs, N J, 1989. 17. P. A. Ioannou and K. S. Tsakalis, "Time and frequency domain bounds in robust adaptive control," Proc. 1988 Amer. Control Conf. 18. K. S. Tsakalis, "Robustness of model reference adaptive controllers: input-output properties," Department of Electrical and Computer Engineering, Arizona State University, Report no. 89-03-01, 1989, to appear in IEEE Trans. Aut. Control). 19. J. M. Krause, P. P. Khargonekar and G. Stein, "Robust adaptive control: stability and asymptotic performance," Proc. ~8th IEEE Conf. Dec. Control, 1989. 20. G. C. Goodwin and K.S. Sin, Adaptive Filtering, Prediction and Control, Prentice-Hall, Englewood Cliffs, N J, 1984. 21. G. Kreisselmeier and D. Joos, "Rate of convergence in model reference adaptive control," IEEE Trans. Aut. Control, vol. AC-27, pp. 710-713, June 1982. 22. B. A. Francis, A Course in Hoo Control Theory, Springer-Verlag, Berlin, 1987. 23. T. Kailath, Linear Systems, Prentice-Hall, Englewood Cliffs, N J, 1980. 24. C. A. Desoer and M. Vidyasagar, Feedback Systems: Input.Output Properties, Academic Press, New York, NY, 1975. 25. P. J. Gawthrop, Continuous-Time Sell-Tuning Control, Research Studies Press, Wiley, New York, NY, 1987. 26. R. H. Middleton and G. C. Goodwin, Digital Control and Estimation: A Unified Approach, Prentice-Hall, Englewood Cliffs, N J, 1990. 27. M. Polycarpou and P. Ioannou, "On the existence and uniqueness of solutions in adaptive control systems," Department of EE-Systems, University o] Southern Cali]ornia, Technical Report no. 90-05-01, 1990. 28. D. G. Luenberger, Optimization by Vector Space Methods, Wiley, New York, NY, 1969. 29. M. Vidyasagar, Nonlinear Systems Analysis, Prentice-Hall, Englewood Cliffs, N J, 1978. 30. M. Vidyasagar, Control System Synthesis: A Factorization Approach, MIT Press, Cambridge, MA, 1985. 31. P. V. Kokotovic, H. Khalil, and J. O'Reiily, Singular Perturbation Methods in Control: Analysis and Design, Academic Press, New York, NY, 1986. 32. M. G. Safonov, A. J. Laub, and G. L. Hartmann, "Feedback properties of multivariable systems: the role and use of the return difference matrix," IEEE Trans. Aut. Control, vol. AC-26, pp. 47-65, Feb. 1981.

152

Ioannou and Datt~

33. J. C. Doyle, K. Glover, P. P. Khargonekar, and B. A. Francis, "State-space solutions to standard //2 and Hoo control problems," IEEE Trans. Aut. Control, vol. 34, pp. 831-847, Aug. 1989. 34. H. Kw~kernaak and R. Sivan, Linear Optimal Control Systems, Wiley, New York, NY, 1972. 35. K. J. Astrom and B. Wittenmark, Adaptive Control, Prentice-Hall, Englewood Cliffs, N J, 1989. 36. G. C. Goodwin and D. Q. Mayne, "A parameter estimation perspective of continuous time model reference adaptive control," Automatica, vol. 23, pp. 57-70, Jan. 1987. 37. P. A. Ioannou and G. Tao, "Dominant richness and improvement of performance of robust adaptive control," Automatica, vol. 25, pp. 287-291, March 1989. 38. C. Samson, "An adaptive LQ control for nonminimum phase systems, ~ Int. J. Control, vol. 35, 1982. 39. D. W. Clarke, P. P. Kanji]al, and C Mohtadi, "A generalized LQG approach to self-tuning control, part I: aspects of design," Int. J. Control, vol. 41, 1985. 40. D. W. Clarke, P. P. Kanjilal, and C Mohtadi, "A generalized LQG approach to self-tuning control, part II: implementation and simulations," Int. J. Control, vol. 41, 1985. 41. J. Sun and P. Ioannou, "Robust adaptive LQ control schemes," Proc. 1989 Amer. Control Conf. 42. Ph. de Larminat, "On the stabilizability condition in indirect adaptive control," Automatica, vol. 20, Nov. 1984. 43. G. Kreisselmeier, "An indirect adaptive controller with a self-excitation capability," IEEE Trans. Aut. Control, vol. 34, pp. 524-528, May 1989. 44. R. Cristi, "Internal persistency of excitation in indirect adaptive control," IEEE Trans. Aut. Control, vol. AC-32, pp. 1101-1103, Dec. 1987. 45. G. Kreisselmeier, "Adaptive control of a class of slowly time-varying plants," Syst. Control Lett., vol. 8, Dec. 1986. 46. K. S. Tsakalis and P.A. Ioannou, "Adaptive control of linear time-varying plants," Automatica, vol. 23, pp. 459-468, July 1987. 47. R. H. Middleton and G. C. Goodwin, "Adaptive control of time-varying linear systems," IEEE Trans. Aut. Control, vol. AC-33, pp. 150-155, Feb. 1988. 48. K. S. Tsakalis and P. A. Ioannou, "Adaptive control of linear time-vaxying plants: a new model reference controller structure," IEEE Trans. Aut. Control, vol. 34, pp. 1038-1046, Oct. 1989. 49. K. S. Tsakalis and P. A. Ioannou, "A new indirect adaptive control scheme for time-varying plants," IEEE Trans. Aut. Control, vol. 35, pp. 697-705, June 1990.

R o b u s t C o n t i n u o u s - T i m e A d a p t i v e C o n t r o l by Parameter

Projection

*

Sanjeev M. Naik, 1 P. R. Kumar, 1 and B. Erik Ydstie 2 1 Coordinated Science Laboratory and Department of Electrical and Computer Engineering University of Illinois, Urbana, IL 61801, USA. 2 Department of Chemical Engineering, Goessmann Laboratory University of Massachusetts, Amherst, MA 01003, USA.

A b s t r a c t . We consider the problem of adaptive control of a continuous-time plant of arbitrary relative degree, in the presence of bounded disturbances as well as unmodeled dynamics. The ~iaptation law we consider is the usual gradient update law with parameter projection, the latter being the only robustness enhancement modification employed. We show that if the unmodeled dynmnics, which consists of multiplicative as well as additive system uncertainty, is small enough, then all the signals in the closedloop system are bounded. This shows that extra modifications such as, for example, normalization or relative dead zones, are not necessary for robustness with respect to bounded disturbances and small unmodeled dynamics. In the nominal case, where unmodeled dynamics and disturbances are absent, the asymptotic error in tra£king a given reference signal is zero. Moreover, the performance of the adaptive controller is also robust in that the mean-square tracking error is quadratic in the magnitude of the unmodeled dynamics and bounded disturbances, when both are present.

1 Introduction Recently, there have been m a n y a t t e m p t s to study the adaptive control of plants with bounded disturbances and unmodeled dynamics. In his pioneering work, Egardt [5] showed t h a t even small bounded disturbances can cause instability in adaptively controlled plants. He further d e m o n s t r a t e d t h a t modification of the a d a p t a t i o n law by projecting the p a r a m e t e r estimates, at each time instant, onto a compact, convex set known to contain the true p a r a m e t e r vector, provides stability with respect to bounded disturbances. Proving the stability of an adaptive control s y s t e m in the presence of unmodeled dynamics is however more difBcult, and researchers have proposed various additional modifications to the adaptation law to analyze and bound the effects of unmodeled dynamics. For example, in [8-10], Praly introduced the device of using a normalizing signal in the p a r a m e t e r estimator, while another notable modification is the idea of a normalized dead-zone [11]. Other modifications used in conjunction with normalization include projection [5] or some form * The research reported here has been supported in part by U.S.A.R.O. under Contract No. DAAL 03-88-K-0046, the JSEP under Contra~t No. N00014-90-J-1270, and by an International Paper Fellowship for the first author.

154

Naik, Kumar, and Ydstie

of leakage, the concept of which was first introduced in [7], such as switching-amodification [12,14]. An excellent unification of these various modifications and results can be found in Tao and Ioannou [18]. In this paper, we show that Egardt's original simple modification of parameter projection is in fact enough to provide stability with respect to small unmodeled dynamics as well as bounded disturbances. Our main results are the following: (i) A certainty equivalent adaptive controller, using a gradient based parameter estimator with projection, ensures that all closed-loop signals are bounded, when applied to a nominally minimum-phase continuous-time system with bounded disturbances and small unmodeled dynamics (Theorem 9.1/. (ii) In the absence of unmodeled dynamics and disturbances, i.e., in the nominal case, the error in tracking a reference trajectory converges to zero (Theorem 10.1). When unmodeled dynamics as well as bounded disturbances are present, the mean-squared tracking error is quadratic in the magnitude of the unmodeled dynamics and bounded disturbances (Theorem 11.11. Thus, the adaptive controller provides robust performance in addition to robust boundedness. While our work thus shows that a simple modification is sufficient to ensure robust boundedness and robust performance, and some early modifications may have been proposed due to the limitations of proof techniques used, we feel that it is nevertheless important for future work to compare the various modifications on the basis of the amount of robustness provided, the resulting performance as a function of unmodeled effects, transient response, etc., as well as the complexities of the modifications themselves. The key stimulus for our work here is the recent paper of Ydstie [19], which showed that parameter projection in a gradient update law is sufficient for ensuring the boundedness of closed loop signals for a nominally minimum-phase, unit delay, discrete-time plant with some types of unmodeled dynamics as well as bounded disturbances. The continuous-time systems studied here give rise to several additional issues such as filtering of signals, parametrization of systems, differentiability considerations of signals, augmented errors, etc., which motivate various changes, and allow us to establish stability for nominal plants with arbitrary positive relative degree, as well as for a class of unmodeled dynamics which is larger than those considered earlier, for example in [20],[14] or [19]. For instance, unlike Ydstie[19] we do not require the true plant to be stably invertible; only the nominal plant is assumed to be minimum-phase. Additionally, in contrast to [14] we also allow the unmodeled dynamics to be nonlinear or time-varying, and do not require differentiability of either the bounded disturbance, which is lumped together with the unmodeled dynamics in our treatment, or the reference input. The rest of the paper is organized as follows: Section 2 introduces the system and reference models. In Section 3, we reparametrize these models, and describe, in Section 4, the adaptive control law. Our analysis starts in Section 5, where we show that all closed-loop signals in the system are bounded by a particular signal re(t). In Section 6, we introduce a signal z(t) defined through a "switched system" which overbounds re(t). In Section 7, we show that the filtered signals

Robust Continuous-Time Adaptive Control

155

are comparable to z over certain bounded intervals of time. To apply these results to the stability analysis of the closed-loop system, in Section 8 we present a nonminimal representation of the closed-loop system, which is then used in Section 9 to complete the boundedness analysis by showing that a certain positive definite function of the signal z(t) and the non-minimal system state error e(t) is bounded. In Section 10, we show that asymptotic tracking is achieved in the nominal case, and in Section 11 we establish a mean-square robust performance result. In Section 12 we present simulation examples to illustrate the results. Finally, Section 13 presents some concluding remarks. Some necessary technical results are collected in Appendices A, B, and C. A list of constants appears in Table D. A preliminary version of the results presented here is contained in [20].

2 System

and Reference

Models

Consider the single-input, single-output system,

y(t)

B(s) = A"~-~_~C1 +/.JmAm(a))B(t) + V(t),

(i)

where B(s)/A(s)is the transfer function of the modeled part of the plant, Am(S) represents the multiplicative uncertainty in the plant, and v(t) represents the effect of additional additive unmodeled dynamics as well as bounded disturbances. We will make the following assumptions on the nominal model of the plant: (A1) A(s) = s" + E n-1 =o B(s) are coprime.

and B(s) = ET=o

o < m 0, and bm > brain > 0. The relative degree of the nominal plant will be denoted by nr := n - m. We will make the following assumptions on the unmodeled dynamics and bounded disturbance of the plant: (A3) The multiplicative uncertainty, Am(S), is a transfer function with relative degree greater than or equal to (1 - nr) such that Am(s -- P0) is stable. (A4) The additive unmodeled dynamics and disturbances give rise to a signal v(t) which satisfies Iv(t)] < Kvm(t) + k~ + k~oexp[-pt], where m(.) is defined by, re(t) -- - d o m ( O + dx(IKOI + ly(t)l + 1), re(O) >_ dx/do,

(2)

All the constants are positive. Furthermore, 0 < p _< do and do q- d2 < P0 < 2d0 + d2, for some d2 > 0. The term kv0 exp[-pt] above allows for the effect of initial conditions.

156

Naik, Kumar, and Ydstie

E x A m p l e 2.1 The above assumptions allow for the class of linear systems

B(s) ( y(t)--~

Nm (s)'~ l+~mD--~fuit

)+vit),

where deg Nm(s) _< deg Dm(s) + n r - 1 (which could be greater than the degree of Dmi s) if nr >_ 2), v(t) = D---~y(t~ NI(S) . . +

~2(s) u(z)N2(s) . . . .. w(t) + ~o exp[-pt],

where ~ , ~ are strictly proper, w ( t ) i s a bounded disturbance, and the polynomials Dm(s - Po), DI(S - Po) and D2(s - 19o)are Hurwitz. D

E x a m p l e 2.2 A class of nonlinear unmodeled dynamics satisfying (A4) is vit) - f ( t, gl it) " n l i ' ) Y i t ) , g2(t) • n2i8)t~(~)) ,

[f(t, Zl(t),z2(t)l
0 and lym(t)l < k~.~,Vt > 0. 3 Parametrization

of System

and

Reference

Models

We now repararnetrize the system and reference models so that they are in a form more suitable for the development of an adaptive control law. To do this, we need to filter the input and output signals. Let a be a positive number satisfying a > d2+2do. We define the "regression" vector~3

¢ :=

(

(. +.)~.-m-1 Y,-", (. + . ) . - m ~,

1

1

)T

(. + . ) ~ . _ m _ l " , . . . , (. + . ) . _ . ~ " 1 z All of the results of this paper continue to hold if instead of the filter (,+~)~_,._. x' we use the filter (,+,)--'~xx(,)' ~ where )~l(s) = (s + a)),l (s) with )~I(s) being monic, of degree ( n - 2), and with all roots having real parts less than or equal to -(2do + d~).

Robust Continuous-Time Adaptive Control

157

We will presently show that there exists a "parameter vector" 0 = (01 . . . . . 02n)T such that the system (1) can be represented as

yCt) = eTCt)0 + ~(t) + ~2~o(t), 02. = b~,

(4)

where eW(t)0 represents the nominal part of the system, ~2do(t) represents the effect of initial conditions arising from the filtering operations, 4 and v~(t) represents the effect of unmodeled dynamics and bounded disturbances. We will also reparametrize the reference model (3) as 1 ym(t) - (s Jr a) nr rt(t) Jr ~2d°(t) '

(5)

where r'(t) := ( s + a ) nr Wm(s)r(t). Note that r'(t) is well defined since the relative degree of Win(s) is hr. Further, r'(t) is bounded since r(t) is bounded. We shall directly suppose from now onwards that rt(t) _< kr,Vt >_O. To see the existence of a 0 for which (4) holds, let F(s) be a monic polynomial of degree (n~ - 1) and G(s) a polynomial of degree less than or equal to (n - 1), such that A(s)f(s) + G(s) : A(s), where A(s) := (s + a) T M . Then, from (1), taking into consideration the effect of the initial conditions introduced by the filtering operation 1/A(s), we have,

A(S)~s_~(s) y(t) = F(s)-~-~y(Q . .A(s) rB(s)

+ ~2cI.(Q .

-- F(s) [ ~--'~u(t) Jr/Jm

B(s)Zlm(S) . . A(s) ..] ~ U(t) Jr ~--~V($)] Jr ~2do(Z).

Thus, if 0 = (0~,..., 02n) T is defined from the coefficients of C(s) -rid B(s)F(~) by, O,(s Jr a) "-1 + . . . Jr 02(s Jr a) Jr 01 = G(s), 02,(s + a) "-1 + . . . + 0,+2(s Jr a) + On+l = B(s)F(s), and v~f(t) = A "x--~v(t) + p m ~ ( ~ Am(s)u(t), then (4) is satisfied. We note that since F is monic, Oun = bin.

4 T h e A d a p t i v e C o n t r o l Law We will use the control law

~T(t)~(t) = ,'(t), to implicitly define the input u(Q, where ¢ ( t ) : = ( ~ = - r Y , . . .

(o) ,Y, ~7~F=-rU,

.... u) T, and 0(t) is an estimate of 0 that we shall presently specify. Note that the "regression" vector ¢(t) defined earlier satisfies ¢(Q = ~ ¢ ( t ) . 4

Here and throughout, ~q(t) will denote a signal which satisfies the following properties: (i) I~(~°(t)l _< co exp[-qt], i = o, 1, where ~O(t) denotes the i - t h derivative of ~q(t). (ii) ~q(t) - 0 when initial conditions are zero. When the value of q is unimportant, we will sometimes drop the subscript on ~.

158

Naik, Kumar, and Ydstie

The adaptive control law (6) is a "certainty equivalent" control law, since if 0(t) = 8 in (6), then the nominal part of the system (4) tracks ym(/), since 1 y(t) = cT(t)0 + ~do(t) = (, + a ) " - " [¢T(t)0] + ~ o ( t ) 1 - (8 + a) - - " ¢(t) + 6do(t) = ~m(t) + ~2ao(t) • Let us now turn to the parameter estimator. We shall use the gradient estimator with projection, ~(t) = Proj [ti(t), a¢(t)e~(t)] II0(0)ll < M, tJ2,(0) > brain, .(t) J' where

a > 0,

ea(t)

(7)

is the "augmented" error,

ea(t) = y(t)

1

-

nCt):=~+

ym(t) -4- (s -4- a)n-rn [cT(t)~(t)]

~

y(t)

+

-

cT(t)0(t) '

u(t)

,~>0,

is a normalizer of the gradient, and Pro j[., .] is a projection whose i - t h component is defined for i ~ 2n by

{

if Hp[I < M or pWz _< 0, pTx zi -- ~ Pi, otherwise,

zl,

Proj[p, x] [i = and for i = 2n by

{

x2,~

Proj[p,x] I~- =

if (IlPll < M or p T x < 0) and (P:n > bmin or z2n >_0),

pTx

pTz-> O)

z2. - II-]-~Tp=.,

~f (llpl[ --- M and

0,

and (P2n > bmin or z2. :> 0), otherwise.

There are several features worth commenting upon. First, the augmented ea(t) consists of the tracking e r r o r y(t) - ym(t), as well as the familiar

error

"swapping" term

1 n-m [¢T(t)0(t)l (s + a)

-

[ (s

+

:),~_mcx(t)]O(t),whichwould

be zero if tJ(t) were a constant. Second, note that n(t) is slightly different from the usual II¢(t)l? since it additionally contains the lower-order filtered terms 1

1

y(t) and ----Tu(t) (s + a) for 1 < i < n - m - 1, which are absent in ¢(t). (8 + a) i Turning to the "projection" mechanism, it has two features. Without any projection, a gradient scheme would simply consist of ~(t)- aC(t)e~(t) ,,(t)

Robust Continuous-Time Adaptive Control

159

However, to keep the estimates inside a sphere of radius M centered at the origin, one would project them according to

~(t) = Vroj' [~(t),

~,~(t)e.(t)] .(t)

J '

where Proj'[p, z] = z if IlPll < M or pTz _< 0, i.e., when the estimate is inside the sphere or not about to leave it, and otherwise

pTx Proj'[p, z] = z - H - ~ p , i.e., one projects the drift term so that it evolves tangentially to the sphere. Our projection Proj[p, z], however, involves an additional feature to ensure that bm(t), the 2 n - t h component of 0(t), is larger than or equal to bmin. So we have Proj [O(t), a~(t)e.(t).]

n(t)

= Proj'

J ~,,

[O(t),oCt)o.Ct).] '*(0 / ~ '

if ~2.(t) > bmin or Vroj' "/O(t), o~b(t)ea(t)] /

,,(t)

J 2.

> O, -

i.e., if bm(t) is not about to become less than brain, and

Proj [O(t), a~(t)ea(t)] = 0 otherwise. n(t) J2. The following easily verified consequences of projection are important. (PO) 02n(t) ~ bmin, II#(t)ll

~¢(t)~,(t)

_< M.

(P1) I1~(011 _< nCt) I (P2) If I1011 _< M and 02n =bm ~ brain, i.e., O lies in the region to which we confine the parameter estimates, then

~r(t)~(t ) < ~r(0 -

[~e(t)~.(t)~ k

-(0

) '

where ~(t) := ~(t) - o

Finally, since the existence of a solution to the above parameter estimator may not be assured due to the discontinuous nature of the projection, we can replace the above projection by the "smooth" projection due to Pomet and Praly [15], for which existence is assured. The key properties (P1) and (P2) continue to hold for the "smooth" projection, and all the results of this paper rigorously hold for this "smooth" projection. In what follows, we will throughout suppose that the nominal plant described by 0 satisfies the assumptions of (P2).

160

Naik, Kumar, and Ydstie

5 Bounding

S i g n a l s by m

In this section we will show that all the signals in the system can be bounded in terms of re(t) given in (2). a For brevity of notation in the proof, we define

~1(~) = (s + ~)"-~, ~(~) = (, + ~)-~, ~(~) = ~ ( , ) ~ ( ~ ) ,

,(t)= A-~(s)[¢r(t)Oct)]-[X~(s)¢V(t)] O(t), ¢'(0 =

v ( t ) , . . • , 5 - V - Z v ( t ) , y S ~ ,,(t),... ,

~(t)

, ~d

el(t) -" y(t) -- Ym(*) • L e m m a 5.1

lu(t)l < K=~,II¢'(t)II + K.~I~(t)I + k . , + coexp[-2dotl,Vt > O.

Proof. Let ¢I := (~-~(S) y,..., ~ - -1~ V , y , A - ~1u , . . . ,

(8)

1 ) be a sub-vec(s+a)U

tor of ¢, and 01 the corresponding sub-vector of 0. Then, the control law (6) gives u = (r ~ - cToT)/bm. From (4), we then obtain lY(t)I < MII¢'II + Iv6l + co exp[-2dot]. Hence ]l~xll < lyl + I1¢'11 _< (1 + M)II¢'II + Iv61+ co exp[-2dot]. Since $ m > brain, and 110111< 11011< M, we thus conclude that I"1 ___m~n (kr + M(1 + M)II¢'II + MIv+l) + coMexp[-2dot], a

Theorem

5.1 For all t > O,

(i) Iv! (t)l _< Kvfm(t) + kvf + Coexp[-(do + dz/2)t] + kvfo exp[-pt],

(ii) Iv~(t)l < K'fm(t) + k~t + co exp[-(do + d~/2)t] + kofo exp[-pt], (iil) I[¢'(t)ll -< K~,..m(t) + co exp[-2dot], (iv) le.(t)l _< Ktmm(t) + k~f + co exp[-(do + d2/2)t] + kvfo exp[-p/] +co exp[-2dot], (v) ly(t)l < Kv,~m(t) + k.f + eo exp[-(do + d~/2)t] + k~fo exp[-pt] (vi)

+co exp [-2dot], In(t)l < K.mm(t) + k... + coexp[-(do + d2/2)t] + K~k~foexp[-pt] +co exp[-2d0t].

Here and throughout, the values of useful constants used in the bounds are specified in Table D in the Appendix. Certain constants whose exact value is unimportant and which do not depend on K~, #m,k~ will be denoted generically by G. Any constant whose exact value is unimportant but which depends on K~, tim, k~, and such that its value decreases as K~,#m,k~ decrease, will be denoted generically by c. Any positive constant which depends only on initial conditions of some filter, and whose exact value is unimportant will be denoted by co. Finally, all constants throughout are positive, unless otherwise noted.

Robust Continuous-Time Adaptive Control

161

The proof of this theorem is based on the following lemma. LeImma 5.2 Let H(s) be a strictly proper, stable transfer function, whose poles

{ps) satisfy rte(p~) < -(do + d~) < 0 for all j. (i) If Win(t) is the input ~o a system with ~ransfer function H(s), which satisfies the bound Iwin(t)l < k.lu(t)l + k~lv(t)l + kmm(t) + k' + k " e x p [ - f t l , V t >__0, for 0 < f (-g2z(t) + K2), = 0, otherwise.

(10)

In Theorem 6.1(ii) below we will show that m itself is bounded in terms of z, thus showing from Theorem 5.1 that all other signals can also be bounded in terms of z. Theorem 6.1

(i) rh(t) _< Kmm(t) + k,~ + co exp[-(d0 + d2/2)t] +Co exp[-pt] + Coexp[-2d0f], Vt _> 0. (i O re(t) < K~,z(t) + k,n, , Yt >_O. (iiO 0.

(11)

(13)

Robust Continuous-Time Adaptive Control

163

Proof. (i)

This follows from in = -dora+ dl(lul + lYl + 1), by making use of the bounds for lul and lYJ in terms of m given in Theorem 5.1(v),(vi). (ii) By the definition of the augmented error ca, we have y = (ca + Ym) - ~, which implies lYl < led + lyml + I¢1. By the Swap.ping 1emma (Morse [6]; Goodwin and Mayne [13]), ((t) = [hTexp(Qlt)] • (H(t)O(t)), where Qx is an (nr x nr) stable matrix such that det(sI - Q1) = A2(s), HT(t) = (¢(t), sd2(t),..., sn-m-l¢(t)), h T is a constant row vector of dimension nr, and "." denotes convolution. Now,

where the last inequality follows from Property (P1) of the parameter estimator, and since IIHII O. Let ta >_ Tt + T' be any instant such that I(tx) = 1. If z(t) >_ L,Vt e (tl --T',tl] where L is a large positive constant (specified in Table D), then there exists a K~max > 0 and a I~m~ > 0 such that for all Kv e [0, Kvmax] and for all Um E [0, Umax], W e have:

.(t)

z~(t----~ > 5(T'),

V t e (tl - T',tl].

(24)

Proof. (i) Let us fix t such that I(t) = 1. By (10), we have [u] + ]eal > - g ~ z + K z ~a . Using the upper bound (20) on u gives, K~¢,1]¢'11 + lea] > (do - ~x)z + /£2 =: RHS. We note that we can take do > 0 and/£2 > 0, by the choices RHS. of g~, Tl, gvmax, and/~max given in Table D. If H¢'H > - - , then the result 2K~0, RHS RHS clearly holds, so we consider He'll < 2K,¢--"-~"Then clearly [e~I >_ ~ . Therefore using the upper bound (16) for v~, we obtain by definition of the augmented error,

~11¢11> I~r¢l > levi - Iv~l - co exp[-2d0TL] > I[(& - ~2)z + k~]/21 - (K',K~zz(t) + K'~k~ +k~f + k ( ~ ) + co exp[-pTl]). Note that (& - ~2) >_ 4K~fKm~ and R2 >_ 2(K~'fkmz + k~f+ k(Tl) + k exp[-pTt]), again from the choices of Ks, Tt, K~m~, and pmax given in Table D.) This implies

Robust Continuous-Time Adaptive Control

165 .(t)

He'[[-> ][¢H -> [.(d04M~2)z]. Hence we obtain, z(--~)2 _> g , z .

n(t)

(ii) First, we will bound the growth rate of z-~(t)" We do not require z to be large for this part of the proof. Note that d ( ~

h _ 2 n ~z < h + 2g2~'~. n Also, ) = ~" 2 z - ~"

IkO~ ~

¢ ' - a¢', which implies h : 2~btT(¢t-a~,bt) "~ ( 1 - 2a)11¢'112 +11¢'112, where, 1 1 1 1 Cei' :__ ((s + a) T M y ..... ~ " ~ Y ' y' (S -I- a) 2n-m-2 u ..... (s + a) u, u) , i.e.,

¢':

¢' := (s + a)¢'. Now, 11¢'11~ < I1¢'112 + y~ + u s. Using this, we get h < 2(1 a)[[¢,ll2 +y2 + u 2. Next, using the upper bounds for u and y from (20), (18), and

z-~(t ) ~ _< K s ( ).)2(t ) + Kb + K c recalling that n(t) = v + II¢'(t)[I 2, we get "~ x(n(t)

Z2(~) "

Next, using the fact that z(t) >_ L, Vt • (tl - T ~,tx], we get

d ~ (

z-,g)n(t)

) < K , ( - 7 7 ~ ) + Kd, where K d := K b +

"-~-'4Kc

z~-(t~) O, for some positive constant K x m . Define the state error e as e := Xc - Xm. This gives,

- Ace - bc(~r~b) + bevy + ~mb¢,l]m , el ---- h : e + V, where we recall that el = 9 - Ym is the tracking error.

(30)

Robust Continuous-Time Adaptive Control

167

9 Robust Ultimate Boundedness Define W, which bounds all signals, by

W = k, eTpe + l z 2 ,

(31)

where P = pW > 0 satisfies PAc + A ~ P = - I . Such a P exists since Ac is a stable matrix. Our main result on robust ultimate boundedness of the overall system is given by the following theorem. It states that eventually W(t) (which bounds all signals) enters a compact set, the size of which is independent of the initial conditions. Furthermore, the size of the allowable unmodeled dynamics for which this is guaranteed is independent of the initial conditions. It should be noted, however, that the time Targe that it takes for W(t) to enter this compact set can depend on the initial conditions. Choose constants 0 < 7 < 1, e0 > 0 small, and e, > 0. Let T,e, Tt, K , max, and/~max be as in Table D. T h e o r e m 9.1 ( R o b u s t U l t i m a t e B o u n d e d n e s s t h e o r e m ) . There ezist a Tlar~ >_ Tt, and positive Kvmax, #max, such that for all Kv E [0, Kvmax], and for all P m e [0, pmax],

W(t) < 8K~,L2exp[4(K, + ez)T], Vt > Tlarse •

(32)

Furthermore, gvmax,lJmax, T, and L are independent of the initial conditions. Proof. The idea of the proof, based on the following lemmas, is to show that whenever W(t) > K w z L 2 throughout an interval of length 2T, then at the end of the interval its value is smaller than at the beginning of the interval. L e n n n a 9.1

W(t) < K w , z2(t) + kwz, Vt > Tt.

(33)

Proof. Since W = keeTpe + ~, z 2 e, = X¢_Xm, l[Xm}l__0. Noting that B'(s) :- S(s)q(s) is such that B'(s - p ) is Hurwitz, and applying Lemma A.1, we get []xp(t)H _< cm(t) + c + co exp[-pt],Vt >_ 0. Finally applying Theorem 5.2(ii), we get the desired bound on [Ixp[[. []

168

Naik, Kumar, and Ydstie

Lemraa 9.2 Consider an interval [a, b]. Then < exp

-W(a) - -

g(7")d

1 + f14 e x p ~ ( b - a)]

(34)

where g(t) := - ~ + Zs I~r(t)¢(t)l + & I~r(0¢(t)l, Z~, Z4 and Z~ bei.g speci~d

~(t)

~(t)

in Appendix C. Proof. See Appendix C.

Lem.-a 9.3 Co.~ider a a.,e inte~at[., b] such tha~ (i) b > ~ > Tt, (ii) W(t) > KwzL2,Vt E [a,b]. Then,

(i) ½z2(t) L/2. The result (ii) follows from (i) because of ~ Kw~L 2 implies z > L/2. To prove (iii), note from Lemmas 9.2, B.4, the discussion preceding Lemma B.4, and by our choices of e,/~max,/¢vmax and L, that

< -~(b - .) + ~3~3/d + ~ 3 b ~ 4 ( T ) / d + ~ 5 ( T ) / v ~ + ~6(T)~" +4."-ink(#, L)l(b - .) + ~5~,7/d + ~[~,=~,~(T)/d + v~l(b - a) (~3 + < - ~" i( b .) + -e 2 ' -

which gives us the desired result after defining k = exp [(f13u3 +_ ~/sur)] Finally,

t

~2

J"

(iv) follows from the bounded growth rate of z(t) shown in (22), and (i).

[]

Lemrrm 9.4 ( C o n t r a c t i o n P r o p e r t y ) . If to > Tl, and W(~) >_ K w z L ~ Vt E [to, to + 2T], then W(to + 2T) _< 7W(to). Proof. There are four possibilities:

Robust Continuous-Time Adaptive Control

169

(1) Suppose I(t) = O, Vt C [to, to + 2T]. Then from Lemma 9.3(ii), we have

(

W(to + 2T) < 8Kw2 exp[-2g2T] +

W(to) T / 2 ) and (t2 - T < T / 2 ) / ( t 2 - T > 7"/2). (Because of symmetry, these reduce to just two). (4) (T < tl < t2 < 2T). Again, we apply the definitions of T and L as in cases 1 and 2 to get W(to + 2T) < 7W(t0) in each case. (a) (I(to + t) = 0 ,Vt • [t2,t2 + T]) (i)(t2 - T < 1) (ii)(t2 - T > 1) (b) (ICto + t) = 1, for some t E [t2, t2 + T]). []

Proof of Theorem 9.1. Lemma 9.4 and the fact that z has a bounded growth rate gives us the desired result since W ( t + 2T) _< 8Kw~ exp[4(K, + e,)T]W(t) whenever W(r) > K w , L2,Vr E [t,t + 2T]. Both K w , and L are independent of initial conditions. However, Tinge can depend on initial conditions. This concludes the proof. 1:3

I1¢'11 C Loo, Oi) v, vs, v~ E Loo, Oil) y C Loo, (iv) u • Loo, (v) e~, e Loo, (vi} i e L ~ , and (vii) "oe goo.

T h e o r e m 9.2 (/)I1¢11,

Proof. The result (i) follows from Theorem 5.1(iii). The result (ii) follows from assumption (A4) and Theorem 5.1(i),(ii). The result (iii) follows since y = ¢T0+ v6 + 6do, and Ilall ___ M. Lemma 5.1 implies that (iv) is true, and since e~ =

_ ¢ T g + v~ + 52do, and Ilffll _< M, (v) holds true. The result (vi) follows since = hTexp[Qlt]. (HT~), and IIHTffll ___C~e~. Finally, result (vii) follows from property (P1) of the parameter estimator.

10 P e r f o r m a n c e

for a N o m i n a l

O

System

We now show that if the system has no unmodeled dynamics and disturbances, then exact asymptotic tracking is achieved. T h e o r e m 10.1 / f K~ = O,k~ = O, and i~m = O, then ly(t) - ym(t)l -+ 0 as t --* oo.

Proof. It suffices to show e~(t) ~ 0 and ~(t) ---* 0 as t ~ oo, since el(t) = e~(t) - ¢(t). Note that since K , = 0, k, = 0, P m = 0 we have v~(t) - O. So, by Lemma A.2, ~n -< - 3 + cexp[-2d0t]. This implies, since I1~1 -< M, that

fo ° e-l-(~v, )d. ~ < ~ ntr)

-

a

+ k . 2do

(35)

Robust Continuous-Time Adaptive Control

171

Boundedness of n(.) .then implies ea • L2. Recall that ea • Loo. ea -- _ ~ r ¢ + ~2do implies ]~al < ]lO"l][l¢l[ + llo]] II¢ll + co exp[-2dot], which implies d~ e Loo. Therefore, by Barbalat's lemma (Popov [1, p. 211], or Sastry and Bodson [17, p. 19]), ea(t) --~ 0 as t --* oo. Now, recall that ((t) = h T exp[Qlt] * ( H ( t ) ~ ( t ) ) , and ][H01[ < (~C[ea[, which implies,

1¢(01 __ c'~

exp[-,,(t - ~')/211e~(")ld".

Using the just established fact that e.~ ~ 0, we will show that ( --* 0. Leg ear, :-- sup{le~(0 [ : t _> 0}. Pick a 6 > 0. Then, 3 a T~ < eo such that (i) C,~le~(t) l (ii) 2Caeam exp[-aT1/2] < 6/4, Vt > TI, and < 6/2. Hence, Vt > 27"1, a a

exp[-a(t-¢)/2]ae+

we have, [¢(0[ -< C'~(e~m i f 2

exp[-.(t-e)/2]de) < 2

2e~,~ e x p [ - a t / 4 ] / a + 6/2 < 2e~m e x p [ - a T / 2 ] / a + 6/2 < 6, thus proving that ~(t) --, o as t ~ o. n

11 R o b u s t

Performance

We now show that the performance of the adaptive tracker, as measured by the mean square tracking error, is robust in that it is a quadratic (hence also continuous) in the magnitude of the unmodeled dynamics and bounded disturbances. T h e o r e m 11.1 lim sup T2.-.* c~

1 --T2 /0

.

(y(t) -- ym(t))2dt < c(K2v + k~ + p2m)

~-2

-2

t)~2

Proof. From Lelnma A.2, we have e~ -

n

±

[T~ d -

T~JT,

v~ n

< -

"

< _ V + co exp[-2d0f], which implies ot

kexpI-2d0T,] T2

'

VT2 > T ~ . -

Hence, lim sup 1 ~ T , e~ -- v~2 < 0 =¢, lima.~_.oosup~1

(36) "

It,-fT'vf-n ' T2 e~n -< limT~..,ooSUp ~l iT,

Noting that z(t) > min{z(0),K2/g2} =: Zmin, and defining Zmax := suPt_>Tt z(t), by (16), we get, v~2 2 (g~vfkm~ + kvf + k(Tt)) ~ z'-"~ " T t .

This implies T2

2

fT e~ < c(K~ + k2 + ~m + k~(T~)), 2 n --

lim sup 1~ T2--,¢¢

which then gives lim sup

! [r~

T~...¢¢ T~jT

2
_ 1, then 3 constants hi(T) > 0 and continuous functions fi(e,T) with 0
0 such that t + 6 E [tj, tj + At] (see Fig. 8). Then,

f+, yT¢,) yw¢ L / 2 implies - 2 K z z < - g 2 z + K2 < ~; < g ~ z -4- kz -4- k(T.L) 1. Let N := LT~J. So we have ON=l[ti,ti+l] = [ti,tl +T'] such that ti+l - t ~ = 1, i = 1 , . . . , N - 1 and t u - iN-1 >_1.

Consider the following sets,

01= l[t~,ti+l] : I¢:O'] < v~,¥t • [ti,ti+x]) O2 = [t~,h + T]\O~. From Lemma B.7 we have that Yt • £21, ~_e for some t* • [ti, t~+l]. Due to (60) max{1,2cMko(T)} _> 1 we have (since ti+l - ti >_. 1) ti+,

s(t)dt >_~ s '

V[t,,ti+l] e ~22.

(60) and the fact

(61)

¢ll

Now let N1, Nu be the number of subintervals contained in £21 02, respectively. Then, from (44) and (61),

N2 < 72ks(cl + IJ2c2(T)T'),

N1 = [T'J - N2 _< T'.

Robust Continuous-Time Adaptive Control

193

Hence, z

jn.,

tr where 0 < ~r < 1 is a real number. This restriction can be interpreted as a stable invertibility condition for the nominal plant. Note that the plant itself need not be stably invertible. Elements which have an unstable inverse may be "hidden" in the unmodeled term. This is especially important since unmodeled high frequency dynamics invariably are present in practical applications. Also, note that in the adaptive design we will not explicitly enforce the stability property to hold for the estimated parameters. It suffices that there exist parameters such that the nominal plant is stably invertible and that the closed-loop system that results from applying a controller derived from such a model does not have "too high sensitivity". These notions will be made more explicit below.

y

*

U +

1

P

m

(X

Y k

I~dt

Fig. 1. Block diagram of nominal feedback system.

Using the definitions above we may rewrite (2) as

y(t) = a(q-1)y(t - d) + ~(q-x)u(t - d) + F ( q - ' ) v ( t ) , and the closed-loop expression for the tracking error becomes

y(t) - y*(t) = F(q-1)v(t) .

(4)

204

Ydstie

From the above we then have

~(t) =

1

(y*(t + d) - ~(q-i)(y*(t) + F(q-1)v(t))) ,

which we may rewrite as

u(t) =

1

(A(q-a)y*(t + d) - o~(q-1)v(~)) .

(5)

It is now clear that the truncation error satisfies

v(~) = A(q-X)A(q-t) B---~----q_t)q-a(A(q-a)y*(~ + d) - a(q-a)v(t)) + A(q-a)b(t), which we write as

v(t) = H0(¢, q-a)u*(t + d) + Ha(e, q-a)b(t) + ~(t),

(6)

where e(t) is introduced to take care of non-zero initial conditions (Fig. 2), a, A(q-1)q-1 rm(q- ) + A(q-1)a(q-a)q-a

H 0 ( ¢ , q - a ) = A(q-a) n ,

is the transfer function from the reference to the disturbance of the closed-loop system, and

B(q-') Hl(~b, q-a) __ Pm(q -1) + A(q-1)a(q-1)q-1 is the closed-loop transfer function from the bounded disturbance input to the truncation error. Note that e(t) is bounded and decays exponentially provided that the transfer function H0(¢, q-a) is exponentially stable. Prom the expressions above it follows that the tracking error satisfies

y(t) - y*(t) = F(q-1)(Ho(¢, q-a)y*(t + d) + H a ( e , q-1)b(t) + ~(~)) , and it is clear that the tracking and disturbance rejection problems can be formulated as optimization problems. In particular, by defining a set ~ so that B(q-1) -1 is a stable polynomial for all ¢ E fir, we solve the tracking problem by minimizing the transfer function F(q-a)Ho(¢, q-a) over ¢ E ~ under a suitable norm. In the multivariable case it is natural to formulate this as an H ~ - t y p e optimization problem. In this paper we work with a SISO system and we will work with a n / 2 - t y p e objective instead. We are now in a position to discuss the "optimal centering problem" and we will use the solution to this to define the concept of a best nominal model. Define the "compressed" /2-gain of a stable linear operator [10] 2

Stability of the Direct SeN-Tuning Regulator

205

Y

V

H0 b

Fig. 2. Block diagram of nominal feedback system.

and the exponentially weighted (truncated) or-norm

II~(t)l12,~-

~-,x(i)2. i----I

Note t h a t in the case or _- 1 this gives the ( t r u n c a t e d ) / z - n o r m of a function. P r o m (4), (5) and (6) we can define the optimal set of model p a r a m e t e r s with respect to the or-norm as 4"

=

arg rain IlH0(~, q--1)[12,a

For future reference let H~(q -1) = optimal control p a r a m e t e r s becomes

.

Ho(~*, q-l). T h e corresponding vector of

0* = (o,~, . . . , ~7~,,o~,..., ,6*+d_~) • 2 The argument that follows can equally well be formulated using the ~compressed" loo norm

IH(q-~)loo,,,= sup (lH(z-~)l) l,l=v~

defined relative to the norm Iz[oo ----supo/~mi. > 0 for all vectors ~ E 8*. The parameter estimates will be projected so that they belong to 8" at all times. The convexity of O* implies that we fix the sign of the high frequency gain of the modeled part of the plant. The lower bound, ~mi,, is introduced to avoid division with small numbers in the control equation and the bounds on 0 are introduced to prevent infinite parameter drift. By noting that we can write the optimized model in the predictive form (2) as

y(t) = ¢(t - d)'O* + 7 ( 0 , where 7(0 = F ( q - 1 ) v ( t ) , the following strategy for direct adaptive control becomes obvious Step O: Initialize 0(0) and ¢(-i), i = O, 1 , . . . , d - 1 and set t = 1 Step 1: O(t) = F o .

O(t - 1) + P ( t - 1)¢(t - d) r + ¢ ( t - d ) ' P ( t - 1)¢(t - d)

"

Stability of the Direct Self-Tuning Regulator

207

Step 2: Solve for

y*(t + d) - ¢(t)'0(t) - 0 Step 3: Set t = t + 1 ~nd go to Step 1.

This strategy is called a "certainty equivalent" strategy since the estimated parameters are used directly in the control law. We notice that the control equation (Step 2) is a re-statement of equation (3) using the estimated parameters. In Step 1, e(0 = Y(0 - ¢(t - d)'0Ct - 1) is the prediction error, O(t) = ( a l ( t ) , . . . ,an(t), ri0(t),...,/3m+a(t))' is a vector of estimated parameters and Fe-{.} is an orthogonal projection which ensures 0(0 E O* for all t. The parameter r _> 0 controls the (local) speed of adaptation. A modification can be introduced to avoid singularity when r = 0 since II ¢(t - d)l I = 0 may ~ v e division with ,.ero. 3 In [1] it was suggested to set r = ~ w h e r e 0 < )~ _< 1 is the forgetting factor, and use recursive least squares to update P(t) so that

1 ( P ( t - 1) - P(t - 1)~b(t_-zd)¢(t_.~ d)'e(_t 2 1 ) ~ P(t)

= -~ _

(A + ¢ ( t - d ) P ( t

-

1 ) ¢ ( t - d))

]

'

The forgetting factor can be updated using the algorithm proposed in [3]. The analysis presented here works for this approach provided P ( t - 1 ) is "regularized" so that 0 < PrainI _< P(t - 1) _< PmaxI, where Pmin ~ Pmax are positive numbers. T h a t is, P(t - 1) should be bounded and have bounded condition number. However, to simplify the algebra we will take P(t - 1) = P, a fixed positive definite matrix. The algorithm described above is known as the Direct Self-tuning Regulator with Parameter Projection. The objective of the paper is to demonstrate that this algorithm is finite gain stable provided that there exist ~ < 1 and a centering so that []H0(¢*, q-1)ll~.,a _< Kv, where Kv is a constant determined in Sect. 4. In other words: The STR with parameter projection solves the optimal centering

problem.

3 Technical R e s u l t s The problem with unmodeled dynamics in adaptive control has attracted considerable research attention. It is an interesting problem, mostly because neglected dynamics give rise to disturbance terms whose magnitude grow at the same rate as the process inputs, their effect persists if they have long memory, and quantitative statements about their magnitude cannot be made until the adaptive system has been shown to be stable. The standard analysis, which relies on decoupling and the use of a smallness in the mean property of the squared prediction error which is independent of stability, then fails. Until recently it was thought 3 In practice a dead-zone is included and we set 0(t) -- 0(t - 1) whenever I[~b(t)l[ _< z~ where A is an arbitrary small number.

208

Ydstie

that the algorithm needed to be modified to recover stability and performance. Several approaches have been suggested including data normalization [9] and relative dead-zones [12]. Both of these methods are aimed towards modifying the adaptive algorithm and in this manner the smallness in the mean property can be restored despite the presence of the unmodeled terms. A Bellman-Gronwall argument can then be applied to demonstrate stability of the closed-loop system. A recent survey of these methods is provided in [7]. In this paper we only use parameter projection, and the decoupling property does not play such an explicit role. Instead, we exploit an exponential stabilizing property of the ideal closed-loop system and the bounded rate of growth that results from the parameter projection and then apply the "switched system" approach presented in [14]. Sets of events where the controller destabilizes the closed loop are isolated and bounded rate of growth arguments are applied to show that on these and a certain number of preceding events the regressor overbounds the unmodeled dynamics. To motivate the development consider an example problem which illustrates, in part, why it is difficult to analyze adaptive control systems. E x a m p l e 3.1. (Ydstie and Golden [15]). Consider adaptive control of the system y(~) Jr- aly($

-

-

1) -- u(t - 1) + v ,

where v is a constant. Furthermore, if the setpoint is a constant different from zero, then the adaptive system can be represented by a mapping in the plane Y* --* -Ol Y + 7 01 --* #l + yl -O' ff + 7 - 1 C-}- y~22

(s)

where 0~ - ~(t)÷al, yJ - y / y * , c - r / y .2, and 7 = l ~ - v / y * . A typical result from a simulation is shown in Fig. 3. What we notice here is the persistent oscillatory phenomenon known as "parameter drift and bursting". While the output appears to converge towards the setpoint, the parameter estimate drifts. Eventually the closed-loop system becomes unstable since a dosed-loop pole migrates outside the unit circle. The output diverges, this causes excitation and the parameter estimate then retunes. This pattern continues indefinitely and the phase plane plot in Fig. 3 indicates that there is a-periodicity and sensitivity with respect to initial conditions. [] The main issues here are the following: - It is not possible to give a complete characterization of the dynamic behavior of even very simple adaptive control systems. - Despite complex dynamics we have a measure of robustness and rapid decay of initial conditions. - The average tracking error is of the order of the external noise.

Stability of the Direct Self-Tuning Regulator

209

2.5 -.. ::...:,..:~. : - , ,~-.:" ,.:..'.'~:.~:a.c,'v ,,.~,." ~-,." :. •

•

•

chaotlc

burst

_

•

.

.,

.

-

1.5 '

~.]"

-

~,.

%..

.'

.

. •

..

...f:~'.~~;~- ~,'~',%, -,~: . .'.. "

:..,~*--:.. ~;.~ .. :." "? " • "~ ".'..

•

:rf'';"'"

.

~" ~- 7 , where m = 2d+lMe pmaxK÷ ,g~ Pmin

.

g2 -- O"

Note that we have settled for expedience rather than optimality in some of these definitions. s

z(i) 2 is the Euclidean n o r m

ilzl I =

iffil

216

Ydstie

P r o p o s i t i o n 4.1 ( I d e a l ease, f a s t a d a p t a t i o n ) . Consider direct adaptive control of (1) with no model mismatch, b(t) -- y*(t) = 0 f o r all t (ideal case) and r = 0 (fast adaptation). Let or* be the smallest real n u m b e r such that II g;(q-X)ll2,o• = 0 and B(z -x) ¢ 0 for I~I > or*. Y or" < I then the adaptive s y s t e m has bounded transient so that

sup II¢(t)ll ~ < uT max I1¢(-i)11 ~ , t>o

--

ii pmax ,I where rn = •,~Id+1 ~wo rt¢ . g~ Pmin

g2

O_

, - * ' max I1¢(-i)112 O0

where, according to the definitions above, N U * < 2d+lMs pmax r" g~ -- m Pmin ~ g ~ -- or -

-

"

T h e first part of the result then follows from L e m m a 3.1 by application of (10). We also must conclude

a(t+l)<e

-'~

gl

a(0),

with =-lng

. 2-(lug1-

in .~2d+lK Ms g~ Pmax u2 z ÷ -ff g~ - or Pmin

Choosing k = 2N* is convenient in this case and gives the conservative bound

= -lug;(1-1/2), which establishes the result.

El

Stability of the Direct Self-Tuning Regulator

217

The estimate for the rate of convergence has not been optimized and is slower than what we would expect if the parameters of the plant where known exactly. (When the parameters are known we have 5 = - I n ~*. In the adaptive case the bound above ensures/~ < -0.5 In x / ~ ) . We note here that exponential stability of the signals does not necessarily imply convergence of the parameter estimates. It is not even clear whether or not the estimates converge. With the deadzone we get lim0(t) = Ooo where 0oo can be viewed as a random variable and

l

sup II (t)ll _< ,a.

Indeed, this point has caused some confusion in the field of adaptive control. Several earlier attempts to demonstrate exponential stability of the closed-loop system depended on exponential stability of the parameter estimator and convergence of the parameters. Such an approach relies heavily on the use of the concept of persistent ezcitation. In one form, persistent excitation implies the existence of positive constants el, c2, n such that

c~z _ 2, er = a* and g2 --- g~ so t h a t -lng2 - B*. In gl - In g2 As in the previous example, the expression for U* simplifies and we get the condition for stability:

U*(R,N,a, g2)
O

where

NU* = kin,

,

.=

and hence

=

g r/pm + r+(k-

1) +

glkl, i=O

'

k

=

2

gives a reasonable bound. T h e first part of the result follows. T h e bound for the transient increases with r/pmax large. This is not surprising since a wrong set of initial conditions for the p a r a m e t e r estimates m a y give a large transient if the rate of a d a p t a t i o n is low. It then takes more time to correct modes leading to instability and the transient performance deteriorates.

Stability o f t h e Direct Self-Tuning Regulator

219

We now turn our attention to the exponential decay property. I m m e d i a t e l y

a(t + 1) _< m a x { R ~ , e - " K ~ a ( O ) + k~} , where the constants need to be computed. First we have 6 = - In

g~ - U*(N, R, a', g2)(ln gl

- In g2).

W i t h R = nR*, n > 1

U*(N'R'a'g2)=B*(

l+k-1)n

so t h a t , according to L e m m a 3.1, 6=--lng~

1--~+~

. n

Clearly, to achieve 6 > 0 we need 1

k-1

or n > k. Somewhat arbitrarily, we set k = 2 and n = 3, which gives

6 = - g 1 In g~. To finalize, we need to determine NU*. F r o m the above, 5

.

5

NU* = -~B < -~m. Hence

R ~ = 3K~R* + k~, where K~ = \ g 2 /

'

i=o \ g z /

and the second p a r t of the result follows. Finally, we show convergence to zero error. First,

v ( t ) 0 b(t) 2 _< kb and

sup,>0 u*(t) 2 ~'- Y ~ * < 1, then

sup I[¢(t)l[ z < t>0

max

t~"~' Ko--<m~d I1¢(-i)11' + 'o,~ + k,,

where Ran, Ko and ko are defined below. Ezponential convergence to a compact set is achieved in the sense that I1¢(t)]12 < m a x { t ~ , e - ° t K ~ omia~xaI1¢(-i)11' + k~) + k , ,

where 6, Pdm,K~ and k~o are positive constants defined below. In addition, we have7 [lu - y*ll~ --- #o + #llF(q-X)A(q-t)l~llb[]2, where #i, i = O, 1 are constants independen~ of the initial conditions. Proof. From Lemma 3.2 with K7 = 0, U*(R,N,(Lg2) -- gz g'- a [ 2 ~ + 2d+t ( K ' + k~ T r/pmax)

,

, ~+~+~"

~ - ' ~ 7 ÷ ~ - T 7 , ". . . .

Pmin

_~..+~,

The adaptive system is stable if we can find N, R, u and g2 such that

U*(R,N,a,g~)
_ d II¢(t)ll ~ < p~, where

P~ = K, max { KoR + ko,g~" ~ ~ ' . Provided that a* < 1, we have Proposition

sup ,>0 II ¢(t)112 g~a(t),

1-

A(t) < .(t + 1------S

(20)

234

Ydstie

To determine the relative occurrence of the events A(t) = 1 we need to relate this equation to a very specific gradient property of the parameter estimator. The swapping error ~(t) = ¢(t - d)'(O(t - 1) - O(t - d)) satisfies d-1

10(t)l _< II¢(t - d)ll ~

IleC(t - d - i)e(t - i ) / n ( t - i)11, i=1 where we used the fact that Fe-{.} projects orthogonally onto 61" so that Ilfo.{O" - O}ll _< lie* - ell for all 8 • O*. Since IIP¢(t - d)ll2/n(t) < pmax w e then have, using (10)

a-1

e(t - i) I

1o(t)l < (Kca(t - d + 1) + L ,1/2_1/2 K " i=1

Here we used the fact that a(t) > re(t). By noting that ¢(t - d)'~(t - d) =

e(t) - 7(t) + r~(t) we have the inequality

I _< le(t +

d)l + IK-~a(t + d + 1) + k.vl 1/2

+ lKi~a(t + 1) + kc] 112,112

d-l[ e ( t + d - i )

[

" m ~ Z.., i=1 I n~" ~ ~-_" i)q'/21 '

so that from (20)

A(t)

1- ~

< a(t + l"------~

n(t + d)ll 2 +

+ (K¢a(t + 1) + '- ,1/2 _1/2

x~)

d-1

+ 2

le(t + i)1

1"maxE n(t i=1

+

i)1/2

Now, a(t) enjoys the same bounded rate of growth property as re(t) (Equation (12)), so that for all t and i > 0 we have i-1

a(t + i) < g~a(0 + k~,

k~ = g~ ~

~kl.

j=0 Since n(t + d) < r + pm~xll¢(t)ll 2 < r + pmax(Kca(t + 1) + ks) this gives, by re-arranging the expression above

A(t) ( 1 - ~a ) < a(t l+ 1------5[IgaKTa(t+l)+k3+kTil/2 a

[ e ( t + i ) [ ]2

+ Ir + pm~(Kca(t + 1 ) + k~){l/2 Ei=l n'~'~ i)'~'2 J "--'--'+--'~[k3 k7

<elK~+2a~t+l)+ d e(t + i) ~× ~i=1 .(t + i) '

2 al (19maxK¢ + r -{- .kC_.Pmax.'~ -a(t + 1) J

(21)

Stability of the Direct Self-Tunlng Regulator

235

where c, = 291a. From this we then have by summing over the interval [t - N, t]

'(

( 1--~2-)' ~

A(i)< Z

im$--N

i=l--N + 2d

ClK'r+ a ( / + l )

(praaxK+ + . +a ( p,,=k+ --'~) ']

e(i + #)') " (22) n-'(i-'+W ./=1

The last t e r m is the sum of squared a posteriori errors. These axe known to satisfy a smallness in the mean condition when the parameter estimates and the unmodeled terms are bounded. Below we show that e(t)2/n(t) is small in the mean on the sets where the adaptive controller is diverging. We use this property to demonstrate that the regressor measures a(t) and from this we get ultimate boundedness of the extended state when there are unmodeled dynamics. To elaborate on this point we introduce another comparison function, namely V(t) = O(t)'P-*O(t). A bit of algebra shows that with the projection we have

V(t) < V(t - 1) - e(t)2 - 2e(t)v(t) -

n(t)

i e(t) 2

0. The expression on the right hand side is positive and makes sense if Kv is small and R is large. We now have, using expression (23) and the inequality derived above, still assuming that a(t + d - i) > R:

(

1-- cr

A(i) V~;-~ ("'-')

× k~;

We now use the inequality (a - b) ~ >_ 0.5a ~ - b~, which gives 2

)'"

,,K.,,

f

ga

>__o.s\~)

V~-~-.(~,--) -~, + ~ , (~.- o)- ~,, (Ko + , ' . " R+ ~o~/ ' T ~

and this establishes Lemma 3.2.

[]

Adaptive-Invariant Discrete Control Systems Ya. Z. Tsypkin Institute of Control Sciences, Moscow, USSR.

Abstract. The paper investigates the structure and properties of selective-invariant and adaptive-invaxia~t discrete control systems in which the effects of external regular disturbances axe eliminated and those of external stochastic disturbances are essentially weakened. Possibilities of eliminating the constraints caused by multiple delays and nonminimum phase of a dynamic plant are described.

1 Introduction A great number of works [1] is devoted to the problem of invariance or, in other words, the problem of compensating the influence of external disturbances both in continuous and discrete control systems. Conditions of absolute or complete invariance, providing accurate compensation of arbitrary disturbances are, as a rule, physically unrealizable. The need to relax these invariance conditions, and ensure compensation of a known class of disturbances, has led to the notion of selective invariance in continuous systems which was introduced by V. S. Kulebakin [2,3]. Availability of a priori information from various disturbance models made it possible to design systems capable of resisting broader classes of disturbances. The theory of such control systems has been developed by Johnson [4,5] for both continuous [4] and discrete systems [5]. With an insufficient level of a priori information on disturbances it is expedient to use the adaptive approach which provides an estimate of the class of disturbances and uses it for disturbance compensation. Systems which use adaptation to achieve invariance are called adaptive-invariant systems. This paper presents design principles of ~2-adaptive-invariant discrete-time control systems. After a brief review of discrete-time control systems the character of a priori and current information about regular invariant systems is considered and structures of selective- and adaptive-invariant systems are established. Then it is shown that an insignificant modification of system structures allows us to remove constraints caused by delays and nonminimum-phase plants. Properties of selective- and adaptive-invariant systems are analyzed when the compensation condition is accurately satisfied. Finally, possibilities of constructing systems with stochastic disturbances which are approximately adaptive-invariant, that is #2-invariant, are analyzed and new problems are formulated.

240

Tsypkin

2 Properties

of Discrete

Control

Systems

Discrete-time control of continuous plants leads to difference equations which can be written in the following form

Q(q)y(n) : ql+kPu(q)u(n) -I- f(n).

(1)

At n-th time instant, y(n) is the output, u(n) is the control, f(n) is an external disturbance and q is the unit lag operator: =

- m),

m = i, 2

....

The polynomials Q(q) and Pu(q) with degrees N and NI, respectively, are such that Q(0) = 1, P~(0) ~: 0. A polynomial satisfying the condition Q(0) - 1 is referred to as a monic polynomial. To simplify the notation the lag operator and the polynomial variable are denoted by the same letter. The integer k :> 0 indicates a multiple lag. The minimum possible lag of a plant is equal to 1, and corresponds to k -- 0 [6]. This fact should be taken into account in the analysis and synthesis of discrete-time control systems. Furthermore, we shall assume that the plant is stable and minimum-phase, i.e., that all the zeros of the polynomials Pu(q) and Q(q) lie outside the unit circle Iql -< 1. For brevity such polynomials are said to be "external" [7] or stable. The requirement of plant stability is not an essential constraint, since a known unstable plant can be always stabilized by feedback. A nonminimum-phase plant imposes some definite limitations to controller synthesis [6,7]. The possibility of overcoming these constraints will be discussed in Section 7. Meanwhile, if not stated otherwise, a plant is assumed to be minimum-phase and stable. The block-scheme of a plant is given in Fig. 1.

-¢-)

.I

[ v ~ /

r

Fig. 1. Plant to be controlled.

Without external disturbances, f ( n ) = 0, the plant equation (1) becomes Q(q)u(-) = ql+kP.(q)u(-).

(2)

As an optimal controller for such a plant we define the controller which satisfies the following condition: T°(q)(Y(") - Y0(n)) = O,

n > 1 + k,

(3)

Adaptive~Invariant Discrete Control Systems

241

where T°(q) is an external monic polynomial of degree/72 > 0, with the assigned distribution of zeros, and yo(n) is the output of a reference model,

Go( q )yo( n ) = qt +~Ho( q)r( n ) ,

(4)

where r(n) is a given reference input, and Go(q) and Ho(q) are external polynomials of degrees Na and/74, respectively. It is known from [6,7] that an optimal controller for plant (1) is described by the equation

Pu(q)S(q)u(n) = T°(q)P(n) - P(q)y(n).

(5)

Here S(q) is a monic polynomial of degree k and P(q) is a polynomial of degree max(N - 1, N2 - k - 1) satisfying the polynomial equation

Q(q)S(q) + ql+~p(q) = TO(q)

(6)

~(n) -- Ho(q) r(n). Go(q)

(7)

and e(n) is defined by

The block-scheme of the optimal system described by equations (2) and (5) is shown in Fig. 2a, and its equivalent form is shown in Fig. 2b. With k = 0, which

i ..( ,)s, , q - (~)

"¢")_1

T.(q) l

J,

,

I

"(")

I P(q) I" (b) Fig. 2. Two equivalent forms of the optimal system. corresponds to S(q) = 1, we obtain from (2) and (5)

Q ( q ) y ( . ) = qP~,(q)u(n),

(8)

242

Tsypkin P.(q)u(n) = T°(q)~(n)

P(q)u(n),

-

(9)

From (6) it follows that

(1o)

Pu(q)--q-'(T°(q)-Q(q)) and (3) reduces to

T°(q)(y(n)

-

yo(n)) --

0,

n ~ 1,

(11)

where

Ho(q)r(n)

yo(.) = q G--~

"

(12)

-

In this case it is not necessary to solve (6) for S(q) and P(q), nor to include a multiple lag in the reference model. The block-scheme of the optimal system (8) and (9) is shown in Fig. 3a and its equivalent form is given in Fig. 3b. Note that for a nonminimum-phase plant the system becomes non-robust [8] due to cancellation of the polynomials Pu(q) in the plant and the controller. Thus a change in the structure of the controller is necessary. Nonminimum-phase of a plant and its multiple lag impose additional constraints upon the choice of a reference model [6,8].

r(.)j~ r l H ~ II P(q)

(~)

~1 ~

TO(q) l rT

rl P(.)

I

(b) Fig. 3. Optimal system forms in the ca~e k = 0. For completely deterministic perturbations f(n) ~ 0 the synthesis problem of optimal control for a plant described by (1), with respect to various optimality criteria, leads to the solution of corresponding variational problems [8].

Adaptive-Invariant Discrete Control Systems

243

For example, if the disturbances satisfy a known bound, then the minimax or game approaches can be used to determine the "best" result for the "worst" disturbance [9-11]. Absolutely invaxiant discrete systems under any arbitrary disturbances are, as a rule, unrealizable, because they require infinite feedback gain [12] which is not allowed because of the lag inherent in the discrete-time plants. The problem of synthesizing an adaptive-invariant discrete system consists in determining the structure and parameters of a controller, such that the control system response coincides with the response of the optimal system in the absence of external regular disturbances. Further problems consist in consideration of possibilities to synthesize the adaptive-invariant systems, where the constraints caused by multiple lag and plant nonminimum-phase would be removed. Solution of these problems will be based on a combination of the adaptive approach [7] with the theory of selective invariance [2,3] for discrete control systems. 3 A Priori

and Current

Information

on Disturbance

A priori information on the disturbance plays an important role in synthesis of the adaptive-invariant systems. Assume that disturbance satisfies a difference equation f ( n ) - D(q)f(n - k - 1), (13) where D(q) is a polynomial of degree M. In the sequel (13) is called the equation of extrapolation or prediction, while the polynomial D(q) is a prediction polynomial. The prediction equation is easily reduced to the compensation equation (1 -

q1+~D(q))f(n) = 0.

(14)

Let us call the polynomial 1 - q1+}D(q) a compensation polynomial. Equation (14) indicates the conditions under which the function f(n) is being compensated. Theorems providing the possibilities to represent a wide class of continuous functions f(n) in the form of solutions of homogeneous differential equations have been formulated by C. Shannon in his investigations of differential analyzers [13] and by V. S. Kulebakin for problems of electromechanics and the development of selective-invariant systems [2,3]. Representation of functions f ( n ) in the form of homogeneous difference equations was considered in [14]. In the presence of sufficiently complete information on disturbance f ( n ) a prediction polynomial is quite easily determined. Thus, ff f(n) - d is constant then O(q) -1, if f ( n ) = d i n + do is linear, then D(q) - 2 - q, if .f(n) = a n is an exponential function then D(q) = a, etc. In some cases it is expedient to use polynomial splines [15] instead of a conventional prediction polynomial D(q). With incomplete a priori information a prediction polynomial is known only up to its coefficients which can be reconstructed from disturbance observations. Thus, there is a need in obtaining information about the disturbance and its estimate. As a rule, direct measurement of a disturbance is impossible. Therefore an indirect adjustable model method

244

Tsypkin

will be employed which is well known in identification [6,7,16]. However, the adjustable model will now play a different role. First, its parameters are not to be identified but rather determined from known values of the plant coefficients. Second, this model is a two-input finite impulse response (FIR) model [16]. Third, it is introduced here to identify a deterministic disturbance, rather than a stochastic one. The equation of this model has the following form: #(n) -- (1 - Q, Cq))y(n) + ql+k PuCq)u(n) .

(15)

We define the error = y ( n ) - v0Cn)

(16)

and obtain from (15) that e(n) = Q(q)y(n) - ql+~ p , ( q ) u ( n ) .

(17)

By virtue of the plant equation (1) the error is equal to the disturbance: e(n) -- f ( n ) .

(18)

Thus, the two-input FIR model provides an indirect measurement of the disturbance. It would seem that only one step is left to achieve invariance - to send a signal of the measured disturbance e(n) with the oppostie sign in order to cancel the effect of the disturbance. But this signal can be sent to the plant only by the controller, and the lag in the plant will cause the difference of disturbances f ( n ) - f ( n - 1) to appear. This difference for an arbitrary disturbance will differ from zero even in the absence of a multiple lag, k = 0. This fact most evidently testifies to the unrealizability of absolute invariance.

4 Selective-Invarlant

Systems

Let us use the above indirect method of measuring disturbances for synthesis of a selective-invariant optimal system. Consider a controller equation which differs from equation (5) in having an additional signal, depending on the error e(n): P u ( q ) S ( q ) u ( , ) = TO(q)~(n) - P ( q ) y ( n ) - D ( q ) ¢ ( , ) .

(19)

Here e(n) is found from equation (17) and the polynomial D(q) is a prediction polynomial, so that e(n) = D ( q ) e ( n - k - 1). (20) Now, taking into account that e(n) = f ( n ) , we obtain the compensation condition (1 - q l + k D ( q ) ) f ( n ) = O. (21) Let us show that controller (19) provides selective invariance for the plant Q(q)y(n) = ql+kpu(q)tt(n) + f ( n ) .

(22)

Adaptive-Invaxiant Discrete Control Systems

245

Indeed, substituting u(n) from (19) into (22) and taking into account polynomial equation (6) and equations of reference model (4) and (7), we obtain

T°(q)y(n) = T°(q)yo(n) - (1 - ql+k D(q))S(q)f(q) .

(23)

But, by virtue of the compensation condition (21), the last term in (23) vanishes and the optimality condition (3) follows from (23). The block-scheme of the selective-invaxiant system, described by equation (22), (14), (17) is shown in Fig. 4. There the processes do not depend on the disturbance f(n). The system

'-'(") ,.+~,~..~¢_¢_. ~j~e.,, ~ ) ~(,.,)

Fig. 4. Selective-invaxiaat system.

is equivalent to the optimal system in Fig. 2. Comparing the controller equation (5) of the optimal system in Fig. 2 with equation (19) of the selective-invariant optimal system in Fig. 4, we conclude that it differs only in the additional signal D(q)S(q)e(n) where D(q) is a prediction polynomial and S(q) is the solution of polynomial equation (6). This signal is generated by the FIR model (15) and a predictor described by equation (20). Equation (19) with ~(n) substituted from (17) can be written in the explicit form as (1 - ql+kD(q))P,~(1)u(n) = T°(q)f'(n) - [P(q) + D(q)S(q)Q(q)ly(n).

(24)

This equation reveals the role of the polynomial D(q) which is closely connected with the main ideas of the K(D)-transform used by V. S. Kulebakin for selective invariance [3]. Equation (24) is also close to the one used by C. Johnson [5] to construct control systems which accommodate disturbances. However instead of a two-input FIR model, [5] uses a reduced-order Luenberger observer and does not point to the connection with selective-invariance results.

246

Tsypldn

5 Adaptive-Invariant Systems If the a priori information on disturbances is not sufficient to determine in advance all predictor polynomial coefficients, then (19) will represent a polynomial with unknown coefficients 0, and so instead of (19) we shall have P u ( q ) S ( q ) u ( n ) = T°(q)~(n) - P ( q ) y ( n ) - D(q, O)S(q)e(n).

(25)

An estimator of the coefficients 0 = O(n) from the observations e(n) = f ( n ) can be realized using the adaptive approach [19,7], by means of a recurrent algorithm. To construct an algorithm, we consider a prediction equation similar to (20), namely ~(n) = D(q, 0),(n - k ~ 1), (20) where 0 is a vector of coefficients of the polynomial D(q,O) of dimension M, equal to a polynomial degree. Introducing the vector of observations' CkCn) = ( e ( n - - k -

1 ) , e ( n - k - 2 ) , . . . , e C n - k - M)) T,

(27)

(26) is written in the following form:

~(.) =

0TCk(.).

(28)

Using [7,19] a general recurrent estimation algorithm is OCn) = O(n - 1) + FoCn) [e(n)

-

0T(n

--

1)¢~(n)] Ok(n),

(29)

where Fo(n) is a gain matrix. In the simplest Kaczmarz algorithm [6,20] this matrix is to(n) =

[¢6T (n)¢k(-)] - 1 Z,

(30)

where I is a unit matrix. For the orthogonal projection algorithm [7,16,17] we have Fo(n) = [ ¢ [ ( n ) F o ( n ) ¢ k ( n ) ] - I F ( n ) F ( n ) - F ( n - 1) - F ( n - 1 ) ¢ k ( n ) ¢ T ( n ) F ( n -- 1) ,

CT(n)F(n - 1)¢k(n)

(31) (32)

if ¢ W ( n ) F ( n -- 1)¢k(n) # 0. Whenever C W ( n ) F ( n -- 1)¢k(n) = 0, then instead of (31) and (28) we let O(n) - 0 ( n - 1),

F ( n ) = F ( n - 1).

(33)

In practice the condition CW(n)F(n -- 1)¢k(n) ¢ 0 is usually replaced by the condition C W ( n ) F ( n - 1 ) ¢ k ( n ) > 6o,

where $0 is some positive number [7,20]. The estimation speed of the Kaczmarz algorithm can be increased by using a modification [21,22]. The orthogonal projection algorithm provides the estimate of coefficients in a finite number of steps

Adaptive-Invariant Discrete Control Systems

-\

247

~

N .f

~)

"X.lJ

E n)

F

~ Fig.

5.

4

Adaptive-invariant system.

[7,20]. The block-scheme of the adaptive-invariant system, described by equations (22), (25) and (24) is presented in Fig. 5. In contrast to the selective-invariant system, the system in Fig. 5 has an adaptive loop which accomplishes parameter estimation in the part of the controller which compensates the disturbance. The adaptive loop identifies the disturbance or, to be more precise, implements a prediction of the disturbance. The adaptive-invariant system becomes equivalent to the optimal system by the end of the adaptation process. Robustness of the adaptive-invariant system coincides with the robustness of the optimal system to which it is asymptotically equivalent. As it is well known [23,24], these optimal systems are robust, i.e., their properties change slightly for small changes in the plant and (or) controller. Hence the adaptive-invariant systems also possess the property of robustness.

6 Elimination of Constraints Caused by the Time Lag The presence of a multiple lag in the plant, k > 0, causes the controller to be more complex. Its order, defined by the degree of polynomial Pu(q)S(q), is equal to (NI + 1 + k), and it increases with the growth of k, while for k = 0, we saw that (9) acquires an extremely simple form. In this case there is no necessity in solving the polynomial equation with respect to S(q) and P(q), since S(q) -- 1 and P(q) is explicitly expressed in terms of T°(q) and Q(q). Let us now consider

Tsypkin

248

the problem of eliminating the influence of a multiple lag on the processes in the control loop. We will prove that a minor change in the equation of the controller of selective- and adaptive-invariant systems allows us to solve this problem. Consider the equation of plant (1)

Q(q)y(n) = ql+kP~,(q)u(n) + f ( n ) .

(34)

The block scheme of this plant (Fig. 1) with no disturbance can be represented in the form (Fig. 6) where the multiple lag block is isolated. To eliminate the

"(") l'l

IIv("+k)"['l

]

Fig. 6. Isolating the multiple lag block. influence of a multiple lag in the feedback loop, the input into the controller should be y(n + k) rather than y(n). This input cannot be obtained directly, but instead it is generated indirectly by means of the two-input FIR model used in the selective-invariant system. Denoting the difference of the two signals by z(n) = y(n) - y(n + k),

(35)

and the error of the simplified model by ,°(n) = O(q)y(n) - qPu(q)u(n),

(36)

Q(q)z(n) = e°(n).

(37)

we obtain z(n) from The equation of an optimal controller of the selective-invariant system, in which the influence of a multiple lag is eliminated, is represented in the following form:

P~,(q)u(n) = T°(q)~(n) - P(q)(y(n) - z(n)) - n(q)e(n)

(38)

where, as it has been seen earlier in (17),

e(n) = Q(q)y(n) - qa+~Pu(q)u(n).

(39)

The prediction polynomial D(q) is such that (1

-

ql+kn(q))f(n) = 0,

(40)

and z(n) is found from (36) and (37). Here we have in mind that the polynomial P(q) is determined by (10). Now the feedback signal is equal to y(n) - z(n) = y(n + k) as is seen from (36). Eliminating u(n) from (34) and (38) and taking into account (36), (37), (33) and (40), we obtain the optimality conditions (11), (12) for the optimal system without disturbance and multiple lag.

Adaptive-Invariant Discrete Control Systems

249

[-~],~

y(n + k)

a)

b)

Fig. 7. Selective-invariant system with multiple lag.

The block-scheme of a selective-invariant system without a multiple lag influence is shown in Fig. 7a, where the two previous models are combined. The block-scheme in Fig. 7b is equivalent except for the location of the lag element outside of the feedback loop. The processes in these systems are shifted relatively to another for an interval equal to the multiple lag. The idea of eliminating the lag influence in continuous systems on the basis of introducing dynamic models of a plant with and without lag was proposed by O. Smith [25,26]. It was also used in discrete systems [27-29]. The adaptive-invariant system which eliminates the influence of multiple lag differs from the selective-invariant system (Fig. 7a) by the presence of an adaptation loop similar to that shown in Fig. 5.

Tsyp]dn

250

7 Elimination of C o n s t r a i n t s Caused by N o n m i n i m u m Phase Plant The optimal controller becomes more complex for a nonminimum-phase plant. Modifications of the appropriate equations are presented in [6] and they will not be discussed here. Instead, we will consider the possibifity of eliminating such complexities. We shall prove that even for a nonminimum-phase plant a relatively simple controller can be used, changing slightly the process in the optimal system with a minimum-phase plant. The equation of a nonminimum-phase plant with no multiple lag, k = 0, is found from (34):

Q(q)y(n) = qPu(q)u(n) + f(n).

(41)

But now the polynomial Pu(q) has zeros lying both outside and inside the unit circle [q[ = 1. This polynomial can be represented as a product of the external P+(q) and internal P~(q) polynomials, i.e.,

Pu(q) = P+(q)P:(q).

(42)

Introduce the notation of a normalized inverse polynomial [6]:

[~:(q) : lqtPu(q-1),

(43)

where g is its degree. Then the polynomial

[~ (q) = P+(q)P~ (q)

(44)

will be external. From (44) and (43) there follows the equality

P~,(q) Pu(q-1) p.(q)

~2

(45)

=

The division of the polynomial/5(q) by the polynomial P~,(q) is realized with the help of a single iteration of factorization [30] with no need for explicitly determining polynomials zeros. Making use of the notation/5~-(q) in (43), we represent the block-scheme of a nonminimum-phase plant described by (41) without disturbances in the form shown in Fig. 8. In this ease the nonminimum-phase

~(.)

Fig. 8. Nonminimum-phase plant without disturbances.

y(n)

Adaptive-Invariant Discrete Control Systems

251

element is isolated. Its transfer function is equal to . With q = e -J~ it follows from (45) that the amplitude frequency of this element is constant, that is it does not depend on the frequency w [6]: yp(n) = P~(q)utn~ Q(q) , ,.

(46)

It can be isolated by changing appropriately a two-input model, and then used as in Section 6 to eliminate the lag influence. In this ease the equation of an optimal controller in the selective-invariant system, where the influence of nonminimum phase is eliminated, has the form Pu(q)u(q) -- T°(q)~(n) - P ( q ) ( y ( n ) - z ( n ) ) - D(q)e(q),

(47)

where e(n), z ( n ) and Q ( q ) z ( n ) = e°(n), are defined from the equations e(n) = Q ( q ) y ( n ) -- qPu(q)u(n) q- f ( n ) . z(n)

=

y(n) --

=

P"(q)

y(n) -- Q(q)

, ,

e°(n) -- Q,(q)y(n) - qPu(q)u(n) -I- f ( n ) .

(48)

(49) (50)

The compensation condition (40) is fulfilled and the polynomial Pu(n) is found from equality (10). The block-scheme of a selective-invariant system with the eliminated influence of nonminimum-phase is shown in Fig. 9a, and the block-scheme which is equivalent to it is given in Fig. 9b. It differs from the optimal control system of a minimum-phase plant in that the phase change is outside the control loop. Of course in this case yp(n) will differ from y(n). All we can say is that amplitudefrequency spectra of the processes yp(n) and y(n) will coincide. This is the price we paid for nonminimum-phase influence elimination. Moreover, elimination of the influence of nonminimum phase and plant lag can be combined if necessary changes are entered in the utilized two-input model. By introduction of the adaptive loop into the selective-invariant system we obtain the adaptive-invariaat system to which the optimal system will be equivalent after the completion of the adaptation process. 8 Approximate Systems

Realizations

of Adaptive-Invariant

Assume that a disturbance f ( n ) is generated by a piecewise continuous function f(t) but not by a continuous one. In this case compensation equation (14) acquires the form (I -

q1+~D(q))f(n)=

x(n),

(51)

where x(n) are jumps caused by the change of intervals of the continuous functions f(t).

252

Tsypkin

b)

Fig. 9. Selective-invariant system with a nonminimum-phase plant.

In this case the optimality condition (3) is replaced by

T ° ( q ) ( y ( . ) - y0(.)) = x ( n )

(52)

I f the change of intervals of f(t) occurs very seldom then the influence of x(n) on the processes in the system is insignificant. In those cases where the disturbance f ( n ) is changing so slowly t h a t the first difference A f ( n ) ----f(n) - - f ( n - 1) is sufficiently small, it is sufficient to assume D(q) -- 1. In this case there will be no need for adaptation and we come to the realization of the simplest selective-invaziant system. If the first difference is not very small but the value A f ( n ) -- a A f ( n -- 1) is small, which is with a - 1 equal to the second difference, then we should assume

Adaptive-Invariant Discrete Control Systems

253

a prediction polynomial of the following form

D(q) = (1 + ~) - ~q.

(53)

Tuning of the parameter a is done by the adaptation loop. In this case we realize the simplest adaptive~invariant system. 9 cr2-Adaptive-Invariant

Systems

Up to now it was assumed that external disturbances f ( n ) are determinatistic. Let us now consider that the disturbances ~(n) are stochastic and they satisfy the conditions E{f(n))=0, E{~(n) 2 } = a ~ . (54) For simplicity we consider a minimum-phase, stable plant without multiple lag (k = 0) and write its equation in the form Q ( q ) y ( n ) = qPu(q)u(n) T ~ ( n ) .

(55)

As it follows from [8,20,31] an optimal controller, minimizing the criterion of the generalized mean square error

(56) coincides with the optimal controller (9) for the same plant but with no disturbance. However, in contrast to (11) we now have

(57)

T ° ( q ) ( y ( n ) - yo(n)) = ~(n),

and, consequently, a minimum value of criterion (56) is equal to the variance of the random disturbance: Jmio ---- E

-- y 0 ( n ) ) ] 2 }

---- E

----

(58)

The block scheme of the optimal system with stochastic disturbance is shown in Fig. 10. It should be noted that, as opposed to systems with deterministic disturbances, where the transient processes were of main interest, in the systems with stochastic disturbances the steady-state processes are in the center of attention. With the stochastic disturbances the notion of invariance loses its meaning. However, the problem of synthesizing the system, that is adaptive-invariant up to a s can be formulated, i.e., the system for which the optimality criterion Jmin reaches the smallest possible value. We assume a priori that the disturbance ~(n) is determined by the autoregression (AR) equation [32,16]: ( ( n ) = C ( q ) ( ( n - I) -i-e(n).

(59)

Tsypkin

Fig. 10. Optimal system with stochastic disturbance.

Here ~ ( nis) a sequence of independent identically distributed random values, satisfying the conditions

E { e ( n ) ) = 0,

E { ~ ( n ) &( nm ) } = 0

with m = 0 with m # 0,

(60)

and C ( q ) is an external polynomial of degree M, with unknown coefficients. This polynomial is actually a prediction polynomial. The variance of ( ( n ) is equal to the minimum of the optimality criterion (58). Substituting C(n) from (59) in (58) and taking into account (60) and equalities E { ~ ( n ) & ( nm ) } = 0 with m >_ 0 we obtain

whence it follows that always

AR-equation (59) leads to the form

This is the equation of incomplete compensation while the polynomial 1- qC(q) of degree M 1 is referred to as a polynomial of incomplete compensation. The problem of synthesizing the a2-adaptive-invariant system consists in determining the structure and parameters of a controller which reduces J,;, = u; to smallest possible value Jmi,= u2. The equation of an optimal controller of the a2-adaptive-invariant system for the plant

+

Q(n)y(n)= nPu(n)u(n)+ C(n)

(64)

Adaptlve-Invariant Discrete Control Systems

255

can be represented in the form Pu(q)u(n) -- T°(q)~(n) - P(q)y(n) - C ( q , O)e(n) ,

(65)

where e(n) is the error e(n) = Q(q)y(n) - qPu(q)u(n),

(66)

which, as it becomes obvious from the comparison of (64) and (66), is equal to the disturbance

,(n) = ¢(n).

(67)

The prediction polynomial C(q, 0) in (65) is not fully determined. The vector 0 of its coefficients is unknown. To obtain the estimate O(n) we shall use the estimation algorithms in the presence of noise [16]. Introducing the observation vector ¢(n) : (e(n - 1) . . . . . e ( n - M)) T ,

(68)

prediction value of e(n) denoted by ~(n) is ~(n) = c ( q , o)~(n - 1) =

OT¢(n).

(69)

The best prediction will be realized with minimization of the prediction criterion

=

} -

{

or, which is equivalent, satisfying the following condition: VJpr(0) = - E { [e(n) - owtb(n)] ¢(n)} = 0.

(71)

On the basis of the adaptive approach [19,16] we can select an estimation algorithm, for example, of the least squares type: 0(n) -- O(n - 1) + / ' ( n ) [e(n) - 0We(n)] ¢(n), /'(n) = / ' ( n -

O(0) ---- 00

1) - ['(n - 1 ) ¢ ( n ) C ( n ) I ' ( n - 1) F(0) --- a i r, a >> 1. 1 + C W ( n ) F ( n - - 1)¢(n) '

(72)

This estimation algorithm is optimal on the class of disturbances e(n) with bounded variance [16]. It realizes identification of the disturbance ((n). The block-scheme of the a2-adaptive-invariant system, described by equations (64)(66), (72) is shown in Fig. 11. The adaptation loop corrects the controller in order to decrease the influence of the disturbance. An equivalent block-scheme for the a2-adaptive-invariant system has a similar form shown in Fig. 10 after a disturbance ((n) was replaces by e(n). If ((n) is a sequence with independent increments [32] then (1 -

q)~(n) -- ~ ( n ) ,

which corresponds to C(q) -- 1 in (59). In this case the optimal system is unfit for operation since ~ = c¢. Assuming that Jmin = ¢x~ in (65) we obtain C(q, 0). The

256

Tsypkin

B~ Fig. 11. a2oadaptive-invariant system.

(n)=(1

D"

Fig. 12. Special case: ¢(n) with independent increments.

-

q)-tc(n)

Adaptive-Invariant Discrete Control Systems

257

block-scheme of the selective-invariant up to ~ system is shown in Fig. 12. There is no adaptation loop in it and the prediction equation realization is extremely simplified. The equivalent block-scheme for the selective-invariant system has the form similar to that shown in Fig. 10 after a disturbance ~(n) was replaced by e(n). In this case the selective invariance up to a2 turns the unfit original system into an optimal system with a minimum error variance. If ((n) is a stochastic disturbance, determined by a moving average autoregression equation (ARMA) [16,32]

((n) -- C(q)((n - 1) + s(n) + B(q)¢(n - 1), where B(q) is an external polynomial, then instead of (63) we obtain the equation of incomplete compensation:

( 1-

B(q) ¢(-) =

(73)

q l + qB(q) ]

and the prediction equation is ~(n) -

C(q)

.4- R(~"l

'---''~n - 1). 1+ qB(q) "

Now, instead of the prediction polynomial there appears a prediction transfer function. The block-scheme of such a a2-adaptive-invariant system coincides with the one shown in Fig. 11, if in the adaptation loop we use C(q) + B(q)

1 + qB(q)

instead of C(q, 0). Surely, in this case the algorithms become complex, because it is necessary to estimate the coefficients of both C(q, O) and B(q, 0). The results obtained can be generalized to the case when the plant possesses a multiple lag. However, we shall not spend time on describing this procedure. Due to incomplete compensation of a stochastic disturbance, eliminating the constraints caused by multiple lag and nonminimum phase will lead to a transformation of the disturbance e(n) and the variance of this transformed disturbance will be greater than a 2. This is a price to be paid for constraints elimination. 10 S o m e

Generalizations

Assume that the disturbance is the sum of a deterministic, f i n ) , and a stochastic disturbance, ¢(n). In this case the compensation equation (14) with ~(n) is replaced by the equation of incomplete compensation (1 - D(q))(f(n) + ~(n)) = (1 - D(q))~(n).

(74)

Thus, it follows that in a selective-invariant system a stochastic disturbance is transformed and the variance of the transformed disturbance is increased because

E {[(1-

> E

=

(75)

258

Tsypldn

Let us find another prediction polynomial with which the variance of the transformed stochastic disturbance would be smaller: (1 - qD(q))(1 - qC(q))(f(n) -I- ~(n)) - (1 - qC(q)) [(1 - qD(q))f(n)] = (1 - qD(q))[(1 - qC(q))f(n)].

(76)

Taking into consideration the compensation equation of a deterministic disturbance (14) with k = 0 and the condition of incomplete compensation of the stochastic disturbance (63) we obtain from the equation of incomplete compensation (1 - qD(q))(1 - qC(q))(f(n) + ~(n)) = (1 - D(q))¢(n).

(77)

It is obvious that E { [ ( 1 - qD(q))¢(n)] 2) < E {[(1 - qD(q))~ (n)] z }

(78)

0.2 < E{[(1-qD(q))¢(n)] 2} < 0.~.

(79)

and, hence:

Thus, using the equation of incomplete compensation (76), the variance of the transformed stochastic disturbance is decreased in comparison with ~r~, but it does not reach the smallest possible value 0,2 . Noting that (1 - qD(q))(1 - qC(q)) = (1 - q( D(q) T C(q)) ) + q2D(q)C(q) , it follows that the prediction polynomial in this case is

A(q) -" D(q) + C(q) - qD(q)C(q) .

(80)

In the adaptive-invariant system we must estimate the coefficients of this polynomial. They enter (80) nonlinearly, or ff overparameterized, they become dependent, which makes the algorithm more complex. Essential simplification is attained in the approximate realization of the invariant systems described in Sections 6 and 10. Thus, assuming D(q) -- 1 in (80) we obtain

A(q) = i q- (1 - q)C(q).

(81)

If, in addition, f(n) is a slowly changing deterministic disturbance, then we can assume A(q) --- 1. The processes in this system will be weakly dependent on

f(n).

Adaptive-Invariant Discrete Control Systems

259

11 E x a m p l e s Consider a dynamic plant described by the equation

Q(q)y(n) = qPu(q)u(n) 4- ((n) + f ( n ) , where

Q(q) = 1 + 0.5q + 0.2q2;

Pu(q) = 1.1 -b 0.3q,

(82)

to which a regular disturbance f ( n ) and (or) a stochastic disturbance ((n) are applied. The regular disturbance is a periodic saw-tooth signal 0.4(n - 50rn)

50k - 12.5 ~ n < 12.5 + 50k,

-0.4(n - 25 - 50k)

12.5 + 50k _< n < 37.5 + 50k,

fl(t) =

(83)

where k is an integer; or the sum of a periodical saw-tooth signal and a harmonic disturbance f ( n ) -- fl(n) 4- f2(n), (84) where

27fn f2(n) = 5sin 25

(85)

Alternatively, the disturbance f2(n) can be stochastic, f2(n) = ((n), with ((n) described by the equation of the form (63), or 1

((n) = 1 - qV(q) e(n)'

C(q) = 2 - q,

(86)

where e(n) satisfies conditions (60) with a = 1. The reference model is determined by equation (4) with

H°(q)=G°(q)=l,

k=O,

yo(n)=qr(n)=r(n-1).

The signals ~(n) = r(n) and yo(n) are presented in Fig. 13. For selective-invariant systems (as it follows from (19)) with k = 0, S(q) = 1, the equation of a controller takes the following form

P~,(q)u(n) = T°(q)~(n) - P(q)y(n) - D(q)e(n) e(n) = Q(q)u(n) - qP,,(q)u(n) ,

(87) (88)

where T°(q) = 1 + 0.5q, Q(q) and P~(q) are assigned by equation (82):

P(q) = q-l(T°(q) - Q(q)) = 0.3 - 0.2q (89) D(q)

= 2 -

q.

Under the control law (87) the responses of the plant y(n) and the reference model yo(n) to f ( n ) (83), which is equal to the sum of regular disturbances (83), (85) (Fig. 14) are presented in Fig. 15. With D(q) = 0, the error of a conventional system, shown in Fig. 16, is increased when compared with Fig. 15.

260

Tsypkin

1.2

J

~

t

i

Reference Model

0.4

- ..........

i

-0.4

-0.8

-1,2

40

80

n

Fig. 13. Reference model signals r(n) and

2.4

~.6

'1

'

r

I

'

L

120

160

200

yo(n).

i

Disturbance

0.8

--

0

-08

-16 I

40

Fig. 14. Regular disturbance

80

f(n).

n

120

:

160

r

200

Adaptive-Invariant Discrete Control Systems

1,2

0.8

261

I

(

Output ~y(n) .......... Yo(n)

0.4 :=" 0 ~-0,4 -0 8 I

-1,2~ 0

4O

,

J

'

f

BO

r

n

I

I

120

J

i

~

p

,

I

160

200

i

1.2

0.8

-0.8

-I .2 J

0

40

80

,

120

n

Fig. 15. Selective-invariant system output and control.

~BO

200

Tsypkin

262

2.Ski

i

2.o/~

~

i

[

Ordinary Output

,sf/~

....... ,

& os

II ~

[

I

/.~

~

l

1

I \

~ln)

,.oV....... I >'.

=

.............. ~o,o,t.......t~

"

o

.......

-1.0

.

.

.

.

"....

.

.

.

...

.

• 1.5

n

i

I

i

I

I

I

J

i

I

( ~

°,L

0,4

~0

-0.4

-0.8

-'L21

I

40

80

I

120

n

Fig. 16. Conventional system output and control.

f

q

160

:

200

Adaptive-Invarlant Discrete Control Systems

263

Finally, we assume that an additional stochastic disturbance (86) (Fig. 17) was applied along with regular disturbance (84) to a plant. In this case the error of the selective-invariant system, as shown in Fig. 18, is quite close to that of the nominal system, i.e., of the system in which regular disturbances are absent and instead of the stochastic disturbance ~(n) we apply to the plant the disturbance e(n) whose variance is close to 1. A conventional system (with D ( q ) = 1), as it can be seen in Fig. 19, is not applicable. Control actions in selective-invariant and nominal systems essentially differ from one another (see Fig. 18 and Fig. 19). The reason is that the variance ~(n) in the course of time tends to infinity. We shall limit ourselves to these simple examples as illustrations of the principle of selective invariance.

1.4

1.2

A c

0.~

0,4

1 40

r

I 80

Fig. 17. Stochastic disturbance ¢(n).

J fl

r 120

I

f 160

200

264

Tsypkin

0.8 0.4

-0.4 -0.8 -1.2

I

40

8O

I

120

n

160

/

-0.2

2OO

k

-0.4

-0.6

-0.8

-

-1.0

-1.2

J

F

40

I

I

80

t n

r

I

120

Fig. 18. Selective-invariant system output and control.

I

160

i

200

265

Adaptive-Invaxiant Discrete Control Systems

1.2

i

0.8

~'- 0.4

A

02

i

I

~

1.0

0.6

I

'

I

i

I

Ordinary Output ~ _

/22 \

-0.2

r

t

!

I

40

I

i

I

i

f I

80

/

\ I

n

I

i

I

r

I

120

200

160

.I

I

~

F

i

4.0 3.2 .~ 2.4

0,~ r

40

80

n

t

120

F i g . 19. Conventional system o u t p u t and control.

I

160

200

266 12 C o n c l u d i n g

Tsypkin Remarks

The main idea of designing adaptive-invariant systems consists in utilizing the error e(n) between the outputs of the plant and the two-input FIR model. This error is equal to the external action f(n) at each moment of time n. Because of the plant lag in discrete systems this signal cannot be directly used to compensate a disturbance. Therefore, a predictor is introduced with which a control law a designed to fulfill the task of disturbance compensation. With the deterministic predictor we come to selective-invariant systems. If the a priori information is not complete and the predictor parameters have to be tuned with the help of the adaptation loop, then we come to adaptiveinvariant systems. These selective- and adaptive-invariant systems are equivalent to corresponding optimal discrete systems without deterministic disturbances. The presence of a stochastic disturbance allows realization of selective or adaptive invariance up to a2. These a2-adaptive-invariant systems are equivalent to corresponding discrete systems, minimizing a quadratic criterion of a generalized error. An attempt to attain in discrete systems the absolute invariance with respect to any deterministic and random disturbances is groundless, because it is impossible to compensate white noise, and forcing the corresponding transfer function to tend to zero via high feedback loop gain is precluded due to the time lag in the plant. Thus, without a priori information on external disturbances their elimination or compensation is impossible in the discrete systems, that is: "there is no invariance without prediction". Utilization of a priori information on disturbances allows the synthesis of selective- or adaptive-invariant systems (with deterministic disturbances) and selective or ~2-adaptive-invariant systems (with random disturbances). The described approach can be, obviously, extended to the uncertainty caused by unknown plant parameters. The usual adaptive approach was connected with identification of plant parameters [6,7,31]. As the basis of our adaptive-invariant systems we utilized the principle of disturbance isolation by means of a two:input FIR model and formulated conditions under which we can design, accurately or approximately, adaptive-invariant and a2-adaptive-invariant systems. With a sufficiently high level of a priori information there may be no need for adaptation and we return to classical selectiveinvariant and cr2-selective-invariant systems. Acknowledgement The author expresses his gratitude to P. V. Nadezhdin for fruitful discussions.

Adaptive-Invaxiant Discrete Control Systems

267

References 1. A. I. Kukhtenko, "The main stages of the invariance theory formation, Parts 1 and 2," Automatika, 1984, no. 2, pp. 3-13; 1985, no. 2, pp. 3-14. 2. V. S. Kulebakin, "On the behavior of the continuously disturbed automatic linear systems," Doklady o/the USSR Academy o] Sciences, col. 68, no. 5, pp. 73-79, 1949. 3. V. Kulebakin, "TheK(D) transform and its practical application," Trudy o] the N. Zhukovsky VVIA, col. 695, pp. 59, 1958. 4. C. D. Johnson, "Theory of disturbance accommodating controllers," in Advances in Control and Dynamic Systems, Chap. 12, New York, NY: Academic Press, 1976. 5. C. D. Johnson, "Discrete-time disturbance accommodating control theory with applications to missile digital control," J. Guidance and Control, col. 4, no. 2, pp. 116-125, 1981. 6. Ya. Z. Tsypkin, "Nonminimum phase in discrete control systems," Itogi nauki i tekhniki., pp. 3-40, 1989. Ser. Tekhn. kibernetiki. Moscow: VINITI, T. 18. 7. Ya. Z. Tsypkin and E. D. Aved'yan, "Discrete adaptive control systems for deterministic plants," Itogi nauki i tekhniki., pp. 45-78, 1985. Ser. Tekhn. kibernetiki. Moscow: VINITI, col. 18. 8. L. N. Volgin, Optimal Discrete Control of Dynamic Plants. Moscow: Nauka, 1986. 9. E. D. Yakubovich, "Solution of an optimal control problem by means of discrete linea~ system," Avtomatika i telemekhanika, col. 9, pp. 73-79, 1975. 10. A. B. Kurzhansky, Control and Observation under Uncertainty. Moscow: Nauka, 1977. 11. V. M. Kuntsevich and M. M. Lytchak, Synthesis o] Optimal and Adaptive Control Systems. Game Approach. Kiev: Naukova dumka, 1985. 12. A. I. Kukhtenko, Problem o] Invariance in Automatics. Kiev: Uk. SSR: GITL, 1963. 13. C. Shannon, "Mathematical theory of the differential analyzer," Journal o/Mathematics and Physics, col. 20, no. 4, pp. 337-352, 1941. 14. V. A. Nikol'sky and I. P. Sevastyanov, "q(e)-transformation of lattice functions discrete systems," in Automatics and Electromechanics, pp. 30-36, Moscow: Nauka, 1973. 15. J. H. Ahlberg, E. N. Nilson, and J. L. Walsh, The Theory o] Splines and Their Applications. New York and London: Academic Press, 1967. 16. J. Zypkin, Grundlagen de, In]ormationellen Theorie der Identification. Berlin: VEB Verlag Technik, 1987. 17. A. G. Ivakhnenko, Electrical Automatics. Kiev: Uk. SSR: Gostekhizdat, 1957. 18. B. M. Mensky, lnvariance Principle in Automatic Regulation and Control. Moscow: Mashinostrojenije, 1972. 19. Ya. Z. Tsypkin, Adaptation and Learning in Automatic Systems. Moscow: Nauka, 1968. 20. G. C. Goodwin and K. S. Sin, Adaptive Filtering, Prediction and Control. Englewood Cliffs, N J: Prentice Hall, 1984. 21. E. D. Aved'yan and Ya. Z. Tsypkin, "The generalized Kaczmarz algorithm," Avtomatika i telemekhanika, col. 3, pp. 71-84, 1979. 22. Y. Marld, "Convergence acceleration of the Kaczmarz algorithm in the case of input process correlation," Avtomatika i telemekhanika, col. 8, pp. 70-73, 1980. 23. K. J. AstrSm, Introduction to Stochastic Control Theory. New York, NY: Academic Press, 1970.

268

Tsypkin

24. V. A. Yakubovich, "Optimization and invariance of hnear stationary control systems," Avtomatika i telemekhanika, vol. 8, pp. 5--45, 1984. 25. O. Smith, "Closing control of loops with dead time," Chemical Engineering Progress, vol. 53, no. 3, pp. 217-219, 1957. 26. O. Smith, Feedback Control Systems. New York, NY: McGraw-Hill, 1958. 27. Ya. Z. Tsypkin, "Lag influence compensation in sampled-data systems," Theory and Applications of Discrete Automatic Systems, pp. 157-171, 1960. Moscow: Izd. of the USSR Academy of Sci. 28. H. Gorecki, Analiza i synteza ukladow regulacji z opoznienie. Warszawa: Wydawnictwa Naucowotechniczne, 1971. 29. E. F. Vogel and T. E. Edgar, "A new dead-time compensator for digital control," in Proceedings of the ISA, Annual Conference, pp. 29-46, 1980. 30. Z. Vostry, "New algorithms for polynomial spectral factorization with quadratic convergence," Kibernetika, vol. 11, no. 6, pp. 415--422, 1975. 31. Ya. Z. Tsypkin, Optimality in adaptive control systems. Uncertainty and Control, Lecture Notes in Control and Information Sciences, vol. 70, Berlin: Springer Vetlag, 1985. 32. T. W. Anderson, The Statistical Analysis of Time Series. New York, NY: John Wiley, 1971.

Stochastic Adaptive System Theory: Recent Advances and a Reappraisal * W. Ren and P. R. Kumar Coordinated Science Laboratory and Department of Electrical and Computer Engineering University of Illinois, Urbana, IL 61801, USA.

A b s t r a c t . Progress has been made in the past year towards the solution of several long standing open problems in stochastic adaptive systems for identification, signal processing and control. We provide an account of these recent advances and a fresh reappraisal of the field. This paper divides itself naturally into two parts. Part I considers identification, adaptive prediction and control based on the ARMAX model. Recent results on the self-optimality of adaptive minimum variance prediction and model reference adaptive control for general delay systems are presented. Both direct and indirect approaches based on non-interlaced extended least squares as well as stochastic gradients algorithms are considered. We emphasize the use of a generalized certainty equivalence approach where the estimates of disturbance as well as parameters are utilized. We also show that self-optimality in the mean square sense in general implies self-tuning, by exhibiting the convergence of the parameter estimates to the null space of a certain covariance matrix. Part II considers stochastic parallel model adaptation problems, which include output error identification, adaptive IIR filtering, adaptive noise and echo cancelhng, and adaptive feedforward control with or without input contamination. Recent results on the convergence of these parallel model adaptation schemes in the presence of nonsta. tionary colored noise are presented.

1 Introduction 1.1 W h a t is S t o c h a s t i c A d a p t i v e S y s t e m T h e o r y ? The problems of recursive identification, adaptive signal processing, and adaptive control of linear stochastic systems have been an active area of research for more than two decades. In this paper, we provide a fresh appraisal of these fields, based on m a n y recent advances, some occurring in the last year Mone. [23,26,20,17,18] We focus on analyzing of the performance of various discrete time parameter * The research reported here has been partially supported by the U.S. Army Research Office under Contract No. DAAL-03-88-K0046, by the Joint Services Electronics Program under Contract No. N00014-O0-J-1270, and by an International Paper Company Fellowship for the first author.

270

Ren and Kumar

adaptation algorithms in the presence of stochastic disturbances. 2 This is due to our belief that adaptive systems are used in many applications to improve system performance. In many such applications, the key to improving performance is to exploit any statistical properties of the disturbances and signals, which can realistically and profitably be assumed in many practical situations, to obtain sharp performance results. One of the main contributions of stochastic adaptive system theory is to show how one may do this. By the "performance" of a stochastic adaptive system, we mean the asymptotic properties of relevant signals and parameters such as the following: i) Stability: Is the system stable in a mean square sense? Does it have bounded states? ii) Self-optimality, or more generally the convergence of signals: Do the signals of interest converge to what they would be if the parameters of the system were known? For example, in the case of prediction, does the adaptive prediction converge to the optimal prediction, say in a mean-square or some other sense? iii) Parameter convergence: Does the parameter estimate converge, and if so to what set? Does it converge to the "true" parameter vector, i.e., is it strongly consistent? iv) Self-tuning property: Does the adaptive filter, predictor or controller converge to what we would have designed had we known the parameters? Note that even if the parameter estimates do not converge to the true parameter, the limit set to which they converge may still yield the self-tuning property. The probabilistic properties that we will exploit in this stochastic adaptive system theory are the following; i) the auto-correlation property of a signal; ii) the cross-correlation property between signals; and iii) any persistence of excitation property of a signal. Property i) characterizes the "predictability" of a signal from its own past. By exploiting such an auto-correlation property, we may be able to estimate the innovation process of a signal, and the signal model which generates it. This could then be employed either for adaptive prediction of a signal, or to identify a system model based on say the prediction error identification method. Or, we could use it to attenuate the effect of the predictable portion of an unmeasured but partially predictable disturbance on a system, using adaptive feedback control. By exploiting property ii), we may be able to define "instruments" to obtain asymptotically unbiased estimates of the parameters in the presence of disturbances, by an instrumental variable or o u t p u t error identification scheme. Alternatively, by making use of an auxiliary correlated measurement, we could extract a signal from noise, as in adaptive noise or echo cancelling, or to cancel the effect of a measurable disturbance, as in adaptive feedforward control. Finally concerning property iii), the persistency of disturbance provides a "natural" excitation to the system, thus allowing us to identify the system better and to establish stronger parameter convergence 2 A disturbance or noise in identification or control of dynamical systems may very well be the signal of interest in a signal processing context. In this paper we mostly adopt a control viewpoint.

Stochastic Adaptive System Theory

271

results. Thus, we could, for example, conclude the self-tuning property of an adaptive controller, once we know that it is self-optimal. 1.2 A D e t e r m i n i s t i c R e d u c t i o n In the stochastic adaptive system theory that we develop below, the key concern is to deal with the stochastically modeled disturbances and signals. However, it is possible to fully dispense with all probabilistic assumptions, and adopt a completely deterministic model of such signals and disturbances. Some readers may find that this is a more convenient point of view to adopt. To start, let us consider the key properties satisfied by a "white noise" sequence w(t). A probabilistic model for "white noise" is a stochastic process which satisfies the following three assumptions. i) w(t) is a martingale difference sequence with respect to an increasing sequence

of a-algebras {~'t}, i.e.,

E[w(t) I .~t_l]

= 0 a.s., Vt.

it) E[w2(t)lF,_l] = a2 > 0 a.s., Yr. iii) suPt E[Iw(t)l°lJ:,_l] < + o o a.s. for s o m e tr > 2.

The key consequences of these assumptions are given in the following lemma, see [1-4]. L e m m a 1 Let ft be a YZt-measurable sequence. I f assumptions i) and iii) above are satisfied, then

a) EtN1 ft_11/J(t)"--0 (EtN=If?-l)+ 0(1) a . s . 3 b)

N E,=I

:,_~(t)

~}, [ft--ll)a.s. on the eve,t{sup,]ftl
1, x(t) and fl are column vectors, a n d x(t) is .Tt-measurable and is available at time t. T h e correspondence b e t w e e n (2.3) and (2.4) with s = d is clear, while for the model (2.1), s = 1, v ( t ) = w ( t ) , z ( t - 1 ) = [y(t- 1),...,y(tp),u(t - d),...,u(td - h)] T, ;3 = [ a x , . . . , a p , b 0 , . . . , b h ] T, £ = r and di --- ci, i = 1, . . . ,r. Note t h a t the regression model (2.4) is a generalization of t h a t considered in [14], which corresponds to the case s = 1 here. Let 00 := [fiT, d x , . . . , (it] T. T o recursively e s t i m a t e 00, n n a t u r a l extension of the algorithm considered in [14], i.e., for the case s = 1, is as follows: Let o(t- 1) be the e s t i m a t e of/70 at time t - 1, and c o n s t r u c t an e s t i m a t e ~(t - 1) of the u n o b s e r v a b l e v(t - 1), ~(t

-

1)

:= y ( t

-

1) -

¢T(t

--

s --

1)O('t- 1),

where ~bT(t -- s -- 1) := [zT(~ -- s -- 1), ~(~ -- s -- 1 ) , - - . , ~(t -- s -- g)]. T h e n we u p d a t e the e s t i m a t e to 0(t) using either the recursive least squares algorithm, 0(t) = 0(t - 1) + P ( t - s ) ¢ ( t - s ) ( y ( t ) - ¢ T ( t -- s)O(t -- 1)) P ( t - s) -1 = P ( t - s - 1) -1 + @(t

-

8)@T(t

--

8);

P(--s) = I

(2.5) (2.6)

or its stochastic gradient c o u n t e r p a r t ,

-s~¢(t 0(t) = 0(t - 1) -F -(-y-(~t )s r) ( t r(t - s) = r(t - s - 1) +

- CT(t - s)lT(t - 1))

II¢(t - s)H 2,

r ( - s ) = 1.

(2.7) (2.8)

In c o n t r a s t to the above algorithms, for s > 1, interlaced a l g o r i t h m s of the form 0 ( 0 = o(t - , ) + P ( t - s)¢(~ - s ) ( y ( t ) - CT(t - s)O(t - s)) (2.9) e-l(t

- 8) = P - l ( t

- 2s) +

¢(t -

s)¢T(t

--

s),

(2.10)

have usually been considered in the literature [22]. Due to the interlacing of s s e p a r a t e recursions, t h e a l g o r i t h m (2.9, 2.10) requires t h e additional storage in 4 Let polynomials F azad G satisfy A F + q--riG = C, where d e g F < d - 1, and let polynomials ~ and ~ satisfy C ~ + q-d-~ = 1, with d e g ~ _< d - 1. Then R -BF-F, D = "~, S = G F + G. See [22].

Stochastic Adaptive System Theory

275

memory of (8(t - 2 ) , . . . , 0(t - s)) and (P(t - s - 1 ) , . . . , P(t - 2s)) at each time t. Since each interlaced recursion effectively sees v(t) as a "white noise", it is clear that the results for s = 1 can be easily extended to the case s > 1 by the use of interlaced algorithms. The consideration of such interlaced algorithms have been purely an artifact of the theoretical inability to demonstrate the convergence of non-interlaced algorithms. In Theorems 2.1 and 2.2 below we resolve the convergence properties of the non-interlaced SG and ELS algorithms defined by (2.7, 2.8) and (2.5, 2.6), respectively. T h e following preliminary fact is useful. L e m m a 2.1 Consider the regression model (2.~). Let D(z) = d l + d2z + . . . +

dtz t. Then (1 + q-*D(q-1)) (~(t) - v(t)) = - - ~ T ( t -- 8 ) 0 ( t ) ,

(2.11)

where ~(t) := O(t) - 0o.

Proof. q-'D(q-1))(Y(t) - v(t)) = ~(t) + C'D(C1)~(t) - U(t) + xr(t - s)~ = - - ¢ T ( t -- s)O(t) + q-'D(q-1)~(t) + zT(t -- s)~

(1 +

= --¢T(t

-- s ) ( 0 ( t )

-- 0 0 ) . I"1

For convenience, let us define N

R(N) := I + ~ ¢ ( t ) f T ( t ) 1 ¢0(t)

:----- [ x T ( t ) , V ( t ) , " ' , V ( t

-- e-I- 1)] T

N

Ro(N) := I + ~ ¢ 0 ( t ) ¢ T ( t ) 1

to(N) := trace ( R o ( N ) ) . T h e o r e m 2.1 Consider the SG algorithm defined by (e.7) and (e.8), and applied to the stochastic regression model (e.4). Assume that 1 + q-aD(q-1) is Hurwitz 5 and is strictly positively real (SPII), i.e., Re[1 + zdD(z)] > O,

for z = e/w,Vw.

(2.12)

s Here and in the sequel, we shall say that a polynomial p(q-1) is Hurwitz if its zeroes are inside the open unit disk, and that a rational transfer function is stable if its denominator polynomial is Hurwitz.

276

Ren and Kumar

Then,

(2.13)

IIO(t)ll =onverges a.s. II0(t) - o(4 - ~)11 ~ < ~ o ,

(2.14)

a.~., v k < c o

+co ,=~ ~ 7 ( t ' - - ~ ' - i S

< ~'

~(0 ~ r0(t) oo

(2.15)

a.s.

(2.16)

a.s.

( v ( t ) - ¢ ~ ( t - , ) 0 ( t - k) - v ( t l ) ~

t=l

r(t - s)

< ~'

a.s.,

(2.17)

Vk < c¢.

Moreover, if

sup ()~max(R0(N))~ < oo, a.s., N ~, ~min(Ro(N) ]

and

ro( N ) ---, oo

(2.18)

then O(O ~

Oo a . s . .

P r o o f . Premultiplying (2.7) by cW(t--s) and subtracting from Y(O, we can rewrite (2.7)

- s)

r ( t (t- s - 1) Taking

yields

squares

IIO'(017 -

Let

~(t) = O'(t - 1).

2¢(t - ~)o'(0~(0 r(* - s - 1)

+

I1¢(~ - s ) l l Z ~ Z ( O r 2 ( t - s - 1)

= I10(~ - 1)112.

V(t):=~(t)ll 2, and for convenience define the operator 7-/t(.) on a random

variable z, as follows, 7/t(z) := z - E [ z I Y , _ , ] .

(2.19)

Making use of (2.19) to rewrite (2.7), We obtain, 2~T(t -- s)0"(t)(~(~)- v(t)) _ 2¢T(t --

V(t) -

r ( t - s - 1)

+

II¢(t - ~ ) l l 2 ~ Z ( t ) -~Ct---s-Z ~

s)E[O(t)lY,_,lv(t)

r(t - s -

---- V ( t

1)

-

2¢a'(t

+

-

1)

s)7-lt(O(t))v(t)

(2.20)

r ( t - s - 1)

We will now show that the last term above is absolutely summable, almost surely. Applying the operator 7/t(.) on both sides of (2.7) and noting that y(t) v ( t ) , rk(t - s), r ( t - s) and 0(t - s) are all ~'t_s-measurable, we have, for i = 0 , . . . , s - 2, 7/t(0(t

i))

~(t - ~ - i)I - ¢(t - s - i)¢T(t r ( t - s - i)

¢(t - s + r(t

s

7"lt(v(t - i))

- ~ - i) u , ( o ( t

/

1)) (2.21)

Stochastic Adaptive System Theory

277

and

¢(t - 2s + 1) 7~,(0(t - s + 1)) - r(t 2s + 1) ~ I , ( v ( t - s + 1)).

(2.22)

Now ,,sing (2.21) with i = 0 ann 7~,(v(0) = v(t), ~T(t -- 8)~,(e(t)),~(t) r ( t - s - 1) _ ~T(t -- d ) ~ , ( e ( t - 1))~(t) ll¢(t - ~)II~ ~(~). r ( t - 8) + r(t : ~ r ~ - ~ 1)

(2.23)

II¢(t - 8)112

Since r(~ - s)r(~ - s - 1) is s u m m a b l e , it follows f r o m L e m m a 1 c) t h a t the last t e r m on the R t I S of (2.23) is s u m m a b l e . To show t h a t the first t e r m on the R H S of (2.23) is also s u m m a b l e , note t h a t

[

I*T('-

As earlier, the first s u m m a t i o n on the RHS above is finite a.s.. To show t h a t the second s u m m a t i o n is also finite, note t h a t from (2.21), for i = 1 , . . . , s - 2,

117/,(0(t - i))1[ -< 117/,(0(* - i - 1))11 +

II¢(t - i)l 117_/,(v( t r(t - ss ---i)

_ i))1

(2.25)

and f r o m (2.22),

117~,(a(t - ,

+ 1))11
o, (2.32)

Stochastic Adaptive System Theory

279

for some constant e > O, and random variable S(O) < co a.s.. Summing by parts, we have s(o

- s ( t - 1)

v(N-s--1)

--

+~SCt) ~=1

r(t-

rCt---s)

-1)

r(-s)"

(2.33)

Since r(t) is monotonically nondecreasing, it follows from (2.32), (2.33) that

k C-~T(t - ,)Y(t) (~(0 - v(0) - '(~(0 - ~(0) 2] S(0) ,=~ r(t - , - 1) >- - r(-,----~"

(2.34)

Using (2.34), (2.31) and (2.28) in (2.20), we can immediately conclude (2.15) and (2.14). (2.16) follows from (2.15). (2.13) can be then shown using Lemma 1 b). The following series of inequalities proves (2.17). ~ - ] [u(t) - ~ T ( t - d)O(t - 8) - v(t)]= r ( t -- s)

t=l

~2[y(t)

- ¢(t -

0, e2 > 0. (2.55) and (2.47) in (2.46), we can immediately conclude (2.39) and (2.41). (2.40) follows from (2.39). To show (2.42), we have the following chain of inequalities, t__~l ( y ( t )

-- cT(t

- - 8 ) O ( t - - ~ ) - - 13($))2

1 + CT(t - s ) P ( t - s - k)4~(t - s)

< 2~(~(t)N _

v(t)) 2 + V"N

[4,T(t -- 8 ) ( 0 ( 0 -- O(t -- k))] 2

2~A-~11 + CT(t -----

~----1

s)P(t

-

s

-

k)¢(t - 8)

_< O(log,-(N - s)) k--1 T N [~"]i=o ~b ( t - s ) P ( t -

+2~

8 - i - 1)~b(t- s - i ) ~ ( t -

i))]2

1 + Cw(t -- s ) P ( t -- s -- k)¢(t - s)

t----1

< O(log r ( N - s))

..-./k-1.-.N

+ 0 ~ _ , ~_ c T ( t - - s - - i ) P ( t - - s - - i - -

) 1)¢(t--s-- i)~2(t--i)

= O(log r ( N - 8)). The parameter consistency when (2.43) holds can be shown as in [14].

D

2.2 A d a p t i v e p r e d i c t i o n We now consider the prediction of a stochastic process y(t) which satisfies the ARMAX model (2.1). Such a model is appropriate, for example, when 9(t) is the output of a dynamical system with a random disturbance and an .Twmeasurable input u(t), or when y(t) contains seasonal fluctuations and u(t) is the appropriate known periodic function. For simplicity, we consider only d-step ahead prediction. Let F'(q - x ) := _~_-i be the solution of 1 -t- f~(q-1) + . . . + f~_lq-(d-1), and Gl(q -1) : - x"P z.~i=091~/ the following "long division",

A(q-~)F'(q -~) + q-dG'(q-~) = 1.

(2.56)

284

Ren and Kumar

Also let F(~ -1) = 1-~t-fl(q-1 ) -~-...-{-fd_lq-(d-1), and G(q-1) = E ~ = o g l q - i be the solution of A ( q - 1 ) f ( q -1) + q-riG(q-t) = C(q-1) . (2.57)

Then (2.1) can be reparameterized as y(t + d) - FCq-1)w(t + d) -- BCq-t)f'Cq-t)u(t) + G'(q-1)y(t) + M(q-X)w(t),

(2.58) where

M(q -1) = q~(C(q-1)F'(g -1) - F(q-~)). From (2.56) and (2.57), A ( q - t )M(q -1) = G(q -1) - Gt(q-1)C(q-t) . Hence, it is clear that M(q - t ) = Truncation (F'(q-1)C(q-1), cO,

(2.59)

where Truncation

aiq -i , ki'--0

:= /

oti+dq - i .

i=O

From (2.59), it follows that the R.HS of (2.58) is .~rt-measurable. Hence the conditional mean, and thus also the minimum variance prediction, .~(t + d) := E[y(t + d)13zt], is given by O(t + d) = B(q-1)f'(q-X)u(t) + G'(q -1) + M ( q - t ) w ( t ) .

(2.60)

Note that though w(t) is not an observed variable, we could estimate w by t~ using the following "observer", V(q-1)~(t) = B(q-1)u(t - d) - A(q-1)y(t).

(2.61)

An indirect approach to adaptive prediction is based on the following procedure: Estimate the parameters A, B, C and the innovation sequence w(t); solve the Diophantine equation (2.56) with the estimated parameters to obtain estimates of F ~ and G'; obtain an estimate of M based on (2.59); replace all the unknowns in (2.60), including w(t), by their estimates to generate adaptive prediction ~(t + d), i.e.,

~(t + d) = ( $ ( q - l , 0 ~ ' ( q - 1 , 0 ) , ( 0

+ G'(q-l,0y(0 + M ( q - l , 0 ~ ( 0 ,

(2.62)

where F' and G' are obtained from

7(q-~,OP(q-~,O + q-dO'(q-~,~) = 1,

(2.63)

285

Stochastic Adaptive System Theory

and M is defined as in (2.59) using if' and ~,.e It should be noted that since w(t) is estimated along with parameters, the linear observer (2.61) based on the estimated parameters need not be used. The use of the disturbance estimate as well as parameter estimate results in a more general certainty equivalence approach than commonly advocated. Alternatively, one could use a direct approach to adaptive prediction based on the reparameterized model (2.3), since then the conditional estimate satisfies fl(t + d) = R ( q - ~ ) u ( t - d) + S ( q - ~ ) y ( t - d) + D ( q - ~ ) v ( t ) .

(2.64)

One can therefore simply estimate R, S, D and v(t) in the model (2.3) and use the estimates in (2.64), to give d-step ahead prediction, ~(t + d) = cT(t)0(t).

(2.65)

Theorems 2.3-2.6 below establish the self-optimality of both the direct and indirect approaches to adaptive predictions. T h e o r e m 2.3 Consider the adaptive prediction based on the direct approach (L65) using the SG algorithm (L7-2.8) with s = d. Suppose that l +q-dD(q -1) is Hur'witz and the following SPR condition holds rain Re [ 1 "Jr z dD(z)] > 0, I~1=i

and

1 N sup ~ ~ (y~(t) + ,~(t)) < oo a.s. . N iv'-~"

(2.66)

Then, the adaptive predictor is self-optimizing in the sense N

+ d)-

+ d))

= 0 a.s. .

(2.67)

Proof. When r(t) is finite, (2.67) holds trivially. When r(t) ~ ~ , the optimality (2.67) follows easily from (2.65), (2.66), (2.16-2.17) and Kronecker Lemma. []

T h e o r e m 2.4 Consider adaptive prediction based on the direct approach (~.65) using the ELS algorithm (2.5-~.6) with s = d. Suppose that 1 + q-riD(q-I) is Hurwitz and the following SPR condition holds rain l~e [1 -t- zdD(z)] -1--" > I~1=1

1

and y2(t)+ u2(t) = O ( ~ )

a.s..

(2.68)

e The multiplication of two time varying polynomials of q-1 is to be understood as ordinary polynomial multiplication disregarding the time dependence of their coefficients.

286

Ren and Kumar

Then the adaptive predictor is self-optimizing in the sense of (2.67). Moreover, if y(t), u(t) and w(t) are uniformly bounded, then the ~accumulated regret" satisfies, N

(~(t + d ) - ~(t + d)) 2 = o (log" U )

,.s..

(2.69)

t=l

Proof. As in the proof of Theorem 2.3, it is enough to consider the case r(t) -* oo. Following [23], let (~(t) - 9(t)) 2 ~' := 1 + OT(t -- d ) P ( t - 2d)¢(t - d) 6, := trace ( P ( t - 2d) - e ( ~ - d)).

Then from (2.65), (2.42), (2.40) and (2.39), we have N

N

(~(t) - ~(t)) 2 < ~ t=r

t=l

N

2~, + ~

~,C(t

- d) (e(t

- 2 d ) - P ( t - d ) ) OCt - d)

t=l

= O(logr0(N)) + o(logro(N) • max I ~.

A2.6) lim inflb0(t)[ > 0. t --*OO A

Then with M designed through (2.76), we have N

1=0 A

With M designed through (2.77), we have N

y~.(y(t) -- y*(t) - w(¢)) 2 = O(N$), t----O

Proof. First note that with the delay equal to one, F ( q - i ) , F i ( q - i ) , F'(q - i ) and their estimates are all equal to 1. Without loss of generality, we can also take ~1 to be 1. Substituting the control law into (2.70), we have

H(q-1)(y(t + 1) - y*(t + 1)) = M(q-l,t)~(t) + R ' ( q - l , t ) C ( q - l , t ) ~ ( $ -b 1) +C(q-1)(w(t + 1) - ~(t + 1)) + c T ( 0 ~ ( 0 , A

with M designed via (2.76),

H(q-1)(y(t + l ) - y * ( t + l)) = ~(t+ l)+C(q-1)(w(t-k l ) - ~ ( t + l))-FcT(t)O(t). T h e rest of the proof then follows Guo and Chen [23].

n

Stochastic Adaptive System Theory

293

It is also possible to reparameterize the model in the form of the controller, so that the controller parameters can be estimated directly. This is the direct approach. Let us consider the direct approach to adaptive MV control for systems with general delay. With the plant parameterized in the form of (2.3), the MV controller for tracking y*(t) is simply given by

y*(t + d) = R(q-1)u(t) + S(q-Z)y(t) + D(q-1)v(t). The corresponding adaptive control law is to replace R, S, D and v(t) by their estimates. If the algorithms (2.5-2.6) and (2.7-2.8) in Section 2.1 are employed for estimation, the adaptive MV tracker simply is given by

y*(t + d) = ~T(~)O(t).

(~.80)

The self-optimality of the direct MV adaptive control (2.80) can be established as for adaptive prediction. For details, we refer the reader to [18]. 2.4 T h e c o n v e r g e n c e o f a d a p t i v e controllers In the previous section, we have established the self-optimality, i.e., the optimal performance with respect to a certain cost criterion on the output y of the system, of a class of certainty equivalence adaptive control. In this section we consider the issue of parameter convergence and show that self-optimality generally implies self-tuning in some sense. Let us consider the indirect adaptive MV control using the stochastic gradient algorithm. In the previous subsection, we have shown that 1 N lira ~ ~ , ( y ( t ) - y°(t))2 = 0 a.s., N --~ o o

t----0

where y°(t) := y*(t)+F(q-1)w(t). Let u°(t) be the input corresponding to y°(t), i.e., it satisfies

A(q-1)y°(t) = q-dB(q-1)u°(t) + C(q-1)w(t).

(2.81)

Since B(q -1) is Hurwitz, it follows that N

lira 1 N--oo ~ ]~(~(~) -- ~'(t))~ = 0 a.s..

De~ne ¢(~) and R°(~) as ~0(t) and Ro(t), using y°(t) and u°(~) in place of

y(t) and u(t), respectively. Then

±N ~ I1¢(~) - ¢(~)11" - o a.s. and

~IIR°(,)

- R(,)II-.-,. o a.s..

294

Ren and Kumar

It then follows from (2.36) that IIIRO(N

- 1)ff(N)ll --* 0 a.s..

For simplicity, assume that the ensemble correlations of the command signal y*(t) exists. Then ~ converges. Let := lim R ° ( N - 1 ) N--.~ N It can then be shown that 0"(t) converges almost surely to the null space of~. The following theorem summarizes the above development and further characterizes the null space of ~lt. T h e o r e m 2.11 Consider the indirect adaptive M V control using the SG algorflhm. Let the assumptions of Theorem ~.7 hold, and suppose that the ensemble correlations of {y*(t)} exist. Then

g(t)

a.s..

Further VO E .N'(~), using the obvious notation, let A, B, C, F and G be the appropriate polynomials corresponding to the parameter vector O+ 0o. Then F(q-1) = p(q-1),

B(q-

)O(q

(2.82)

(2.83)

=

and

o.

(2.84)

t.7.0

Moreover, i f B ( q -1) and G(q -x) do not have a common factor, and {y*(t)} is persistently ezciting of order lp >_min(deg C(q-1), deg A( q-1) ) + 1, then o(t) - , Oo a.s..

Proof. Since O"E .A/(~), we have N

1 y ~ ( ( A - i)y°(t) + q-~(B - B)u°(t) + (C - C)w(t)) ~ -* 0 a.s.. N t----1 So, 1 ~v -ff Z ( ( A - . 4 ) y

* ( t ) + q - d ( B - B ) u ° ( t ) + ( ( A - A ) F + C - C ) w ( t ) ) 2 ~ O. (2.85)

¢-----1

Since u°(t) is Y't-measurable, it follows as in [3,13] that the first d coefficients of (A - A ) F + C - C have to be zero, which yields (2.82).

Stochastic Adaptive System Theory

295

Multiplying (2.85) by B(q -1) and substituting (2.81) into (2.85), we obtain 1

N

--N ~

[(BA - B.A)y*(O + ( B ( C - .4F) - B ( C - AF))w(t)] 2 --~ 0 a.s..

t=O

Hence

1

N

I2 t----O

2

-.0

and 1

N

So

B ( C - AF) = B ( C -- A F ) ,

(2.87)

which is equivalent to (2.83). If B and G do not have a common factor, then

for some scalar A. Iftp > d e g A + 1, from the above and (2.86), we obtain A A - A = 0. It then follow from (2.87) that A = 1, .4 = A, and C = C. From (2.87), we obtain

( B . A - .BA)F -- B C - B C . Hence if ~p > deg C + 1, we can proceed similarily. This completes the proof. []

The parameter convergence results above can be easily extended to more general adaptive control laws for which signal convergence in the mean square sense can be established, e.g., model reference adaptive control. The extension to the ELS algorithm is also possible. For these extensions, we refer the readers to Ren and Kumar [18]. In the case of direct adaptive MV regulation of systems with general delay, based on the SG algorithm using a-priori estimates, the parameters of C(q -1) need not be estimated, and it can then be shown that if there is no overpararneterization, the limit set # + 00 of 0(0 , is a line passing through the origin and the true parameter. The fact that II0"(t)l] converges can then be used to establish the convergence of 0(0 to a particular point on the line. This result then generalizes to the case of general delay the result of Becker, Kumar and Wei [3], which establishes the convergence of the parameter estimates to a random multiple of the true parameter for the unit delay case.

296

Ren and Kumar

,1 aCq-~)~ + ,~(,)

! Fig. 1. Problem A: Output error identification and adaptive IIR filtering.

z(t) ID

~l H(q-~)! i

i

,'r,(~) [KCq I

I

/

I~(t)

Fig. 2. Problem B: Adaptive feedforwaxd control.

s(t)

,.[ G(q-~)I Y(~)~

(•)

+-...-

I~(z)

Fig. 3. Problem C: Adaptive feedforward control with input contamination.

Stochastic Adaptive System Theory 3 Stochastic

Parallel

Model

297 Adaptation

Problems

In this section, we consider three successively more complicated parallel model adaptation problems which are all inspired by practical problems and are illustrated in Figs. 1, 2 and 3. All the fixed systems in Figs. 1-3 are unknown but stable, linear and time invariant. In all three figures, G(q -1) is the system whose output y(t) is to be matched by ~(t), the output of the (partially) tunable system in the lower channels of the figures. The signal s(t) is the common input to both the upper and lower channels, while v(t) is the disturbance. Problem A, which is the simplest of the three and is depicted in Fig. 1, encompasses both output error identification as well as adaptive filtering. In the identification context, G(q -1) is the unknown dynamical system to be identified, and z(t), s(t) and v(t) are the output, input, and disturbance plus measurement noise of the system, respectively. In the filtering context, v(t) is the signal to be extracted, y(t) is the noise corrupting the measured signal z(t), and s(t) can be regarded as the source of the noise. In both cases, v(t) can be quite arbitrary, except for the reasonable and practical assumption that {v(t)} and {s(t)} are stochastically uncorrelated. The challenge here is to exploit this correlational property to provide unbiased estimation of the dynamical system G(q-1), in the context of identification, or undistorted extraction of the signal v(t), in the context of filtering. Problem B, depicted in Fig. 2, is clearly a generalization of Problem A, and arises in practice as an adaptive feedforward control scheme for disturbance cancelling, which has the following physical interpretation. P(q-1) is an unknown "plant", for which u(t) is the input, and e(t) is the output, which is subject to additive disturbances v(t) and y(t) which are mutually uncorrelated. A signal re(t) related to s(t), the source of the disturbance y(t), is measured. The goal of adapting the feedforward controller K(q -1, t) is to eliminate the part due to y(t) from the output e(t), leaving only the uncancelable disturbance v(t). Problem C, depicted in Fig. 3, represents a further complication that is inspired by practical applications of adaptive feedforward control in the areas of active acoustic noise and vibration control [24]. The feedback F(q -~) in Fig. 3 models the contamination of the measurable signal re(t) by the control input u(t). In active acoustic noise control this effect is called acoustic feedback. More generally, it has been termed as input contamination in [24]. Section 3.1 considers Problem A and develops tools that allow us to exploit the uncorrelatedness between signals. The extension to Problems B and C is considered in Section 3.2. 3.1 T h e c o n v e r g e n c e o f o u t p u t e r r o r r e c u r s i o n s In this subsection, we consider Problem A, the output error identification and adaptive IIR filtering problem. Referring to Fig. 1, let G(q -1) -

B(q -1) A(q-1) '

298

Ren ~nd Kumar

where A(q -1) = 1 + alq -x + . . . + a n q - " B(q -1) = bo + blq -1 -4r... -1- b,~q - m . Hence, A(q-1)y(t) = B(q-1)s(t) ,

(3.1)

and z(t) = u(t) + v(t). As mentioned, the goal is to estimate the parameters of A and B, and the signal v(t). We consider the following algorithm which is a slight variation of Landau's output error identification algorithm [25] in that a projection of the parameter estimate onto a compact convex set A4 known to contain 00 = [ a l , . . . , a n , bo,...,b,n] is employed. We estimate 0o by O(t):= [ ~ x ( t ) , . . . , ~ ( t ) , ~0(t), ... ,~m(t)] T, as follows, ¢(t - 1) O'(t) = O(t - 1) + -~(t- iSe(0,

0(0) e M

(3.2)

0 ( 0 = F[O'(t)l, where F[-] denotes the projection onto A4, r(t - 1) = r(t - 2) + II¢(t - 1)ll 2, r(-1) e(t) :: z(t) - (,bT(t -- 1)0(t -- 1)

= 1

¢(t -- 1) := [--y'(t -- 1 ) , . . . , --y'(t -- n), s ( t ) , . . . , s(t -- re)l,

(3.3)

and u'(t) = ¢ r ( t

-

1)¢(0.

Because the estimate of v(t) is given by e(t), we shall say that the algorithm (3.2-3.3) is self-optimizing if, 1 t¢ lim ~ E ( e ( t )

N---*oo

- v(t)) 2 - 0 a.s..

(3.4)

t=l

Theorem 3.1 below establishes this optimality under the following set of assumptions. Assumptions +oo

A3.1) The disturbance is of the form v(t) = E c i ( t ) w ( t

- i), where {w(t)} is a

i=0

martingale difference sequence with respect to the increasing sequence of ~fields 5~ generated by (w(0),..., w(t), {u(k)}~), and {ci(t)} is deterministic and satisfies Ici(t)l < / ( c a -i, for some a > 1, Ke < co, and Vt >_ 0. A3.2) The sequences {[y(t)[}, {Is(t)[}, {Iv(t)[}, and {[w(t)l}, are uniformly bounded by a finite number F, almost surely.

Stochastic Adaptive System Theory

299

A3.3) A(q -~) is Hurwitz and satisfiesthe following strictly positive real (SPB.) condition, Re[A(e/~)] > 0,Vw.

A3.4) Oo E .A,4. A3.5) limN-.co inf ~ ~"~=1 s2(t) > 0. Note that the set .A,4 can be constructed from minimal a-priori information about the transfer function ~ , for example, from the knowledge of a finite upper bound for its Hco-norm. T h e o r e m 3.1 Let the assumptions (,43.1) to (A3.5) hold. Then for the algo-

rithm described by (3.~ - 8.3}, the self-optimality result (3.4), as well as the following results hold: sup r( N) N -"-~ O, an integer T < so, such that I+T

~[u(t),...,

u(t - n - m)]Eu(t), ... , u(t - n -- m)] T _ eI,+m+,,

i=t

for all t su~fficiently large; or it) u(t) is an A R M A process, i.e., u(t) = C(q-X)~(t), where C(q -1) is a stable rational transfer function, and e(t) is a martingale difference sequence with finite but nonzero variance. Then, O(t) --* Oo a.s..

Proof. As

in Section 2.1, it can be shown that

1

~

~---'1") ¢(t -- l)(e(t) -- v(t))

~b(t- 1)~b(t- l)Tft(t-- I) -I-~

I--~-NR(tr(t-- 1)1)¢(t -

+ - ~1 [ ~7.~

1)v(t) I + ~1l l R:v( t

- 1)[1 [lO'(t) - O(t)[ I + o(1)

I

where R ( N ) := ~"~ ¢(t)¢W(t). The first two terms on the RHS above converge to zero by Schwarz inequality and the self-optimality established earlier. The third term can be shown to converge to zero by the techniques in the proof of Theorem 3.1, i.e., the decomposition of v(t), the backward recursion for ~b(t - 1), and Lemma 1 a). We turn next to the fourth term on the RHS above. Let 7D(Z) denote the indicator function of the complement of a set D. Clearly

lift'(t) - ft(t)ll ___L,,(o'(t))llza'(t)ll. Hence, N l ~ - ~ l l R ( t - 1)ll [[e'(t) - e(t)[I 1 1

o

r(-d) : 1,

where F(.) denotes the projection onto a convex compact set .A4. The following theorem establishes the optimality of the above adaptive control law. T h e o r e m 3.3 Let the following conditions hold:

i) {v(t)} and {s(t)} satisfy the assumptions (A3.1), (AS.e) and (A3.5). i V Assumptions (,43.6) and (,43. 7) hold. iii) min Re[n(z)] > ( d - 1 )

a

(3.9)

Izl=l

iv) Oo E f14, and "bo ~ O, VO E f14 . Then N

lim ~1 ~ [ ~ ( t ) - ,~(t)y = 0 ~v--.~o

(3.~o)

a.$..

t=1

Proof. Let us denote v(t) := IIO'(t)lr, and

z~(O := II~S(t - d)e(t)ll

~(t - d)

Then V(t) satisfies the recursion

V(N) _< V(0) + )--~A2(t) + 2a t=l

+2a~-"@(t Z.~ ,=i

@T(t

d)e(t)

t=l

- d)('g(t - 1) - ~(t - d))c(t) r(t - d)

(3.11)

304

Ren and Kumar

The last term on the RHS above can be bounded as follows,

N T(t- d ) ( O ( t - 1) - 8"(t -d))e(t) 2a~_~ r(t - d)

,=, N

d-1

tw.1

i=1

N d-1

< ~y:C~2(t) + ,~2(t

i))

-

t=l i=1 N

< 2(d - llY:za2(0 + o(1).

(3.12)

t=l

As in the proof of Theorem 3.1, we can show that

~(t -- ~) d)v(t) I

_ 0. Substituting (3.12) and N

t=l

-- d)ff(t - d)(e(t) - v(t)) r(t -- d)

~¢T(t t=l

r(t -- d)

)

(3.14)

+ O(1).

Now, N

~za~(t

) = a~

t=l

t=l


]T071~'''(1-t'0Tw n-l~)Oq%~-l~['0(Z)+0T/~(Z)]U "-I-~On(X , 0 1 , . . . ,

0n_l) -I- 0Ten (~, 0 1 , . . . , 0 n _ l ) .

(5.15)

Let On be a new estimate of 0 and define the control u as 1 u = ~(z, d , , . . . ,

On) [-CnZn - ~n - 0 , ~ ¢ n ] ,

(5.16)

where

~(Z,01

O n ) : ( 1 ± 0 TOTI'~ "'" ( l q - o T _ I ~ ) [ f l O ( Z ) q ' o T f l ( Z ) ]

(5.17)

Adaptive Feedback Lineaxlzation of Nonlinear Systems

327

Substitute (5.16) into (5.15) to obtain T 0'Y1'~

• (1 "{-

O"t'n-l'~,(z)u]

-~" --CnZn 4" (0 -- 0n)Twn(x, t91,... , l~n) ,

(5.18)

where (5.16) is used in the definition of w,. Finally, let the update law for the estimate t9n be ~n = zn wn(z, Ox,..., On). (5.19) Feasibility of this design procedure and the stability of the resulting closedloop adaptive system are analyzed in the next section.

6 Feasibility and Stability The above design procedure has introduced a control law defined by (5.16)(5.17) and a set of new coordinates Z l , . . . , z n defined by (5.12). In order to ensure that the procedure is feasible, we construct in Proposition 6.1 an estimate 5 C IRn(l+p) of the feasibility region such that for all (z, t91,... ,tgn) • ~ the denominator in (5.16) is nonzero and the coordinate change (5.12) is one-to-one, onto, continous and has a continuous inverse. P r o p o s i t i o n 6.1 Suppose the pure-feedback form (5.1) of the system (~.1) exists in Bz, and let Bo C IFtp be an open set such that [1+~

I#0(z) +

Ozi+l

>0,

VzEBz,

VtgiEBo, l o, vz • B . . vo. • , o

(6.1)

(62)

Then, the set 9: = B, x B~ is a subset of the region in which the design procedure of Section 5 is feasible. Proof. Obvious, since (6.1) and (6.2) guarantee that in B~ x B~' the denominator in (5.16) is nonzero and (5.12) is uniquely solvable for zi. [] R e m a r k 6.2. In general, the feasibility region is not global. However, this is not due to the adaptive scheme, because, even when the parameters 0 are known, the feedback linearization of the system (5.1) can only be guaranteed for 0 E B0 C ll~p, an open set such that

1 + 0 w O"[i(Z)ozi+l > 0 , Vz E B~ , VO E Bo, 1 < i < n - 1 trio(Z) + 0Wfl(Z)[ > 0, Vz • B~, V0 • Be.

(6.3) (6.4) []

328

Kokotovi~, KaneUakopoulos, and Morse

In the feasibility region, the adaptive system resulting from the design procedure can be expressed in the z-coordinates as ~'1 -- --ClZl "JrX2 -~"(0 -- 01)Ttol(Xl, Z2, 81)

~n-1 = --Cn-lZn--1 "Jc Xn "~" (0 -- 0 n _ I ) T t O n _ I ( X l , . . . , Xn -- --CnXn JC (0 -- on)TtOn(X, O1, ....On)

Oi=z~w~(z,,...,xi+l,01,...,O~),

Xn, 01, . . . , 0 n _ l )

(6.5)

l 2, 1 < j < m .

-wW(z,01,...,d,~-m)d,_m+l},

(7.14)

Substitute (7.14) into (7.12) and rewrite the last m equations of (7.12) as d

i } clxl 1 Xlkl :

Xrn L k,,,

=-

kl

:

kl

+ {w +

pu]} (0 - d,_m+l)

/ crn .r~ i L k,~'vkm J cl x l "1 kl. kl ]

+ w T _ m + l ( ~ , d , , . . . , d,_~+~)(0 - ~,-m+~), (7.15) I Cm

2: m

I

L km km J

where (7.14) was used in the definition of for the estimate d,-m+l be

Wa-m+l. Finally,

~,-m+1=w,_,~+~(~,o~,...,o,_~+~)

:

let the update law

.

(7.16)

Xrrt Lk.,

Note that this procedure is feasible only in the region in which the matrix P --" BO °c ~ Sitgn-m+l,i is invertible. The stability properties of the resulting i=1 closed-loop system are analogous to those listed in Theorem 6.4, and can be similarly established using the Lyapunov function 1 T

I

n-m+l

(7.17)

i----1

8 Global

Regulation

There are strong theoretical and practical reasons for investigating whether the stability properties of an adaptive system can be made global in the space of the states and parameter estimates. Systems with a finite region of attraction may not possess a wide enough robustness margin for disturbances and unmodeled dynamics. Furthermore, it is usually hard to find nonconservative estimates of finite regions of attraction. Another aspect of the global stability issue is the comparison of the proposed adaptive controller with its deterministic counterpart, that is, the controller that would be used if the parameter vector 0 were known. Suppose that for all values

Adaptive Feedback Lineaxiz~tion of Nonfiuear Systems

333

of 0 there exists a deterministic controller which achieves global stabilization and regulation of the system. If, with 0 unknown, the proposed adaptive controller does not achieve the same global stability, this loss is clearly due to adaptation. The stability result of Theorem 6.4 is not global. However, as pointed out in Remark 0.2, this is not due to adaptation, because for pure-feedback systems global stability may not be achievable even with 0 known. In Proposition 8.3 we define the class of "strict-feedback" systems, for which a globally stabilizing controller exists when 0 is known. We then prove that for this class of systems our adaptive scheme guarantees global stability when 0 is unknown. In order to characterize the class of "strict-feedback" systems, we use the following assumption about the part of the system (4.1) that does not contain unknown parameters: A s s u m p t i o n 8.1 There exists a global diffeomorphism z = ff(~), with ~b(O)= O, transforming the system = f0(¢) + g0(¢)u, (8.1)

into the system £'i = zi+l , l < i < n - 1

(8.2) z, = 70(z) + ~0(z)u,

with

-to(O) = o, ~o(z) ¢- o vz ~ ~t".

(8.3)

R e m a r k 8.2. The local existence of such a diffeomorphism is equivalent to the feedback linearization condition (4.4). At present there axe no necessary mad sufficient conditions verifying the global validity of this assumption. Sufficient conditions for Assumption 8.1 are given in [21], while necessary and sufficient conditions for the case where ~0(z) - const, can be found in [22,23]. [] P r o p o s i t i o n 8.3 Under Assumption 8.1, the system (g.1) is globally diffeomorphieally equivalent to the "strict-feedback" system l 0 there exists a time T such that: Ix,(t)l _< 6 Vt _> T. (63)

And, with (55), we have:

It follows that:

1~2(t)[ > 6 ( 1 + Icsl)

-~

~t)

< - 6 ~.

(65)

Therefore, with (62), defining r(~) by:

r = ~1 (7 2 _ e 2 (1 + icsl)2),

(66)

V6 > 03T, 3 r : Vt_> T+r,[x2(t)[_< 6(l+[csD.

(67)

we have established:

This is CO.

[]

Ad~ptive Stabilization of Nonlinear Systems

357

This Example illustrates a typical fact about assumption CO: it is usually difficult to prove that it holds. Above we used a Total Stability argument, here is another example invoking Barb,~ilat's Lemma [26, p.211]:

(6s)

Example: Consider the following system: ~ = x2

+ p°

(69)

•z 2 = u .

We wish to stabilize the equilibrium point zl = 0, xu = -p*. Assumptions BO and PRS are satisfied when we choose:

y(xl,~2,pl = x ~ + ( ~ 2 + p + x l ) ~ , ~.(~l,~,P) = - 2 ( ~ 2 + p + ~ 1 ) .

(70)

To check that CO is also met, we follow the same steps as the ones proposed by Kanellakopoulos, Kokotovic and Marino for the proof of [9, Theorem 1]. Clearly, if V(Zl(t), z2(t), p~)) tends to 0, the same holds for zl(t) and x 2 ( t ) + z l ( t ) + ~ ( t ) . To obtain our conclusion, it is sufficient to prove that zl(t) tends to 0. Indeed, in such a case, the first equation in (69) implies in this case that z2(t)+p* tends to 0. Since we have: lira g--*-bco

/'z,is)ds z l ( t ) = 0 = xl(O) + t .lira + o o J0

,

(71 /

from Barb~tlat's Lemma, xxi$) tends to 0 if this time function is uniformly continuous. This is indeed the case since its time derivative is:

• ~'~t) = ~ , ~ / =

~(t),

(72)

where, by assumption, the time function u(t) is bounded.

[]

Assumptions BO, PBS and C O a r e w e a k e r than (All, (A2) and (A3). Indeed, it follows from linear systems theory [7] and Total Stability theorems [11] that assumptions (A1 / and (A3) imply the existence of: -

$'2, an open neighborhood of ~, / / , an open neighborhood of p*, P, an n x n positive definite matrix, C, an n x m matrix, and k, a strictly positive constant,

such that, by letting:

V(x,p) = V(z) = ( z - ~ ) W p ( z _ e ) ,

un(z,p) = - C ( z -

£) + u0(p), (73)

we have:

~v(x)[.0(~)+..(x,p)ob(~)+a0(x/p]

< kv(~),

v(~,p) •

a×u.

(74)

358

Praly, Bastin, Pomet, and Jiang

This implies that assumption PItS is satisfied. Also, the function V is positive definite and rndially unbounded. Hence, assumptions BO and CO hold with $2o = 12 and a0 the largest positive real number a such that:
0 with D * defined in (95), 4. ( P - P * ) T M - 1 p r o j ( M , p , y ) 0 and "~pt, p) M ZT e < 0 With the definition (102) of Proj and Point 3 in Lemma (103), we get: (if_ p,)W (_ZWe) _ (~_ p,)W M _ l P r o j (M, ~, - M ZWe) 19"P ^ 07~ ~ T -~v (P)M-K'Fv (P) o~,

^

p,)] /

-

op ,_~M#9

(276)

t'~--'~T

Then, the expression of ~ in (261) and the inequality on /~r in (263) give with (262):

}~I'--IIProj(M,~,--"MZTe) I[ 1: ~2 ~1 = v / T l e l ,

ax = 2, S i x =

~-

~2 ~! = (1 +~).v"71e] + clel,(1 = 2, 321 = 2 ( ( 1 - ~ ) 2 Jr-C2) T •

(342)

It follows that there exists a constant T, depending only on the initial condition, such that, for all t in [0, T), we have: 0 < -

,~ v(x(t), ~(t)) 41 - v ( x ( 0 , ~ ( 0 )

= h(x(t),~(t))

= h(t) -

e ( t ) < r + ft.

(343)

Adaptive Stabilization of Nonlinear Systems

395

Hence, we have established, for all t in [0, T):

V(x(t),i(t)) < a1(T+ fl) < at a,+T+9 -8 o,

(383)

Adaptive Stabilization of Nonlinear Systems

401

with:

h(x,p) =

m v(x,p) ,~I - v(x,r,) '

~(~,p) = p(z,p)2,

(384)

,

(385)

and p(z,p) = 1 +

where:

cgV

(hi - V ( x , p ) ) 2

•

(z,p)Ao(x))

OV

-~p (z,~)~ + ~-z (x,~)v(x,~,l~) ® b(z) = 0.

(386)

Since r does not depend on u, ~ is explicitly defined. Then, from our smoothness assumptions on the various functions and with Lemma (103), this system (383) has a locally Lipschitz-continuous right-hand side in the open set defined by: (z, Ff, Zf,~,M) E f 2 x I R x ] R t x H x A d

and

V(z,~) < a l .

(387)

It follows that, for any initial condition in this open set, there exists a unique solution (z(t), ~(t), Zf(t), ~(t), M(t)), defined on a right maximal interval [0, T), with T possibly infinite, and satisfying (387). Applying Points 1 and 2 of Lemma (264), we also know that, for all t in [0, T):

p(t) E / / i 6211~(t)-p*ll 2 + el

I'

r Ilelf _ 0, V/C compact subset of H, the set {h [U(h,p) 1 depending only on dg(p). Since the functions di, i = 8, 10 are continuous and e and ff are bounded from (523), there exists a constant k depending only on the initial conditions such that: ..k

u _< -cO

"~' (1"~-0')[(I"~-0") Hell]'~(l]ell l'l'n'-' Jrllell I-~'F,-')

+k (, + o,) I1:11.

(526)

420

Praly, Bastin, Pomet, and Jiang

Finally, since r satisfies (514), e is bounded, and from GC2 we have A < 1, we get more simply:

We may now apply Lemma (583) in Appendix B with: X = 0

(528)

and:

0x = kllell + 2kv~llell, ~x - 2, Sxl = 10k 2~-~-2 2 ~2 ~ 2~I -~2 - ~(v~llell) ~ , ~ - ],Si~ = da

i, S,a = k2/cl(p*,~0)) (529)

~

= 2kvTIlell,

~2 = ~ (vTIlell) x ,

G =

2, s~l = 4 k ~ - ~

(2 =

2 E ' $22 =

II -_ II

~x ~i

k2 kt(p*,p~O) ) .

With property 3 of U in assumption RURS, it follows that there exists a constant T depending only on the initial conditions such that, for all t in [0, T), we have:

Then, with (523), this implies the existence of a constant c~, depending only on the initial conditions, such that, for all t in [0, T):

[Ih(x(t),~t))l[
0

}

,

there exists a constant kc bounding the Jacobian matrix:

[[VProj(M,p,y)[[ ~_ k¢

V(M,p,y) E C.

(561)

426

Praly, Bastin, Pomet, and Jiang

Table 3. Summary of results Fundamental assumptions

]

o.n

I Bic Istimtion

method

assumptions

Additional assumptions

algorithm

V ind. of p MC MC

G c (3~s) ICS ]

~1 - V

MC

CO

GC (380)

RMC, GC (438) Estimation h n o t rel. V

] ]

RBO -~ R U R S RCO

GC (438)-(439) RMC, GC (510) GC (510)-(511)

BO EEF GC MC R

: Boundedness Observability : Equation Error Filtering : Growth Condition : Matching Condition : Refined

CO A-LP ICS PRS RF URS

: Convergence Observability : A-Linear Parameterization : Imbedded Convex Sets : Pointwise Reduced-order Stabilizability : Regressor Filtering : Uniform Reduced-order Stabilizabihty

Now, let (Ml,Pl,Yl) and (Mo,Po,Yo) be two points such that, for any cr in [0, 1], the point (Ma,pa,y~,) is in the set .&4 x / / x IRI, with: M~ : a M 1 + ( 1 - ~ ) M 0 p~ = ~ p , + (1 - ~ ) p o

(562)

y~ = tr yl + (1 - a) y0Four cases must be distinguished: Case h (Pl, Yl) and (P0, Y0) are not in S. Then, we have trivially: {[Proj(Ml,pl, Yl) - Proj(M0,Po, Y0)H = {{Yl- Y0{I •

(563)

Adaptive Stabilization of Nonlinear Systems

427

Case 2: For all a in [0, 1], (p~, y~) lies in S. Then, from the above preliminary remark and the Mean Value Theorem, we get: [IProj(M1, pa, yl) - Proj(Mo, Po, Yo)[[ < k [lIMa - M0[[ + liP1 -Po[[ + [[Yl - Yo[[] (564) with the constant k given by (561). Case 3: Say (Po, Y0) belongs to S but (Pl, Ya) does not. Then, we define a* by: ex* =

min a. O p(p) > o.

(575/

(576)

It follows that q is not an interior point of the set //0 and, therefore, IIq - P'II is larger or equal to D*, the distance from p* to the boundary of the closed set /7o = {p EP(p) = 0 }, i.e., we have:

> D "2 .

~(Pll i1o,

(577)

_

The conclusion follows from the fact that, p* being an interior point of the set 170 (see (95) and (93)/, the Basic Separation IIahn-Banach Theorem [4, Theorem V.1.12] implies: OP

0~(p)(p-p*) > 0 v p : p(p) > 0.

(578)

Point ~: Again, point 4 is clear for all (p, y) not in S. And, for (p, y) in S, we get, using point 3: (p - p*)W M - 1 P r o j ( M , p, Y)

~(p) = (P -- P * ) T M - l y --

(p--p')TM-~y.

(~---~(p)v)(~-~p(p)(p-p')) O~ OP T -~p (p)M-~p (p)

(579)

Adaptive Stabilization of Nonlineaz Systems

429

Point 5: Let us compute the time derivative of~(p(t)) along a solution of (104).

We get:

P(~t)) = ~(~(t))y(t)

if ~(p~(t)) < 0 or ~ ( ~ ( t ) ) y ( t )

< 0

-- -~-~p~(ff(t))y(t)(1-T'(ff(t))) ff P(ff(t)) > 0 and ~ ( p ~ t ) ) y ( t ) > O.

(580) Therefore, we have: P(~(t)) _< 0

if

P(~(t)) >__ 1.

(581)

Since the initial condition satisfies: 7~(~(0)) ___ 1,

(582)

a continuity argument proves that the same holds for all t where if(t) is defined. f]

B: A Useful Technical L e m m a L e m m a : (see also [3, T h e o r e m 137.1.91) (583) Let X be a C 1 time function defined on [0, T) (0 < T 0, -90 < Ys < 90. The inverse transformation is defined in as W=yl

¢~ = v ~ c o s y~ Cb = x/-~sin Ys ia=~ ib=~

1

sinys\

(20) 1

~-

]

~ sin ys

~

)+--#cosys

Tj____N_)N) Y2+~

•

The dynamics of the induction motor with nominal parameters are given in the new coordinates by ~/1 = Y 2

Y2 = L~1¢1 + LgaLf¢lua + Lgb LI¢lUb

(21)

~t3 = Y 4

il4 = L~¢2 + Lg,,LI¢2u~ + Lgb Lt¢2Ub i15 = L!¢a. The first four equations in (21) can be rewritten as

('.~)

L2¢1

u~

(22)

where D(z) is the decoupling matrix given by

[Lg~,LlCt LgbL/¢I] D(x) = [Lg~Lf¢2 LgbLJ¢2j Since

[ -#--¢b 2o~¢7 #-'M¢a 2a~ffa ¢~

aM# 2 det [D] = - 2 - - ~ - ( ¢ . + ¢~),

~ Cb

"

(23)

(24)

Induction Motors: Adaptive Nonlinear Control

441

D(z) is nonsingular everywhere in/2. The terms LffCt and Lff~b2 are given by Lff¢l = -/~flnpw(¢~ + Cb2) - / ~ ( a + 7)(¢aib

-

Cbia)

-- I~npW(¢aia +

Cbib),

L~¢2 = (4a 2 + 2a2/~M)(¢a2 + ¢~) + 2aMnpw(¢aib -- Cbia)

(25)

--( 6a2M + 2(~TM)(¢aia + Cbib) + 2°t2M2(i2a + i~). The dynamics of the flux angle Y5 = $3(z) are d¢3 = dy5 =

dt

dt

~M now + - -2" ~ ( ¢ a i b c a -- Cbia) = npyl + npRrNyz(JY2 + TLN). (26)

The difference between flux angular speed ~3 and rotor speed npw is usually called slip speed, ws, which can be expressed, recalling the expression of a, as

RrNM Caib -- Cbia -- RrN T ¢3 - np~ = ~, -

L-----/- ¢ ~ + ¢?,

(27)

n o I¢12,

where

T - - npM

Lr (¢rai,b -- ~/'rbisa)

(28)

represents the electric torque. The input-output linearizing feedback for the system (21) is given by

( Uo a- l) ( x:) ~ b

[(--L~I~_L~2)--~ (13a)]vb ,

(29)

where v = (v~, Vb)T is the new input vector. Substituting the state feedback (29) in (21) the closed-loop dynamics become, in the y-coordinates,

Yl ~ Y2 y2=-Va = u4

(30)

Y4 =Vb ~]5 : npYl +

RrN 1 , j

--(Y2+TLN).

np Y3

Equation (26) or (27) represents the dynamics which have been made unobservable from the outputs by the state feedback control (29). In order to track desired reference signals Wvef(t) and [,~[ref(t) 2 for the speed Yl = w and the square of the flux modulus Y3 = ¢~ + Cb2, the input signals va and Vb in (29) are designed as va = - k a l ( ~ X

- ~ror(t)) - ka:(U2 - ~ f ( t ) )

•2 Vb = --kbx(Y3 --IC[~f) -- kb2(Y4 -- [¢Ivef) +

+ ~ref(t)

.-2

loiter,

(31)

442

Marino, Peresada, and Valigi

where (kal, ka2) and ( k b l , ks2) are constant design parameters to be assigned in order to shape the response of the decoupled, linear second-order systems

d~ dt 2

d -

ref)

=

--

re,)

--

d2

--

d

ro,)

~-~(1¢12 - ICier) = --kb~(l¢l 2 --I¢lLe) -- k b = ~ ( l ¢ l

2

-- I¢lLt).

(32)

Remarks 1) The system (30) is input-output decoupled; the input-output mapping is a pair of second-order linear systems. This allows for an independent regulation (or tracking) of the outputs according to (32). Transient responses are now decoupled also when flux weakening is performed. This is an improvement over the field oriented control (see also [9]). 2) State-space changes of coordinates both in the field oriented control and in the decoupling control (i.e., (11) and (20)) axe valid in the open set /2 = {x e IR 5 : ¢~ + ¢~ ¢ 0}; notice that ¢~ + ¢~ = 0 is a physical singularity of the motor in starting conditions. 3) While measurements of (w, ia, ib) axe available, measurements of (Ca, Cb) pose some problems (see [17]). As far as parameters are concerned, variation in load torque TL and rotor resistance R, cause a loss of input-output decoupling and steady-state regulation errors. This calls for an adaptive version of the control (29),(31), which is given in the next Section. 4) Easy computations show that the induction motor model (1) is not feedback linearizable. The necessary and sufficient conditions given in [20] fail; in fact the distribution G1 = span {ga, gb, adlga, adfgb} is not involutive since the vector field [adlga, adlgb] does not belong to G1 (adxY or IX, Y] denotes the Lie bracket of two vector fields; one defines recursively adlx Y = adx(aclixlY)). Following the results in [21], since Go = span {g~,gb} is involutive and rank G1 = 4, it turns out that the largest feedback linearizable subsystem has dimension 4. This shows that the control (29),(31) provides the largest lineaxizable subsystem in the closed loop. 5) The state-feedback control (29),(31) is essentially the one proposed in [9]. It is made clear that the decoupling control makes the angle ¢a unobservable from the outputs, and that (1) is not feedback linearizable. Exact input-output decoupling controls for induction motors are proposed also in [7,8] with reference to a simplified model: the mechanical dynamics in (1) are not considered and w is viewed as a parameter in the last four equations of (1).

4 Adaptive Input-Output Linearization In this section we develop an adaptive version of the decoupling control (29) under the assumptions that TL and Rr are unknown constant parameters. Let us rewrite the system (4) in the V-coordinates defined by (20); since the Lie

Induction Motors: Adaptive Nonlinear Control

443

derivatives L / 2 ¢ t , LI~Lf¢I, LI~¢2 , L I ~ L I ~ , LI~¢3 , LItLI2¢~ , LgaC3, Lgb¢ 3 all vanish, we have

~]1 = Y2 + PtLl~¢t Y2 = L~¢t + P2LI2LICI + LgaLtCtua + LgbLl¢lUb YS : Y4 + P2LI2¢2 Y4 = L ) ¢ 2 + p2LI2LI¢~ + LgaLl¢2ua + Lgb Ll¢2Ub = LI¢3 +p2Ll~d~a.

(33)

Let 15(t) = (151(t), i ~ ( t ) ) w be a t i m e varying e s t i m a t e of the p a r a m e t e r s a n d let

\ep~/

p2

~(t)]

be the p a r a m e t e r error. Following [3,6] we now introduce a t i m e - v a r y i n g statespace change of coordinates depending on the p a r a m e t e r s e s t i m a t e ~(t) Zl -- Yl

Z2 =--Y2 q- plL]t q~l za = ya

(35)

z4 = y4 + 1~L~,2¢2

zs=ys. In the z-coordinates t h e s y s t e m (4) b e c o m e s Zl = Z2 + e p , L l , ¢ l

.~ + L~aLI¢lUa + LgbLl¢lUb z'2 = L~qJl + p2Lf~L/¢t + ~dPl L lt~t /~S = z4 + ep~Ll~¢2

(36) L

+u. (L,~L~,2 + ~L~L~.¢2) + ~'b (L~L~¢~ + ~L~bL~¢~ ) L1¢3 + p2Ly~¢a • Let

where D(x,~) =

LgaLI¢ t LgbLl~t ] LgaL]~b2 + #2LgaLl2q~2 Lgb Ll¢2 + p'~LgbL]2qJ2 J

-L~,I - ~LI~LI~,1 - ~ Lj, ¢1 a~ ---L~b2

~LI2LI~b2- --~-Lf2¢2

-k~,(z, - z,,~) - k.2z2 Vb --- - k b l ( Z S

- zsref) - k b 2 z 4 ,

(38) :2L2/~v2 ~LIL/~¢2 _ p2

444

Marino, Peresada, and Vallgi

(kal, ks2), (kbl, kb2) are control parameters to be designed and zl ref and z3 ref are the desired values for the rotor speed and the square of the rotor flux amplitude respectively. Since det D(x,/~2) = - 2 - ~ ( R r N + ~ ) ( ¢ ~ + ¢~),

(39)

the decoupling matrix is singular not only when (¢~ + ¢~) = 0 as in the nonadaptive case but also when ~(~) = --RrN; this additional singularity has to be taken into account in the design of the adaptive algorithm. Defining the regulation error t~ = (Zl -- Zl ref, Z2, 2:3 -- 2:3 ref, 2:4) T ,

(40)

the closed-loop system becomes e l = e2 + epl

L/I ¢1

e2 - - - - k a l e l

-- k a 2 e 2 +

%2L/2L.f¢l

~s = e4 + %~L12¢2 e4 = --kble3 -- kb~e4 + %2 (LI~Lf¢2 + 152L~¢2) i~5 = L I e s + p2Ly~¢3.

(41)

While the dynamics of z5 are (42)

z5 = n p z x + R.r J z 2 + T L -- ep, , Bp Z3

the dynamics of the vector e can be rearranged as

i3 e4

= K

= g

c3 e4

e3 e4

+

L12¢2 / Lf~LI¢2 + ~ L ~ ¢ 2 J

+ W(z,p

)

~ eP2

ep~

(43)

~

where K = block diag (Ks, Kb), Ka =

_kal

L], ¢I = - L]2Lf¢I--~

_ka2

,

_kbl

_kb2

,

1

d' ~r

1+

(¢aib -- ¢bia) -- --

1+

(44)

445

Induction Motors: Adaptive Nonlinear Control 2 LI2¢2 = ~ (M(¢aia + Cbib) -- (Ca2 + ¢~)) , Lj~Lj¢2 = RrN~, ¢2 = '1, '1 =

4~rLr + 2M 2 . . ~ 2M2- .2 .~aL~ ('P~ + ebb) + " ~ ' 2 ('a + 'b) 6aMLr + 2M 3 trL3r (¢a/a 4" Ibbib),

W(z,~2) is called the regressor matrix and is a function of the x-variables (and therefore of the z-variables). Let P = block diag (Pa, Pb) be the positive definite symmetric solution to the Lyapunov equation KTp + PK = -Q, (45) with Q = block diag (Q,, Qb), Q~ and Qb positive definite symmetric matrices. Consider the quadratic function

(46)

V = eTpe + eWF-tev,

where T' is a positive definite symmetric matrix. The time derivative of V is d_~V = ,T (KTP + PK)e + dt

r ey/W ee + / - - l d % ] L

dt J "

(47)

If we now define dev = _ F W T p e dt

(48)

or equivalently dt

= FWTpe

(49)

which defines the dynamics of the parameter estimate 15(t), and use (45), equation (47) becomes dV dt

_eWQ e

(50)

This guarantees that e(t) and 15(t) are bounded and that e(t) is an L 2 signal; it follows from (43) that the first four state variables ( z l , . . . , z4) are bounded. We are guaranteed to avoid the singularities z3 = 0 and ~ = --RrN for the decoupling matrix, and therefore for the control (37j as well, if the initial conditions (e(0),%(0)) are in S = {(e,%) e IR6 : e ' P e + e T F - l e v < c}, the largest set entirely contained in {(e,ev) e IR6 : ep2 < R¢ + ~ 1 , ea > a 2 - Z3ref}, where a l > 0 and as > 0 are arbitrary. Since W(z,152) is continuous, contains only bounded functions of z5 (sine and cosine), and (zl,z~,.z3, z 4 , ~ ) are bounded, it follows that W(z,~2) is bounded and therefore ~ and 15are bounded as well; since, according to (42), k5 is bounded for (e, %) E S, it follows that

446

Marino, Peresada, and Valigi

= ~ [ K e + W(z,f~)ep] is bounded as well. Now, since e is a bounded L 2 signal with bounded derivative ~, by Barbalat lemma ([22], p. 211) it follows that lim Ile(t)ll = 0,

(51)

i.e. zero steady-state regulation error is achieved. Since g is bounded as well, is uniformly continuous and (51) implies by Barbalat lemma again that lh-n Ile(t)ll = o.

(52)

t--tOo

Therefore, it must he that

\ ~,(t) ]

,-.~

= 0.

(hZ)

Equation (53) implies, from (44), that

lim L1,¢l%,(t) = hm ( - j e , , ( t ) )

=O

,-.~limLj, L1*~e,,(t) - , n ~

1+

t----* OO

-TZ'~,

(54)

T(Oep,(O

= 0,

i.e., lira ep, (t) = 0,

(55)

and, since limt--.oo T(t) = TL, whenever TI. ~ 0, i.e. in any physical situation,

,Era ep~(t) = o,

(56)

that is, parameter convergence is achieved. The difficulty in identifying rotor resistance under no-load condition is a common problem ([23]) and it is related to physical reasons. If the motor is unloaded, when speed and rotor flux regulation is achieved, the slip frequency in (27) is zero so that the flux vec.tor rotates at speed npw and we have Rrird = 0, Rri, a = 0; it follows that rotor currents are zero and therefore rotor resistance is not identifiable in steady-state. It is proposed in [23] to track a sinusoidal reference signal for ¢2 + ¢~ so that rotor currents are different than zero and rotor resistance can be identified. In summary, we have shown that the adaptive feedback control (37), (49) gives the closed-loop system = Ke + W%

~p = - F W T p e .

(57)

If the initial conditions (e(O), %(0)) e S we have

lira lie(Oil = o,

~.--*oo

lim lep~(t)l = o.

t--*O0

(58)

Induction Motors: Adaptive Nonlinear Control

447

Moreover, if TL ~ 0 we also have

lim lew(t)l = O.

(59)

t---*oo

Prom (51) and (52) it follows that in any case we have

lira I'~(01 = {gref,

t--*OO

l~

~t.--* O0

I~1 = o,

lim I¢1 = ICier,

(60)

mn I¢1- O.

lt--~ O0

5 Simulations The proposed control algorithm has been extensively simulated for a 15kW motor, with nominal torque of 70 Nm and nominal speed of 220 rad/sec, whose parameter values are reported in the Appendix. The simulation test reported involves the following operating sequences: the unloaded motor is required to reach the nominal speed and the nominal value of 1.3Wb for the rotor flux amplitude ]¢], with the initial estimate of rotor resistance R~ in error of +50%. At t = 2sec, a 40Nm load torque, which is unknown to the controller, is applied. At t = 5sec the speed is required to reach 300rad/sec, well above the nominal value, and rotor flux amplitude reference is weakened accordingly to the rule I¢] = ~ The reference signals for flux Wre f" amplitude and speed, reported in Fig. 1, consist of step functions. A small time delay at the beginning of the speed reference trajectory is introduced in order to avoid time overlapping of flux and speed transients.

4OO

FluxAmplitudeRefem,nce

.Speed Reference. 1.4

300

I

200

1.2

100 0

~

0

Figure l.a

~ Time(sex)

8

080

~ Figure l.b

~ Time(sec)

Fig. 1. Reference trajectories.

Both the non-adaptive (29), (31) and adaptive (37), (49) control laws have been simulated.

448

Maxino, Peresada, and VMigi

In the non adaptive case, we observe from simulations that parameter errors cause a steady-state error both in speed and flux regulation and a coupling between speed and rotor flux which is noticeable both during speed transient and at load insertion (see Fig. 2, where dashed lines stand for reference trajectories, solid lines correspond to simulated behavior and the dotted line represents the electric torque). To clarify the effects of unknown parameters, consider the closed-loop system, obtained applying feedback control (29), (31) to the motor (4) (recall (33)):

e2 e3 : K

it1)rL l l 0 (;:) e2

e3

e4

+

[ i

e4

=K

e2 + W*(y) p

e3 e4

Ly2L!cbl L1~¢2 L!2L! ¢2

(61)

9s ----L f ¢ 3 , w h e r e e = (el, e2, e 3 , e 4 ) T ~--- (Yl - Ylref, Y2, Y3 - Y3ref, Y4) T is t h e regulation error vector and Ylref and Y3rd are the desired values for the speed and the square of the flux amplitude respectively, K has the structure given in (44) and represents the linear part corresponding to (32), while W*p takes into account the effects of parameter uncertainties (see (3)). Matrix W* entries are given in (44), from which we see that LI~ ¢1 is constant and LI~L! ¢1 is proportional to the electric torque T. The entry L12¢2 is proportional, via a non-zero constant, to the derivative of the squared flux amplitude and therefore, once flux steady-state is achieved, L!2¢2 - - 0. The entry L12L! ¢2 can be rewritten as L!~L! ¢2 - - 6~1 d¢_[~. dt "1" c2T2, with el and c2 nonzero constants, that is, as the sum of a term proportional to the derivative of the squared flux amplitude plus a term proportional to the square of the electric torque. When electric torque is zero and flux amplitude steady-state is achieved, Ly2Lyd?2= 0. Up to 2sec there is no load torque so that pl = 0, the electric torque T is zero (except for a short transient after the first step in desired speed, when a coupling is noticed) and the rotor flux dynamics reach a steady state (Fig. 2.b): this implies zero steady-state error according to the above analysis. This is confirmed by simulation; we see (Fig. 2, 0 < t < 2) that speed and flux steady-state error is zero even if rotor resistance is in error of +50 %. From load insertion (from t = 2 sec) on, the electric torque and Pl are different than zero, which causes, according to (61), a coupling and steady-state errors, as confirmed by Figs. 2.a and 2.b. Notice that even if the load torque were known (and therefore Pl = 0), rotor resistance error (p2 # 0) would still cause a speed steady-state error due to the entry L!~L! ¢1 which is proportional to the electric torque (see Figs. 3.a and 3.b). The dynamic responses when both parameters are known are reported in Fig. 4 for comparison. The adaptive case simulations are reported in Figs. 5: speed and flux amplitude behavior are shown in Figs. 5.a and 5.b respectively, where solid lines

Induction Motors: A d a p t i v e Nonlinear Control

449

, Flux Amplitude.

Speed • Torque ,

,_, 4OO u 300

1

0.5 0

................

i ............

i ...............

2 4 6 Figure 2.a Tim-"(sec)

0

8

8

Figure 2.b

Time (scc)

F i g . 2. Nonadaptive case (with both load torque and rotor resistance error).

A

.Speed &.Torque,

40O

~'V-

I...........

30O

• Flux Amplim&,

1.5

/

I

V"

I.I o

i

0

0.5

i

a

,

2

4

6

Figure3.a

0

8

0

Figure 3.b

Time (sec)

I

I

4

6

Time (sec)

F i g . 3. Nonadaptive case (with rotor resistance error).

Speed ~, Torque,

~. 400 300

Y

1.5

Flux,~plitu&

1 0.5

.".........................

0

I

0

2 Figure 4.a

~

..'~........................

6

0 0

Time (see)

F i g . 4. Nonadaptive case (without pa.raxneter error).

Figure 4.b

Time (sec)

450

Marino, Peresada, and Valigi

represent actual variables, dashed lines the corresponding reference values and dotted line the electric torque; Figs. 5.e and 5.d report load torque and rotor resistance respectively, where solid lines represent true parameter values and dashed lines the corresponding estimates. In Fig. 5.c the load estimator was disabled until t = 2 sec. T h e Q matrix in (45) has been chosen equal to the identity matrix, the gain matrices K , and Kb have been chosen as (k~l, ka2) = (400, 40), (kbl, kb2) = (900, 60) and the parameter update gain matrix F has been chosen as F = diag (71,72) ----diag (0.1, 7 x 10-s).

,.., 4OO

,Speed& Torque

80

3oo

6O

2O0

o

4O

wl°

0~

•

i

loo

0

True & Estimated Load i T

100 ~

20 0

i

0

Figure 5.a 1.5

2 4 6 Figure 5.c Time(sec)

0

Time(sec)

Flux Amplitude,

v~ O'lS

1

0.1 05

True & EstimatedResistance

0.2 i

i

0.05

0 0

i

00 Figure 5.b

Time (sec)

i

i

2 4 6 Figure 5.d Time(s¢c)

8

Fig. 5. Adaptive case.

T h e dynamic performance of the adaptive control law is satisfactory: no steady-state errors occur and transient responses are decoupled, excepting for an initial short time interval. During the first speed transient, due to a wrong initial resistance estimate, a small flux error occurs. At the same time, due to the torque required to increase speed, the rotor resistance estimate quickly converges to the true value and complete decoupling is achieved: no flux error occurs during the subsequent load insertion. Notice that the estimation algorithm (49) is driven by the regulation error e defined by (40); more precisely, in view of the structure of the regressor matrix W and the structure of the P matrix solution of (45), the

Induction Motors: Adaptive Nonlinear Control

451

load estimate dynamics are driven by the regulation error on speed and speed time derivative, i.e. by el and e2, while the rotor resistance estimate dynamics are driven by the whole error vector e. It is clearly impossible for the adaptation law to distinguish between regulation error due to change in reference value (e.g., step function references) and regulation error due to parameter variations. As a result, the rotor resistance estimate slightly varies both after the load insertion (at t = 2 sec) and after a step change in speed reference (at t = 5 sec). Similarly, the step change in speed reference at t = 5 sec causes a peak in load torque estimate, even though actual load does not change. Figures 6 show the control input signal us. Control action consists in varying amplitude and frequency of the applied voltage. The nonlinear state-feedback control law results in voltage supply signals which are well within the capabilities of actual inverters and therefore can be easily implemented by current power electronic technology.

Appliedvoltageua

4OO

400

Appliedvoltageua

~. 200

~=, 200

0

-200

o

0:5

i

1'.5

"4002

2

200

:> -200

400

Appliedvoltageua

Nil

-4004

45 5 5.5 Figure 6.c Time(sec)

~

' 3

, 3.5

4

Figure 6.b Time (sec)

Figure 6.a Time (see)

400

2

E

Appliedvoltageua

200 0

-200 6

"4006

615

7

715

Figure 6.d Time (sec)

Fig. 6. Applied input voltage ua.

Figures 7 show the i~ current waveform. The step functions used as reference trajectories in the simulation clearly are the "worst case" signals. In a real implementation smoother reference signals should be used, improving transient responses and reducing stator current and stator voltage peaks.

_50

452

Marino, Peresada, and Valigi

StatorCurrentia

100

l{lO

50 -~.... '

0

6

6

-50 -I00

0

2

,

2.5

,

3

,

3.5

4

Figure 7.b Time(see)

StatorCurrentia

I00

5 -1017_

|

0.5 I 15 Figure 7.a Time (sec)

, StatorC-M,rrent ia,

I00

oi Ww I

, StatorC-Mrrentia,

50

i ii .....

-50

-I004

415

5

5:5 Figure 7.c Time (sec)

6

6

6.5 7 7.5 Figure 7.d Time (sex:)

Fig. 7. Stator current ia.

6 Conclusions In this paper it is shown how the theory of input-output decoupling and its adaptive versions lead to the design of a satisfactory controller for a detailed nonlinear model of an induction motor. T h e control is adaptive with respect to two parameters which cannot be measured and is based on a converging identification algorithm. The main drawback of the proposed control is the need of flux measurements. However, nonlinear flux observers from stator currents and rotor speed measurements have been determined [19]. Preliminary simulations show that a good performance is maintained when flux signals are provided by the observers to the adaptive control algorithm. This is a direction of further investigation. Another direction of research is the real implementation of the control in order to verify the influence of sampling rate, truncation errors in digital implementation, measurement noise, simplifying modeling assumptions, unmodeled dynamics and saturations.

Induction Motors: Adaptive Nonlinear Control

453

Appendix

Induction Motor P a r a m e t e r s (two-phase data) P~ Rr is ~bs ir ~br u w np Ls Lr M J TL T

stator resistance rotor resistance stator current stator flux linkage rotor current rotor flux linkage voltage input angular speed number of pole pairs angle of rotation stator inductance rotor inductance m u t u a l inductance rotor inertia load torque electric motor torque P a r a m e t e r Values /~ Rr Ls Lr M J

= = = = = =

0.18 ~2 0.15 f2 0.0699 H 0.0699 H 0.068 H 0.0586 K g m 2

References 1. A. ]sidori, Nonlinear Control Systems, 2nd ed., Berlin, Springer-Veflag, 1989. 2. H. Nijmeijer and A. van der Schaft, Nonlinear Dynamical Control Systems, Berlin, Springer-Verlag, 1990. 3. R. Marino, I. Kanellakopoulos, and P. Kokotovic, "Adaptive tracking for feedback llnearizable SISO systems," in Proe. ~Sth IEEE Con]. Dec. Control, Tampa, FL, pp. 1002-1007, Dec. 1989. 4. S. Sastry and A. Isidori, "Adaptive control of linearizable Systems," IEEE Trans. Aut. Control, vol. 34, pp. 1123-1131, Nov. 1989. 5. D. Taylor, P. Kokotovic, R. Marino, and I. Kanellakopoulos, "Adaptive regulation of nonlinear systems with unmodeled dynamics," IEEE Trans. Aut. Control, vol. 34, pp. 405-412, April 1989. 6. I. Kanellakopoulos, P. Kokotovic, and R. Marino, "An extended direct scheme for robust adaptive nonlinear control," Automatica, to appear, March 1991.

454

Marino, Peresada, and Valigi

7. A. De Luca and G. Ulivi, "Dynamic decoupling of voltage frequency controlled induction motors," in 8th Int. Conf. on Analysis and Optimization of Systems, INRIA, Antibes, pp. 127-137, 1988. 8. A. De Luca and G. Ulivi, "Design of exact nonlinear controller for induction motors," IEEE Trans. Aut. Control, vol. AC-34, pp. 1304-1307, Dec. 1989. 9. Z. Krzeminski, "Nonlinear control of induction motors," in lOth IFAC World Congress, Mfinich, pp. 349-354, 1987. 10. R. Maxino, S. Peresada, ~nd P. V~llgi, "Adaptive partiM feedback linearization of induction motors," in Proe. P9th IEEE Conf. Dec. Control, Honolulu, HI, Dec. 1990. 11. A. Fitzgerald, C. Kingsley Jr, and S. Urea, s, Electric Machinery, McGraw-Hill, 1983. 12. P. Kranse, Analysis of Electric Machinery, McGraw-Hill, 1986. 13. W. Leonhaxd, Control of Electrical Drives, Berlin, Springer-Veflag, 1985. 14. P. Kranse and C. Thomas, "Simulation of symmetrical induction machinery," IEEE Trans. Power Apparatus and Systems, vol. PAS-84, no. 11, pp. 1038-1053, 1965. 15. F. Blaschke, ~Das Prinzip der Feldorientierung, die Grundlage ffir die Transvector Regelung von Asynchronmaschienen," Siemens-Zeitschri]t, vol. 45, pp. 757-760, 1971. 16. F. Blaschke, "The principle of field orientation applied to the new transvector closed-loop control system for rotating field machines," Siemens-Rev., vol. 39, pp. 217-220, 1972. 17. R. Gabriel and W. Leonhaxd, "Microprocessor control of induction motors," in Proc. I E E E / I A S Int. Semiconductor Power Converter Conf., Orlando, FL, pp. 385-396, 1982. 18. W. Leonhard, "Microcomputer control of high dynamic performance ac-drives: a survey," Automatica, vol. 22, no. 1, pp. 1-19, Jan. 1986. 19. G. Verghese and S. Sanders, "Observers for flux estimation in induction machines," IEEE Trans, Industr. Electronics, vol. 35, pp. 85-94, Feb. 1988. 20. B. Jakubczyk and W. Respondek, "On linearization of control systems," Bull. Acad. Pol. Sci., Ser. Sci. Math., vol. 28, no. 9-10, pp. 517-522, 1980. 21. R. Marino, "On the largest feedback linearizable subsystem," Syst. Control Lett., vol. 6, pp. 345-351, Jan. 1986. 22. V. Popov, Hyperstability of Control Systems, Berlin, Springer-Verlag, 1973. 23. H. Sugimoto and S. Tamai, "Secondary resistance identification of an inductionmotor applied model reference adaptive system and its characteristics," IEEE Trans. Industr. Application, vol. IA-23, pp. 296-303, March 1987.

G l o b a l A d a p t i v e O b s e r v e r s and O u t p u t F e e d b a c k S t a b i l i z a t i o n for a Class of N o n l i n e a r Systems* Riccardo Marino and Patrizio Tomei Seconda UniversitY, di Roma, "Tor Vergata" Dipartimento di Ingegneria Elettronica Via O. Raimondo, 00173 Roma, ITALY.

A b s t r a c t . We address the problem of determining global adaptive observers for a class of single-output nonlinear systems which are linear with respect to an unknown constant parameter vector. Sufficient conditions are given for the construction of a global adaptive observer of an equivalent state, without persistency of excitation. Under additional geometric conditions the original (physical) state can be asymptotically observed as well. The results obtained axe based on nonlinear changes of coordinates driven by auxiliary filters (filtered transformations). When only a single input is allowed and it is assumed to enter linearly in the state equations, we determine via geometric conditions a more restricted class of nonlinear single-input, single-output systems which can be globally stabilized by a dynamic (adaptive) observer-based output feedback control. Linear minimum-phase systems with unknown poles and zeroes, known sign of the high-frequency gain and known relative degree belong to such a class of systems. Systems which axe not feedback lineaxizable may belong to such a class as well. 1 Introduction In the last few years several papers have developed adaptive versions of nonlinear control algorithms designed via the so-called geometric approach to nonlinear systems (see [1] and [2] for an introduction to the field). Those nonlinear algorithms very often rely on the cancellation of nonlinear terms by feedback or o u t p u t injection: when those terms contain unknown parameters the development of adaptive versions is very much needed as simple examples can show. Under the assumption that unknown parameters enter linearly in the nonlinearities to be cancelled, adaptive versions of state feedback linearizing controls have been developed in [3-7], of input-output linearizing controls were reported in [8], and of nonlinear observers in [9,10]. Nonlinear adaptive stabilization using the "control Lyapunov function" technique was studied in [11,12]. More recently, an adaptive version of a static output-feedback linearizing control given in [26] has been proposed in [13] and [14], under the restriction of sector-type nonlinearities and of so-called output matching conditions. * This work was supported in part by Ministero della Universitg e della Ricerca Scientifica e Tecnologica and in part by the U.S. Air Force Office of Scientific Research under Grant AFOSR 90-0011.

456

Maxino and Tomei

In this paper, we address the problem of global adaptive output feedback stabilization of a class of nonlinear systems, that is, the design of a dynamic output feedback control (compensator) such that for any initial condition of the closed-loop system and for any unknown value of the parameter vector, the closed-loop system has the property that the state of the system tends to zero and the state of the compensator is bounded.

The main result presented determines via geometric conditions a class of nonlinear single-input single-output systems, linear with respect to parameters, which are globally stabilizable by an (adaptive) dynamic observer-based output feedback control for every unknown constant value of the parameter vector. Those systems are characterized by a global, asymptotically stable, linear zero dynamics and are such that global observers with linear error dynamics [19] can be designed. The only restriction imposed on the nonlinearities is their dependence, in suitable state coordinates, on the output only. No sector-type restrictions or output matching conditions [13,14] are required. Systems which are not feedback linearizable may be included in such a class. Linear minimumphase systems with known relative degree, known sign of the high-frequency gain, known upper bound on the number of poles but unknown poles and zeroes are included in the class of nonlinear systems considered.

We make use of nonlinear adaptive observers developed in [10,15] and of filtered transformations into adaptive observer forms. In the case of relative degree greater than one, we use techniques recently developed in [16-18] for the construction of Lyapunov functions of cascade systems and, following [5], we employ several "parameter estimators" in the design of nonlinear adaptive controls.

The paper is organized as follows. In Section 2 some preliminary results on adaptive observers and on the adaptive observer form are recalled. In Section 3 we recall from [15] the concept of filtered transformation and show that a class of nonlinear systems can always be globally transformed into an adaptive observer form by a filtered transformation so that global adaptive observers can be designed. This generalizes a well-known result on adaptive observers for linear systems [21]. Section 4 addresses the problem of observing the original "physical" state. In Section 5 we further restrict the class of nonlinear systems considered. In Section 6 we show how to design an adaptive output-feedback stabilizing control, in the relative-degree-one case, by using filtered transformations. In Section 7 we show how to design adaptive output-feedback stabilizing controls for the class of nonlinear systems determined in Section 5, when the relative degree is greater than one. The results presented in Sections 6 and 7 generalize well-established results for hnear minimum-phase systems [21].

A d a p t i v e Observers and O u t p u t Feedback for Nonlinear Systems

2 Adaptive

Observers:

Basic

457

Results

We consider nonlinear single-output systems P

-- f ( x ) "4"qo(x,u) "4"Z O i q i ( z , u ) i=l

~ f(z)+qo(x,u)+O(x,u)O,

v=h(x),

xEIRn,u~IRm,OEIR

p

(27)

veIR,

with qi : IRn x IR'~ --+ IR",O < i < p, f : IR a ---* IRn, h : IRn ~ IR smooth functions, h(xo) = O, qo(z,O) = 0 for every x E IR"; x is the state, u(t) : IR+ IRm is the control, 0 is the parameter vector which is assumed to be constant, and y is the scalar output. We assume that the system (,U) with u = 0 and O = O: =

I(~)

v = h(~),

satisfies the condition

rank{d(niyh(x)) : O --

d(L -lh) Proof. Conditions (i) are shown in [19] to be necessary and sufficient for (Z) with u = 0 and 0 = 0 to be transformable via a local diffeomorphism (2.10) in U~o into the system = A¢~ + ¢0(Y)

y - C¢¢. The needed local coordinates as those in which

¢ = T(x) are defined, by virtue of assumption (i),

0 a ~ r -- ( - 1 ) / 04n-i '

O_ O, Vwo E IR"

(//) limt-..oo Ilx(t)- e(t)ll = o, Vwo e m ' , w(o) e m " .

460

Marino and Tomei

D e f i n i t i o n 2.2 The constant integer v such that Lq~Li.( f +qo ). h = O

'

Vi, 1 -_ O, and lim IIx(t)ll = o, |....-* O 0 for the closed-loop system.

[]

474

Marino and Tomei

We now first establish a technical result which generalizes L e m m a 3.2 and will be one of the basic tools in the next two sections• L e m m a 5.3 Consider the system P

= Acx + fo(Y, u, O) + Z f i ( y ,

u)0i

i=l

(5.10)

y=Ccz. Let fij, 1 < j < n, be the components of the vector fi. I f

Were, r u e m,

f, Au, u) = 0,

l

II~KelI2,

(6.27)

~z ~Se

then there exists a positive constant kL such that

v O .

This system is both globally state-feedback linearizable and input-output linearizable by state feedback for any known value of the parameters O and b. Nevertheless, the adaptive algorithms proposed in [3,4,8] do not apply. A global adaptive stabilizing state-feedback control for a simpler example (b=l) was first obtained in [11]; this system belongs to the class of systems in the so-called "pure feedback form" which can be globally adaptively stabilized by state feedback via the systematic algorithm proposed in [5]. According to Theorem 7.1 this example can be adaptively globally stabilized by output feedback: in fact, the example is of relative degree p = 3 and belongs to the class of systems (SR), with b - [0, 0, 1]T a Hurwitz vector of degree p = 3. [] R e m a r k T.1. Let us examine how Theorem 7.1 applies to a single-input, singleoutput linear system characterized by the transfer function of known relative degree p b , s " - ' + . . . + b, ,(s) W(~) = ~" + , , * " - 1 + . . . + a, = d(~)'

(7.57)

where the numerator is a Hurwitz polynomial with unknown coefficients but with bp of known sign, while the denominator is a polynomial with unknown coefficients ai, 1 < i < n. The above assumptions are standard ones in adaptive

Adaptive Observers and Output Feedback for Nonlinear Systems

491

control of linear systems [21, p. 183]. A realization in observer form of (7.57) can be written as

f01001 [i] [al} 00 1 . . . 0 [

~=

:::'.

"

:[~+

bp u -

/ o o o..: {/ LOOO...oJ

a2

a3 y

(7.s8) b

y= [100...0]_ 0. However, the initial conditions of (2.1) are unknown and the tracking can be achieved only asymptotically, that is, y(t) = yr(t) +

.-, y,(t)

as t

-.,

(2.1o)

where e(t) is the exponentially decaying tracking error caused by the mismatch of the initial conditions. When the parameter 9 is unknown, we replace it in (2.5) by its estimate ~, to be obtained from a parameter update law. To this "certainty-equivalence" part of our control we add a term fi which will be a handy tool later. So, our adaptive control will be of the following form: G

u = -~-~2 y + r - ~v(y) -

F B - E2 u + f~

E2

(2.11)

When applied to the nonlinear plant (2.1), this control yields the following inputoutput description of the resulting feedback system: 1 where, as in the case when 0 was known, e(t) contains all the exponentially decaying terms caused by the mismatch of the initial conditions. It should be observed that with an exact estimate 0 = 0 the linearization of (2.12) is achieved. Introducing the error variables e=y-yr,

0=0-~,

(2.13)

and taking the difference between (2.12) and the reference model (2.3), we obtain the tracking error equation: 1 e --- (D -t- 1) 3

[0v(y) + "~] "1"e(t).

(2.14)

2.3. E r r o r a u g m e n t a t i o n a n d s w a p p i n g . Following the standard practice in adaptive control, we now set out to construct an error equation in which the parameter error is filtered only by a strictly positive real (SPR) transfer function. As a first step, we rewrite (2.14) in the form e -- (D ÷ 1-----~ 0~-~ -i- 1) 2 1 -t- (D -'t-1) 3 [0v(y)]

(D - ' ' ' '+' ~ 1) fi

~

(D 1Jr 1) [ ( D ÷

1

1) ' v ( y ) ] ÷ e(t).

(2.15)

The first term in (2.15) is in the desired S P R form, while the second term is due to the additional control term ft. As for the third and fourth terms, these are the

Adaptive Output-Feedback Control for Nonlinear Systems

499

familiar swapping terms, whose presence is caused by the time-varying nature of t~: if 0 were constant, these two terms would cancel out. Let us therefore define the augmented error ff as = e + ~/o,

(2.16)

where the term Oo represents all the undesirable terms in (2.14):

T/o=-

1 _ 1 ( D + I ~ u + ( D + I ) s [0v(y)]

[8(DT

1 (D+I)

1 1)2v(y)] ]

(2.17)

The signal multiplying 0 in the first brackets is of particular importance and is denoted by 1

hi - (D + 1) 2 v(y).

(2.18)

Considering v(y) as the input and hi as the output, we represent (2.18) in the state-space form

[ ttl ] = ]~ = A°h + bv(y)

(2.19)

where

It can now be verified that ~/0 is the output of the third order system //0 = -~/0 + rh

(2.21) =

= il = Aorl- b ~ - hO.

(2.22)

The variables hi and }71 from (2.19) and (2.22) allow us to express the tracking error as

1 ~ e- D +'-----

[0hi - 71] "4"~(t).

(2.23)

The analogous expression for the augmented error is e-

D-41 1 [0hl] + e(t) ,

(2.24)

and it has the desired SPK form: the parameter error {~multiplied by the "regressor" hi is the input into the SPR filter 1/(D + 1).

500

Kandl~kopoulos, Kokotovi6, and Morse

2.4. U p d a t e law. From this point on, the route prescribed by most of the adaptive linear control literature is to choose a normalized gradient update law and to set ~ -- 0 (thus returning to a pure certainty equivalence control). In the case of adaptive linear systems, boundedness of the closed-loop signals can then be established using the Gronwall lemma or some type of small-gain argument. Attempts to apply this type of stability proof to nonlinear systems have so far been successful only when conic constraints are imposed on the nonlinearities. Without such linear growth constraints, the term ( 0 - 0) v(y) can cause some /

signals to escape to infinity in finite time if the parameter error 0-/t is not rapidly decreased. The difficulty with normalizations of update laws is that they don't allow a rapid enough decrease of the parameter error when this error is most harmful. A simulation example of instability of a full-state-feedback scheme with normalization [10] is given in our companion paper [20], where it is also shown that an unnormalized update law preserves global stability. We are, therefore, motivated to look for an unnormalized update law. The SPR form of (2.24 / suggests the unnormalized gradient update law 0 = ht~.

(2.25)

Given the complexity of the nonlinear system (2.1), it is likely that such a simple update law will shift the difficulties in the adaptive design to the proof of stability. Indeed, this is the case. A simple Lyapunov-like function involving ~2 and ~2 has a nonpositive derivative, but fails to prove boundedness of y. It clearly shows, however, that r/0 must be taken into account. Our next attempt is with the function 1 ~2 + ~2(.r)d.r 1 T (2.26)

(

/,=

)

where P is the positive definite solution of the Lyapunov equation o.= Using (2.22), (2.24) and (2.25), we compute l~, = - ~1 (~_ e(t))2 _ ~ 1[ ~ 2 +

2r}Tphhle+ 2~/Wpb~] ,

(2.28)

and try to render it nonpositive. The tool we have prepared for this task is ft. However, it turns out to be impossible to counteract the effects of the twodimensional vector hhl$ by fi alone. Hence, we need an additional degree of freedom involving hi. This prompts us to replace ~1in (2.26) by the new vector (2.29)

¢1 "-- T}I

where the nonlinear function ~1(hl) is at our disposal. With the new variables ¢, the nonnegative function to be used in our proof is V=~

,( ,,/= e2+

+

e2(r)dl.

), +

;Tp¢.

(2.30)

Adaptive Output-Feedback Control for Nonlinear Systems

501

To evaluate ~Ywe need ~, which is obtained by differentiating (2.29) and using (2.19)-(2.22): tl = & - hi0 - e(hl)¢~ ~2 = - ~ 1 - 2C2 + 2~1(hx)~1 - fi - h ~

(2.31)

Introducing the notation [ -h~ ] w = [-hlh2 h~l J '

(2.32)

-

we compute the time derivative of V as:

[

~

0~lh~]}. (2.33)

2.5. Design equation. W e now have two tools to make V nonpositive: the function ~1(hi) and the control term ft. With these tools we will attempt to represent the quantity enclosed in braces in (2.33) as the s u m of two squares. It turns out that this is possible to achieve by decomposing P as P = P1 + P2 such that the following design equation holds:

[

~t~x

0~1 h2(~1] . (2.34)

The substitution of (2.34) into (2.33) yields the desired form for lY: 0 is the adaptive gain, guarantees boundedness of all the closed-loop signals and convergence of the tracking error e to zero [18, Chap. 5], and the control augmentation is not needed: fi ~ 0. 3.3. E r r o r a u g m e n t a t i o n a n d swapping. For relative degree higher than one (n - m > 1), the design becomes considerably more complicated, since IbmI/E1 is no longer SPR.. We first rewrite (3.17) in the form 1 [ e= D+Ao

~1

+ D ~" ,~o "~o

]

-

bmfi

¢ +

'

where

Eo(D)(D + ~o) = E1 (D).

(3.20)

In contrast to the example of Sect. 2, where the high-frequency gain bm was known, here it is unknown. Therefore, using an estimate b,n and denoting b,n = bm - bin, we rewrite (3.19) in the form

1

e= D+A0

¼

"~b~r ~b+bm

(~ o a - 0 w¼~b+~--~0

-

E

Since the first summand in (3.21) is in the desired SPR form, we define the augmented error e = e + 70,

(3.22)

where the term 70 represents all the undesirable terms in (3.21): 70 =

D + :~0

~_.~0fi_ ~r

¢ + ~00 ~W¢

.

(3.23)

The vector multiplying 0 in the second term is denoted by 1 ¢ = 70 ¢(y, u, r).

(3.24)

508

Kanellakopoulos, Kokotovi~, and Morse

Considering ¢(y, u, r) as the input and ¢ as the output, we represent (3.24) in the matrix state-space form: [-t

AoH

=

+

b~bT(y,u, r)

(3.25) cT -- cTH, where (c, A0, b) is a minimal realization of 1/Eo : c T ( s I _ A0)_lb _

(3.26)

1 Eo(s) "

Now ~/0 is the output of the (n - m)-dimensional system ~0 : --)lOgO "4- bmcT~

(3.27)

il = Ao~l - b~ - H O .

(3.28)

The variables ¢ and cTt/from (3.25) and (3.28) allow us to express the tracking error as (cf. (3.19)) 1 [bmST¢- bracts] + e(t) e -- A-----~ D+

(3.29)

The analogous expression for the augmented error is 1 [bm~~ - A-------~ D +

b~T.]

+e(t)

and it has the desired SPR form: the parameter errors by the SPR filter 1 / ( D + Ao).

(3.30)

~ and bm are filtered only

3.4. U p d a t e law. As in Sect. 2, we choose the unnormalized gradient update laws suggested by the SPK form of (3.30): k 0 = sgn(bm)FCe

bm=

(3.31)

--Tcw~ff,

(3.32)

where F = F T > 0 and 7 > 0 are the adaptive gains. From Sect. 2 we know that in the proof of stability there will be a need to balance the interaction between y and H. Therefore, we introduce the new variables ~ : = S-lrl, S ~ C~Iscn, (3.33) where h -

n - m - 1, Ci=

[c,A 0 •c,...,

A

i-lc

1" ,

i=l,...,fi

(3.34)

Adaptive Output-Feedba~:k Control for Nonlinear Systems

509

[A°] ELi----[Ij×j, 0]j×i

,

O<j_ j.

(3.53)

k=l

But from (3.52) we have J k=l

which, combined with (3.53), results in (3.49). Having established (3.48) and (3.49), we now set out to find ~i(H) and fi which, along with Pi defined by (3.46)-(3.47), satisfy the design equation (3.44). Substituting (3.46) into (3.44) we obtain a

We + S-Xbfi = p-1 ~-~ CT MiCiwwTCT MTCi .

(3.54)

Using (3.41) we rewrite the design equation (3.54) as a

(3.55) i=l

Premultiplying both sides of (3.55) by cTAio-1 for { ----1 , . . . , fi, we obtain

+S] ~ + cWA~-iba,

1 < i < ~.

(3.56)

From (3.33), (3.35) and (3.51) we have

~101 (2C2

CaS = Y.Cn = Ca +

(3.57)

(.-iC.-i which gives

cTAio-IS = cTAio-1 + ~ i - l C / - 1 ,

1 < / < fi,

(3.58)

where we have defined for notational convenience ~0=0,

C0=0.

(3.59)

512

Kanellakopoulos, Kokotovid, and Morse

Furthermore, 0 cTAio-lS = cTAio-lC~l~--"Cr1 _ eTAio-Ic~I

:

~r~-lCr,-I =~i-tCi-1,

1 < i < fi,

(3.60)

where we have used the definition of Cn to obtain the last equality. Finally, from the definition (3.26) of the triple (c, A0, b) we have: l