Techniques in Discrete-Time Stochastic Control Systems, Volume 73: Advances in Theory and Applications (Advances in Theory & Applications)

CONTROL AND DYNAMIC SYSTEMS Advances in Theory and Applications Volume 73 CONTRIBUTORS TO THIS VOLUME MOHAMMED ABBAD...

Author: Cornelius T. Leondes

39 downloads 700 Views 10MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

CONTROL AND DYNAMIC SYSTEMS

Advances in Theory and Applications Volume 73

CONTRIBUTORS TO THIS VOLUME MOHAMMED ABBAD ZIJAD A GANO VIC SIVA S. BANDA BOR-SEN CHEN GIANNI FERRETTI JERZY A. FILAR ZORAN GAJIC LANG HONG P. J. LYNCH CLAUDIO MAFFEZZONI SEN-CHUEH PENG RICCARDO SCA TTOLINI XUEMIN SHEN JITENDRA K. TUGNAIT EDWIN ENGIN YAZ HSI-HAN YEH

CONTROL AND DYNAMIC SYSq'EMS ADVANCES IN THEORY AND APPLICATIONS

Edited by

C. T. LEONDES University of California, Los Angeles Los Angeles, California

VOLUME 73:

TECHNIQUES IN DISCRETE-TIME STOCHASTIC CONTROL SYSTEMS

ACADEMIC PRESS San Diego New York Boston London Sydney Tokyo Toronto

This book is printed on acid-free paper.

Copyright 9 1995 by ACADEMIC PRESS, INC. All Rights Reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Academic Press, Inc. A Division of Harcourt Brace & Company 525 B Street, Suite 1900, San Diego, California 92101-4495

United Kingdom Edition published by Academic Press Limited 24-28 Oval Road, London NWl 7DX

International Standard Serial Number: 0090-5267 International Standard Book Number: 0-12-012773-3

PRINTED IN THE UNITED STATES OF AMERICA 95 96 97 98 99 00 QW 9 8 7 6

5

4

3

2

1

CONTENTS CONTRIBUTORS ................................................................................... vii PREFACE ................................................................................................ ix

Techniques for Reduced-Order Control of Stochastic Discrete-Time Weakly Coupled Large Scale Systems ..................................................

Xuemin Shen, Zijad Aganovic, and Zoran Gajic Techniques in Stochastic System Identification with Noisy Input & Output System Measurements ................................................................ 41

Jitendra K. Tugnait Robust Stability of Discrete-Time Randomly Perturbed Systems

89

Edwin Engin Yaz Observer Design of Discrete-Time Stochastic Parameter Systems ...... 121

Edwin Engin Yaz The Recursive Estimation of Time Delay in Sampled-Data Control Systems .................................................................................................... 159

Gianni Ferretti, Claudio Maffezzoni, and Riccardo Scattolini Stability Analysis of Digital Kalman Filters

207

Bor-Sen Chen and Sen-Chueh Peng Distributed Discrete Filtering for Stochastic Systems with Noisy and Fuzzy Measurements ............................................................................... 237

Lang Hong

vi

CONTENTS

Algorithms for Singularly Perturbed Markov Control Problems: A Survey ..................................................................................................

257

Mohammed Abbad and Jerzy A. Filar Control of U n k n o w n Systems via Deconvolution

................................

289

Hsi-Han Yeh, Siva S. Banda, and P. J. Lynch

I N D E X .....................................................................................................

315

CONTRIBUTORS

Numbers in parentheses indicate the pages on which the authors' contributions begin.

Mohammed Abbad (257), Departement de Mathematiques et Informatiques, Faculte des Sciences de Rabat, Morocco Zijad Aganovic (1) Cylex Systems, Inc., Boca Raton, Florida 33487 Siva S. Banda (289) Flight Dynamics Directorate, Wright Laboratory,

Wright-Patterson Air Force Base, Ohio 45433

Bor-Sen Chen (207) Department of Electrical Engineering, National Tsing Hua University, Hsin Chu, Taiwan, R.O.C. Gianni Ferretti (159) Dipartimento di Elettronica e Informazione, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milano, Italy Jerzy A. Filar (257) School of Mathematics, University of South Australia, The Levels, South Australia 5095 Zoran Gajic (1) Department of Electrical and Computer Engineering, Rutgers University, Piscataway, New Jersey 08855 Lang Hong (237) Department of Electrical Engineering, Wright State University, Dayton, Ohio 45435 P. J. Lynch (289) Flight Dynamics Directorate, Wright Laboratory, WrightPatterson Air Force Base, Ohio 45433 Claudio Maffezzoni (159) Dipartimento di Elettronica e Informazione, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milano, Italy Sen-Chueh Peng (207) Department of Electrical Engineering, National YunLin Polytechnic Institute, Huwei, Yunlin, Taiwan, R.O.C. vii

viii

CONTRIBUTORS

Riccardo Scattolini (159) Dipartimento di Informatica e Sistemistica, Universitgt degli Studi di Pavia, 27100 Pavia, Italy Xuemin Shen (1) Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1 Jitendra K. Tugnait (41) Department of Electrical Engineering, Auburn University, Auburn, Alabama 36849 Edwin Engin Yaz (89, 121) Electrical Engineering Department, University

of Arkansas, Fayetteville, Arkansas 72701 Hsi-Han Yeh (289) Flight Dynamics Directorate, Wright Laboratory, Wright-Patterson Air Force Base, Ohio 45433

PREFACE Effective control concepts and applications date back over millennia. One very familiar example of this is the windmill. It was designed to derive maximum benefit from windflow, a simple but highly effective optimization technique. Harold Hazen's 1932 paper in the Journal of the Franklin Institute was one of the earlier reference points wherein an analytical framework for modem control theory was established. There were many other notable milestones along the way, including the MIT Radiation Laboratory series volume on servomechanisms, the Brown and Campbell book, Principles of Servomechanisms, and Bode's book, Network Analysis and Synthesis Techniques, all published shortly after mid-1945. However, it remained for Kalman's papers of the late 1950s (which provided a foundation for modem state-space techniques) and the tremendous evolution of digital computer technology (which was founded on the continuous giant advances in integrated electronics) to establish truly powerful control systems techniques for increasingly complex systems. Today we can look forward to a future that is rich in possibilities in many areas of major significance, including manufacturing systems, electric power systems, robotics, aerospace systems, and many other systems with significant economic, safety, cost, and reliability implications. Thus, this volume is devoted to the most timely theme of "Techniques in Discrete-Time Stochastic Control Systems." The first contribution to this volume is "Techniques for Reduced-Order Control of Stochastic Discrete-Time Weakly Coupled Large Scale Systems," by Xuemin Shen, Zijad Aganovic, and Zoran Gajic. There are cases of large scale complex stochastic systems whose discrete-time control can be greatly simplified by approaching their design through decomposition into weakly coupled systems with no significant impact on overall system performance. "Techniques in Stochastic System Identification with Noisy Input and Output System Measurements," by Jitendra K. Tugnait, describes parameter estimation and system identification for stochastic linear systems, a topic of active research for over three decades. In earlier works it was often assumed that the measurements of the system output are noisy, but measurements of the input to the system are well defined. This contribution is an in-depth and comprehensive treatment of the important problem of the identification

x

PREFACE

of stochastic linear systems when the input as well as the output measurements are noisy. Illustrative examples demonstrating the effectiveness of the results are presented. The next contribution is "Robust Stability of Discrete-Time Randomly Perturbed Systems," by Edwin Engin Yaz. Maintaining a control system's stability in the presence of parameter perturbations (i.e., the issue of robustness) is. of prime importance in system design, and therefore, much research effort to improve sufficient conditions for stability robustness has been expended in this area. In addition to presenting a comprehensive treatment of important results to date a number of significant new results are presented. "Observer Design for Discrete-Time Stochastic Parameter Systems," by Edwin Engin Yaz, is an in-depth treatment of linear system observer design to reconstruct the state vector of discrete-time systems with both white (timewise uncorrelated) and jump-Markov-type parameters based on noisy observations. Such stochastic parameter systems occur in many significant applications as noted in this contribution. The significant area of robustness in observer design is also examined. The next contribution is "The Recursive Estimation of Time Delay in Sampled-Data Control Systems," by Gianni Ferretti, Claudio Maffezzoni, and Riccardo Scattolini. Many engineering systems have inherent varying time delays in their system dynamics. This contribution is an in-depth treatment of the principal approaches to the recursive delay estimation problem, their applicability assumptions, and their related performance in a number of simulation experiments. The next contribution, "Stability Analysis of Digital Kalman Filters," by Bor-Sen Chen and Sen-Cheuh Peng, is an in-depth treatment of methods to ensure the stability of state estimators by Kalman filters with the finite word length inherent to digital (computer) filters. Means for treating (stabilizing) filters which might otherwise be unstable are also presented. "Distributed Discrete Filtering for Stochastic Systems with Noisy and Fuzzy Measurements," by Lang Hong, presents algorithmic techniques for distributed discrete filtering of both noisy and fuzzy measurements. Illustrative examples of application of the techniques to the multisensor integration problem are presented. The computational efficiency and capability of dynamic sensor integration are clearly demonstrated by these examples. Mohammed Abbad and Jerzy A. Filar present a comprehensive treatment of algorithms for controlled Markov chains in "Algorithms for Singularly Perturbed Markov Control Problems: A Survey." Since the uncontrolled case forms a fundamental building block, this area is also examined in depth, and numerous illustrative examples are presented. The final contribution to this volume is "Control of Unknown Systems via Deconvolution," by Hsi-Han Yeh, Siva S. Banda, and P. J. Lynch. This contribution deals with methods for on-line real-time control of systems or

PREFACE

xi

plants which are unknown, a problem which occurs in diverse engineering system problems, as noted in this contribution. The technique of on-line deconvolution generates control signals that steer the output of an unknown plant to follow a reference trajectory as closely as the numerical accuracy of the instrumentation allows, provided that the unknown plant is stable and has minimum phase, both of which might be generally expected in practice. The contributors to this volume are all to be highly commended for their contributions to this comprehensive treatment of discrete-time stochastic control systems. They have produced a work which should provide a unique and most highly useful reference on this broad subject internationally for many years to come.

This Page Intentionally Left Blank

Techniques for Reduced-Order Control of Stochastic Discrete-Time Weakly Coupled Large Scale Systems

Xuemin Shen Department of Electrical and Computer Engineering University of Waterloo Waterloo, Ontario, Canada, N2L 3G1

Zijad Aganovic Cylex Systems Inc. 6001 Broken Sound Parkway Boca Raton, FL 33487

Zoran Gajic Department of Electrical and Computer Engineering Rutgers University Piscataway, NJ 08855-0909

I. INTRODUCTION The weakly coupled systems were introduced to the control audience by (Kokotovic et al., 1969). Since then many control aspects for the linear weakly coupled systems have been studied (Medanic and Avramovic, 1975; Ishimatsu et al., 1975; Ozguner and Perkins, 1977; Delacour et al., 1978; Mahmoud, 1978; Petkovski and Rakic, 1979; Washburn and Mendel, 1980; Arabacioglu et al., 1986; Petrovic and Gajic, 1988; Gajic and Shen, 1989, 1993; Harkara et al., 1989; Gajic et al., 1990; Shen, 1990; Shen and Gajic, 1990a, 1990b, 1990c; Su, 1990; Su and Gajic, 1991, 1992; Qureshi, 1992). The general weakly coupled systems, in different set ups, have been studied by Siljak, Basar and their coworkers (Ikeda and Siljak, 1980; Ohta and Siljak, 1985, Sezer and Siljak, 1986, 1991; Kaszkurewicz et al., 1990; Siljak, 1991; Srikant and Basar, 1989, 1991, 1992a, 1992b; Skataric, 1993; Skataric et al., 1991, 1993; Riedel, 1993). The weak coupling has been also considered in the concept of multimodeling (Khalil and Kokotovic, 1978; Ozguner, 1979; Khalil, 1980; Saksena and Cruz, 1981a, 1981b; Saksena and Basar, 1982; Gajic and CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.

2

XUEMIN SHEN ET AL.

Khalil, 1986; Gajic, 1988; Zhuang and Gajic, 1991) and for nearly completely decomposable Markov chains (Delebecque and Quadrat, 1981; Srikant and Basar, 1989; Aldhaheri and Khalil, 1991). The nonlinear weakly coupled control systems have been studied in only a few papers (Kokotovic and Singh, 1971, Srikant and Basar, 1991, 1992b; Aganovic, 1993; Aganovic and Gajic, 1993). The discrete-time linear control systems have been the subject of recent research (Shen and Gajic, 1990a, 1990b). In this chapter we first give an overview of the obtained results on filtering and control of discrete-time stochastic systems and then present some new results. For the reason of completeness, we study the main algebraic equations of the linear control theory, that is, the Lyapunov and Riccati equations. Corresponding parallel reduced-order algorithm for solving discrete Lyapunov and Riccati equation of weakly coupled systems are derived and demonstrated on the models of real control systems. Algorithms for both the Lyapunov and Riccati equations are implemented as synchronous ones. Their implementation as the asynchronous parallel algorithms is under investigation.

II. RECURSIVE METHODS FOR WEAKLY COUPLED DISCRETE-TIME SYSTEMS In this section parallel reduced-order algorithms for solving discrete algebraic Lyapunov and Riccati equations of weakly coupled systems and the corresponding linear-quadratic optimal control problem are presented.

A. PARALLEL ALGORITHM FOR SOLVING DISCRETE ALGEBRAIC LYAPUNOV EQUATION Consider the algebraic discrete Lyapunov equation

ATpA-P---Q,

A_0

(1)

In the case of a weakly coupled linear discrete system the corresponding matrices are partitioned as

eA3

A4'

Q-

eQ~

Q3 '

eP~

Pa

where Ai, i = l, 2, 3, 4, and Qj, j = l, 2, 3, are assumed to be continuous functions of e. Matrices /91 and /93 are of dimensions n • n and m • m, respectively. Remaining matrices are of compatible dimensions. The partitioned form of (1) subject to (2) is -

t,1 +

+

+

+

-

0

(3)

DISCRETE-TIME WEAKLY COUPLED SYSTEMS

3

AT p1A2 - P2 + Q2 + AT p2A4 + AT pDA4 + e2AT p T A 2 -- 0

(4)

ATpDA4 - P3 + Qa + e2(ATp1A2 + A T p 2 A 4 + A T p T A 2 ) -- 0

(5)

Define, O (e 2) perturbations of (3)-(5) by

A T P l A I - Pl + QI -- O

ATl f'IA2 + AT['2A4 + ATpDA4 -

#2 +

(6)

Q2 - 0

(7)

A T PaA4 - P3 + Qa - 0 !

(8)

!

Note that we did not set e - 0 in Ais and Q:is. Under the assumption made in (1), A < 0, it follows that for sufficiently small e the matrices A1 and A4 are stable. Then the unique solutions of (6)-(8) exist. Define errors as

1'1- 1'1 + eEl _

-192-- P2 + e E 2

(9)

P3 - PD + eEa Subtracting (6)-(8) from (3)-(5), the following error equations are obtained

AT E1A1 - E1 - - A T peA3 - AT p T A 1 -- AT pDA3 AT EDA4 - Ea - - A T p1A2 - AT p2A4 - AT p T A 2

(10)

AT E2A4 - E2 - -AT1E1A2 - AT p T A 2 -- AT EaA4 The proposed parallel synchronous algorithm for the numerical solution of (10) is as follows (Shen, et al., 1991).

Algorithm 1: AT E ~ i + I ) A 1 _ E~ '+1)

_

_ A T p(2i)Aa _ A T P(i)TA1

_

AT p(i)Aa

= - ~ 2 " l **2 - **2 " 2 **4 - A T

A2

(11)

AT E~i+I)A4 - E~ i+1) = - A T E~i+I)A2 - AT p (i)T A2 - A T E(3i+l)A4 with starting points E~ ~ - E~ ~ - E (~ - 0 and p(i)_pj+e2E~i),

j_1,2,3;

i-0,1,2...

(12)

4

XUEMIN SHEN ET AL.

Now we have the following theorem. Theorem 1 Under stability assumption imposed in (1) and for ~ sufficiently small, the algorithm (11)-(12) converges to the exact solutions for Ejs with the rate of convergence of O(e2). For the proof of this theorem see (Gajic and Shen, 1993). 1. CASE STUDY: DISCRETE CATALYTIC CRACKER A fifth-order model of a catalytic cracker (Kando et al., 1988), demonstrates the efficiency of the proposed method. The problem matrix A (after performing discretization with the sampling period T = 1) is given by

Ad

--

0.011771 0.014096 0.066395 0.027557 0.000564

0.046903 0.056411 0.252260 0.104940 0.002644

0.096679 0.115070 0.580880 0.240400 0.003479

0.071586 0.085194 0.430570 0.178190 0.002561

-0.019178"] -0.022806| -0.11628 / -0.048104| -0.0006561

The small weak coupling parameter is e = 0.21 and the state penalty matrix is chosen as Q = I. The simulation results are presented in Table 1.

B. PARALLEL ALGORITHM FOR SOLVING DISCRETE ALGEBRAIC RICCATI EQUATION The algebraic Riccati equation of weakly coupled linear discrete systems is given by

P - A T P A + Q - A T P B ( B T P B + R ) - I B TPA,

R > O, Q >_ 0

(13)

with cA3

Q-

A4

~2

'

eB3

Q3

0

B4

(14)

R2

and e is a small weak coupling parameter. Due to block dominant structure of the problem mamces, the required solution P has the form P-

[

ePf

eP2] /'3

(15)

DISCRETE-TIME WEAKLY COUPLED SYSTEMS .

5

..

F i 1.00030 0.00135 0.00135 1.00540

0.54689 0.40537 -0.10944 2.08640 1.54650 -0.41752

1.93020 0.68954 -0.18620 0.68954 1.51110 -0.13802 -0.18620 -0.13802 1.03730

1.01390 0.05290 0.052897 1.20180

0.66593 0.49359 -0.13322 2.54040 1.88290-0.50820

2.20320 0.89183 -0.24071 0.89183 1.66100 -0.17841 -0.24071 -0.17841 1.04820

1.016200.06184 0.06184 1.23600

0.69091 0.51209 -0.13821 2.63570 1.95350-0.52722

2.26010 0.93400 -0.25208 0.93400 1.69230 -0.18683 -0.25208 -0.18683 1.05040

,,

I/

iJ

ii

2

!

3

1

.

0

1

6

7

0

0

.

0

6

3

7

1

i 006 , 4 i[ 1.01680 0.06409 / 0.06409 1.24450 i

,

ii

0.69604 0.51590 -0.13923 2.65520 1.96800 -0.53113

2.27170 0.94260 -0.25439 0.94260 1.69860 -0.18855 -0.25439 -0.18855 1.05090

0.69710 0.51668 -0.13944 2.65930 1.97100-0.53193

2.27410 0.94437 -0.25487 0.94437 1.70000 -0.18891 -0.25487 -0.18891 1.05100 2.27460 0.94473 -0.25497 0.94473 1.70020 -0.18898 -0.25497 -0.18898 1.05100

..

5

1.016800.06417 0.06417 1.24480

0.69731 0.51684 -0.13948 2.66010 1.97160-0.53210

6

1.01680 0.06418 0.064 18 1.24490

0.69736 0.51687 -0.13949 2.66010 1.97170-0.53213

7

1.016800.06419 0.06419 1.24490

0.69737 0.51688 -0.13950 2.66030 1.97180-0.53214

i

ii

2.27470 0.94481 -0.25499 0.94481 1.70030-0.18899 -0.25499 -0.18899 1.05100 I

II

2.27470 0.94482 -0.25499 0.94482 1.70030 -0.18900 -0.25499 -0.18900 1.05100 ,.

Table 1" Reduced-order solution of discrete weakly coupled algebraic Lyapunov equation (P(') = P,=,,a)

The main goal in the theory of weakly coupled control systems is to obtain the required solution in terms of reduced-order problems, namely subsystems. In the case of the weakly coupled algebraic discrete Riccati equation, the inversion of the partitioned matrix B T P B -t- R will produce a lot of terms and make the corresponding approach computationally very involved, even though one is faced with the reduced-order numerical problems. To solve this problem, we have used the bilinear transformation to transform the discrete-time Riccati equation into the continuous-time algebraic Riccati equation of the form

ArT P~ + Pr

+ Qr - P~S~Pr - O,

S~- B~R[1B T

such that the solution of (13) is equal to the solution of (16).

(16)

6

XUEMIN SHEN ET AL.

The bilinear transformation states that equations (13) and (16) have the same solutions if the following relations hold, that is

Ac = I -

2D -T

S~ - 2(I + A ) - I S d D -1, Sd -- B R - 1 B T

Q~ - 2D-XQ(I + A) -1

(17)

D - (I + A) T + Q(I + A ) - I S d assuming that (I + A)-1 exists. It can be seen that for weakly coupled systems the matrix

(I + A)_~ _ [ 0(1) O(e)

O(e)] 0(1)

(18t

is invertible for small values of e. It can be verified that the weakly coupled structure of the matrices defined in (14) will produce the weakly coupled structure of the transformed continuous-time matrices defined in (17). It follows from the fact that Sd from (17) and Q from (14) have the same weakly coupled structure as (18), so does D in (17). The inverse of D is also in the weakly coupled form as defined in (18). From (17) the weakly coupled structure of matrices Ac and Qc follows directly since they are given in terms of sums and/or products of weakly coupled matrices. Using the standard result from (Stewart, 1973), it follows that the method proposed in this section is applicable under the following assumption. Assumption 1 The system matrix A has no eigenvalues located at-1. It is important to point out that the eigenvalues located in the neighborhood o f - 1 will produce ill-conditioning with respect to matrix inversion and make the algorithm numerically unstable. Let us introduce the following notation for the compatible partitions of the transformed weakly coupled matrices, that is

[All ,A12] sc [sll ,s12] Ac

-

P~-

6A21

A22

5

P3 '

~2

'

eS~2

$22

~Q~2

Q22

[

(19)

(20)

These partitions have to be performed by a computer only, in the process of calculations, and there is no need for the corresponding analytical expressions. The solution of (16) can be found in terms of the reduced-order problems by imposing standard stabilizability-detectability assumptions on the subsystems. The efficient recursive reduced-order algorithm for solving (16) can be found in


7

(Gajic and Shen, 1993). It will be briefly summarized here taking into account the specific features of the problem under study. The O(e 2) approximation of (13) subject to (14)-(15) can be obtained from the following decoupled set of reduced-order algebraic equations P 1 A l l -+- A1T1P1 -- P 1 S l l P 1 + Qll - 0 PaA22 -+- A~2Pa

-

(21)

PsS22Ps + Q92 - 0

and (22)

P2A2 + A~P2 - - ( P I A 1 2 + A ~ P , + 012 - P1s12P,) where /~1(~)- [All(e)-

S~(e)P~(e)],

A 2 ( e ) - [A22(e)- S22(e)Ps(e)]

(23)

The unique positive semidefinite stabilizing solutions of (21) exist under the following assumption. Assumption 2 The triples (All(e), v/Sii(e), v/Qii(e)), i = 1, 2, are stabilizable-detectable. Under Assumption 2 matrices/~1 (e) and A2(e) are stable so that the unique solution of (22) exists also. If the errors are defined as

Pi = Pj +,2Ei,

j - 1, 2, 3

(24)

e(P2 + e2E2)] Ps + e2E3 J

(25)

then the exact solution will be of the form P

P1

e(P2

+e2E1 -+- eZE2) T

The fixed point parallel reduced-order algorithm for the error terms, obtained by using results from (Gajic and Shen, 1993), has the form. Algorithm 2:

EI'+I)A1 + A~EI '+~) _ p~,)s12p~,) ~ + P~')~P~') '.-722.r-2

--

1 -- -t-x21-,- 2

E(i+I)A2 + ' '~2 ~'3 ~-('+1)-

-]-P(i)Ts12p(i)

"F-

(26)

$11

_,,,-(')~s, 1-2 -(') +

(27)

--12~2 q-

P(i)TA12

$22

A~E~'+I) + E~'+I)A: + E~'+I)AI: + A~,EI '+~) :P(~')r,-'12- 2 +,~(~'+1)s11~(')2

+

~i '+~) $12 ~(;)

+

E~')$22 ~i '+~) / #

(28)

8

XUEMIN SHEN1~1 AL.

with E[ ~ - 0, E~ ~ - 0, E (~ - 0, where

P~') -

Pj + , ~ E J ' ) ,

j - 1, 2, 3;

i = 1, 2, 3, ...

(29)

and

(3o)

This algorithm satisfies converges to the exact solution of E with the rate of convergence of O(e~), that is

II

or

("11

i = 0, 1, 2,..

(31)

or equivalently

II

II- o

(32)

In summary, the proposed parallel algorithm for the reduced-order solution of the weakly coupled discrete algebraic Riccati equation has the following form: 1) Transform (13) into (16) by using the bilinear transformation defined in (17). 2) Solve (16) by using the recursive reduced-order parallel algorithm defined by (26)-(30). 1. CASE STUDY: DISCRETE MODEL OF A CHEMICAL PLANT A real world physical example (a chemical plant model (Gomathi et al., 1980)) demonstrates the efficiency of the proposed method. The system matrices are obtained from (Gomathi et al., 1980) by performing a discretization with a sampling rate T = 0.5.

A-

B T _

10 -2

95.407 40.849 12.217 4.1118 0.1305

10_2[ 0.0434 [ -0.0122

1.9643 41.317 26.326 12.858 0.5808 2.6606 -1.0453 Q = Is ,

0.3597 16.084 36.149 27.209 1.8750 3.7530 -5.5100 R = I~

0.0673 4.4679 15.930 21.442 3.6162

0.0190 1.1971 12.383 40.976 94.280

3.6076 -6.6000

0.4617 -0.9148

]


9

The small weak coupling parameter e is built into the problem and can be roughly estimated from the strongest coupled matrix (matrix B). The strongest coupling is in the third row, where e-

b31 3.753 -- = 0.68 b32 5.510

Simulation results are obtained using the MATLAB package for computer aided control system design. The solution of the algebraic Riccati equation, obtained from Algorithm 2, is presented in Table 2. For this specific real world example the proposed algorithm perfectly matches the presented theory since convergence, with the accuracy of 10 -4 , is achieved after 9 iterations (0.68 is = 10-4). Note very dramatic changes in the element P~J) per iteration. Thus, in this example only higher order approximations produce satisfactory results. The obtained numerical results justify the necessity for the existence of the higher order approximations for both the approximate control strategies and the approximate trajectories.

III. PARALLEL REDUCED-ORDER CONTROLLER FOR STOCHASTIC DISCRETE-TIME SYSTEMS In this section, we study the linear-quadratic Gaussian control problem of weakly coupled discrete-time systems. The partitioned form of the main equation of the optimal linear control theory m the Riccati equation, has a very complicated form in the discrete-time domain. In Section II, that problem is overcome by using a bilinear transformation which is applicable under quite mild assumption, so that the reduced-order solution of the discrete algebraic Riccati equation of weakly coupled systems can be obtained up to any order of accuracy, by using known reduced-order results for the corresponding continuous-time algebraic Riccati equation. Although the duality of the filter and regulator Riccati equations can be used together with results reported in (Shen and Gajic, 1990b) to obtain corresponding approximations to the filter and regulator gains, such approximations will not be sufficient because they only reduce the off-line computations of implementing the Kalman filter which will be of the same order as the overall weakly coupled system. The weakly coupled structure of the global Kalman filter is exploited in this section such that it may be replaced by two lower order local filters. This has been achieved via the use of a decoupling transformation introduced in (Gajic and Shen, 1989). The decoupling transformation of (Gajic and Shen, 1989) is used for the exact block diagonalization of the global Kalman filter. The approximate

10

XUEMIN SHEN ET AL.

20.9061 0.9202 0.9202 1.2382

1.8865 1.4365 18.5536 0.5259 0.3219 2.1852

1.2937 0.1971 1.2516 0.1971 1.1514 1.2887 1.2516 1.2887 21.0090

39.2244 2.5453 2.5453 1.5406

3.4212 2.3932 28.8267 0.7575 0.4428 3.3277

1.4754 0.2982 2.0621 0.2982 1.2067 1.7456 2.0621 1.7456 25.1919

4.2746 2.8594 32.9119 0.8637 0.5006 3.8272

1.5558 0.3423 2.4450 0.3423 1.2304 1.9451 2.4450 1.9451 26.7777

4.6785 3.0634 34.4250 0.9111 0.5250 4.0179

1.5911 0.3609 2.5959 0.3423 1.2399 2.0161 2.5959 2.0161 27.2107

I !

50.6375 3.6481 3.6481 1.6827

] ]

L

u

3

56.1732 4.2167 4.2167 1.7492

4

58.6366 4.4773 4.4773 1.7788

4.8566 3.1498 34.9986 0.9314 0.5351 4.0888

1.6063 0.3686 2.6519 0.3423 1.2436 2.0416 2.6519 2.0416 27.3486

5

59.6956 4.5906 4.5906 1.7915

4.9327 3.1858 35.2222 0.9400 0.5392 4.155

1.6127 0.3717 2.6727 0.3423 1.2451 2.0510 2.6727 2.0510 27.3982

60.1433 4.6387 4.6387 1.7969

4.9646 3.2008 35.3112 0.9436 0.5409 4.1258

1.6154 0.3729 2.6800 0.3729 1.2451 2.0546 2.6800 2.0546 27.4171

i

u

ii

i i

!~ 9

60.4410 4.6707 4.6707 1.8004

! i

1 ||

i

12 i_1_

i

1.6171 0.3737 2.6853 0.3729 1.2461 2.0567 2.6853 2.0567 27.4288

4.9857 3.2106 35.3676 0.9459 0.5420 4.1321 .....

60.4621 4.6730 4.6730 1.8006

4.9872 3.2113 35.3715 0.9461 0.5420 4.1326

1.6172 0.3738 2.6857 0.3729 1.2461 2.0569 2.6857 2.0569 27.4295

16 [~ 60.4636 4.6732

4.9873 3.2113 35.3717 0.9461 0.5420 4.1326

1.6172 0.3738 2.6857 0.3729 1.2461 2.0569 2.6857 2.0569 27.4296

ii

4.6732 1.8006 u

i

P, =

l

=

i

p

=pl

Table 2: Reduced-order solution of the discrete weakly coupled algebraic Riccati equation

feedback control law is then obtained by approximating the coefficients of the optimal local filters and regulators with the accuracy of O(e.N). The resulting feedback control law is shown to be a near-optimal solution of the LQG by studying the corresponding closed-loop system as a system driven by white noise. It is shown that the order of approximation of the optimal performance is O (eN), and the order of approximation of the optimal system trajectories is O(E.2N).


11

All required coefficients of desired accuracy are easily obtained by using the recursive reduced-order fixed point type numerical techniques developed in Section II. The obtained numerical algorithms converge to the required optimal coefficients with the rate of convergence of O(e2). In addition, only loworder subsystems are involved in the algebraic computations and no analyticity requirements are imposed on the system c o e f f i c i e n t s - which is the standard assumption in the power-series expansion method. As a consequence of these properties, under very mild conditions (coefficients are bounded functions of a small coupling parameter), in addition to the standard stabilizability-detectability subsystem assumptions, we have achieved the reduction in both off-line and online computational requirements. The results presented in this section are mostly based on the doctoral dissertation (Shen, 1990) and on the recent research papers (Shen and Gajic, 1990a, 1990b). Consider the linear discrete weakly coupled stochastic system

x~(n + 1) = Al~xl(n) + eA~2z2(n) + Bllul(n) + eB~2u2(n) +Gttw~(n) + eG~2w2(n) x2(n + 1) = eA2xz~(n) + A22x2(n) + eB2~u~(n) + B22u2(n)

(33)

with the performance criterion

s

(34)

-

n--0

where xi E ~ n i , i -- 1, 2, comprise state vectors, ui E ~ rl~i , i = 1, 2, are control inputs, Yi C ~ti, i = 1, 2, are observed outputs, wi E ~"~ and vi E ~t~ are independent zero-mean stationary Gaussian mutually uncorrelated white noise processes with intensities Wi > 0 and Y~ > 0, respectively, and zi E ~,i, i = 1, 2, are the controlled outputs given by

zx(n) = Dllxl(n) + eD12x2(n) z2(n) = eD21zl(n) + D22x2(n)

(35)

All matrices are bounded functions of a small coupling parameter e and have appropriate dimensions. In addition, it is assumed that Ri, i = 1, 2, are positive definite matrices.

12

XUEMINSHENETAL. The optimal control law is given by (Kwakernaak and Sivan, 1972) (36)

u(n) = - F ~ ( n )

with ~(n +

I) =

(37)

A ~ ( n ) + B u ( n ) + K[y(n) - C~(n)]

where

A_[All ,A12] B_[Bll ,B12] C_[Cll cA21

A22 '

eB21

B22

[eK2x K22 '

'

eC21

eF21

~C12 ] C22

(38)

F22

The regulator gain F and filter gain K are obtained from F - ( R + B r P B ) - 1BT P A

(39)

K - A Q C T ( V + C Q C T ) -1

(40)

where P and Q are positive semidefinite stabilizing solutions of the discrete-time algebraic regulator and filter Riccati equations, respectively, given by P - D TD + A TPA - A ~'PB(R + B TPB)-xB TPA

(41)

Q - AQA T - AQC r (V + CQC T)-xCQAT + GWG T

(42)

with R = diag(R1 R2),

and

W - diag(W~ W2),

o_[oll ,o12] eD21

D22

V - diag(V~ V2)

[ 11 c12]

'

eG~l

G22

(43)

44,

Due to the block dominant structure of the problem matrices the required solutions P and Q have the form

[ (e12],

P- ~P/~2 P22

0-

[Qll eQ12] EQ~2 Q22

(45)

In order to obtain the required solutions of (41) and (42) in terms of the reduced-order problems and to overcome the complicated partitioned form of the

DISCRETE-TIMEWEAKLYCOUPLEDSYSTEMS

13

discrete-time algebraic Riccati equation, we have used the method developed in the previous section, to transform the discrete-time algebraic Riccati equations (41) and (42) into the continuous-time algebraic Riccati equations of the form P S R P + D T D R -- O, SR -- B R R R 1 B T

(46)

A F Q + Q A T - QSFQ + G F W F G T -- O, SF -- C T V F 1 C F

(47)

ATRp + P A R -

such that the solutions of (41) and (42) are equal to the solutions of (46) and (47), that is P-P,

Q=Q

(48)

where AR -

BRR;~B~ -

I-

2(I +

T

A)-~BR-~BT/x~~

(49)

DT DR -- 2 A ~ I D T D ( I + A) -1 A R -- (I + A T) + D T D ( I + A ) - X B R - 1 B T

and - Z-

2(A

- 2(I + A

)-Ic

v-IcAT

G F W F G T - 2 A F 1 G W G T (I + AT) -1 A F -- (I + A) + G W G T (I

(50)

+ AT)-IcTv-1c

It is shown in Section II that the equations (46) and (47) preserve the structure of weakly coupled systems. These equations can be solved in terms of the reduced-order problems very efficiently by using Algorithm 2, which converges with the rate of convergence of O(e2). Solutions of (46) and (47) are found in terms of the reduced-order problems by imposing standard stabilizability-detectability assumptions on subsystems. Getting approximate solution for P and Q in terms of the reduced-order problems will produce savings in off-line computations. However, in the case of stochastic systems, where the additional dynamical s y s t e m - f i l t e r - has to be built, one is particularly interested in the reduction of on-line computations. In this section, the savings of on-line computation will be achieved by using a decoupling transformation introduced in (Gajic and Shen, 1989). The Kalman filter (37) is viewed as a system driven by the innovation process. However, one might study the filter form when it is driven by both

14

XUEMIN SHEN ET AL.

measurements and control. The filter form under consideration is obtained from (37) as ~ ( n + 1) - (AI~ - B ~ F ~

- e2B~9.F~2)~(n)

+e(A12 - BllF12 - B12F22)~2(n) + K l l v l ( n ) + eK12vz(n)

(51) ~2(n + 1) = e(A21 - B21Fll - B22F21)~l(rt) +(A22 - e2B21F12 - B22F22)~,2(n) + e K e l v l ( n ) + K22vz(n) with the innovation process

,~(,~) = y~(~) - , c ~ , e ~ ( n )

(52)

- c~e~(,~)

The nonsingular state transformation of (Gajic and Shen, 1989) will block diagonalize (51) under the following assumption. Assumption 3 The subsystem matrices ( A l l - B l l F l l - E2B12F21) and ( A 2 2 - B 2 2 F 2 2 - e2B21F12) have no eigenvalues in common. The transformation of interest is given by

[~l(n)

-

E2/~H e/-/

1

-~/;

(n) ]

~2 1

(53)

with

T -

I,,, -ell

eL ] I,~ - eZHL

(54)

where matrices L and H satisfy equations L(a22 -I- e.Hal2)

_

(all --

~2

a l 2 H ) L 2t- a12 -- 0

H a l l - a22H + a21 - e2Ha12H = 0

(55)

(56)

with a l l -- A l l -

BllFll-

e2B12F21

a12 -- A 1 2 - B11F12 - B12F22

a2x = A2x -

B2xFll

-

B22F21

a22 = A22 -

B22Fz2

-

e2BzlFx2

(57)


15

The optimal feedback control, expressed in the new coordinates, has the form

ui(n) - - f i . ~ i ( n ) - efi2r u2(n) : - - , f 2 i ~ l ( n ) - f2zPp.(n)

(58)

with ~)i(n + 1) = a i O i ( n ) + / 3 i i v i ( n )

+ eflizv2(n)

(59)

~)2(n + 1) = az~)2(n) + eflzivi(n) + fl22v2(n) where

f i i - Fll -e2F12H.

A2-

f2~ = F2~ - F22H, C~l -"

F12 + (Fll - e2F12H)L

f22 = F22 + e2(F2~ - F 2 2 I t ) L

-- e2al2H,

all

flii = K i i - e 2 L ( H + K z i ) .

(60)

c~2 : a22 -~- e2Hal2 fli2 = K i 2 - L K 2 2 - e 2 L H K i 2

fl2i = H K i l + Kzi,

fl22 = K22 + e2HKi2

The innovation processes vi and v2 are now given by v i ( n ) = y i ( n ) - diiT)i(n) - edi2~2(n)

(61)

v2(n) = y2(n) - ed2i~i(n) - d2z7)2(n) where d i i -- C i i - e~Ci2H,

di2 = C i i L ,u Ci2 - e2C12HL

d21 = C21 - C2~H.

d22 = (722 + e 2(C2i - C22H)L

(62)

Approximate control laws are defined by perturbing coefficients F i j . Kij. i . j = 1. 2; L and H by O(ek). k = 1. 2 . . . . . in other words by using k-th approximations for these coefficients, where k stands for the required order of accuracy, that is

u~)(~)

_

-

e.(k)^(k). ,2~ ,i t ~ ) -,

(63)

.(k)^(k ~2~ 0~ )(~)

with _

~k)(,j,/,

+

+ e,~(k)

~(~)v(~)(n)

^(k )(~/') 1) _ O~k)'2

+

~,(k) Vl(k),lJ/,), ']'~21

(k)

(,~)

(64)

(k)v~k) (7"/,) +/~22

where

v~k)(n)

-

yl(n)

-

d (k);'(k) ~i .,~ (n) - e d(k);'(k ..~ .,~ )(,~)

,,~)(,~)

--

y~(,~)

--

e,(k) ^(k) ( n ) , ~ ,7~

-

,(k) ~^(k) (,~) 12 a22

(65)

16

X U E M I N SHEN ET AL.

and f};) - s.~ + o(,~),

dl~ ) - d,j + o(, ~) (66)

.,~ - .,~ + o(,~1. ~}~) - ~,~ + o(, ~1 i,j = 1,2

The approximate values of j(k) are obtained from the following expression j(k)

(~

1 _ -~E E

(n)DT Dz(k)(n) + o,k, (n)Ru(k)

(67)

n--O

1

T

)r

= -~tr{D Dq~)-q-f,k Rf(k)q~k)} where z2

T

r f~k)

u(k)(n) u(k)_ [u!k)(n)],

f(k,_ [ef2~)

^(k)) (68)

,jr(k)

f~2)]

The quantities qlk) and qlk) can be obtained by studying the variance equation of the following system driven by white noise ,~(~)(n + 1)

f~(~)C

~(~) - t~(~)d(k)

,~(~)(n)

(69)

+ [ G0 /3(k)] 0 [ w(n)

~(,,)]

where c~(k)_

c~ k)

0

0 4~' '

O(k)_

,~l(k )

)

,~,~ ; ~ , ~ , ~ - , ~ ,

11

;:~

(~0)

Equation (69) can be represented in the composite form

r(~)(,~ + :) - A(~)r(~)(n) + rt(k)w(n)

(71)

with obvious definitions for A(k), II (k), I'(k)(n), and w(n). The variance of F(k)(n) at steady state denoted by q(k), is given by the discrete algebraic Lyapunov equation (Kwakernaak and Sivan, 1972)

q(k)(n + 1) -- A(k)q(k)A(k)r + II(k)WII (k)T, W - diag(W, V)

(72)

DISCRETE-TIME W E A K L Y C O U P L E D SYSTEMS

17

with q(k) partitioned as q(k) 11 T q(k) -- [q~k2)

q~k2) q~)]

(73)

On the other hand, the optimal value of J has the very well-known form, (Kwakernaak and Sivan, 1972)

jopt _ l t r [ D T D Q + P K ( C Q C T + V ) K T]

(74)

where P, Q, F, and K are obtained from (39)-(42). The near-optimality of the proposed approximate control law (63) is established in the following theorem. Theorem 2 Let z x and x2 be optimal trajectories and J be the optimal value of the performance criterion. Let x~k), z~k), and j(k) be corresponding quantities under the approximate control law u (k ) given by (63). Under the condition stated in Assumption 3 and the stabilizability-detectability subsystem assumptions, the following hold

jopt _ j(k) __ O(ek) Var{z,-x,

(k)

}-O(e2k),

i-1,2,

k = O, 1, 2, ....

(75)

The proof of this theorem is rather lengthly and is therefore omitted here. It follows the ideas of Theorems 1 and 2 from (Khalil and Gajic, 1984) obtained for another class of small parameter problems n singularly perturbed systems. These two theorems were proved in the context of weakly coupled linear systems in (Shen and Gajic, 1990c). In addition, due to the discrete nature of the problem, the proof of our Theorem 2, utilizes a bilinear transformation from (Power, 1967) which transforms the discrete Lyapunov equation into the continuous one and compares it with the corresponding equation under the optimal control law. More about the proof can be found in (Shen, 1990). A. C A S E S T U D Y : D I S T I L L A T I O N C O L U M N A real world physical example, a fifth-order distillation column control problem, (Kautsky et al., 1985), demonstrates the efficiency of the proposed method. The problem matrices A and B are

A-

10 -3

989.50 117.25 8.7680 0.9108 0.0179

5.6382 814.50 123.87 17.991 0,3172

0.2589 76.038 750.20 183.81 1.6974

0.0125 5.5526 107.96 668.34 13.298

0.0006 0.3700 11.245 150.78 985.19

18

XUEMIN SHEN ET AL.

BT_10_3[

0.0192 --0.0013

6.0733 --0.6192

8.2911 --13.339

9.1965 --18.442

0.7025 ] --1.4252

These matrices are obtained from (I~autsky et al., 1985) by performing a discretization with the sampling rate A T - 0.1. Remaining matrices are chosen as

6"-

[11000] 0

0

1

1

1 '

Q-Is,

R-h

It is assumed that G - B, and that the white noise intensity matrices are given by Wz-1, W2-2, Vz-0.1, V2-0.1 The simulation results are presented in Table 3. In practice, how the problem matrices are partitioned will determine the choice of the coupling parameter which in turn determines the rate of convergence and the domain of attraction of the iterative scheme to the optimal solution. It is desirable to get as small as possible a value of the small coupling parameter. This will speed up the convergence process. However, the small parameter is built into the problem and one can not go beyond the physical limits. The small weak coupling parameter e can be roughly estimated from the strongest coupled matrix ~ in this case matrix B. Apparently the strongest coupling is in the third row, that is b3z 8.2911 -- ~ 0.62 b32 13.339

e-

It can be seen that despite the relatively big value of the coupling parameter e = 0.62, we have very rapid convergence to the optimal solution.

j(k)

y(k) _ j

0.6989 x 10 -3

0.80528 x 10- 2 ,-

1

4'

0.75977 x 10-2 .

.

2

.

"

0.2438 x 10 -3

.

0.74277 x 10- 2

0.7380 x 10 -4

0.73887 x 10-2

0.3480 x 10 -4

,,

.

optimal

.

.

.

.

0.73546 x 10- 2

0.5000 x 10 -6

0.73539 x 10 -2

< 1.000 x 10 -7

0.73539 x 10- 2 =

.,-

Table 3: Approximate values for criterion


19

IV. NEW METHOD FOR OPTIMAL CONTROL AND FILTERING OF LARGE SCALE SYSTEMS In this section the algebraic regulator and filter Riccati equations of weakly coupled discrete-time stochastic linear control systems are completely and exactly decomposed into reduced-order continuous-time algebraic Riccati equations corresponding to the subsystems. That is, the exact solution of the global discrete algebraic Riccati equation is found in terms of the reduced-order subsystem nonsymmetric continuous-time algebraic Riccati equations. In addition, the optimal global Kalman filter is decomposed into local optimal filters both driven by the system measurements and the system optimal control inputs. As a result, the optimal linear-quadratic Gaussian control problem for weakly coupled linear discrete systems takes the complete decomposition and parallelism between subsystem filters and controllers. In this section, we introduce a completely new approach pretty much different than all other methods used so far in the theory of weak coupling. It is well known that the main goal in the control theory of weak coupling is to achieve the problem decomposition into subsystem problems. Our approach is based on a closed-loop decomposition technique which guarantees complete decomposition of the optimal filters and regulators and distribution of all required off-line and on-line computations. In the regulation problem (optimal linear-quadratic control problem), we show how to decompose exactly the weakly coupled discrete algebraic Riccati equation into two reduced-order continuous-time algebraic Riccati equations. Note that the reduced-order continuous-time algebraic Riccati equations are nonsymmetric, but their O(~ 2) approximations are symmetric. The Newton method is very efficient for solving these nonsymmetric Riccati equations since the initial guesses close O(e z) to the exact solutions can be easily obtained. It is important to notice that it is easier to solve the continuous-time algebraic Riccati equation than the discrete-time algebraic Riccati equation. In the filtering problem, in addition of using duality between filter and regulator to solve the discrete-time filter algebraic Riccati equation in terms of the reduced-order continuous-time algebraic Riccati equations, we have obtained completely independent reduced-order Kalman filters both driven by the system measurements and the system optimal control inputs. In the last part of this section, we use the separation principle to solve the linear-quadratic Gaussian control problem of weakly coupled discrete stochastic systems. Two real control system examples are solved in order to demonstrate the proposed methods.

20

XUEMIN SHEN ET AL.

A. L I N E A R - Q U A D R A T I C C O N T R O L P R O B L E M In this section, we present a new approach in the study of the linearquadratic control problem of weakly coupled discrete systems. By applying the new algorithm the discrete algebraic Riccati equation of weakly coupled systems is completely and exactly decomposed into two reduced-order continuous-time algebraic Riccati equations. This decomposition allows us to design the linear controllers for subsystems completely independently of each other and thus, to achieve the complete and exact separation for the linear-quadratic regulator problem. Consider the weakly coupled linear time-invariant discrete system described by (Gajic et al., 1990; Gajic and Shen, 1993) x l ( 0 ) = ~1o x2(0) = ~2o

(76) with state variables xt E R "1, x2 E R n~, and control inputs ui E R mi, i = 1, 2, where e is a small coupling parameter. The performance criterion of the corresponding linear-quadratic control problem is represented by

J-

1 oo [~ k)~ u(k)rRu(k) E ( o (k) + ]

(77)

k=0

where

z(k) - [xl(k)] u(k) - J ut(k) ] Q _ [ Q1 x2(k) ' u2(k) ' eQ~

R_[R1

0

O]

R2

eQ2 ] > 0 Qa -

(78)

> 0

It is well known that the solution of the above optimal regulation problem is given by

u(k) - -R-XBT)~(k + 1) -

-(1{ + B T p r B ) - t B T p r A x ( k )

- -Fx(k) -

[Ft -

~F3

eF2 ] F~

(79) ~(k)

where )~(k) is a costate variable and P,. is the positive semi-definite stabilizing solution of the discrete algebraic Riccati equation given by (Dorato and Levis, 1971; Lewis, 1986)

P~ - Q + A T P~[I + SP~]-IA = Q + A r P ~ A - ATprB[R + BTp~B]-tBrP~A

(80)


21

The Hamiltonian form of (76) and (77) can be written as the forward recursion (Lewis, 1986) +

with

1)

Hr _ [A + BR-'~BTA-TQ, _A_TQ

-BR-1BTA -T] A_ T

(82)

where Hr is the symplectic matrix which has the property that the eigenvalues of n r can be grouped into two disjoint subsets P1 and r2, such that for every A~ E 1"1 there exists Ad E P2, which satisfies A~ • Aa = 1, and we can choose either P1 or P2 to contain only the stable eigenvalues (Salgado et al., 1988). For the weakly coupled discrete systems, corresponding matrices in (79)-(82) are given by

cA2] cA3 A4 ' B -

A-[A1

eB2 ] 1BT _ [ $1 6S2] eB3 B4 , S - BReS~2 $3

[B1

(83) The optimal open-loop control problem is a two-point boundary value problem with the associated state-costate equations forming the Hamiltonian matrix. For weakly coupled discrete systems, the Hamiltonian matrix retains the weakly coupled form by interchanging some state and costate variables so that it can be block diagonalized via the nonsingular transformation introduced in (Gajic and Shen, 1989), see also (Gajic and Shen, 1993). In the following, we show how to get the solution of the discrete-time algebraic Riccati equation of weakly coupled systems exactly in terms of the solutions of two reduced-order continuous-time algebraic Riccati equations.

R "~

Partitioning vector A(k) such that A(k) - [AT(k ) A~(k)] T with Al(k) E and A2(k) E R "~, we get

x2(k + 1) ~l(k + 1) A2(k + 1)

--

H~

x2(k) ~l(k) A2(k)

(84)

It has been shown in (Gajic and Shen, 1993; in the chapter on the open-loop control, page 181) that the Hamiltonian matrix (82) has the following form

H~-

"Air eA2~ Sir

ES2~

fA3r A4r

~4r

~3r

0~r eQ2~ AT~ eAzT~ eQ3r Q4r EAT2r AT2~

(85)

22

XUEMIN SHEN ET AL.

Note that in the following there is no need for the analytical expressions for matrices with a bar. These matrices have to be formed by the computer in the process of calculations, which can be done easily. Interchanging second and third rows in (85) yields

I

xl(k 1) 1 Al(k ++ 1) x2(k + 1) )~2(k + 1)

_

A2r S rlIXl 'I

I Air ATllr eQ2r eATlr Qlr 6A3r eS3r A4r ~ 603r eATI2r Q4r A T 2 r

(86)

eT3r T4r where

Air

Tit' T3r

Ax(k) _ x2(k) A2(k)

)~2x21k)k)

S:~r

S~r

(87)

A3r

---

AT2r'

~

AT22r

Introducing the notation

v(k)- ~(k)

v(k)-

(88)

we have the weakly coupled discrete system under new notation U(k + 1) = V(k + 1) =

TlrU(k) -~-F.T2rV(]r ,T3~U(k) + T~rV(k)

(89)

Applying the transformation from (Gajic and Shen, 1989, 1993), defined by

T~

_

[I-e2LrHr

6Hr

l

--eLr] I

~(k/

_a

'

Tr I _

I

-eH~

'Lr ]

I - e2ttr Lr

(90)

v(k/l

to (89), produces two completely decoupled subsystems [ 771(k +772(k + 1)1)] - 77(k + 1) - (Tlr -e2T2~H~)~7(k)

(91)

[

(92)

~i(k + 1) ] ~2(k + 1) - ~(k + 1) - (T4r + E2H,-T2r)~(k)


23

where Lr and Hr satisfy (93)

Hr Txr -- T4r Hr + T3r --~.2HrT2rHr -- 0

L~ (T,~ + , ~ z ~ T ~ ) - ( T .

-,~T~rZ~)Lr - ~

- o

(94)

The unique solutions of (93) and (94) exist under condition that matrices TI~ and -T4~ have no eigenvalues in common, (Gajic and Shen, 1989). The algebraic equations (93) and (94) can be solved by using the Newton method (Gajic and Shen, 1989), which converges quadratically in the neighborhood of the sought solution. The good initial guess required in the Newton recursive scheme is easily obtained, with the accuracy of O (e2), by setting e -- 0 in those equations, which requires the solution of linear algebraic Lyapunov equations. The rearrangement and modification of variables in (86) is done by using the permutation matrix E of the form

il(k,1

o o ollxl(k1 E[~(k)

~(k) ]

(95)

From (88), (90)-(92), and (95), we obtain the relationship between the original coordinates and the new ones

~,(k) _ ETT, EFx(k)]

_ii~[x(k)

-

[Hlr n2r]Ix(k)] II3r

II4~

,~(k)

(96)

Since A(k) = P,.x(k), where Pr satisfies the discrete algebraic Riccati equation (80), it follows from (96) that

[ ~l(k) ]

(97)

In the original coordinates, the required optimal solution has a closed-loop nature. We have the same characteristic for the new systems (91) and (92), that is,

,72(k)

[~(~)] -

0

0]

P~b

~(k)

(98)

24

XUEMINSHENET AL.

Then (97) and (98) yield [P~o

P~bO] __ (II3r + II4rPr)(IIlr + II2~P~) - t

(99)

it can be shown from (21) that II~ = I + O ( e ) ~ IIt~ = I + O ( e ) , II2~ = O(e), which implies that the matrix inversion defined in (99) exists for sufficiently small e. Following the same logic, we can find P~ reversely by introducing [f~tr

E T T'~ z E - ft,. -

~2r]

(100)

f~3,. f~4,.

and it yields Pr -

([

f~3r + f~4r era 0

0 P~

])(

~Zr + fl2r

0

o])-,

Prb

(101)

The required matrix in (101) is invertible for small values of e since from (100) we have f~r = I + O(e) =~ f~tr = I + O(e), f~2~ = O(e). Partitioning (91) and (92) as Laar [~t(k +

I)

:::

r/2(k) ]

(k)

-

(k)

(102)

1o3,

and using (98) yield to two reduced-order nonsymmetric algebraic Riccati equations Praazr - a 4 r P r a - a 3 r + Praa2rPra = 0 (104) P~bbt~ -b4~ P ~ b - b3~ + P~bb2~P~b = 0

(105)

It is very interesting that the algebraic Riccati equation of weakly coupled discrete-time control systems is completely and exactly decomposed into two reduced-order nonsymmetric continuous-time algebraic Riccati equations (104)(105). The latter ones are much easier to solve. It can be shown that O(E 2) perturbations of (104) and (105) lead to the symmetric reduced-order discrete-time algebraic Riccati equations obtained in (Shen and Gajic, 1990b). The solutions of these equations can be used as very good initial guesses for the Newton method for solving the obtained nonsymmetric algebraic Riccati equations (104) and (105). Another way to


25

find the initial guesses which are O(e 2) close to the exact solutions is simply to perturb the coefficients in (104)-(105) by O(e2), which leads to the reducedorder nonsymmetric algebraic Riccati equations

p(O)-~r,~ ,r - A~lrP (~ - Olr + p(O)~-~-p(O)r,, -r r,, - 0 T D( 0 pr(O)-y-.(0)-5--.(0) _ 0

(106)

Note that the nonsymmetric algebraic Riccati equations have been studied in (Medanic, 1982). An efficient algorithm for solving the general nonsymmetric algebraic Riccati equation is derived in (Avramovic et al., 1980). The Newton algorithm for (104) is given by p(i+l) (air + a2rP (/)) - ( a 4 r - P ( i ) a 2 r ) p ( i + l ) _ aar + P(ia)a2rP(ia) i - 0, 1, 2, ... (107) The Newton algorithm for (105) is similarly obtained as

b

(blr+b2r

-

- rb b2r) p(i+rb l,"

(lo8)

i - 0, 1, 2, ... The proposed method is very suitable for parallel computations since it allows complete parallelism. In addition, due to complete and exact decomposition of the discrete algebraic Riccati equation, the optimal control at steady state can be performed independently and in parallel. The reduced-order subsystems in the new coordinates are given by (109) ~l(k + 1) = (blr + b2rPrb)~l(k)

(110)

In summary, the optimal strategy and the optimal performance value are obtained by using the following algorithm. Algorithm 3: 1) 2) 3) 4) 5)

Solve decoupling equations (93)-(94). Find coefficients air, bit, i -- 1, 2,3, 4, by using (102)-(103). Solve the reduced-order algebraic Riccati equations (104)-(105), which leads to Ira and Prb. Find the global solution of Riccati equation in terms of Pra and Prb by using ( 101). Find the optimal regulator gain from (79) and the optimal performance criterion as Jopt = 0.5xT(to)Prx(tO).

26

XUEMIN SHEN ET AL.

Example 1: In order to demonstrate the efficiency of the proposed method, a discrete model (Katzberg, 1977) is considered. The problem matrices are given by A-

I 0.964 -0.342 0.016 0.144

0.180 0.802 0.019 0.179

Q = 0.1/4,

0.017 0.162 0.983 -0.163

R =/2,

0.0191 0.179 0.181' 0.820

B-

e = 0.329,

0.019 0.180 0.005 -0.054

nl = 2,

0.0011 0.019 0.019 0.181

n2 = 2

The optimal global solution of the discrete algebraic Riccati equation is obtained as

Pexact

-

1.3998 -0.1261 0.1939 -0.4848

-0.1261 1.2748 0.5978 1.3318

0.1939 0.5978 1.1856 0.7781

-0.4848] 1.3318 0.7781 1.9994

I

Solutions of the reduced-order algebraic Riccati equations obtained from (104) and (105) are

[0.7664 oos,o] b_[12042 0140]

P~a- 0.2287 0.3184 '

1.1374 2.4557

By using the formula of (101), the obtained solution for Pr is found to be identical to Pexact and the error between the solution of the proposed method and the exact one which is obtained by. using the classical global method for solving algebraic Riccati equation is given by P e x , c t - Pr - O(10 -13)

Assuming the initialconditions as :ca'(to) = [1 m a n c e value is

Jopt =

1 1

1] the optimal perfor-

0.sxT(0)P,-z(0) = 5.2206.

B. N E W F I L T E R I N G M E T H O D F O R W E A K L Y C O U P L E D LINEAR DISCRETE SYSTEMS The continuous-time filtering problem of weakly coupled linear stochastic systems has been studied by (Shen and Gajic, 1990c). In this section, we solve the filtering problem of linear discrete-time weakly coupled systems using the problem formulation from (Shen and Gajic, 1990b). The new method is based on the exact decomposition of the global weakly coupled discrete algebraic Riccati equation into reduced-order local algebraic Riccati equations. The


27

optimal filter gain will be completely determined in terms of the exact reducedorder continuous-time algebraic Riccati equations, based on the duality property between the optimal filter and regulator. Even more, we have obtained the exact expressions for the optimal reduced-order local filters both driven by the system measurements. This is an important advantage over the results of (Shen and Gajic, 1990b; 1990c) where the local filters are driven by the innovation process so that the additional communication channels have to be used in order to construct the innovation process. Consider the linear discrete-time invariant weakly coupled stochastic system x l ( k + 1) = Alxl(k) + eA2x2(k) + Glwl(k) + eGzw2 (k)

x2(k + 1) = eAaxl(k) + A4x2(k) + eGawl(k) + G4w2(k) ~(0) = ~0, ~ ( 0 ) = ~0

(111)

with the corresponding measurements

y ( k ) - [ y l [(yk2) (] k- )

ecaC1 eC2]c4[ x+lx( 2k () ]k )

[vl(k)]v2(k)

(112)

where Xi E Pt, ni are state vectors, wi C lZrl and vi E lZII are zero-mean stationary white Gaussian noise stochastic processes with intensities Wi > 0, V~ > 0, respectively, and yi E R II, i = 1, 2, are the system measurements. In the following Ai, Gi, (7/, i - 1, 2, 3, 4, are constant matrices. The optimal Kalman filter, driven by the innovation process, is given by (Kwakernaak and Sivan, 1972) :~(k + 1) = A~(k) + K[v(k) - C:~(k)]

(113)

where A

[A1 A2] eAa

A4 ' C -

[cl

eC3

C4 ' K -

[K1 K2] eK3

K4

(114)

0]

(115)

The filter gain K is obtained from

K - AP/CT ( v + cP/CT) -1 '

v

0

V2

where P! is the positive semidefinite stabilizing solution of the discrete-time filter algebraic Riccati equation given by

t'I - APIA T

APIC T(V + C P / C T ) - I c P I A T + GWG T

(116)

28

XUEMIN SHEN ET AL.

where G -

6G3

G4

W -

0

W2

(117)

Due to the weakly coupled structure of the problem matrices the required solution P! has the form

el--

e]l

6Pf2 ] PIa

(118)

Partitioning the discrete-time filter Riccati equation given by (116), in the sense of weak coupling methodology, will produce a lot of terms and make the corresponding problem numerically inefficient, even though the problem orderreduction is achieved. Using the decomposition procedure proposed from the previous section and the duality property between the optimal filter and regulator, we propose a new decomposition scheme such that the subsystem filters of the weakly coupled discrete systems are completely decoupled and both of them are driven by the system measurements. The new method is based on the exact decomposition technique, which is proposed in the previous section, for solving the regulator algebraic Riccati equation of weakly coupled discrete systems. The results of interest which can be deduced from Section IV.A are summarized in the form of the following lemma. Lemma 1 Consider the optimal closed-loop linear weakly coupled discrete system z l ( k + 1) -

(A1 - B1F1 - e 2 B 2 F 3 ) z , l ( k ) + ~.(A2 - B1F2 - B 2 F 4 ) z 2 ( k )

x2(k + 1) - e(A3 - B3F1 - B4F3)xl(k) + (A4 - B2F2 - e 2 B 3 F 2 ) x 2 ( k ) (119) then there exists a nonsingular transformation Tr

(~2o) such that

q" 1) -- (al,. + a2rPra)~l(k) (2(k + 1) : (bl,. + b2,.P,.b)(2(k)

~'l(k

(121)

where Pra and Prb are the unique solutions of the exact reduced-order completely decoupled continuous-time algebraic Riccati equations P~al~ - a 4 ~ P ~ -a3~ + P~a2~P~ = 0 Prbblr - b4r Prb -- b3r q- Prbb2r Prb -- 0

(122)


29

Matrices air, bit, i = 1, 2, 3, 4, can be found from (102)-(103). The nonsingular transformation Tr is given by

T,= (II1r+ II2rPr)

(123)

Even more, the global solution Pr can be obtained from the reduced-order algebraic Riccati equations, that is

Pr -- (~3r + ~4r[ PraO

0

Known matrices f~i,., i = 1, 2, 3, 4 and II1,., II2r are given in terms of the solutions of the decoupling equations defined in Section IV.A.

The desired decomposition of the Kalman filter (113) will be obtained by using duality between the optimal filter and regulator, and the decomposition method developed in Section IV.A. Consider the optimal closed-loop Kalman filter (113) driven by the system measurements, that is

:~l(k + 1) -- (A1 - KIC1 - e2K2Ca)~l(k) + e(A2 - KIC2 - K2C4)~,z(k) + g l y l ( k ) + eg2y2(k)

+ ,g3vl(k) + g,v,~(k)

(125)

By using (111)-(112) and duality between the optimal filter and regulator, that is A ---. A T, Q ~ G W G T, B ---, C T B R _ I B T ---, c T v _ I c

(126)

the filter "state-costate equation" can be defined as

[

x(k + 1)

z(k)

where Hf --

AT + CTv-1CA-1GWGT _A_IGWGT

-cTv-tCA-I

A_ 1

]

(128)

Partitioning A(k)as A(k) - [AT(k) AT(k)]w with Al(k) E R n' and A2(k) E R n2, (127) can be rewritten as following I xxl2((kk++ l1)1 ) _

Al(k + 1)

)12(~ -~- 1)

Sl,

A~! eA2T! ~Sas- &s Qlf ~Q2! All! eA12! eQaI Q4I eA21y A22!

:~2(k)

1

Al(k) A2(k)

(129)

30

XUEMIN SHENET AL.

Interchanging the second and third rows yields

ixlkl,1 xz(k + 1)

QI! eATI

~2(]g -+- 1)

"Q3f

)~i(k+ 1)

_

sl,

All! eS3!

eQ2! eA-12! ATI $4! 6A21f Q4f A22f

__ [ TI]

6T2.f]

)~i(k)

z2(k) )i2(k)

(130)

Al(k)

ET3s T4S

z2(k)

where

T3I-

Q~s Aiif ' At21

Q2I

Q,3I A21] '

Q,4I A22]

A12]

(131)

These matrices comprise the system matrix of a standard weakly coupled discrete system, so that the reduced-order decomposition can be achieved by applying the decoupling transformation from Section IV.A to (130), which yields two completely decoupled subsystems r/l(k+ 1) r/2(k + 1)

al! a2! a3! a4!

_

[

~l(k + 1) ,~2(k + 1)

_

[

r/l(k) r/2(k)

_

[TI! -e2T2IHI] r/2(k)

bi! b2] b4l ] [ ~l(k)~2(k) -- [T4/ + E2I'IfT2~] [ ~(k)

l b31

(132)

(133)

Note that the decoupling transformation has the form of (90) with H I and L.t matrices obtained from (93)-(94) with T/I's taken from (131). By duality and Lemma 1 the following reduced-order nonsymmetric algebraic Riccati equations exist (134) Pl.al! - a4] Pla - a3I + Plaa2! PI~ = 0

Pybbl] - b4I Pfb -- b3y + Pybb2] Pyb = 0

(135)

By using the permutation matrix

ixl,k,1 9~(k) ~(k)

-E

~(k) ~(k)

1 i,.1 o o -

0 0

Z,,~ 0

0 0

o o

~(k) ~(k)

(136)


31

we can define HI_[II~/ IIaf

H2f]_ET[I-e2L!H ! ell!

II4/

-eLf] I E

(137)

Then, the desired transformation is given by (138)

T f = (IIl! -J- II2fP.f)

The transformation Tr applied to the filter variables (125) as 1)1

produces

T

~2]

(139)

'/)l(k 1) ] _ [~2(k + 1) +

[A1-K1CI-e2K2C3 e(A2-K1C2-K2C4)] [~)1(k) ] TfT [e(A3- gac~ - K4C3) A 4 - K4C4- e2K3C2 TT 7)2(k) +TfW[ K1

eK3

eK2 ]

K4 y(k) (140)

such that the complete closed-loop decomposition is achieved, that is ?)l(]g "]- 1) -- ( a l / -~- a2fPfa)T~l(k) 'Jl-K l y ( k ) ?02(k q- 1) - (bl! + b2yPfb)T~]2(k)-Jr- K 2 y ( k )

[K,]

where

Kz

- TfTK

(141)

(142)

It is important to point out that the matrix P/ in (138) can be obtained in terms of Pla and Plb by using (124), with f~lf, f~2f, f~af, ~41 obtained from

~3f

~4f

-eHf

[ -- e2Hf Lf

A lemma dual to Lemma 1 can be now formulated as follows. Lemma 2 Given the closed-loop optimal Kalman filter (125) of a linear

discrete weakly coupled system. Then there exists a nonsingular transformation matrix (138), which completely decouples (125) into reduced-order localfilters

32

XUEMINSHENETAL.

(141) both driven by the system measurements. Even more, the decoupling transformation (138) and the filter coefficients given in (132)-(133) can be obtained in terms of the exact reduced-order completely decoupled continuous-time Riccati equations (134) and (135). It should be noted that the new filtering method allows complete decomposition and parallelism between local filters. The complete solution to our problem can be summarized in the form of the following algorithm. Algorithm 4: 1) 2) 3) 4) 5) 6) 7)

Find Ttl, T21, 7"3I, and 7"4I from (131). Calculate I,! and H I from (93)-(94) with T/l's obtained from (131). Find ail, bil, for i = 1, 2,3,4 from (132)-(133). Solve for PI,, and Plb from (134) and (135). Find Tr from (138) with P! obtained from (124). Calculate KI and K2 from (142). Find the local filter system matrices by using (141).

C. LINEAR-QUADRATIC GAUSSIAN OPTIMAL CONTROL PROBLEM This section presents a new approach in the study of the LQG control problem of weakly coupled discrete systems when the performance index is defined on an infinite-time period. The discrete-time LQG problem of weakly coupled systems has been studied in (Shen and Gajic, 1990b). We will solve the LQG problem by using the results obtained in previous sections. That is, the discrete algebraic Riccati equation is completely and exactly decomposed into two reduced-order continuous-time algebraic Riccati equations. In addition, the local filters will be driven by the system measurements, on the contrary to the work of (Shen and Gajic, 1990b) where the local filters are driven by the innovation process. Consider the weakly coupled discrete-time linear stochastic control system represented by (Shen and Gajic, 1990b)

zl(k + 1) = Alzi(k) + EA2z2(k) + Btut(k) + EB2u2(k) z2(k + 1) = eA3:vt(k) + A4:v2(k) + eB3ux(k) + B4u2(k)

(144)


33

with the performance criterion

1

~ [~(k)~(k) + ~(k)R~(k)]

J--~E

,

k=o

R > 0

(145)

where zi E R nl, i = 1,2, comprise state vectors, u E R 'hi, i = 1, 2, are the control inputs, y E R z~, i -- 1, 2, are the observed outputs, wi E R "i, i = 1, 2, and vi E R t~, i = 1, 2 are independent zero-mean stationary Gaussian mutually uncorrelated white noise processes with intensities Wi > 0 and V / > 0, i = 1, 2, respectively, and z E R '~, i = 1, 2 are the controlled outputs given by (146)

z(k) - Dlxx(k) + D2:e2(k)

All matrices are of appropriate dimensions and assumed to be constant. The optimal control law of the system (144) with performance criterion (145) is given by (Kwakernaak and Sivan, 1972)

u(k) = -F~,(k)

(147)

with the time-invariant filter ~(k + 1) = As

+ Bu(k) + K[y(k) - C&(k)]

(148)

where

A-

cA3

A4'

B-

C-

eC3

C4

, K-

eB3

B4

eK3

(149)

K4

The regulator gain F and filter gain K are obtained from

F - (R + B T P r B ) - I B TPrA

(150)

K - A P I C T (V + CPyCT) -1

(151)

where Pr and P! are positive semidefinite stabilizing solutions of the discretetime algebraic regulator and filter Riccati equations, respectively given by

Pr -- D T D + A T p r A -- A T p r B ( R + B T p r B ) - I B T p r A

(152)

Pj - A P j A ~ - A P j C ~ (V + C P j C ~) -~ C P j A ~ + V W V ~

(153)

34

XUEMIN SHEN ET AL.

where eD3

D4'

eG3

G4

The required solutions P,. and 1:'I have the forms

[~rl P"-

eP T

esx ,PI ]

~Pr2] Pr3

' P!-

eP 2 Pya

(155)

In obtaining the required solutions of (152) and (153) in terms of the reduced-order problems, (Shen and Gajic, 1990b) have used a bilinear transformation technique introduced in (Kondo and Furuta, 1986) to transform the discrete-time algebraic Riccati equation into the continuous-time algebraic Riccati equation. In our case, the exact decomposition method of the discrete algebraic regulator and filter Riccati equations produces two sets of two reducedorder nonsymmetric algebraic Riccati equations, that is for the regulator Praalr - a 4 r P r a --aar -+" Praa2rPra -- 0

Prbblr -b4rPrb-

bz~ +

Prbb2rPrb = 0

(156)

(157)

and for the filter P/aal! - a4! P1a - a3! + Plaaz! P/a = 0

P]bbl] -- b4y P/b

- bay +

Pybb2]P]b = 0

(158)

(159)

where the unknown coefficients are obtained from previous sections. The Newton algorithm can be used efficiently in solving the reduced-order nonsymmetric Riccati equations (156)-(159). It has shown in the previous section that the optimal global Kalman filter, based on the exact decomposition technique, is decomposed into reduced-order local optimal filters both driven by th.e system measurements. These local filters can be implemented independently and they are given by 7)1(k + 1) - ( a l /

+ az1P1a)rCll(k)+ Kly(k) + Bxu(k)

~)2(k + 1) - ( b l / + b2/Pfb)T~2(k) -t- K , y ( k ) + B , u ( k ) where

3

BI| _ T_TB_ B2 J

(n 1 + II2/ ns )-TB

(160)

(161)


35

The optimal control in the new coordinates can be obtained as

u(k)- -F~,(k)-

-FTrT [7)1(k)] -

[7)l(k) ]

(162)

where F1 and F2 are obtained from [Vx

F2 ] - F T ~ - (R + B w P~B)-IB r P~A(III! + II2!PI) T

(163)

The optimal value of J is given by the very well-known form (Kwakernaak and Sivan, 1972)

1

Jopt - ~tr[D TDP! + P~K(CPIC T + V ) K T]

(164)

where F, K, Pr, and Pr are obtained from (150)-(153). 1. CASE STUDY: DISTILLATION COLUMN In order to demonstrate the efficiency of the proposed method, we consider a real world control system - - a fifth-order discrete model of a distillation column (Kautsky et al., 1985). This model is discretized with the sampling rate of AT -- 0.1 in (Shen and Gajic, 1990b; see also, Gajic and Shen, 1993, page 153). The system matrices are given by

A = 10 -3

989.50 117.25 8.7680 0.9108 0.0179

5.6382 814,50 123.87 17.991 0.3172

0.2589 76.038 750.20 183.81 1.6974

6.0733 -0.6192

B T _ 10_3 [ 0.0192 [ -0.0013

0.0125 5.5526 107.96 668.34 13.298

8.2911 -13.339

0.0006 0.3700 11.245 150.78 985.19 9.1965 -18.442

0.7025 ] -1.4252

]

and the other matrices are chosen as C-

1 0

1 0

0 1

0 1

0] D T D - I s , 1 '

R-I2

It is assumed that G - I~ and that the white noise processes are independent and have intensities

W--Is,

V - - 0.1 • I2

36

XUEMINSHENETAL.

It is easy to see that this model possesses the weakly coupled structure with ni = 2, 2 = 3, ande = 8.2911/13.3~9= 0.62. The obtained solutions for the LQG control problem are summarized as following. The completely decoupled filters driven by measurements y are given by ~l(k+l)-

[0.3495 -0.2126

-0.5914] [0.6426 0.4551 ~l(k)+ 0.3277

+

7)2(k + 1) -

0.6607 -0.0833 -0.6043

[0.0163 -0.0478 -0.1302

0.0065 0.0027

0.0298 0.4020 -0.6039

0.0670] 0.0403] y(k)

-0.012] u(k ) 0.0054

-0.1464 ] -0.1255 ,~2(k) 0.3915

0.1037] [0.064-0.0080] 0.2649 / y(k)+ 0.0085 -0.0165 0.57991 0.0013-0.0103

u(k)

The feedback control in the new coordinates is obtained as u(k) -

[0.37320.4318] [_0.0739_0.17550.4207] -0.4752

-0.5981 ~x(k)+

0.0961

0.2230

-0.6128 ~2(k)

The difference of the performance criterion between the optimal value, Jopt, and the one of the proposed method, J, is given by Jopt = 216.572 J - Jopt - 2.6489 x 10 -11

Simulation results are obtained by using the software package MATLAB (Hill, 1988).

V. REFERENCES 1.

Aganovic, Z. Optimal Control of Singularly Perturbed and Weakly Coupled Bilinear Systems, Ph. D. Dissertation, Rutgers University, 1993.

2.

Aganovic, Z. and Z. Gajic. Optimal control of weakly coupled bilinear systems. Automatica, 29(1993): 1591-1593. Aldhaheri, R. and H. Khalil. Aggregation of the policy iteration method for nearly completely decomposable Markov chains. IEEE Trans. Automatic Control, AC-36(1991): 178-187.

3.


4.

5.

6. 7. 8. 9.

10. 11. 12.

13. 14.

15.

16. 17. 18. 19.

20.

37

Arabacioglu, M., M. Sezer, and O. Oral. Overlapping decomposition of large scale systems into weakly coupled subsystems, 135-147, in Computational and Combinatorial Methods in System Theory, C. Byrnes and A. Lindquist, Eds., North Holland, Amsterdam, 1986. Avramovic, B., P. Kokotovic, J. Winkelman, and J. Chow. Area decomposition for electromechanical models of power systems. Automatica, 16(1980): 637--648. Delebecque, F. and J. Quadrat. Optimal control of Markov chains admitting strong and weak interconnections. Automatica, 17(1981): 281-296. Delacour, J., M. Darwish, and J. Fantin. Control strategies of large-scale power systems. Int. J. Control, 27(1978): 753-767. Dorato, P. and A. Levis. Optimal linear regulators: the discrete time case. IEEE Trans. Automatic Control, AC-16(1970): 613-620. Gajic, Z. Existence of a unique and bounded solution of the algebraic Riccati equation of the multimodel estimation and control problems. Systems & Control Letters, 10(1988): 185-190. Gajic, Z. and H. Khalil. Multimodel strategies under random disturbances and imperfect partial observations. Automatica, 22(1986): 121-125. Gajic, Z. and X. Shen. Decoupling transformation for weakly coupled linear systems. Int. J. Control, 50(1989): 1517-1523. Gajic, Z., D. Petkovski, and X. Shen. Singularly Perturbed and Weakly Coupled Linear Control Systems ~ A Recursive Approach. Springer-Verlag, Lecture Notes in Control and Information Sciences, 140, New York, 1990. Gajic, Z. and X. Shen. Parallel Algorithms for Optimal Control of Large Scale Linear Systems, Springer-Verlag, London, 1993. Gomathi, K., S. Prabhu, and M. Pai. A suboptimal controller for minimum sensitivity of closed-loop eigenvalues to parameter variations. IEEE Trans. Automatic Control, AC-25(1980): 587-588. Harkara, N., D. Petkovski and Z. Gajic. The recursive algorithm for optimal output feedback control problem of linear weakly coupled systems. Int. J. Control, 5t)(1989): 1-11. Hill, D. Experiments in Computational Matrix Algebra, Random House, New York, 1988. Ikeda, M. and D. Siljak. Overlapping decompositions expansions and contractions of dynamic systems. Large Scale Systems, 1(1980): 29-38. Ishimatsu, T., A. Mohri, and M. Takata. Optimization of weakly coupled systems by a two-level method. Int. J. Control, 22(1975): 877-882. Kando, H., T. Iwazumi, and H. Ukai. Singular perturbation modeling of large-scale systems with multi-time scale property. Int. J. Control, 48( 1988): 2361-2387. Kaszkurewicz, E., A. Bhaya, and D. Siljak. On the convergence of parallel asynchronous block-iterative computations. Linear Algebra and Its Applications, 131(1990): 139-160.

38

XUEMIN SHEN ET AL.

21. Katzberg, J. Structural feedback control of discrete linear stochastic systems with quadratic cost. IEEE Trans. Automatic Control, AC-22(1977): 232-236. 22. Kautsky, J., N. Nichols, and P. Van Douren. Robust pole assignment in linear state feedback. Int. J. Control, 41 (1985): 1129-1155. 23. Khalil, H. Multi-model design of a Nash strategy. J. Optimization Theory and Applications, 31( 1980): 553-564. 24. Khalil, H. and P. Kokotovic. Control strategies for decision makers using different models of the same system. IEEE Trans. Automatic Control, AC-23(1978): 289-298. 25. Khalil, H. and Z. Gajic. Near-optimum regulators for stochastic linear singularly perturbed systems. IEEE Trans. Automatic Control, AC-29(1984): 531-541. 26. Kokotovic, P., W. Perkins, J. Cruz, and G. D'Ans. e---coupling approach for near-optimum design of large scale linear systems. Proc. IEE, Part D, 116(1969): 889-892. 27. Kokotovic, P. and G. Singh. Optimization of coupled nonlinear systems. Int. J. Control, 14(1971): 51-64. 28. Kondo, R. and K. Furuta. On the bilinear transformation of Riccati equations. IEEE Trans. Automatic Control, AC-31(1986): 50-54. 29. Kwakernaak, H. and R. Sivan, Linear Optimal Control Systems, Wiley Interscience, New York, 1972. 30. Lewis, F. Optimal Control, Wiley, New York, 1986. 31. Mahmoud, M. A quantitative comparison between two decentralized control approaches. Int. J. Control, 28(1978): 261-275. 32. Medanic, J. Geometric properties and invariant manifolds of the Riccati equation. IEEE Trans. Automatic Control, AC-27(1982): 670--677. 33. Medanic, J. and B. Avramovic. Solution of load-flow problems in power stability by E---coupling method. Proc. lEE, Part D, 122(1975): 801-805. 34. Ohta, Y. and D. Siljak. Overlapping block diagonal dominance and existence of Lyapunov functions. J. Math. Anal. Appl., 112(1985): 396-410. 35. Ozguner, U. and W. Perkins. A series solution to the Nash strategies for large scale interconnected systems. Automatica 13(1977): 313-315. 36. Ozguner, U. Near-optimal control of composite systems: the multi timescale approach. IEEE Trans. Automatic Control, AC-24(1979): 652-655. 37. Petkovski, D. and M. Rakic. A series solution of feedback gains for output constrained regulators. Int. J. Control, 29(1979): 661-669. 38. Petrovic, B. and Z. Gajic. Recursive solution of linear-quadratic Nash games for weakly interconnected systems. J. Optimization Theory and Applications, 56(1988): 463--477. 39. Power, H. Equivalence of Lyapunov matrix equations for continuous and discrete systems. Electronic Letters, 3(1967): 83.


39

40. Qureshi, M. Parallel Algorithms for Discrete Singularly Perturbed and Weakly Coupled Filtering and Control Problems, Ph. D. Dissertation, Rutgers University, 1992. 41. Riedel, K. Block diagonally dominant positive definite approximate filters and smoothers. Automatica, 29(1993): 779-783. 42. Saksena, V. and J. Cruz. A multimodel approach to stochastic Nash games. Automatica, 17(1981a): 295-305. 43. Saksena, V. and J. Cruz. Nash strategies in decentralized control of multiparameter singularly perturbed large scale systems. Large Scale Systems, 2(1981b): 219-234. 44. Saksena, V. and T. Basar. A multimodel approach to stochastic team problems. Automatica, 18(1982): 713-720. 45. Salgado, M., R. Middleton, and G. Goodwin. Connection between continuous and discrete Riccati equation with applications to Kalman filtering. Proc. IEE, Part D, 135(1988): 28-34. 46. Sezer, M. and D. Siljak. Nested E---decomposition and clustering of complex systems. Automatica, 22(1986): 321-331. 47. Sezer, M. and D. Siljak. Nested epsilon decomposition of linear systems: Weakly coupled and overlapping blocks. SlAM J. Matrix Anal. Appl., 3(1991): 521-533. 48. Siljak, D. Decentralized Control of Complex Systems, Academic Press, Cambridge, MA, 1991. 49. Shen, X. Near-Optimum Reduced-Order Stochastic Control of Linear Discrete and Continuous Systems with Small Parameters, Ph. D. Dissertation, Rutgers University, 1990. 50. Shen, X. and Z. Gajic. Optimal reduced-order solution of the weakly coupleddiscrete Riccati equation. IEEE Trans. Automatic Control, AC35(1990a): 60(0602. 51. Shen, X. and Z. Gajic. Approximate parallel controllers for discrete weakly coupled linear stochastic systems. Optimal Control Appl. & Methods, 11(1990b): 345-354. 52. Shen, X. and Z. Gajic. Near-optimum steady state regulators for stochastic linear weakly coupled systems. Automatica, 26(1990c): 919-923. 53. Shen, X., Z. Gajic, and D. Petkovski. Parallel reduced-order algorithms for Lyapunov equations of large scale linear systems. Proc. IMACS Symp. MCTS, Lille, France, (1991): 697-702. 54. Skataric, D. Quasi Singularly Perturbed and Weakly Coupled Linear Control Systems, Ph.D. Dissertation, University of Novi Sad, 1993. 55. Skataric, D., Z. Gajic, and D. Petkovski. Reduced-order solution for a class of linear quadratic optimal control problems. Proc. Allerton Conf. on Communication, Control and Computing, Urbana, (1991): 440--447.

40

XUEMIN SHEN ET AL.

56. Skataric, D., Z. Gajic, and D. Arnautovic. Reduced-order design of optimal controller for quasi weakly coupled linear control systems. Control Theory and Advanced Technology, 9(1993): 481-490. 57. Srikant, R. and T. Basar. Optimal solutions of weakly coupled multiple decision maker Markov chains. Proc. Conf. on Decision and Control. Tampa, FL, (1989): 168-173. 58. Srikant, R. and T. Basar. Iterative computation of noncooperative equilibria in nonzero-sum differential games with weakly coupled players. J. Optimization Theory and Applications. 71(1991): 137-168. 59. Srikant, R. and T. Basar. Asymptotic solutions of weakly coupled stochastic teams with nonclassical information. IEEE Trans. Automatic Control. AC37(1992a): 163-173. 60. Srikant, R. and T. Basar. Sequential decomposition and policy iteration schemes for M-player games with partial weak coupling. Automatica. 28(1992b): 95-105. 61. Stewart, G. Introduction to Matrix Computations. Academic Press, New York, 1973. 62. Su, W. Contributions to the Open and Closed Loop Control Problems of Linear Weakly Coupled and Singularly Perturbed Systems. M.S. Thesis, Rutgers University, 1990. 63. Su, W. and Z. Gajic. Reduced-order solution to the finite time optimal control problems of linear weakly coupled systems. IEEE Trans. Automatic Control, AC-36(1991): 498-501. 64. Su, W. and Z. Gajic. Parallel algorithm for solving weakly coupled algebraic Riccati equation. AIAA J. Gudance, Dynamics and Control, 15( 1992): 536--538. 65. Washburn, H. and J. Mendel. Multistage estimation of dynamical weakly coupled in contiuuous-time linear systems. IEEE Trans. Automatic Control, AC-25(1980): 71-76. 66. Zhuang, J. and Z. Gajic. Stochastic multimodel strategy with perfect measurements. Control ~ Theory and Advanced Technology, 7(1991): 173-182.

Techniques in Stochastic System Identification with Noisy Input 8z Output System Measurements Jitendra

K. Tugnait

D e p a r t m e n t of Electrical Engineering A u b u r n University Auburn, A l a b a m a 36849

I. I N T R O D U C T I O N Parameter estimation and system identification for stochastic linear systems have been a topic of active research for over three decades now [7],[19],[20],[37],[45]. It is often assumed that the measurements of the system output are noisy but the measurements of the input to the system are perfect. The problem considered in this chapter is that of identification of stochastic linear systems when the input as well as the output measurements are noisy. An interesting example of system identification with noisy input may be found in [38] where the problem of (off-line) estimation of certain parameters associated with the dynamics of a submerged undersea vehicle is studied. The various control inputs (the rudder angle, the stern-plane angle, etc.) and the corresponding motion variables (pitch angle, yaw rate, etc.) can only be remotely sensed, hence, are contaminated with sensor noises. This is a typical example of situations where the model considered in this chapter should prove to be useful. In multivariate time series problems where one is interested in exploring the relationship (transfer function) between two groups of variables, it is CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.

42

JITENDRAK. TUGNAIT

more logical to "symmetrically" model the system by allowing all measured variables to be noisy [29]. Such models are called errors-in-variables models in the econometrics literature [6]. In this chapter we consider a specific class of systems, namely, those where the input process is non-Gaussian and the measurement noise at the input as well as the output is Gaussian if the input process has symmetric probability density function (PDF). The noise processes are allowed to be non-Gaussian with symmetric PDF if the input process has asymmetric P DF. Clearly, this model may not be always appropriate but there are several situations of practical interest where such assumptions are valid. For instance, a pseudo-random binary sequence is often used to probe a control system for identification purposes [7], [19], [20]; such sequences are clearly non-Gaussian with nonvanishing trispectrum. We will now consider another such case in some detail. A signal processing application of considerable interest is that of differential time delay estimation in oceanic environments for passive sources. In this problem it is desired to estimate the time-difference of arrivals of a passive acoustic source signal at two sensors. There exist several approaches to accomplish this by exploitation of the second-order statistics (correlation and cross-correlation functions) of the data [10],[12]. The mathematical model underlying this approach is that of linear system identification with measurements at one of the sensors being modeled as the input to the system and the measurements at the other sensor being modeled as the system output. In a non-dispersive medium the linear system is just a delay (time delay to be estimated), whereas in a dispersive medium it is a general linear system [26]. A frequently made assumption is that the noises at the two sensors are uncorrelated Gaussian processes. This is clearly true for receiver induced noises. But it is not true for ambient noise. Under certain conditions, the ambient noise has been found to be a colored Gaussian process that is correlated both temporally as well as spatially [10],[11],[28]. The model considered in this chapter is quite appropriate for this problem. Past approaches to the problem of stochastic linear system identification may be divided into two classes: those that exploit only the second order statistics, and those that use higher (higher than second) order cumulant statistics. A good survey of the work done prior to about 1980 is given in [2]. For later work, see [3]-[9], [16]-[18], [29] and [32]. Higher order statistics have been exploited in [5], [6], [12]-[14], [31], [33]-[35], [39], [40] and [44]. Shderstrhm [1],[2] allows only white additive noise at the input, and furthermore, the input and the output noises are assumed to be mutually uncorrelated. Most of the early work in this area has been done in econometrics where such models have been called errors-in-variables models. When only second order statistics are exploited, it is known that, in general, there does not exist a unique solution [3]-[6],[9]. Therefore, atten-

STOCHASTIC SYSTEMIDENTIFICATION

43

tion has been focused on characterization of the class of transfer functions which fit the data. Fairly complete results are available for scalar dynamic case [3]-[6] but the multivariable case remains incomplete[9]. The use of higher order cumulant statistics ([23],[24])can, in principle, yield consistent parameter estimates. Deistler [5] (see also [14]) has shown how to estimate the transfer function of an SISO system in the frequency domain by use of the higher order cumulant spectrum of the output and the higher order cumulant cross-spectrum of the input-output record. That is, [5] presents a non-parametric approach. In [12] a special case (FIR filters) has been considered where output third-order cumulants and input-output third-order cross-cumulants have been used to derive linear parameter estimators. Implicit in [12] is the use of a (higher order) persistency of excitation condition. No general results are currently available to indicate when such a condition would hold. Consistency can be shown for the case of fourth-order cumulants [44]. Instrumental variable type approaches have been presented in [31] using third-order cumulants and in [35] using fourth-order cumulants of the inputoutput record. The consistency results of [31] and [35] have been proven only for i.i.d. (independent and identically distributed) inputs. Otherwise one needs to check the invertibility of a m a t r i x which depends upon the unknown parameters to be estimated; therefore, the invertibility condition check is not practical. The consistency results of the approach proposed in this chapter hold under far more general conditions. On the other hand we confine our attention to scalar models whereas [31] and [35] deal with multivariable models. In [34] a novel cost function involving the third-order cumulants of the input-output data has been proposed and it has been shown to be proportional to a conventional mean-square error criterion based upon noiseless data. Its extension to the fourth-order cumulant case has not been provided in [34]. Note that when the system input has a symmetric PDF, its thirdorder statistics will vanish rendering the approaches based upon the thirdorder statistics useless. One example is that of a pseudo-random binary sequence as the system input. Consistency of the approach of [34] has been established under several restrictive conditions such as the system input u(t) is a linear process given by u(~) - G(q-1)e(~) (see Sec. II for notation) where {e(t)} is a zero-mean, i.i.d, process with nonzero third cumulant at zero lag and G(q) represents a stable, linear model such that G(1) ~= 0. It should be noted that unlike the second-order statistics case, one can not, in general, model a stationary random process with a given higher-order cumulant spectrum as having been generated by driving a linear system with an i.i.d, sequence [27]. In this chapter we do not require any such constraint. Moreover, our approaches also apply to fourth-order statistics case unlike [34]. Also [34] requires that the noise processes, if non-Gaussian

44

JITENDRA K. TUGNAIT

with symmetric PDF, should be linear processes. We do not need such an assumption in this chapter. In [40] several linear/iterative approaches using the auto-and/or cross- third-order cumulants of the input-output processes have been presented. Conditions under which the proposed approaches will yield consistent parameter estimators have not been provided in [40]. Indeed we have given examples in [44] where the cross-cumulant method of [40],[12] may yield biased results. The simulation results presented in [40] show that the cross-cumulants-based approach of [40] outperforms the approach of [34] by a wide margin for the presented example. In [28] and [39] the square root of the magnitude of the fourth cumulant of a generalized error signal is proposed as a performance criterion for parameter estimation. Both single-input single-output and multiple-input multiple-output models have been considered in [39]. Strong consistency of the proposed parameter estimator has been established for linear inputs in [39] for Gaussian noise processes. The approach of [281,[39] results in a nonlinear estimator that requires a good initial guess for convergence; unfortunately, no method for reliable initialization was provided in [39]. The approach of [28],[39] is briefly discussed in Sec. II. In this chapter two new classes of parametric frequency domain approaches are proposed for estimation of the parameters of scalar, linear errors-in-variables models. One of the proposed approaches is a linear estimator where using the bispectrum of the input and the cross-bispectrum of the input-output, the system transfer function is first estimated at a number of frequencies exceeding one-half the number of unknown parameters. The estimated transfer function is used to estimate the unknown parameters using an overdetermined linear system of equations. In the second approach a quadratic transfer function matching criterion is optimized by using the linear estimator as an initial guess. Both the parameter estimators are shown to be consistent in any measurement noise that has symmetric PDF. The input to the system need not be a linear process but must have nonvanishing bispectrum. These two classes of approaches can be modified to exploit integrated polyspectrum, either bispectrum or trispectrum. The integrated polyspectrum is defined as a cross-spectrum between the process and a nonlinear function of the process; see Sec. III for further details. As discussed in Sec. III, integrated polyspectrum (bispectrum or trispectrum) is computed as a cross-spectrum; hence, it is computationally cheaper than the corresponding polyspectrum particularly in the case of the fourth-order cumulant spectrum. If the non-Gaussian input to the linear system has a symmetric P DF, its bispectrum and the integrated bispectrum will vanish whereas its trispectrum and the integrated trispectrum will not, provided that the fourth cumulant of the input 0'4= is nonzero. Extension of the bispectrum-based approaches to trispectrum-based approaches is computationally complex and the resulting estimators are likely to have poor statis-

STOCHASTIC S Y S T E M IDENTIFICATION

45

tical performance because of the high variance of the trispectrum estimators [24]. Herein lies the significance of the integrated polyspectrum-based approaches which apply with almost equal computational and programming ease to both cases, those involving integrated bispectrum as well as those concerned with integrated trispectrum. Compared to [39] we do not need the input to be a linear process for consistency to hold true. Compared to [5] we use parametric models. The chapter is organized as follows. The approach of [28],[39] is briefly discussed in Sec. II. In Sec. III a more precise statement of parameter estimation problem under consideration is provided along with a definition and some analysis of the integrated polyspectrum of interest in this chapter. The bispectrum-based approaches are described in Sec. IV. Sec. IV includes estimation of the auto- and cross-bispectrum, their large sample properties, a linear parameter estimator and a nonlinear parameter estimator which is asymptotically equivalent to a negative log-likelihood function. The integrated polyspectrum (bispectrum and trispectrum) based approaches are described in Sec. V which includes estimation of the integrated bispectrum and integrated trispectrum, their large sample properties, two linear parameter estimators and two nonlinear parameter estimators which are asymptotically equivalent to some negative log-likelihood functions. Consistency of the proposed parameter estimators is established is Sec. VI under some mild sufficient conditions. Finally, two simulation examples are presented in Sec. VI to illustrate the proposed approaches and to compare them with several existing approaches. Certain technical details may be found in the Appendix.

II. A T I M E - D O M A I N T E C H N I Q U E II.A. M O D E L A S S U M P T I O N S To keep the notation and the basic ideas simple, we initially consider only a causal SISO model; later, we generalize to MIMO models. Let u(t) and s(t) denote the "true" input and output, respectively, at (discrete) time t. The two are assumed to be related via a finite-dimensional ARMA(na, nb) model

A(q- 1)s(t) : B(q- 1)u(t)

(ILl)

A(q -1) "- 1 + ~ aiq -i

(II-2)

where

/-----1

46

JITENDRA K. TUGNAIT

B(q- 1) ._ ~

biq-i

(11-3)

i=1

and q-1 is the backward shift operator, i.e., q-is(t) - s ( t - 1), etc. As in [1] and [7], for example, we have assumed for convenience only that no term bou(t) is present, i.e., that there is at least one delay in the system. Explicitly, we may write (II-1) as 'r~ffi

nb

s(t) + E ais(t - i) - E biu(t - i). i=1

(11-4)

i=1

It is assumed that the input u(t) is generated by driving an asymptotically stable linear filter H(q -1) by another process e(t)

u ( t ) - H(q-1)e(t)

(II-5)

H(q -1) -- C(q-1)/D(q-1),

(11-6)

where

and C(q- 1) and D(q- ~) are finite-dimensional. The processes {u(t)} and {s(t)} are not available for measurement. But we can measure noise-contaminated input and output

9(t) - u ( t ) + vi(t)

(11-7)

y(t) - s(t) + vo(t)

(11-8)

The following conditions on the above model are assumed to hold.

( H I ) A(z) ~ 0 for [z[ _< 1 where z is a complex variable. Moreover, A(z) and B(z) are coprime (they have no common factor).

(H2)

The process {e(t)} is a zero-mean, i.i.d, non-Gaussi=r~ sequence with 74~ := E{e4(t)} - 3[E{e2(t)}] 2 76 0 and E{e8(t)} < oo.

(H3) The noise processes {vi(t)} and {v0(t)} are stationary, zero mean, jointly a=ussi=~ and independent of {e(t)}. Moreover, there exist 0<M 0 for every l 1 -- ~

(A-7)

86

J I T E N D R A K. T U G N A I T

and P{[ [B'(ej(''''+''')) - H(eJ("+"")leo)]&ma~

I< ' ) > 1 - 6,

for every N > N(e, 6). Hence it follows that given any el (= > 0 there exists an integer N ( e l , 6 ) such that P{[Dm~(O)l < el} > 1 - 6 V0 E |

VN > N(el,~).

e+

(A-8)

2Me) and (A-9)

That is, convergence of D,.,.,,.,(O) is uniform in /9. The desired result now follows by using the uniform convergence of D,~,~ (0), Lemma 2(i) and noting that

IJ(oo')(0)-

ID.~.(0)I o - &)(Oo)

where we have used Lemma 10 in deducing that #(p) > 0. By Lemma 9, given any e 0 there exists an integer gl(e, ,5) such that V0 6 @(p) and VN _> N1 (e, 6)

P{J(B)(O) > J(s)(o)

-

e >_

I~(P) e} > 1 - ,5.

(A-12)

The above equation may be rewritten as inf

P{ o ~ e(o)

j(s )

(o) > ~ ( p ) - ~} > 1 - ~

VN _> N~(~, 6).

(A-~3)

Similarly, by Lemma 9, given any e < #(p)/3 and 6 > 0 there exists an integer N2(e, ,5) such that V0 E Oc and VN > N2(e, 6) we have

P{J~B)(8)< ff(off)(O)+ e} >

1

-

6.

(A-14)

In particular, we have

P{J(s)(oo) < J(off)(Oo)+ e - e} >

1-6

VN > N~(E, 6).

(A-~S)

However, since

~f j~B~(0) _<s~(Oo), 06(90 it follows from (A-14) and (A-15) that

J(s)(o)

P{ O6|

VN _> N 2 ( e ,

< e _ 1 - - 6

6).

(A-16)

Consider 0N as defined in (V-13). Then we have inf 0COo

P{0N 6 S ( D o , p ) } -- P

>P

inf

OEec

J(B)(o) < e -
0. This stability margin will be used to find estimates of # which is the bound on the second moment of the perturbation vector in (3). We first present the robustness results given in [21]: Theorem 1. Let (18) hold. The perturbed system (1) is m.s.e.s, and a.s.e.s. if ~, < ~,~ = [ ~ . ( Q ) ( t

+ ~)-~A~-~(P)] ~/~

(19)

where P, Q are given by equation (2). Proof. The quadratic Lyapunov function candidate ~ -- xkTpxk, where P > 0 satisfies (2) due to (18) and the inequality

(a.i/2Axj: - oFV2f k) rP(at/2Axk - oFV2f ~:) >_0 used in the form

xkTATpfk lead to

+

fkTpAxk 0, is necessary and sufficient for the m.s.e.s, and sufficient for a.s.e.s, of system (63). So, we have

p(A | A + d2D | D) < 1

(66)

STABILITY OF RANDOMLY PERTURBED SYSTEMS

105

as our stability condition. However, when dk is given by (64), then the steady state second moment (if it exists) expression for (63) yields

Y - - A YA T + ~loo(A YD r + D YA T) 4- doo2D Y D T

(67)

where doo - limk-~ooE{dk} - 0 due to the stability condition I~l < 1. From (64), we also have k-1

dk2 -~- ~2k'd024. E

(~)k-i-l~rr

(68)

' 1 _.~k dk2 -- ~r'2kd024- 1 - ~2 crr 2

(69)

/=0

where ar 2 is the variance of Ck or

for k E A/'. If we consider the limiting case using the monotonicity of the right side in the above expression, we obtain

doo 2 = lira E { d k 2 } . - - (1 - r 1 6 2 k--*oo

(70)

in (69). Therefore, comparing with (66), even if the noise is not white but has the model (64), the m.s.e.s, and a.s.e.s, of the system (63) will be preserved provided that d2 assumed for the white noise is such that > (1 -

or

in which case, X > Y will be obtained. In other words, in the case of an over-estimation of the steady state second moment of the colored noise by the variance of the erroneously assumed white noise, the condition (66) will be sufficient to preserve stochastic stability in the senses defined above.

106

EDWIN ENGIN YAZ

V. J U M P

MARKOV

PERTURBATIONS

Let equation (18) hold for some c~ > 0. This stability margin for the nominal system will allow us to accomodate perturbations of the form (4). Consider the equations N

(1 + a ) A r ( E

(71)

PJTrij)A- p i = Qi

j=l

for some Qi > 0, i = 1, ...,N. We will show that the stability margin of A (18) is sufficient to guarantee the existence of pi, i = 1, ...,N that satisfy (71) for some given H matrix. Let us observe that (71) can be obtained if a recursion like (15) converges to steady state values. The necessary and sufficient condition given in (17) for the convergence yields

p(

/

7ql(l+a)A r|

T

9

...

7rlu(l+a)A v|

...

7rNl(1 -+- ol)AT| A T

"'"

9 7rNN(1

+ ol)A T | AT

/

)
0, pi > 0 is true for all

i = 1, ..., N. So, we have proved the following result. Lemma 1. Let inequality (18) hold for some a > 0. Then, for given Qi > 0 and H, there exist matrices p i > 0 i - - 1, ..., N that satisfy (71). Consider the system (1) with jump nonlinear perturbation satisfying (4). We can state the following results: Theorem 12. Let equation (18) hold, then system (1) perturbed in the way defined by (4) remains m.s.e.s, and a.s.e.s, if N It i ,( ltl i -- [o~,,~n(Qi)(1 .+. o / ) - 1 ~ 1 - 1 ( E PJ 7rij)] 1/2 j=l

(75)

for i = 1, ...,N and where Qi and p i are given by (71). The proof of this result as well as the following two use lemma 1 together with techniques similar to the ones employed in finding perturbation bounds for white noise perturbations, so will not be repeated. Theorem 13. Let equation (18) hold. Then system ( 1 ) p e r t u r b e d in the sense of (4) remains m.s.e.s, and a.s.e.s, if

Iti < ,a2 i = - a l ( A ) + {a~2(A)

+ [a,\n(~ P3~O) + A.(Q')](1 + c~)-t,~(~ PJ 7r,j)} ~/2 j=l j=l

(76)

for i - - 1,...,N. We have the following result for unstructured linear perturbations given by (5):

108

E D W I N ENGIN YAZ

Corollary 3. Let (18) hold.

Then system (5) where Dk(w) is a matrix

random sequence with elements depending on states of a finite Markov chain is m.s.e.s, and a.s.e.s, if

a,(D,) < max {ltli,lt2 i}

(77)

for i = 1, ...,N. The proof of this result directly follows from the proofs of Theorems 12 and 13 and the use of (5) in (4). Another result for this class of perturbations is as follows" Theorem 14. Let (18) hold. The results of corollary 3 follow if

p{II T | A | A + a-lIIa | (st(I)stW(I))} < (1 + a) -1 where 7rllal2(Sl) H~

:'-"

9

...

(78)

7rNlal2(DN)I

o~

~

...

Proof.

From (15) and (17), it follows that the necessary and sufficient

condition for the m.s.e.s, and a.s.e.s, is the existence of X i > 0, 1 _< i _< N such that N

XJ = E (A + D,)riiXi(A + Di)T + Vj

i--1

for some ~ > 0. To find a sufficient condition for these types of stability in terms of bounds on the perturbations, we consider N

XJ = E (A + Di)rijX'(A + Di)T + Vj

i=l

N

_< (1 + a)A E i=1 N

< (1 + c~)AE /-=1

N 7rijXiAT + (1 + a -1) E DirqXiDiT i=1 N

7rijXiA r + (1 + ~- 1) E a12(D')Trijtr[Zi] i=1

STABILITY OF RANDOMLY PERTURBED SYSTEMS

So, if there exist X j

109

> 0, 1 < j < N, such that N

X d = (1 + c~)A~

N

7rijXiAr + (1 +

a12(Di)riitr[Xi]I

c~-1) ~

i=1

/=1

then this would be sufficient to guarantee stochastic stability in the senses mentioned above. The rest of the proof uses Kronecker product and stack operator properties to describe the stability criterion similar to equations (17a) and (17b). Structured linear perturbations of the type (6) can be treated similarly as follows: Corollary 4. Let (18) hold. The perturbed system (6) remains m.s.e.s, and a.s.e.s, if

Idol < An(Q~)(1 §

N o~-I)-I)~I-I(DT E

7rijPj)

j=l

for all 1 _< i _< N when

Qi and pi

are given by (71).

Proof. The proof follows from that of Theorem 12 by using the perturbation structure in (6). Corollary 5. Let (18) hold. Then the results of Corollary 4 are true if N

N

N

[di[ < [2al(DTE 7rijPJD)]-I{-al(ATE ~rijPJD+ DT E 7rijPJA) j=l N

j =1 N

+ [a12(ATE 7rijPJO+ or E 7rijPJA)+ 4 a l ( D r j=l

j=l

j =1 N

E

7rijPJD)

j=l

N

x An(Qi +

AT E 7rijPJA)]t/2} j=l

for 1 < i _< N with

Qi and pi

in(71).

Proof. The proof follows from that of Theorem (13) by taking the structure of the perturbations into account.

110

EDWIN ENGIN YAZ

VI. WHITE

AND JUMP

MARKOV

PERTURBATIONS

TOGETHER

Let us now consider the case where both types of perturbations exist in the system. This can happen e.g. in the following form [20],[45]" m

Xk+l

: (A -1-E dkl(W)D'i)xk

(79)

l=1

where

dkZ(w)

is a standard (zero mean and unit variance) scalar white

noise sequence independent of the elements of

Dl i,

i = 1, ..., N which are

dependent on a Markov chain with N states. Both random sequences are assumed to be independent of x0. For systems like (79), the second moment evolution can be written as N

m

N

(80)

SJk+l : A E pikTrijAT--[- Z E nlipkiTrijnliT i:1

l=l i:1

We have the following results: Theorem 15.

Let (18) hold.

Then the perturbed system (79) remains

m.s.e.s, and a.s.e.s, if m

N

D, irD, i)

AI(E

< (1 +

a)-lk.(Qi + aPi)kl-l(E 7rijPi)

l=l

(81)

j=l

for 1 _< i 0 that solve (71) for some

Qi > o, 1 0. In the case where X E Mn L+, the spectral radius p ( X ) =]] X II. Let s space of all linear transformations r [[ ~4 II. ~4 E s (~4~(.) = A(r

Mnrn L} denote the Banach

9Mnm L ---* Mnm L with supremum norm

Mn L} is called stable if for some k > 1, II r

]]< 1

or equivalently the spectral radius of .A, p(jt) < 1.

For such linear transformations,-we have p(A) = limk_~ [I Ak ]]l/k - 0, we have L

qt

L

Xl = E 7rmlAmXmAmT"3r-E Z 7rrnl[J~miXmJ~miTo'aint-7rmFm~vFm~ m=l

i=1 m = l

L

>-- E

7rml[AmXmAm T 4- 7rmFm~'vFmI]

m=l So, gg is a stable transformation with K - 0, meaning that (At, 6}) is m.s. detectable. T h e o r e m 5. Let p(~) < 1 and FtS,~Fz T> O, 1 __n x - n~ for the system model (1) based on noisy measurements (60), where C and H are full-row

rank, then the steady state solution of the conditional covariance pt in (56) is given by equation (42) with the matrices F in (54), Kt = KI 2 in (62) and NI, /14l, el in (73)-(75). A given conditional covariance set P C A([nL+ is assignable by a gain sequence K if and only if either equations (43) with E, C iL(rL,+n.)n~, and (44) (or (59)) or equation (45) holds. If a P C ll,'lnL§

is assignable, then it can be assigned by the nonunique gain sequences tft 2 given by (46)-(49) and Kt 1 and Kt 3 in (70)-(71). The P E Mn L§ with minimal trace satisfies (50) and if it is assignable, it can be assigned by the observer gain Kt 2 given by (51) and Kt 1 and K1:3 given by (70)-(71). The proof of the above result can be done by first imposing the unbiasedness condition (69) on the estimation error evolution, next finding the expression for the conditional covariance pt, and then applying Lemmas 3-5. As commented in the previous subsection, the need to process the current measurement instantaneously can be circumverted by introducing the auxiliary variable ~. given by (66). VIII. R O B U S T E S T I M A T I O N In this case, we assume that the unknown (co)variances belong to nonempty compact sets a,~ i E 0,~ i, az j E Ozj, L'~ E O., and r ~ E O~ and minimize the performance index j l _. E{li eN II2

Iro--l}

(76)

for some fixed positive integer N over the worst possible values of these (co)variances where ek is given by equation (6) for the linear unbiased full

146

EDWIN ENGIN YAZ

order estimator in section III. So, the mathematical problem we are tackling is to optimize JmM l = rain max K

jt, 1 rrd _~•

.ryu

0

-0.01

-0.02 0

200

400 d

Fig. 10 System $2 - crosscorrelation in experiment 3

600

RECURSIVE ESTIMATION OF TIME DELAY

193

NONLINEAR LEAST SQ UARES METHODS

A. NTDI: NONLINEAR 7IME DELAY/DENTIFICATION

S1

Experiment 1 The free design parameters were chosen as follows: hl=8S, L=0.9, drain=20, f--O.O 1Hz. The delay estimate, reported in Fig. 11, shows that the true delay value d=-40 is achieved in about 5800s. The evolution of the estimate is quite slow, but essentially monotone.

S1 - e s t i m a t e d d e l a v w i t h NTDI

40

~lid

9 ,,,

'

'1

9

-

9

...........

30

!" 0

-

0

, i

2000

i

4000

6000

8000

time (s) Fig. 11 System S1 - estimated delay in experiment 1

Experiment 2 With the same setting of the free design parameters as in experiment1, convergence towards the true delay value is still essentially achieved, see Fig. 12, although the presence of noise makes the estimated value fluctuate around the true one.

Experiment 3 Unsuccessful, we were unable to obtain convergence of the algorithm to any estimated delay value.

194

GIANNI FERRETTI ET AL.

S l - e s t i m a t e d d e l a y with.NTDI

50 40 30 20 0

0.5

I

t i m e (s)

1.5

2

xlO 4

Fig. 12 System S1 - estimated delay in experiment 2

$2 In this case, experiments 1, 2 and 3 were unsuccessful, even though many simulation runs were performed with different tunings of the free design parameters.

B. EA4I/7~: ESTIMATED MODEL VARIABLE REGRESSION ESTIMATOR

S1

Experiment 1 The free design parameters were chosen as follows: hl=lS, ~=0.94, drain=l, dmax=50, f--O.O1Hz. The delay estimate is reported in Fig. 13, which shows the extremely quick convergence to the true delay value d=-40.

Experiments 2 and 3 These experiments were unsuccessful. This may lead to conclude that the surprisingly good performance in the deterministic case must be paid by a very small robustness with respect to the presence of noise.

RECURSIVEESTIMATIONOF TIME DELAY

50

195

$1 - e s t i m a t e d d e l ~ w i t h EMVRE

o__J0

200

400

600

time

800

I000

(s)

Fig. 13 System $1 - estimated delay in experiment 1

$2

Experiment 1 The free design parameters were chosen as follows: h2=O.ls, ~=0.94, dmin=lO0, dmax=600, f=0.025Hz. As depicted in Fig. 14, the true delay value d=300 was estimated very quickly.

$2 - es)imated d d a y w!th I}MVI~.

400 300 200

100

0

50

I O0 time

150

(s)

Fig. 14 System $2 - estimated delay in experiment 1

200

196


Experiments 2 and 3 In this case too we were unable to obtain convergence of the delay estimate, thus confirming the results of Experiments 2 and 3 associated to S 1.

METHODS BASED ON RATIONAL APPROXIMA TIONS

A. DPDIA" DISCRETE-TIME PADI~-BASED DELAY /DENTIFICATION ALGORITHM

SI

Experiment 1 The free design parameters were chosen as follows: h l=lS, L=0.98, din=20, f~.00125Hz. The delay estimate is reported in Fig. 15; it is apparent that the true value d=40 is identified after about 10000s.

50

-

SI

-estimated

delay wi.thDpDIA r

it a

,

0

l

i

,

=

O.S

i

1.5

1 time

{s)

xlO 4

Fig. 15 System $1 - estimated delay inexperiment I

Experiments 2 and 3 These experiments were unsuccessful.


197

$2 The free design parameters were chosen as follows: h2=O.ls, L=0.96, din=150, f~.0016Hz. The delay estimate d=-329 was obtained after 4000s. Note that in this case the true delay value d=-300 tends to be overestimated, due to the nonminimumphase characteristics of the system $2, see Fig. 16.

$2- estimat~

400

delaywith DPDIA

200

0 0

I000

2000

3000

4000

time (s) Fig. 16 System $2 - estimated delay in experiment 1

Experiments 2 and 3 These experiments were unsuccessful.

VARIABLE REGRESSOR METHOD

TIDEA: TIME DELAY ESTIMATION ALGORITHM

S1 In this case, the initialization of the discrete time model was chosen as the discretization of the continuous time system described by

198


1 G(s) = l + 3 0 0 s Furthermore, it was set h~=2s, the RLS method was implemented by keeping constant to l O0 the trace of the covariance matrix and the filter L(z) appearing in TIDF~ was chosen as the discretized version of a continuous time third order Butterworth filter with cutoff frequency O.078rad/s.

Experiment 1 The results of experiment 1 are reported in Fig. 17, which shows that the continuous time delay estimate after 3000s is about 39.5s and that the correct estimate is practically achieved after about 800s.

S l - e s t i m a t e d d d . a y w i t h TIDEA

50

J

,

0

j

t

1000

2000

time

3000

(s)

Fig. 17 System S1 - estimated delay in experiment I

Experiments 2, 3 Both experiments were successful; so, only the most severe experiment 3 is reported in Fig. 18. The estimate of the continuous time delay at the end of the experiment is about 38.4s. This demonstrates that the robustness of the method with respect to additional noise is quite good.


$1 - e s t i m ~

50

,

0

5OO

199

d e l a y w i t h TIDEA

,

i

1000

,

i

1500

2000

t i m e (s) Fig. 18 System S1 - estimated delay in experiment 3

50

$ 2 - e s t i m a t e d del

w i t h TIDEA

0 0

2000

4000

6000

t i m e (s) Fig. 19 System $2 - estimated delay in experiment 1

$2 The initialization of the discrete time model was chosen as the discretization of the following discrete time system:

200


1 G(s) = (1 +s)(1 +3 00s) The estimation sampling time h2, the RLS algorithm and the filter L(z) were kept the same as for system S 1.

S ~ m a g r f l t u d e of zthe true and estimated model I

0 -50

10-4

l o l I I I I | I I I I I I I *

I

l l I l I I I , I I I I I I I J

I I I .... l l I I I I I

I I I I I I I *

I l I l I i I

l l I l I i I

e e I e i i I

l II l lO l ll i Ii i II i ii III

I I I I I I I I

I I I I I I I I

I I I I I I I II

I II I II III I II I II I II III jj

I

e l l l I I I

9 w i illw

w

I I I I I I I

I

10-3

I

I

I I I IIII I

I

I I I I I

I I I II I I I III I I I I II I I I III I I I III . - - . - - ~ I I I III I I I III I I I III I I I III I I I III I I I III I I I III i i i i ii

.....

I I I I I i I I I I I I I l

I I I I I I I i

I

I III

,,,,

I

I

,

,, ......

I

I I I I I

I

I I I I" I

I I I I I I I I I -- "II --i-_ I I I I I I I I I I I I I I I I I I .. I i

I

I

I II

I I I I I I" I

I I I I I I" I

I II I II I II I II I II l'eT I II

I I I I I i

l'Vlt'l,,4. I I II I I II I I II I I II i i i i

10-2

10 -1

|,|,l

frequ~

~2 -phase of the true and estimated model

-200

[

[~ : I ; ; ; ; Qi

I i i i i i i

I i i i i i i

I I i i i i i

I i i i i i i

I i i i i i i

I i i I i i i

III i ii i ii i ii iii i ii i ii

"

~

I i i i I i i

I i i I I i i

I i i i i i i

I I

i I

I I

I I

I I

i iii I I II

I I

I I

I I

|

|

"Irl-ll.~_lll i i i i i i Yl~Mlll,l._ i i i i i i iii - ' ~ i i i i i i iii 1 ~ i i i i i iii i-"%ii4~ i i i i i ii i ~ i i i i i ii i i -~i

[ [ : [[

J i i i I

i i i i i i i

I I

I I

-400 10-4

I I I

I I I

I I I

I I I

I III I I II I I II

10-3

I I I

I I I

I I I

I I I

I I I

|

I III I III

I I

I I

I I

I I

I

I III I I II I III

I I I I

I I I I

I I I I

I I I I

I I I I

. . I. .I .I .I I. I .II . . . I. . I. .I .I I. I. III. . . .

I I I I

[

10-2

ii i i ii ii ii i i

ii l llJ

I I I I

~,'I I~I~ II.11 II| I I

10-1

frequency Fig. 20 System $2 - Bode diagrams of the true (continuous line) and estimated (dashed line) model

Experiment l

The result of system 1 is reported in Fig. 19, where the continuous time delay estimate at the end of the experiment is about 43s. In order to evaluate the performance of TIDEA, it is worth recalling that this method is based on an allpoles approximation of the rational part of the true sampled transfer function. Since


201

the model used here is simply of second order, the presence of a large nonminimum phase zero in $2 is indirectly accounted for by overestimating the time delay. The comparison between the frequency responses of the true and of the estimated models (Fig. 20) clearly illustrates the global accuracy of the estimate.

Experiment 3

Even in this case, experiment 2 is omitted because experiment 3 is still successfull and more significant. The delay estimate at the end of the experiment, shown in Fig. 21, is about 42.5s. Thus, the noise has not any practical effect on the estimate accuracy.

50

$2 - e s t i m a t e d d e l a ,,-

O 0

with TIDF~

w

2000

4000

6000

time (s) Fig. 21 System $2 - estimated delay in experiment 3

IX. COMPARISONS AND CONCLUDING REMARKS In the introduction of this survey, many reasons have been recalled which motivate the development of efficient and reliable recursive methods for the estimation of time delay in discrete-time systems. Then, the main approaches proposed in the literature have been presented together with the analysis of some specific algorithms. Finally, some simulation results have been reported to illustrate the performance of these techniques. It now comes natural to wonder whether some of the algorithms here considered can be of help in the fulfillment of the initial goals.

202


To this regard, it is ditiicult to draw definite conclusions from simulation results. However, we believe that some guidelines can be identified for the proper use of the methods here examined. In particular, it seems apparent that correlation methods guarantee excellent performance even in the noisy case provided that the input signal is correctly chosen and the system operates in open loop. Conversely, in closed loop operations this approach can only be used provided that some very fight assumptions are made. Then, correlation techniques can be considered in a preliminary analysis or, e.g., for diagnosis purposes. As a further conclusion, we can say that the methods based on the overparametrization of the B-polynomial in general fail to be successfull for the proper delay detection, since they require to discriminate the zero/non zero pattern of the estimated coefficients, which is a hard task whenever some noise acts on the system. However, if the model has to be estimated for control synthesis purposes and the proper delay estimation is not strictly required, what really matters is that the frequency response of the estimated system matches the true one, at least in the frequency band of interest. In these cases, even the extended B-polynomial method, see [101-[131, as well as other techniques, can be of help. Nevertheless, if a fixed structure PI or PID regulator has to be synthesized from the estimated model, an overparametrized B-polynomial is not the best starting point, while a first or second order model plus a time delay is to be preferred. From this point of view, other algorithms, e.g. TIDEA [6], [421, should be considered. Most of the methods here analysed have shown to be extremely sensitive to the presence of noise; as such, some of them, e.g. EMVRE, although very fast and of high performance in the noise-free case, must be used with great care in practical situations. In general, robustness and performance are conflicting properties, as quite usual in identification (compare the results of NTDI and EMVRE). In order to achieve robusmess and reliability with respect to the presence of noise, we believe that a significant filtering action must be employed in all practical circumstances, as illustrated by the results reported for TIDEA. This has recently been demonstrated for any estimation technique based on Least Squares minimization in [431, where it is shown that only filtering can eliminate local minima in the performance index to be minimized. Finally, it has to be recalled that the aim of this survey was only to present and discuss recursive estimation techniques, while in the literature other off-line methods have been proposed for the delay estimation, see [44]-[50]. However, most of these approaches can still be plugged into one of the basic categories presented here.


203

X. R E F E R E N C E S

G. Ferretti and C. Maffezzoni, "Monitoring and Diagnosys of a Pulverization Process", 1FAC/1MACS Symposium SAFEPROCESS'91, Baden-Baden, Germany, pp. 199-204 (1991). O. J. M. Smith, "Closer Control of Loops with Dead Time", Chem. Eng. Progress, 53, pp. 217-219 (1957).

.

3.

K. Yamanaka and E. Shimemura, "Effects of Mismatched Smith Controller on Stability in Systems with Time-delay", Automatica, 23, pp.787-791 (1987).

4.

C. Santacesaria and R. Scattolini, "Easy Tuning of Smith Predictor in Presence of Delay Uncertainty", Automatica, 29, pp.1595-1597 (1993).

.

K. J. Astrom and B. Wittenmark, Adaptive Control, Addison Wesley, Reading, MA (1989).

6.

A. Leva, C. Maffezzoni and R. Scattolini, "Self-Tuning PI-PID Regulators for Stable Systems with Varying Delay, Automatica, 30, pp.1171-1183 (1994).

7.

H. Kurz and W. Goedecke, "Digital Parameter-Adaptive Control of Processes with Unknown Dead Time",Automatica, 17, pp. 245-252 (1981).

8.

R.M.C. De Keyser, "Adaptive Dead-Time Estimation", 2nd 1FAC Workshop on Adaptive Systems in Control and Signal Processing, pp. 209-213 (1986).

.

L. Keviczlqr and C. Bfiny~isz, "An Adaptive PID Regulator Based on Time Delay Estimation" Proc. 31st Control and Decision Conf., pp. 3243-3248 (1992).

10. E. F. Vogel and T. F. Edgar, "Application of an Adaptive Pole-Zero Placement Controller to Chemical Processes with Variable Dead Time", Proc. Amer. Control Conf., pp. 786-791 (1982). 11. C. Chandra Prasad, V. Hahn, H. Unbehauen and U. Keuchel, "Adaptive Control of a Variable Dead Time Process with an Integrator", 1FAC Workshop on Adaptive Control of Chemical Processes, pp. 67-71. (1985). 12. D. E. Clough and S. J. Park, "A Novel Dead-Time Adaptive Controller", IFAC Workshop on Adaptive Control of Chemical Processes, pp. 19-24 (1985).

204


13. I.L. Chien, D. A. Mellichamp and D. E. Seborg, "A Self-Tuning Controller for Systems with Unknown or Varying Time Delays", Proc. American Control Conf., pp. 905-912 (1984). 14. K. K. Biswas and G. Singe "Identification of Stochastic Time Delay Systems", IEEE Trans. Aut. Control, AC-23, pp. 504-505 (1978). 15. F. Abrishamkar and G. A. Bekey, "Estimation of Time Varying Delays in Linear Stochastic Systems", Proc. 7th 1FAC Symp. on Identification and System Parameter Estimation, pp. 1793-1798 (1985). 16. W.X. Zheng and C. B. Feng, "Identification of Stochastic Time Lag Systems in the Presence of Colored Noise",Automatica, 26, pp. 769-779 (1990). 17. G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco. 18. G. A. Dumont, A. Elnaggar and A. Elshafei, "Adaptive Predictive Control of Systems with Time-Varying Time Delay", 1st European Control Conference, pp. 1081-1086 (1991). 19. A. Elnaggar, G. A. Dumont and A. L. Elshafei, "Adaptive Control with Direct Delay Estimation", Proc. 12th IFAC World Congress, 9, pp. 19-23 (1993). 20. A. I. Elnaggar, Variable Regression Estimation of Unknown System Delay, Ph.D. Thesis, Dept. of Electrical Engineering, University of British Columbia, Vancouver, B.C. Canada (1990). 21. C. H. Knapp and G. C. Carter, "The Generalized Correlation Method for Estimation of Time Delay", 1EEE Transactions on Acoustic, Speech and Signal Processing, ASSP-24, pp. 320-327 (1976). 22. G. C. Carter, "Coherence and Time Delay Estimation", Proc. 1EEE, 75, pp. 236-255 (1987). 23. L. Ljung and T. S&terstrfm, Theory and Practice of Recursive Identification, The MIT Press, Cambridge, Mass. (1983). 24. C. B~iny~sz and L. Keviczl~, "A New Recursive Time Delay Estimation Method for ARMAX Models", 8th 1FAC/1FORS Syrup. on Identification and System Parameter Estimation, 3, pp. 1452-1457 (1988). 25. J. Bokor and L. Keviczlqr "Recursive Structure, Parameter and Delay Time Estimation using ESS Representations", Proc. 7th IFAC Syrup. on Identification and System Parameter Estimation, pp. 867-872 (1985).


205

V. Kaminskas, "Parameter Estimation in Systems with Time Delay and Closed Loop Systems", Proc. 5th IFAC Symp. on Identijication and System Parameter Estimation, pp. 669-677 (1979). R. Pupeikis, "Recursive Estimation of the Parameters of Linear Systems with Time Delay", Proc. 7th IFAC Symp. on Ident~jicationand System Parameter Estimation, pp. 787-792 (1985). A. Elnaggar, G. A. Dumont and A. L. Elshafei, "Recursive Estimation for Systems of Unknown Delay", Proc. 28th Control and Decision ConJ, pp. 1809-1810 (1989).

Z. Y. Zhao, S. Sagara and K. Kumarnaru, "On-Line Identification of time Delay and System Parameters of Continuous Systems Based on Discrete-Time Measurements", Proc. 9th IFAC/IFORS Symp. on Identijication and System Parameter Estimation, pp. 1145-1150 (199 1). S. Sagara and Z. Y. Zhao, "Recursive identification of transfer function matrix in continuous systems via linear integral filter", International Journal of Control, 50, pp. 457-477 (1989). Dj. Juricic, "Recursive Estimation of Systems with Time Varying Parameters and Delays", Proc. 10th IFAC World Congress, Vol. 10, pp. 265-270 (1987). M. E. Salgado, C. E. de Souza and G. C. Goodwin, "Issues in Time Delay Modelling", Proc. 8th IFAC/IFORS Symp. on Identification and System Parameter Estimation, pp. 786-791 (1988). S. Roy, 0.P. Malik and G. S. Hope, "Adaptive Control of Plants using AllZero Model for Dead Time Identification", IEE Proceedings-D, 138, pp. 445452 (1991). M. Bjarklund, M. Nihtilii and T. Siiderstrom, "An Algorithm for Identification of Linear Systems with Varying Time-Delay", 9th IFAC/IFORS Symp. on Identijication and System Parameter Estimation, pp.1254-1259 (199 1). M. Agarval and C. Canudas, "On-Line Estimation of time Deley and Continuous-Time Process Parameters", Int. J. Control, 46, pp. 295-31 1 (1987). C. E. de Souza, G. C. Goodwin, D. Q. Mayne and M. Palaniswami, "An Adaptive Control Algorithm for Linear Systems having Unknown Time Delay", Automatica, 24, pp. 327-341 (1988). W. R. Robinson and A. C. Soudack, "A Method for the Identification of Time Delays in Linear Systems", IEEE Transactions on Automatic Control, AC-24, 97-101 (1970).

206

GIANNI FERRETTI ET AL

G. C. Goodwin and K. S. Sin, Adaptive Filtering Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ (1984). E. Gabay and S. J. Merhav, "Identification of Linear Systems with TimeDelay Operating in a Closed Loop in the Presence of Noise", IEEE Transactions on Automatic Control, AC-21,711-716 (1976).

P. J. Gawthrop and M. T. Nihtilq "Identification of Time Delays Using a Polynomial Identification Method", System & Control Letters, 5, pp.267-271 (1985). E. Boje and E. Eitelberg, "On Identification of Time Delay via the Approximation e-Ts z (1-sT/2)/(l+sT/2)", 9th IFAC/IFORS Symp. on Identification and System Parameter Estimation, pp. 1501-1505 (199 1). G. Ferretti, C. Maffezzoni and R. Scattolini, "Recursive Estimation of Time Delay in Sampled Systems", Automatica, 27, pp. 653461 (1991). G. Ferretti, C. Maffezzoni and R. Scattolini, "On the Identifiability of Time Delay with Least Squares Methods", Internal Report Dipartimento di Elettronica e Infomazione, Politecnico di Milano (1994).

G. P. Rao and L. Sivakumar, "Identification of Time Lag Systems via Walsh Functions", IEEE Trans. Aut. Control, AC-24, pp. 806-808 (1979). E. V. Bohn, "Walsh Function Decoupled Parameter Estimation Equations for Dynarmc Continuous-Time Models with time-Delay", Proc. 7th IFAC Symp. on Identrfication and System Parameter Estimation, pp. 799-802 (1985). Y. P. Shih, C. Hwang and W. K. Chia, "Parameter Estimation of Delay Systems via Block Pulse Functions", ASME Jou. of Dyn. Sys. Meas. and Control, 102, pp. 159-162 (1980).

M. Marchand and K. H. Fu, "Frequency Domain Parameter Estimation of Aeronautical Systems with and without Time Delayn, Proc. 7th IFAC Symp. on Identification and System Parameter Estimation, pp. 669-674 (1985). A. E. Pearson and C. Y. Wuu, "Decoupled Delay Estimation in the Identification of Differential Delay Systems", Automatica, 20, pp. 761-772 (1984). W. Song and X. Yu, "Structure and Parameter Identification for a Kind of Multivariable Linear Systems with Unknown Time Delays", Proc. 7th IFAC Symp. on Identification and System Parameter Estimation, pp. 793-798 (1985).

P. Nagy and L. Ljung, "Estimating Time-Delays via statedpace Identification Methods", 9th IFAC/IFORS Symp. on Identification and System Parameter Estimation, pp. 1141-1144 (1991).

St ability Analysis of Digit a1 Kalman Filters Bor-Sen Chen* Sen-Chueh Peng t "Department of Electrical E ~ ~ g i n e e r i n g National Tsing Hua University Hsin Cllu, Taiwan, R.O.C.

t Department of

Electrical Engioeering

National Yun-Lin Polytecll~iicInstitute Huwei, Yunlin, Taiwan, R.O.C.

Abstract A new stability collclitio~~ is introduced for the Iuter. In these situations, state values ant1 coefficie~ltsare stored in registers with a finite ~lutrlber

of bits. Tlle values rnust be quantized, because the size of the available register for each value is finite. Extremely, the results ~ l i u s talso be quantized after every aclditioll or nlultiplicatioli (floating- or fixed-point arithlnetic.). These q~lantiz;~tions cal.lse rourlcloff errors. LVhru implemeatiug a filtering algorithm

011

a floating-point conil>uter., our slloillcl cousicles the ~>roblems t h a t arise in dealing with floati~ig-point coml,utation and finite wordlengtli.

A great deal of works [1],[2],[:3], [4],[5],[6], have bee11 clevotecl to the an:t.lysis of rouncloff error in digital signal processing. Tliese c1igita.l signal processing ideas serve as the ha~ ~ e s wit11 ro~~ncloff noise for optimal state sis of t e ~ h u i c ~ clealiug estiniatiou in this 11aper. Sonic rrsearc:hs have also been clone on finite-wortllent ar~alysisof cligital controllers. Those controllers riiaybe be in fixecl- and floating-poilit arithmetic [l],[S], [9],[I 11,[12]. For exanll>le, to1.1ntloffnoise a ~ i dscaling in the digic coml>ensators ital iml,lementation of linear q ~ ~ a t l r a t C:aussian are co~~siclerecl by Moroney et al. [9]. Rink ancl C l ~ o n g[ l l ] have deterlniliecl a11 upper borincl on the mean-squa.re error in state regulator systems. This influence of linear optinlal cliscretetime systenls cl1.1r to finite wortllength is introcl~lc:ec\ by Van

209

STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS

IVingercleri and De Iutecl gains. Williar~lsori[4] has presented a Illore specific form~.llationof a sirnilar problem in [7]. In particular, be llas taliell the state souncloff quantizatiori into account arid applierl the error spectrur~ishaping technique

[lo] in the cligital Kalman filter tlesign. However, less research has been ~>~.iblislietl on tlie s~.lt>jectof cligital Iiits, ol.ltputs, coefficients, ancl stat.es. In considering floating-point arithn~etic:,sllcl~dyna.nlic ra.nge consiclerations generally can be r~eglecteclbec:a~.ise of tlie large ra11ge of rel)reseutablr nl.lmbers, but cluantization is ititrocl~iceclfor both m~.iltiplicationand acldition. These quantizations leacl to a cleterioration

i11

the icleal

(i.e., irifiriite worrllength) performances of filteririg algorithms.

In extreme cases, tliese cluantizations car] even lead to an iclea.lly stable filtering algorithln 1,ecoming ~lnstable.This instability issilt. cleeply infl~lenc:esthe cIesig.11of I Iolltl>~lt ~'CI

y ( k ) , ancl tlie AT)C is 11si11ga synlmetrical refereuc-e system. T h a t is,

y a ( k ) = y(k) + d(k)

(10)

where rl(k) is the AD(: cluautiz:ttiou error occurring a t the kt11 sample time and satisfying Irl(k)l

5 TLof full scale (FS). T h e

probability distributioli of d(k) is uniform over tlie clila~ltizatioli error and also a white noise process [:3]. Sometimes, the consicleratiou of AT)C pro1,lems is very troubleso~ne,since an ADU may reach saturation ariil clepenil highly on tlie liarilware specifications. However, it is necessary to convert analog signals to digital from using a n AD(> in most digital systeuis containing analog as well as digital colnponents. Tlleli tlie iliscrete state eil~~at,ions [Eils. 7 ari~l81 c:ali then he rewritten as

where

€(A*) = e(X)

+ rl(X.)

is a w l ~ i t erioise process with mean zero ancl covariance [.C2

+

(2-L . F,C' . 2)' . I h l / ~ ' ] , wI~ereIn, is all iclrntity niatris with approlxiate cliuleusioils. I11

tlie one--step-alie:t(l prerlic-tiou I>rol)lerii,let tlie process 1)e

described by Eqs. (7) and (8). The state estinlator has the form

aricl the perforn1;tnct.t is defined as

213


where t ( k ) = i ( k ) = . c ( k ) , called estilllation error. Tlieli tile I i a l ~ n a nfilter is to clioose l i ( k ) in E(l.(1:3) to ruininlize the perforlua~iceI , and in an infinite-prec-ision case, tlie solution is given as

To implemerit a digital Ii

(26)

216

BOR-SEN CHEN AND SEN-CHUEH PENG

Using the represelitatiolis of Eqs. (21) alicl (23) in Le~llllla

1, Eq. (1 7) beco~lies

Upori substitution of Cx(k)

+ e ( k ) for y8(k), we liave

+ terliis of higher-orcler

A" A', A4

Co~nl,iningEcls. (1 I ) an11 (28), it follows tliat

where 0 is the zero matrix, ant1 I is the identity lnatrix with appropriate dimensiou. In Eq. (%9), because tliese t e r ~ n sof higher-orcler A', A', arlcl A4 are small, tliey are ig~iorecl.Let


217

The11 Eq. (29) car] be rewritten as

Next, we shall al~alyzethe esti~nationerror dile to finitewordlength irl~ple~lle~ltatioi~ of cligi tal I(a11nan filters. Although tile allalysis gives precisely t l ~ rerror clue to finite-worcllength effects, the calculation is slightly c~.in~l,ersome. Tllris, by ignoring higher quantities iu Eq. ( a s ) , the ac:tr.lal t.stin~at,ionerror of the digital I\;a.l~nanfilter clan be rxpressecl by

where


218

BB

= [ A ( I C H 3 C + K R 1 C : + R 2 1 i C ' )i A ( R 2 A + AHI - KH:,C' - Ii'RIC - Iressiou of h, will be rather conlplicatecl. Therefore, we ignore the s~nitllhigher-ortler terms in Ecls. (59) and (GO), ancl each

11,

can he exl)ressed al,proxirnately by

si + IT 7.k , 2

hi =

for i = 1 , 2 , - . . , h i l - 1

k=l

ILM

=

b+

n

M-1

k= 1

T~I~ each I I 12, has zero Ineau ant1 the variances are approximately E[hf] =

(1

+ 1)2-~'"/:i

T h e r o ~ ~ n t l e~ni~lti~,lic-ation d of an !\/-vector

.E

l)y an iL1 x ill

matrix A call be written by

wliere n ; j is an e l e m r ~ of ~ t A, ancl h's

i11 clifferent

rows are statis-

tically inclepenclent and ~.lncorreIate-ttl.Hence, we s i ~ b s t i t r ~each te bI-tuple (Al, h2, . . . , h i t I ) for each hf-tuple (Ilil, hz2,- . . , 1 2 ; ~ ) ancl o l ~ t a i uthe per~~littecl rel'resentation

which is c o ~ l v e ~ l i efor ~ i t rnanipr.llation in the clerivatio~iof oirr results.

STABILITYANALYSIS OF DIGITAL KALMAN FILTERS

VII.

23 1

Appendix B : Proof of Theorem

Cousides tlie combine-?tlstate Eq. (30)

l7

=

[i]

Solving tlie ~)rrceclingtlifferruc-eecl~~:~tiou, we o1,tain the solution

Taliing norms, we get

232


Using Eq. (37), it is

fo111id

that

Divicliiig both sides of Eq. (65) by

7."

we obtain

Applying tlie Re1narl;s of Lemma 2, we have

+

(

I

)

{'

-" 1-

+ '"l "l/'~"k +

1I I1 ,::J:

(7~~11~11/7'>1


Next, ~ ~ ~ ~ l l t i p leither y i ~ l gside of Eq. (67), it follows that

When k

t

m,

if

I-

+ ~ r b l l P j l (< 1, then

the state estimatol. will

be r o b ~ ~ s t lstable. y

VIII. As k

-t

oo,

Appendix C : Proof of Theorem 2 Eq. (68) becomes

233


234

Hence, the bound of

1 I (

ask-tm. Fro111 Eq. (;31), t h e actual estimation error is

Taking nornls, we I ~ a v t .


235

TIie~iEqs. (69) to (71) together iniply that

References [I] G. Aillit a ~ i dIJ. Shamkeel,"Sunall rou~idoffnoise realization of fixetl-point cligital filters aucl controller," IEEE Trans. OIL Ac~ustics,Speech, and Signal Proces.si.r~g,Vol. ASSP36, pp. 880-891 ,(1988). [2] L. R. R.al>iner alicl 11. Golcl, Theory and applicatiorl o j digital signal pi-octcs.sir~g,Prentice-Hall, El~glttwoodCliffs, NJ. (1975). [:3] A. v. Oppeuheinl aucl R. FV. Scliafer, Digital Sigrlnl Praoc.f:ssing,Prentice-Hall, E~iglewooclCliffs, N.J , ( 1975). [4]

r). Willia.ulison a~iclS. Sriclliara.n, "An approat:li to coefficient worcllength retl1tc:tiotl in cligital filters," IEEE Tr.ans. Vol. CAS-32, pp. 893-90:3,(1!185). or1 Cir.cuit.5 nrtcl S;y.tit~vz..s,

[5] H. Homar, "(:om~~l.ltatio~ially efficient low rouncloff rioise secontl-orcler state spa(-e str~.lctures,"IEEE Trans. on Circuits aild ,S'ystems, Vol. (;AS-3, p1). 35-41,(1986). [6] D. V. Bliaskar Rao, "Analysis of coefficient quaritizatiorl el.rors in state-space digital filters," IEEE T r a ~ ~OIL s . Acoustics, Speech, a,kl ,5'ign;l ~ r o c c . s . s iVol. ~ ~ ~ASSP-34, , pp. 1311:39,(1956).

[7] A. B. Stripacl, "Prrfor~nanc-etlegraclation in cligitally implrnientecl Iialnia~ifilters," IEEE Trarls. on Aerospace arld Electror~ic.,Systents, Vol. AES-17, pp. 626-6334,(1981).

[8] P. Morouey, A . S. lVillsl;y, and P. I m a x ( a l . b l . c l )

and d2 < ma:r(a2, b2. c2) -+ bounded

c) Ld - (L~ N Lb C/Lc) ---+ continuously increasing, where A is an intersection operator Rd = (Ha N Rb A Rc) --+ continuously decreasing

d) from b and c --+ D" is convex

9 D" is a fuzzy number. Fig. 5 illustrates the proof.

DISTRIBUTED DISCRETE FILTERING

Lot Lb

Ra

/' "

9 . '~

' i ; ",

cl bl

al (dl)

247

m

Rb Rc

,

c2 (d2)

b2 a2

Figure 5: An illustration of the intersection of A*,/3* and C*. The newly generated fuzzy number represents the most likely confidence interval of i~jk. Losing some information is unavoidable while taking the intersection of three fuzzy sets, but the advantage on the controllability of data fuzziness will cover this loss. The next example will show that this approach takes all the possible fuzziness sources into consideration, and still keeps the fuzziness in a reasonable range during long-term predication and propagation of the Kalman filter. 4. D i s t r i b u t e d D i s c r e t e F i l t e r i n g of N o i s y a n d F u z z y M e a s u r e m e n t s A discrete stochastic system with N sensors can be modeled by 3gk • z_.ik =

~k-lf-k-1

--~ Gk-lWh-1,

/~{Wk-1} -- 0, J~{Wk_lWkL1} -- Q ; - 1 ,

i i + v-k, i E{v~} = O, E { ~ (i~ i)T } I R k ,.i i - 1 , Hkz-~ -

---, x

where z_k E 7~~ is to be estimated at the kth moment. The system matrix ~

(13) (14) E T~~x~ is

determined by the relative dynamics between sensors and the sensed environment, and the modeling error is characterized by the error covariance matrix Q;. There are N measurement models in the sensor system, each of which corresponds to a sensor. The symbol z*~ C T~"~ .i describes the noisy and fuzzy measurements acquired by sensor i specified by R k- Note

248

LANG HONG

that there are two sources of possible measurement errors: one is stochastic and the other is fuzzy. The stochastic error is used to describe the degree of our knowledge of the measuring instruments, say a meter with +5% of accuracy. Fuzziness is employed to describe the error introduced during the instrument reading (or data recording) process. Even if we have a perfect meter, the reading from the meter may not be perfect. In Eq. 1:3. vector x k is represented in the central coordinate system at the kth moment, and ~xk i

~,

i = 1 ~ . . . , N in Eq.

moment.

14 are represented in the local coordinate systems at the kth

Each sensor is associated with a local coordinate system, and the integration

process in the central coordinate system is called the central process. The communication network from the central coordinate system to the ith local coordinate system (the forward communication network) is specified by function Wi xik -

~--i(s k, k) + dik, i = 1 ..... N

(15)

where d i denotes the uncertainty of the communication network whose mean and variance are assumed known

E{d' ~}

=

0, and

E { ~ (i~ ) i T } = U k ~

The communication network from the ith local c o o r d i n a t e s y s t e m to the central coordinate system (the backward communication network) is represented by function r i x~ k - ~ i ( x ~ , k )

+ a__ik, i -

1 .... , N

(16)

where -~--C x i k is the variable of x~ represented at the central coordinate system, and c( is the backward communication uncertainty with its statistics given by E{a_jr = 0,

and E{c~(a_~) r} = Vk i.

It is assumed that forward and backward communication network uncertainties are uncorrelated, i.e.,

E { 9_~(~_~) i i T } = o. Usually, the forward and backward communication networks satisfy the following constraint x k = ~ i ( ~ , i ( x k , k ) , k ).

However, in general, they could be any function.

(17)


249

In this chapter, the communication networks are assumed to be linear transformations

(Fig. 6)

i

x k = Jikxk + Tik , i = 1,...,N,

(18)

where Jik is a rotation matrix and Tik is a translation vector, and both Jik and T__ik are assumed exactly known.

,hk,_Tlk ZT

Figure 6: A multicoordinated multisensor system. In centralized filtering, local sensors take only measurements and most of the work of filtering is left to the central processor, which causes a heavy calculational burden in the central unit. In order to achieve computational efficiency, distributed filtering is employed. In the distributed algorithm for dynamic sensor systems, each sensor has its own processor and the noisy and fuzzy measurements acquired by the sensor are locally processed. Only the statistics of local process results are sent to the central site for integration. A functional diagram of distributed filtering is shown in Fig. 7, where general backward communication links (Eq. (16)) are used. Since local processors can run in parallel, the distributed algorithm is more computationally efficient. In distributed filtering of noisy and fuzzy measurements, there are N local processors, each of which implements the fuzzy Kalman filtering equations ^.i in Table 2 and generates N local updated estimates z_ klk and P .iklk, i = 1, ..., N at the kth

moment. By mapping N local updated estimates as well as N local propagated estimates to the central integration site by the backward communication networks, Fig. 7, we have -

:~,iCklk' P*iCktk , and Z__ ^.iCklk-l' P*cklk-l' i = 1, ..., N ready for integration, here the subscript C

also denotes the quantities at the central site before integration.

250

LANG HONG

The integrated estimate ~*klk is derived as follows N i~lk = p~lk[(p;ik_l)-l_b;ik_ 1 + ~--~((p.i . c'/r i=1

9 -- (P .iGI~-, )-lx---~*Cklk-1 i ix-- Ckl/~ )]

(19)

which generates a m i n i m u m central error covariance

(eT~la-)

-1

= (e2l*-,)-

N ' -1 + Z[(P*~:klk ) - - ( e i=1

1

no,syand.11 no,syand"'l

fuzzy data z k

^"l Xkl k

^.1 ( Xkl k , k ) + _

.1

~Ckl

1 processor local i J

^"1

(2o)

I processor ,~ N I

1^'

Xklk

Xklk

^.i i _ ( Xkl k , k ) + ~k klk

G,k-,)-']"

noisy and .N fuzzy data z k

fuzzy data z k

I processor local 1 I

.i

^.N N Xkl k , k ) + ~_k

1

Cklk

[ estimate integration[

A*

integrated estimate Xklk

Figure 7: ,.\ scheme of distributed filtering of noisy and fuzzy measurements. The distributed filtering algorithm described here is optimal in the sense of minimizing local error covariances and the central error covariance. The algorithm is also dynamic, i.e.. the integrated results are colltinuouslv updated when new m e a s u r e m e n t s are available. The dvnainic capability of an algorithTn is important for real-time applications, because adding new measureme~llsdoes ilot reqtzit'e recomt)uting from scratch.


251

5. A n E x a m p l e This example illustrates an application of the algorithm to multisensor integration. Consider one object moving on a two-dimensional surface in a near elliptical course with a constant speed, Fig. 8. Five sensors are located at five different places measuring the location of the object. Since each sensor provides only partial information about the object due to uncertainties in the sensor and fuzziness of measurement readings, we combine the measurements from these five sensors to get more accurate information about the object. The dynamics of the object is modeled by [ xk l Yk

=

I c~176 2 sin(1 ~

~k

I Xk-1 ~]k-1

- - 2l s i n ( l ~ )

cos( 1~) (I~k-- 1

.-~

~k--1

Wxk-1

,

__W_Wk_I

e~,

N(0, Q~-I)

(21)

Wyk_ 1 Wk--1

where z_k_1 and w__k_1 are 2 by 1 vectors, and Ok-1 is a known 2 by 2 non-fuzzy matrix. The measuring processes of the five observers are described by the following five measurement models, each of which is represented at their own local coordinate systems xk

=

xk

z yk

+

Yk

xk i ' v__~ ~ X(0, n*~),

i = 1,...,5

(22)

Vyk

i and v ki are 2 by 1 vectors (the fuzziness of the system parameters enters through where z'k, measurements z__. ik and Q~-I) and measurement matrices H i, i=1,...,5 are identity matrices. The communication networks among the sensors are given by z__.ik = J i z _ k +

T i'

(23)

i = 1, ..., 5

where the rotation matrices ji are

sin(0/)

cos(0/)

where 01 = 4 5 ~ 0 2 - - - 1 5 ~ 0,3=0 ~ 0 4 = 9 0 ~, and 0 5 = - 4 5 ~. The translation vectors

T i

are 10

T ~ = [010 ' --T2=

9

T 3

24] T4 [ 51 10

' --

-8

[6] ' --

-13

"

252

LANG HONG

The communication networks are assumed to be known exactly. Fig. 9 (a) presents the noisy measurements (due to stochastic uncertainties only) from each sensor, and Fig. 9 (b) shows the noisy and fuzzy measurements (the dotted line denotes the noisy measurements containing stochastic uncertainties and the squares represent the fuzziness). The a-cut of the fuzziness considered in this chapter is c~ = Pip-0; the other or-cut levels can be easily obtained using the c~-cut representation during the process. The fuzziness intervals for the example are given as follows.

Fuzzy elements

Fuzzy intervals

q;_,

0.1

R k . i = l

.....

0.2

5

Polo

0.3

The stochastic uncertainties for the example are specified by

Qk-1

=

[0.2 0

0 0.2

,

R/k

=

10 0

1

i=l

'

.... 5 a n d P o t o =

1 0] 0

1

9

Fig. 10 gives the result of distributed integration based on the noisy and fuzzy data. It's easy to see that the fuzziness of Fig. 10 is reduced compared to that of Fig. 9 (b). Fig. 11 depicts the optimally integrated estimate without considering the fuzziness of the data. It should be noticed that taking the measurement fuzziness into consideration doesn't mean that the estimated result of the Kalman filter should be improved. Actually, the estimation of the Kalman filter and the data fuzziness don't interfere with each other. The Kalman filter finds out the optimal estimation (in the sense of minimizing the trace of the estimate error covariance matrix), while the fuzziness represents the vagueness of the estimation. 6. C o n c l u s i o n s This chapter presents an algorithm for distributed discrete filtering of noisy and fuzzy measurements and applies the algorithm to multisensor integration. Both computational efficiency and the capability of dynamic sensor integration are achieved.

The approach

discussed here keeps data fuzziness away from the effect of long-term fuzzy arithmetic and compresses the fuzziness in a reasonable range.

DISTRIBUTED DISCRETE FILTERING 30

>,

.

.

.

253

True trajectory .

0

-10

-20

%

-~'0

-,'0

;

i;

2'o

~0

x

Figure 8: The true trajectory of the object.

40

.

Mutlsensor noisy measurements . . . .

.

20

10

>,

0

-10

-20

-30

-40 -40

-30

-20

-10

x

0

10

20

30

Figure 9 (a)" Noisy measurements of the object locations (containing only stochastic uncertainties).

254

LANG HONG Multisensor fuzzy and noisy measurements

4~I 3O

>"

0

-10

-20

-30

-40 -40

-30

-20

-10

0

10

20

30

Figure 9 (b)" Noisy and fuzzy measurements of the object locations (containing both stochastic uncertainties and fuzziness). Centralized integration of fuzzy and noisy data

>,

0

-10 -15 -20 -25

5

-10

-5

0 x

5

10

Figure 10: The result of integrating the estimates derived from the noisy and fuzzy measurements.

DISTRIBUTED DISCRETE FILTERING 25

.

255

Integrated estimate from noisy measurements only . . . .

0

-10 -15 -20

-25 •

Figure 11" The result of integrating the estimates derived from the noisy measurements only.

References [1] D. Dubois and H. Prade, Fuzzy Sets and Systems: Theory and Applications, Academic Press, New York, 1980. [2] L. Hong, "Centralized and Distributed Multisensor Integration with Uncertainties in Communication," IEEE Trans. Aerospace and Elec., Vol. 27, pp. 370-379, 1991. [3] R. E. Kalman and R. S. Bucy, "New Results in Linear Filtering and Prediction Theory,"

J. Basic Engin., Vol. 83D, pp. 95-107, 1961. [4] A. Kaufmann and M. M. Gupta, Introduction to Fuzzy Arithmetic, Van Nostrand Reinhold, New York, 1984. [5] S. Schnatter, "On Statistical Inference for Fuzzy Data With Applications to Descriptive Statistics," Fuzzy Sets and Systems, Vol. 50, pp. 143-156, 1992.

256

LANG HONG

[6] S. Schnatter, "Linear Dynamic System and Fuzzy Data," in Cybernetics and Systems (R. Trappl, Ed.), World Scientific, pp. 147-154, 1990. [7] H. W. Sorenson, Ed., Kalman Filtering: Theory and Application, IEEE Press, 1966. [8] L. A. Zadeh, "The Concept of Linguistic Variables and Its Application to Approximate Reasoning, Parts 1, 2, 3," Inform. Sci., Vol. 8, pp. 199-249, Vol. 8, pp. 301-357, and Vol. 9, pp. 43-80, 1975.

Algorithms for Singularly Perturbed Markov Control Problems: A Survey 1 Mohammed Abbad Departement de Mathematiques et Informatiques Faculte des Sciences de Rabat, Morocco

Jerzy A. Filar School of Mathematics University of South Australia

Introduction Finite state and action controlled Markov chains (CMC's, for short) are dynamic, stochastic, systems controlled by a controller, sometimes referred to as "decision-maker". These models have been extensively studied since the 1950's by applied probabilists, operations researchers, and engineers. Operations Researchers typically refer to these models as ::Markov decision processes". The now classical CMC models were initially studied by Howard [25] and Blackwell [7] and, following the latter, were often referred to as "Discrete Dynamic Programming". During the 1960's and 1970's the theory of classical CMC's evolved to the extent that there is now a complete existence theory, and a number of good algorithms for computing optimal policies, with respect to criteria such as maximization of limiting average expected output, or the discounted expected output (e.g, see Derman [14]). These models were applied in a variety of contexts, ranging from water-resource models, through communication networks, to inventory and maintenance models. Furthermore, in his book published in 1977 Kushner [31] proposed a computational method for solving controlled diffusions based on approximations by Markov decision processes. The effectiveness of this approach is now well accepted (e.g, see Kushner and Dupuis [32]). Over the past two decades there has been a steady stream of graduate-level texts and monographs by Ross [35], Federgruen [18], Denardo [13], Kallenberg [26], Heyman and Sobel [24], Tijms [40], Kumar and Varaiya [30], Hernandez-Lerma [23], White [41], Sniedovich [39] and Puterman [34] that indicate the continued research interest in these topics. However, in recent years a new generation of challenging problems in CMC's began to be addressed. One class of these problems focussed around the following question: In view of the fact that in most applications the data of the problem are known, at best, only approximately; how are optimal controls from the complete information models affected by perturbations (typically 1We are indebted to M. Haviv for a number of helpful discussionsand for his comments on an earlier draft of the manuscript. CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.

257

258

MOHAMMED ABBADAND JERZYA. FILAR small) of the problem data?

From the practical point of view the above question is of vital importance; however, it leads to challenging mathematical problems arising from the following natural phenomenon: 9 If the perturbation of a Markov Chain alters the ergodic structure of that chain, then the stationary distribution of the perturbed process has a discontinuity at the zero value of the disturbance parameter. This phenomenon was illustrated by Schweitzer [37] with the following example:

P(e) - ( 1- e/2e/2 1- e/2e/2)

Let

be the perturbed Markov Chain whose stationary, distribution matrix is p . ( ~ ) _ ( 1 / 2 1 / 21/2 1/2) for all e E (0, 2]. Thus we have

~,0

1/21/2

-~ P ' ( 0 ) -

01

"

It will be seen in sequel that the above difficulty leads to a very natural, and challenging problem in the theory of singularly perturbed Markov Chains. As with all surveys, it is a difficult problem to determine which of the many interesting contributions should be explicitly cited and/or discussed. In order to keep the size of this survey manageable, and also to increase its potential appeal to control theory practitioners, we have tried to follow the guidelines below: 1. Only finite state/finite action CMC's were considered. 2. We have focussed on constructive methods and algorithmic procedures. 3. ~Ve emphasized the results which analysed, the more difficult, singular perturbations. While the decisions to include or exclude any given result were (of necessity?) made subjectively, we hope that the bibliography is comprehensive enough to enable the reader to at least begin tracing the contributions of most of the researchers active in this field.

SINGULARLYPERTURBED MARKOVCONTROLPROBLEMS

2

Singularly

Perturbed

Markov

259

Chains

In order to analyse the perturbed Markov control problems, we must first understand the uncontrolled case that is equivalent to the controller having only a single strategy at his/her disposal. This sub-topic is sometimes called the perturbation theory of Markov chains and is of interest in its own right. In what follows, we concentrate only on the results that have found extensions to the controlled case. We begin by considering a specially structured class of Markov chains that has received the most attention in the literature. A nearly decomposable Markov chain is defined by an N x N irreducible transition probability matrix P(e) of the form P(e) = P + eA, where

p_

/

P~ o ... o / o

P2 ...

o

0

0

P~

...

Pi is an ni x ni irreducible transition probability matrix, i = 1,2, ..., n. This class arises naturally in many applications of large scale finite state Markov chains. It is characterized by a decomposition of the states into groups, with strong interactions between states in the same group, and weak interactions between states in different groups. The strong-weak interaction structure was first introduced by Simon and Ando [38]. Courtois [9] developed the first analytical techniques for this class, and applied it to many problems in queueing networks and computer systems. The fundamental problem to be analyzed for this class of Markov chains is the computation of the stationary distribution. This problem suffers from high dimensionality and ill conditioning. Courtois gave an aggregation procedure for the computation of an o(c)-approximation of the stationaw distribution of P(e). His procedure is as follows: 1) Compute the stationary distribution p~ of Pi, i - 1, 2, ..., n. 2). Form an aggregated transition matrix

P(E) = s~ + ~VAW,

(1)

260

MOHAMMED ABBAD AND JERZY A. FILAR

where V is an n x N matrix and W is an N x n matrix, which are defined by:

V-

0 I~7 p[

... ...

0

... PT,

0

and 0 W=

In 2 ... 0

0

...

0 / 0

(2)

0/ 0

(3)

l n.

where In, is a column vector consisting of ni ones, i = 1, 2, ..., n. 3) Compute the stationary distribution ~" of P(e). Note that ~)" does not depend on e. 4) The approximate stationary distribution is given by: O'V.

(4)

The appeal of the nearly decomposable case stems from the fact that the originally uncoupled groups of states can still be treated in a largely independent manner, with the results connected only later on as is done in steps 2-4 of the above algorithm. This can be easily seen from the following simple example. E x a m p l e 2.1

P(e) = P + eA =

IZ I I II I~110 I 1

0

1

0

0

0

0

~

0

0

0

1 0

1

0

-i

+ e

I) 1 P T = (~

2 3 2 ~ ) and p ~ = ( -g -~) .

0

0 0

9

SINGULARLYPERTURBED MARKOV CONTROLPROBLEMS

261

2)

)

and

W =

I1~ 1 0 0

0 1 1

"

1 1 ) VAW

-

23 5

~2

9

5

3) ~) T h e a p p r o x i m a t e s t a t i o n a r y d i s t r i b u t i o n is:

i--i I---i

)"

Of course, it can be easily checked that Courtois's algorithm will fail in many cases where the nearly decomposable structure is no longer present. This points to the need to develop an anologous theory for the more general perturbed Markov chains. Based on the theory of Kato [27] for the perturbation of linear operators, Delebecque [10] derived a more general formula for the approximation of the stationary distribution matrix. Below, we present his approach. Let, d(e) = do + edl + e2A2 -k-...,

e e [0, eo] ,

be a family of perturbed generators of Markov chains 2. The unperturbed generator Ao is an mo x mo which contains my ergodic classes and possibly some transient states. It is assumed that A1, A2, ... are also generators of Markov chains. Let P~ be the stationary distribution matrix corresponding to the generator Ao. We define: 2Recall that a generator of a Markov chain is obtained by subtracting the identity matrix from its probability transition matrix

262

MOHAMMED ABBAD AND JP_,RZYA. PILAR

qk := (qk(s), s = 1, 2, ..., m0)T; k = 1,2, ..., m I where qk(s) is the probability (with respect to the Markov chain defined by the generator A0) to be absorbed in the class k starting from s. We define an m0 x m1 matrix W1, and an mx x m0 matrix V1 by: ~V1 "-- [qk(8 I l k = l - m 1 \ /is=l-too

(5)

ls=l-mo ksJk-l-ml

(6)

and rm

Vl "--[

where, inks "-

{ [P0k]s, 0,

if s is in the ergodic class k otherwise

(7)

where P0k is one of the identical rows of Po corresponding to the ergodic class k. Note that from the definitions of W1 and Vl, it follows that:

Po = W1V1 and VI W1 =

I m ~x m , .

(8)

Let Ho be the matrix defined by: Ho := (Po - Ao)

- Po.

(9)

A "reduced series" is constructed by: A I ( c ) - A~ + cA I + e2A{ + ..., where A~ := VI[PoAIPo]W1

AI "- VI[P~A2Po + PoA,HoA,P~]W1

(i0)

SINGULARLYPERTURBEDMARKOVCONTROLPROBLEMS

263

n

AI-1 :-- V I [ E p=l

P~ (Akl HoAk= ) ... (Ak,_, HoAk, ) P~ ]W1 E kl +...kp=n

Now, repeat the construction above with the series A(e) replaced by Al(e). A new series A2(e) and new matrices Pi*, H1, 1/2, W2 are constructed. By induction a sequence of series of mi x mi matrices is defined, where mo>_ml >_m2>_ .... Now, the following results were established by Delebecque[10]. T h e o r e m 2.1 (i) Every series in the construction above is a generator of a Markov chain, for e small enough. (ii) The process of reduction described above has only finitely many steps; say, k steps. (iii) Let P* (e) be the stationary distribution matrix corresponding to the generator A(e). Then, lim P*(e) e.-+o

-

W1W2...WkP~VkVk_I...V1.

/ 1~ o/ / ~

E x a m p l e 2.2 let,

0100 0010 0 0 0 1

P(e)-

O0 0 10-1 0 1 0

+

o / / -lo 1 o/

0 0 -1

0 -1

+e

1

--101 1 00 -I 0 0

"

Hence,

/000 0/ Ao-

0 0 0 1 0 -1 0 1 0

0 0 -1

A1 =

/10 10/

'

0 -1 1

-1 1 -1

0 1 0 0 0 0

,

Ak--0,

From (5) and (6), we have that,

V1--

1

0

0100

0

O)

and W I =

I

i 0 01 I 0 01

o

k>2.

264

M O H A M M E D ABBAD AND JERZY A. FILAR

From (8), we have that,

P o ~"

I 1 00 010 0

I

0

1 0 0 0

"

0 1 0 0

From (9), it follows that,

I

H0

O 0 00 O 0 00 1 -i 0 10 " 0

-101

The coefficients of the reduced series (10) are: A~=

00

1

-1

....

Now, new matrices are constructed:

(~0 1 o I ' ~1_(o0 0 o I "

PI=P~=V2=W2=

The coefficients of the next reduced series are:

-11

A2~ =

- 11

i

- Ai2 - """

The matrices that correspond to the above series are:

~_(Ol ol)

,P~=

~

, H~=

- ~, ~

Also, we have that:

V3=

(,1~ ~

~

, l~V3=

(1) i

"

-i ~

.

SINGULARLY PERTURBED MARKOV CONTROL PROBLEMS

265

The process of Delebecque stops at this stage, therefore k = 2. Hence:

limP*(e) e--+0

-

W1W2 P:~V2 V1 =

/ii~176 / 0 0 ~ ~ 0 0

"

Note that, when the procedure of Delebecque is applied to the case of nearly completely decomposable Markov chain, it gives exactly the method of Courtois. Let P be a transition probability matrix for some Markov chain. Suppose that the perturbation is such that P(E) - P + eA is a transition probability matrix for some irreducible Markov chain for all 0 < e < e~ Let 7r(e) be the unique stationary distribution of P(e). Schweitzer [37] showed that for all e > 0 small enough, or

i

i=O

(ii)

where 7r(O = 7r(o)Ui for some fixed matrix U. Since 7r(e) satisfies, ~r(e) = 7r(e)(P + cA),

(12)

it follows from (11)and (12)that,

(I - P ) = 7r(j+l)(i_ P) - ~(J)A = 0 j _ 0 , 1 , . . . Since P may contain more than one ergodic class, the set of equations, { 7r(~ P) = 0 7rOe = 1 where e is the vector of ones, is not sufficient to determine 7r(0) uniquely. Hassin and Haviv [19] developed an algorithm for computing k" such that:

266

MOHAMMED ABBAD AND JERZY A. FILAR 7r(~ e - 1 P) =

7r(~

0

vr(J+')(I- P) - 7r(J)A = O, j - O, 1,..., k" - 2 is the minimal set of equations needed to determine rr (~ uniquely. If k" = 1, then the only set of needed equations is: rr(~ e = 1 7r(~ P) - 0 Let M(e) be the matrix with entries Mij(e), where :YIij(e) is the mean passage time from state i to state j when transitions are governed by P(e). From the theory of Markov chains, it is well known (e.g see [17], [28]) that,

( I - P(e))M(e) = J - P(E)Md(e),

(13)

where Md(e) is a diagonal matrix whose diagonal coincides with the diagonal of M(e), and all entries of J are one. From (13), it is clear that M(e) admits Laurent series expansion. Let uij be such that M~j(e) = o(e-u'J) which means that u~j is the order of the pole of M~j(e) at zero. The following relation is due to Hassin and Haviv [19],

k" = 1 + max(uij

-- ujj).

The algorithm for the computation of uij for all i and j is stated below (see [19]). Fix a state s. Consider the graph G = (V, Er, E~) where V is the set of states, E r = {(i,J)l pij > 0} is the set of regular arcs, and E~ = {(i,j)[ Aij > 0} is the set of epsilon arcs. For a subset C E V, let 6(C) = { ( i , j ) e E.r U Er e C,j cI C}.

Step1

(Initialization) Construct a graph G'--- (V', E',., Ere) from G by deleting all loops (i, i) e Ee and all edges going out of s. Set u(i) := 0 and S ( i ) : = i for all i E V. S t e p 2 (Condensation of cycles)

SINGULARLYPERTURBED MARKOV CONTROLPROBLEMS

267

If G' contains no directed cycles of regular arcs, go to Step3. Let C be a directed cycle of regular arcs. Condense C into a single vertex c. If ~(C) A E'~ -r q}, set u(c) := max{u(i)li E C}. If ~(C) C E'~, set u(c) = 1 + max{u(i)li E C}, and set E'~ := E'r U ~(C),

E'~:=E'~\6(C). Set S(c) := U,ecS(i). Repeat Step2. Step3 Set T := V'. Let u(j) (i, j) E E'~ where i E T, u(i) := max{u(/), u(j) If T = (s}, go to Step4.

= max{u(i)li E T}, and set T := T \ { j } . For set u(i) := u(j). For (i,j) E E'~ where i E T, set 1}. Otherwise, repeat Step3.

Step4 (Computation of u(i) : uis) {S(vf)lv' E V'} is a partition of V. For each v E

that v E S(v') and set u(v):= u(v'). Set u(~):: m~x{m~• i) e E~},m~•

V\{s},

find v' E V' such

~r(~,i) e E~}}.

oo

The computation of the sequence {~r(0}i=0 by the minimal set of equations is given in [21]. The following example, based on an example given in [19], illustrates the above procedure. Note that the data of this example is the same as Example 2.2 which was used to demonstrate Delebecque's procedure. E x a m p l e 2.3 Let,

P(e)

/ 1000/ / -1010 /

= P + eA

=

0100 1000 0100

+e

0 -1 1

-101 1 00 -100

(i4)

The graph G is defined in Figure 1 where the regular arcs are represented by bold arrows and epsilon arcs by dashed arrows. We proceed through the steps of the algorithm with the fixed state s = 2. In Step1, we consider the graph represented in Figure 2. We have that u(1) = u(3) = u(4) = 0.

268

MOHAMMED ABBAD AND JERZYA. FILAR

s

s

s

" 0 and ~s'esp(s'ls, a) = 1. Henceforth, such an CMC will be synonymous with the four-tuple r' - (S, {A(s) 9 s e S}, { r ( s , a ) ' s e S,a e A(s)}, {p(~'l~,a) " e

s, a

e

A decision rule 7rt at time t is a function which assigns a probability to the event that any particular action is taken at time t . In general ~Tt m a y depend on all realized states up to and including time t, and on all realized actions up to time t. Let ht - (So, a0, s l , . . . , at-l,St) be the history up to time t where ao e A(so),... ,at-y e A(st-1). ~Tt(ht, at) is the probability of selecting the action at at time t given the history hr. A strategy 7r is a sequence of decision rules ,'T = (~0 . r l , . . . , 7rt,...). A Markov strategy is one in which 7rt depends only on the current state at time t. A stationary strategy is a Markov strategy with identical decision rules. A deterministic strategy is a stationary strategy whose single decision rule is nonrandomized. Let C, C(S) and C(D) denote the sets of all strategies, all stationary strategies and all deterministic strategies respictivelv. Let Rt and E,~(Rt, s) denote the random variable representing the immediate reward at time t and its expectation when the process begins in state s and the controller follows the strategy ft. The overall reward criterion in the limiting average CMC F is defined by 1 J (s, ,-r) := lim inf T + I T--,oo

T

~E~(Rt's)'

seS,

,'reC

t--O

A strategy 7r~ is called optimal if

J(s ,'1~ - m a x J ( s , ,'1) for all s e S rt E C

(16)

The overall reward criterion in the discounted CMC F is defined by

J'~ ( s) = m a x J'~ ( s, zr) for all s E S ~rE C

(17)


271

where,

s~

~) := Z

~'E~(R,, ~) , ~ e S, ~ e C

t=O

A strategy ~r~ is called a-optimal if jO(s ~ o ) = ma• J~(s, ~) for m s e S ~EC

It is well known that there always exist optimal and a-optimal deterministic strategies and there is a number of finite algorithms for its computation (e.g., Derman [14], Howard [25], Kallemberg [26], Ross [35]). We shall now consider the situation where the transition probabilities of F are perturbed slightly. Towards this goal we shall define the disturbance law as the set

D = {d(s'ls, a)ls, s' e S, a e A(s)} where the elements of D satisfy (i) ~s, es d(s']s, a) = 0 f o r all s 6 S, a e A(s) (ii) there exists eo > 0 such that f o r all e E [0, e0] p(s']s, a) + ed(s']s, a) >_ 0 f o r all s E S, a e A(s). Now, we have a family of perturbed finite markovian decision processes F~ for all e E [0, e0] that differ from the original CMC F only in the transition law, namely, in r~ we have p~(s'[s, a) := p(s'ls , a) + ed(s'ls, a) f o r all s, s' e S, a E A ( s ) . The limiting average CMC corresponding to r~ is the optimization problem J~(s) := max 4(s, rr) s 6 S ~EC

(L~)

where J~(s, 7r) is defined from F, in the same way as J(s, 7r) was defined in F.

Similarly, the discounted CMC corresponding to F~ is the optimization problem

272


J:(s) := maxJ,~(s 7r) s E S rrEC

(D~)

~

For every 7r E C(S) we define: (i) The Markov matrix P(Tr) = (pss,(Tr))s.N,=l, where pss,(Tr) :=

X~ P(S'ls, a)Tr(s,a)

aEA(s) for all s, s ~ E S; (ii) the Markov matrix P,(,'r) = (p]s,(Tr)),~,=l where Pess,(~) :- F-,aEa(s)Pc(St[S, a)7[(s, a) for all s, s' E S; (iii) the perturbation generator matrix D(Tr) - (d~,(,'r))~v~,=, where d~,(Tr) "-EaEa(s) d(stlS, a)Tr(S, a); and (iv) the stationary distribution matrix of P~(,-r): P,'(,'r) -(p;;,(,-r)) ~.,,=1 N := lim 9

t ~

1 t +

1

P~(,'r) k=o

where P~ := IN an N x N identity matrix. The stationary distribution matrix P'(,'r) of P(Tr) is defined similarly. Note that for every ,'r E C(S)

P,(,'r) = P(,'r) + eD(Tr).

(18)

With every 7r E C(S) we associate the vector of single stage expected rewards r(Tr) = (r~ (Tr),..., rN(,'r)) T in which rs(,'r) := ~aeA(s) r(s, a),'r(s, a) for each

sES. It is well known that for each stationary strategy. 7r E C(S)

J,(s, 7r) = [P,'(,-r)r(Tr)]s, s E S

(19)

Just as in the case of uncontrolled, singularly perturbed, Markov chains the nearly decomposable assumption leads to elegant solution methods. Furthermore, this assumption has the following appealing physical interpretation: Imagine that the controlled system consists of a number of, largely


273

autonomous, components that are coupled by infrequent "disturbance" from a central unit. Clearly this induces the nearly decomposable structure. We now formally define the nearly decomposable controlled Markov chain and devote the remainder of this section and the entire following section to the discussion of the many desirable properties of this special and important class of CMC's. Let F be an (unperturbed) CMC. In addition, we shall assume the following: A1) S = U ~ = l ~ where Si n sj = O i f / n l + n2 + ... + nn = N.

76 j, n > 1, card S/ = ni,

A2) p(s'ls , a) = 0 whenever s E Si and s' E Sj, i -76 j. Consequently, we can think of F as being the ':union" of n smaller CMC's ri, defined on the state space Si, for each i = 1, 2, ..., n, respectively. Note that if 1"Ii is the space of stationary strategies in Pi, then a strategy 7r E C ( S ) in F can be written in the natural way as 7r = (7rl, 7r2, ...TRY), where 7F E Ili. The probability transition matrix in Fi corresponding to 7~; is of course defined by P~(Tri) := (ps,s,(Tri))s,s, es~. The generator G~(rri) and the stationary distribution matrices P/" (Tri) matrices can be defined in a manner analogous to that in the original process F. A3) For every i = 1,2, ...n, and for all 7ri E Hi the matrix Pi(Tr ~) is an irreducible matrix. In view of A3), P~(Tr/) is a matrix with identical rows. We shall denote any row of PT(Tr i) by p~(Tri). We shall now consider the situation where the transition probabilities of F are perturbed slightly. Towards this goal we shall define the disturbance law as the set

D = {d(s'ls , a)ls , s' E S, a E A ( s ) } where the elements of D satisfy (i) E~,~s d(s'ls, a) = 0 f o r all s E S , a E A(s). (ii) - 1 _ n', 7r, - 7r,., and 7r,. is optimal in the limit Markov control problem (L). However the problem of estimation of n', or equivalently of e,., is still an open problem. In the discounted case, it was shown in [1] that the perturbation is which means that for any maximizer 7r0 in the original problem (Do), limmax i V~(s, 7r~ - V ~ ( s ) 1 = ~0

s6S

stable,

O.

The result above, shows that any optimal strategy in (Do) is an approximate solution of the perturbed problem (D~) for e sufficiently small, It is also possible to consider the general perturbation namely:

p~(Jl~, a)

-

p(s'l~, a) + d(s'ls, a),

s, s' e S, a e A(s).

The corresponding limiting average Markov control problem by:

Jd(s) := max [P~(~)r(~)]~ ~6C(S)

sES

(Ld')

(28) is defined

(29)

where P~(Tr) is the stationary distribution matrix of the transition matrix Pd(Tr) defined by: [Pd(Tr)]ss, "-- ZaeA(s)P~(*'I*, a)Tr(s, a). In general, P~(n) may not have a limit when I]dll -~ 0 as is illustrated by the following example.

282

MOHAMMED ABBAD AND JERZYA. FILAR

E x a m p l e 5.1 Assume that d = (dl, d2) and that there is only one strategy ~r which induces the perturbed transition matrix

Pd(Tr) -- ( 1-did2 1 -dl

"

(30)

The stationary distribution matrix is now given by d2

P~ (Tr) =

d_.._4z__)

d,~d2 d,~d~ dl +d2

(31)

dl +d2

but ( all+d2) dl has no limit as I[dl[--+ 0. However, in [1] it was shown that the general perturbation is stable in the case of unichain or communicating CMCs. That is, for any maximizer no in the unperturbed problem (L~) lim IIPd(Tro)r(Tro)IIdll-~O

Jd[I = O.

We recall that CMC is unichain if for any deterministic strategy n E C(D), the transition matrix P(Tr) has only one ergodic class. Similarly, an CMC is called communicating if for any s, s I E S, there exist a deterministic strategy ,-v E C(D) a natural number n such that [P"(Tr)]ss, > 0.

In the rest of this section, we consider only the nearly decomposable CMCs. In this case, from Section 2, we have the following formula for the limit stationary matrix/5~(7r) for every stationary strategy 7~ E C(S)" (32)

-

where, P ' ( ~ ) is the stationary distribution matrix of the irreducible transition matrix defined by the generator

V(~)D(~)W.

Based on the formula (29), the following two algorithms were developed in [2], for the computation of an optimal deterministic strategy in the limit Markov control problem.


5.1

283

Linear Programming Algorithm

Consider the following linear programming problem (P)" maximize Ei~=l Esesi E.c:A(s) r(s, a)z~ subject to:

~

i (5~s, - P(s'ls, a))z~,

- - 01

S' E Si;i = 1 , . . . , n

sESi aEA(s) n

E

E

E

i =0; d(s'ls, a)zs~

E

j-1,...,n

i=l s'6S i s6Si aEA(s)

n

E E E i=1

i =1

sESi sEA(s)

i >0;i=1

Zsa u

~ 99

. n;

seSi, aeA(s).

It can be shown (see [2]) that an optimal strategy in the limit Markov control problem (L) can be constructed as follows.

Theorem 5.4 Let {zs,i li = 1 , . . . , n's E Si; a

E A(s)} be an optimal extreme solution to the linear program (P), then the deterministic strategy defined by:

7r(s) - a,

s E Si, i = l , . . . , n v=v z~i > O

is optimal in the limit Markov control problem (L). The linear program (P) is similar to the one given by Gaitsgory and Pervozvanski [18]. However, these authors used techniques different from those in

[2]. 5.2

Aggregation-Disaggregation Algorithm

The following algorithm is a policy improvement algorithm which converges in a finite number of iterations to an optimal deterministic strategy of the

284


limit Markov control problem. Step1 Select an arbitrary deterministic strategy ~r in the CMC F, and set: [Tr(i)]~ : : 7r(s);

s 6 S,; i : 1, . . . ; n.

Step2 Compute p~(,'r(i)); qij(,x(i)); and c(i, zr(i)), i = 1,... ,n, j = 1 , . . . ,n. For each i = 1 , . . . , n, the computation of p~(Tr(i)) is done by solving the linear system:

{ x~p,(~(i)) - x ~ i

~-,s6Si X s

1 -

-

Step3 Solve for the unknowns A, Y l , . . . , Yn-1; the linear system:

I

n

A + y~ = c(i, 7r(i)) + E i = I q~j(Tr(i))yj

i = 1,...,n

y~ = 0

Step4 For each i = 1 , . . . , n , compute the deterministic strategy ,x'(i) obtained after one iteration of the simplified policy improvement algorithm, when the starting strategy is zr(i), for the CMC Fi with rewards:

j=l s'ES i

s e s,,

Step5 If 7r'(i) = lr(i) for a l l / = 1 , . . . , n . Stop. Otherwise 7r(i) +-- 7r'(i); i = 1 , . . . ,n and go to Step2.

a e A(s).


285

References [1] M. Abbad and J. A. Filar, "Perturbation and Stability Theory/or Markov Control Problems", IEEE Trans. Automat. Contr., Vol. 37, NO. 9, pp. 1415-1420, 1992. [2] M. Abbad, T. R. Bielecki and J. A. Filar, "Algorithms for Singularly Perturbed Limiting Average Markov Control Problems", IEEE Trans. Automat. Contr., Vol. 37, NO. 9, pp. 1421-1425, 1992. [3] R. W. Aldhaheri, H. K. Khalil, "Aggregation of the Policy Iteration Method for Nearly Completely Decomposable Markov Chains ", IEEE Trans. Automat. Contr., Vol. 36, NO. 2, pp. 178-197, 1991. [4] E. Altman and V. A. Gaitsgory, "Stability and Singular Perturbations in Constrained Markov Decision Problems", IEEE Trans. Automat. Contr., Vol. 38, NO. 6, 1993. [5] E. Altman and A. Shwartz, "Sensitivity of Constrained Markov Decision Problems", Annals of Operations Research, Vol. 32, pp. 1-22, 1991. [6] T. R. Bielecki and J. A. Filar, " Singularly Perturbed Markov Control Problem: Limiting Average Cost", Annals of Operations Research, Vol. 28, pp. 153-168, 1991. [7] D. Blackwell, "Discrete Dynamic Programming", Annals of Mathematical Statistics, Vol. 33, pp. 719-726, 1962. [8] M. Cordech, A. S. Willsky, S. S. Sastry and D. A. Castanon, "Hierarchical Aggregation of Singularly Perturbed Finite State Markov Processes", Stochastics, Vol. 8, pp. 259-289, 1983. [9] P. J. Courtois, "Decomposability: Queueing and Computing Systems", Academic Press, New York, 1977. [10] F. Delebecque, "A Reduction Process for Perturbed Markov Chains", SIAM J. App. Math., Vol. 48, pp. 325-350, 1983. [11] F. Delebecque and J. Quadrat, "Optimal Control of Markov Chains Admiring Strong and Weak interactions", Automatica, Vol. 17, pp. 281-296, 1981. [12] F. Delebecque and J. P. Quadrat, "Contribution of Stochastic Control Singular Perturbation Averaging and Team Theories to an Example of LargeScale Systems: Management of Hydropower Production", IEEE Tranc. Automat. Contr., Vol. AC-23, NO. 2, pp. 209-222, 1978. [13] E. V. Denardo, "Dynamic Programming ", Englewood Cliffs, NJ: PrenticeHall, 1982.

286

MOHAMMEDABBADAND JERZYA. FILAR

[14] C. Derman, "Finite State Markovian Decision Processes", Academic Press, New York, 1970. [15] N. V. Dijk, "Perturbation Theory for Unbounded Markov Reward Processes with Applications to Queueing ", Adv. Appl. Prob., Vol. 20, pp. 99-i 11, 1988. [16] N. V. Dijk and M. Puterman, "Perturbation Theory for Markov Reward Processes with Applications to Queueing Systems", Adv. Appl. Prob., Vol. 20, pp. 79-98, 1988. / [17] J. L. Doob, "Stochastic Processes", Wiley, New York, 1953. [18] A. Federgruen, "Markovian Control Problems", Mathematical Centre Tracts 97, Amsterdam, 1983. [18] V. G. Gaitsgori and A. A. Pervozvanskii, Theory of Suboptimal Decisions, Kluwer Academic Publishers, 1988. [19] R. Hassin and M. Haviv, "Mean Passage Times and Nearly Uncoupled Markov Chains", SIAM J. Disc. Math., Vol. 5, NO. 3, pp. 386-397, 1992. [20] M. Haviv, "Block-Successive Approximation for a Discounted Markov Decision Model". Stochastic Processes and their Applications, Vol. 19, pp. 151-160, 1985. [21] M. Haviv and Y. Ritov, "Series Expansions for Stochastic Matrices", Unpublished Paper, 1989. [22] M. Haviv, "An Approximation to the Stationary Distribution of Nearly Completely Decomposable Markov Chain and its Error Analisis", SIAM J. Alg. Disc. Math., Vol. 7, NO. 4, 1986. [23] O. Hernandez-Lerma, "Adaptive Markov Control Processes", in Applied Mathematical Sciences, Vol. 79, Spring Verlag, New York, 1989. [24] D. P. Heyman and M. J. Sobel, " Stochastic Models in Operation Research", Vol. 2, 1984. [25] R. A. Howard, "Dynamic Programming and Markov Processes"~ M. I. T. Press, Cambridge, Massachusetts, 1960. [26] L. C. M. Kallenberg, "Linear Programming and Finite Markovian Control Problems", Mathematical Centre Tracts 148, Amsterdam, 1983. [27] T. Kato, "Perturbation Theory for Linear Operators ", Spring-Verlag, Berlin, 1980. [28] J. G. Kemeny and J. L. Snell, "Finite Markov Chains", Van Nostrand, New York, 1960. [29] P. Kokotovic and R. G. Phillips, '9t Singular Perturbation Approach to Modeling and Control of Markov Chains ", IEEE Trans. Automat. Contr.,


287

Vol. AC-26, pp. 1087-1094, 1981. [30] P. Kumer and P. Varaiya, "Stochastic Systems: Estimation, Identification and Adaptive Control", Englewood Cliffs, NJ: Prentice Hall, 1986. [31] H.J. Kushner, "Probability Methods for Approximations in Stochastic Control and for Elliptic Equations", Academic Press, New York, 1977. [32] H.J. Kushner and P.G. Dupuis, "Numerical Methods for Stochastic Control Problems in Continuous Time", Springer Verlag, New York, 1991. [33] J. B. Lasserre, "A Formula [or Singular Perturbations of Markov Chains", Unpublished Paper, 1993. [34] M. L. Puterman, "Markov Decision Processes: Discrete Stochastic Dynamic Programming", John Wiley and Sons, New York, 1994. [35] S. M. Ross, "Introduction to Stochastic Dynamic Programming", Academic Press, New York, 1983. [36] P. J. Schweitzer, "Perturbation Theory and Finite Markov Chains", J. Appl. Probability, Vol. 5, pp. 401-413, 1968. [37] P. J. Schweitzer, "Perturbation Series Expansions for Nearly Completely Decomposable Markov Chains", Teletrafic Analysis, Comput. Performance Evaluation, pp. 319-328, 1986. [38] H. A. Simon and A. Ando, "Aggregation of Variables in Dynamic Systems", Econometrica, Vol. 29, pp. 111-138, 1961. [39] M. Sniedovich, "Dynamic Programming", M. Dekker, New York, 1992. [40] H. C. Tijms, "Stochastic Modelling and Analysis", Wiley, New York, 1986. [41] D. J. White, "Markov Decision Processes", John Wiley and Sons, Chichester [England], New York, 1993.


CONTROL OF U N K N O W N SYSTEMS VIA DECONVOLUTION Hsi-Han Yeh Siva S. Banda P . J . Lynch

Fright Dynamics Directorate, Wright Laboratory Wright-Patterson AFB, Ohio 45433

I. INTRODUCTION

Recently, a class of heuristic control schemes which have been widely used in industrial processes [ 1-10] have found applications in the aircraft control systems [2, 4-7]. This class of control schemes include Model Predictive Control [1, 8-9], Model Algorithmic Control [2-3, 6-7], Dynamic Matrix Control [10], and Output Predictive Algorithmic Control [4-5]. These schemes share a common feature of the on-line calculation of future control inputs that will steer the system along a desired reference trajectory, based on a given mathematical model of the plant. The control loop is closed by updating the calculation at each sampling instant with new measurements of the actual input and output of the plant. Generally, the algorithms involved make the system nonlinear and time-varying. They are usually too complicated for the system to submit to analytical treatment [ 1, 3]. However, if the prediction scheme is linear and if the CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.

289

290

HSI-HAN YEH ET AL.

C(z) z1 -1

G(z)

( 1- z )G(z)

I Fig. 1. Unconstrained Model Algorithmic Control with Closed-Loop Linear Prediction

control and output variables are unconstrained, the closed-loop system becomes linear [3-4, 8-10]. For Model Algorithmic Control (MAC), the controller for this special case of linear prediction is realizable by cascading an integrator with the inverse of the mathematical model of the plant [3]. (See Fig. 1) In MAC and its variants [1-10], the plant model is identified off-line. Where the unitsample response model (or, loosely speaking, impulse-response model) is used [2-3, 6-7], the inverse of the plant model is implemented as a deconvolution between the incoming signal and the unit-sample response model. Since at the n th step of a deconvolution, only the first n-1 signals in the input sequence and the first n signals in the unit-sample response sequence are needed, it is logical to question whether it is necessary to have the unit-sample response of the plant identified off-line and installed in the memory of the controller in advance. To put it more pointedly: Can the controller compute the unit-sample response sequence of the plant term-by-term while the system is in operation, and use the partially identified unit-sample response to generate the next-step control variable in real time? This chapter answers the above questions affirmatively, and goes on further to show that the output error sequence which contains the information on

CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION

291

the unit-sample response of the plant may be used directly in a deconvolution to generate the optimal next-step control. Section II shows that the deconvolution of two causal signals can be implemented in real time to obtain the unit-sample response sequence of a system, if a sufficient number of delays are allowed. Section III presents a design of self-adaptive control which computes the unitsample response sequence of the unknown plant on-line, and uses it to generate the next step control. In Section IV, a model reference control of an unknown plant via deconvolution is presented.

Section V gives numerical examples.

Comparison with unconstrained Model Algorithmic Control is made in Section VI. Section VII presents the conclusions.

II. REAL-TIME DECONVOLUTION

Deconvolution of causal signals C(z) and G(z) results in a causal signal U(z) given by

c (z) U(z) = G(z)

( 1)

Therefore, the implementation of a deconvolution involves the realization of 1/G(z). Since G(z) is a causal signal, it is represented by an infinite series

G(z) = gk z k + gk+l z ( k + l ) + gk+2 z(k+2) + ---

(2)

for some k > 0, where m may be called the number of delays in G(z). If G(z) is the output of a stable system then lim gn = g, and G(z) may be approximated n~oo by

292

HSI-HAN YEH ET AL.

N-l G(z) ---

E

.

z-(k+N)

gk+iz'(m+l) + gk+N"''-'--~ 1 - fz

(3)

i=0 for sufficiently large N. The constant f is a forgetting factor, 0 < f 0 it can be easily verified that z-k/G(z) can be realized as in the diagram of Fig. 2, where the unit delay is represented by a triangle.

gk+N z

+

+

gk+N-1 -1 z

I Q O

O O

O

O

D O O

gk+2 +~

t

~

+

gk+l

z -1

z

1

gk

L

Fig 2. Realization of z-k/G(z)

f


293

Thus, the deconvolution of two causal stable signals can be implemented on-line with a shift-register and a memory bank of finite memory storages, by allowing a sufficient number of delays. Since the computation is hardware implemented, the computation time required is negligible.

The

following development explores the use of on-line deconvolution in two approaches to the adaptive control of unknown systems, namely, self-adaptive control and model reference control.

III. SELF-ADAPTIVE CONTROL A(z)

R(z>

_~

+

% (Z> i

E(zk/ !

r- [ B(z)

l ]

U(z~

~ ^ ..- G(z)

[C(z) Unknown System

Fig 3. Simultaneous Identification and Control

Consider the block diagram of Fig. 3 where G(z) is the unknown system, A(z) and B(z) are arbitrary design parameters, and Gc(z) is the adaptive controller to be designed. It is readily seen that

E(z) = B 1- ~ G(z) = (1- G(z)Gc(z))R(z)- G(z)A(z)

(4)

294

HSI-HAN YEH ET AL.

Choosing -1

B(z) =A(z)

(5)

[Gcl(z) - G(z)]Gc(z)R(z) + [(3(z) - G(z)]Afz) = 0

(6)

and rearranging Eq. (4) give

If the controller is built in such a way tllat

Gc(z) - ^

1

G(z)

(7)

in other words, if the output of the controller is the deconvolution of R(z) and Cr(z), then Eq. (6) may be rewritten as

[CJ(z) - G(z)]U(z) = 0

(8)

where U(z) is the control signal to the unknown system. The implication of Eq. (8) under the conditions Eq. (5) and Eq. (7) may be interpreted as follows: (a) the negative of the inverse of the z-transform of the excitation signal A(z) is implemented as the transfer function of the error processor B(z), (b) the inverse of the z-transform of the output of the error processor is implemented as the transfer function of the controller Gc(z) and (c) if the input to the plant or the controlled system is non zero for some t = nT, n > 0, then the output of the error processor is the unit-sample response of the unknown plant or controlled system. Furthermore, in this implementation, the error signal E(z) is given by


E(z)

295

B(z) (9)

= -A(z)6(z) = -A(z)G(z)

Since A(z) is arbitrary, it may be chosen to minimize the error signal e(nT). Controller

+ V Et

~(z) Identifier

/A(z)

A

G(z) = G(z) If U(z)~f 0

Fig. 4 Identification and Control via Deconvolution

The above observation leads to a scheme of an adaptive control system which identifies the unit-sample response sequence of an unknown plant while at the same time uses this sequence in the adaptive controller to force the plant output to follow the command signal. The plant transfer function is unknown but its unit-sample response can be measured as the output of the error processor, if the controller transfer function is the inverse of the plant transfer function. Without loss of generality, one may assume that the unknown system has one delay (k=l) in its unit-sample response. An implementable adaptive control system can be drawn from the above analysis. (see Fig. 4) The error processor unit in Fig. 3 is labeled "identifier," because its output is exactly the

296

HSI-HAN YEH ET AL.

unit-sample response of the plant if U(z) r 0, which is easily satisfied by applying A(z) in advance of R(z).

A. REALIZATION OF THE CONTROLLER

Note that an additional delay is assigned to the controller, in order to accommodate the computation time needed for the deconvolution.

The

controller may be realized as in Fig. 2, with the additional delay connected to the output. Since the unit-sample response of G(z) has one delay, the first non zero value at the output of the identifier is available to the controller one sampling period after the application of A(z). Therefore, the furst non zero signal of R(z) must be three sampling periods behind that of A(z), in order for the computation time to be accommodated and the deconvolution to be synchronized (the first non zero signals of~(z) and R(z) arrive at the controller at the same time). This can be verified as follows: (Assume A(z) has no delay).

R(z)_ r3 z-3 + r4z'4 +

...

Cr(z) - g l z" 1 + ~ 2 z - 2 + ...

y0 = y~ = 0

r3

Y2 = 7"gl

= Y(z)

(10)

(11)

(12)

Where Y(z) is the controller output. It is easily seen that if rl * 0, then at t=0 one must have

rl

y o = : -gl

(13)


But gl is not available until t-T.

297

Therefore, the controller is not

realizable. If r 0 = r I - 0, one must have, at t=T r2

Yl =7-gl

(14)

This means r2 must arrive the controller at t=T, and the computation time is not accommodated. It should be noted that, since the computation time is small compared to the sampling period, it can be accommodated by staggering the input and output sampling of the block G(z), thus saving one delay. However, this is not the issue of this chapter.

B. CONSTRAINTS ON THE UNKNOWN PLANT

The implementability of the block diagram of Fig. 4 imposes two major constraints on the unknown plant: The plant must (1) be of minimum phase and (2) have a convergent unit-sample response sequence. The minimum phase requirement arises as the controller transfer function is the inverse of the estimated plant transfer function.

Any non

minimum phase zero of the plant therefore is an unstable pole of controller. Since at this stage of the control scheme the control process is open-loop, a non minimum phase system is a major obstacle. This is common to other existing model-reference and self-tuning adaptive control systems. As non minimum phase effects are also major contributors to the low performance of optimal control systems and modem robust designs, the minimum phase requirement for the unknown plant should not be regarded as a restriction unique :to the technique presented here. It is a restriction to design procedure, however,

298

HSI-HAN YEH ET AL.

The requirement of a convergent unit-sample response sequence arises because the controller is implemented from the output sequence

of the

identifier. A finite memory controller can be implemented only if the output sequence, which is the unit-impulse response sequence of the plant, is convergent. This type of plant includes those that are input-output stable, or have at worst a pole at the origin of the s-plane (a "type 1" system).

C. THE STARTER SIGNAL

A system involving a deconvolufion such as the one in Fig. 4 must be started by applying the signal A(z) first. (Otherwise, the two signals involved in deconvolution wait indefinitely for one another to start.) Therefore A(z) may be called the "starter signal." Equation (9) implies that -A(z) may be regarded as a linear operator which operates on the unit-sample response of a transfer function G(z) to generate the error signals of the adaptive control system. Since the plant is unknown, selection of A(z) can only be made on the basis of the general properties of the plant, such as the convergence and smoothness of the unitsample response. The magnitude of the excitation signal A(z) may be made small to give a small error sequence. However, the practical implementability of

-l/A(z) imposes a lower limit to the magnitude of the time signal represented by A(z). One possible choice of A(z) is

A(z) =-Uo(1 - z"l)

(15)

This starter signal has non zero values at t=0 and t=T only. This A(z) being a differentiator, gives

E(z) - Uo(1 - z"l)G(z)

(16)


en = uo(gn - gn-l)

=

uoAgn

299

(17)

The error signal is the first order difference of the unit-sample response sequence of the unknown plant. The error sequence vanishes as n ~ 0- whether the controlled system is type one or type zero.

The higher the sampling

frequency, the smaller the output error. The error is also proportional to uo, which can be made as small as realistically possible. One can also choose A(z) to be a double differentiator. The choice of a starter signal that will make the system error small and quickly vanishing is practically unlimited.

D. UPDATING THE CONTROLLER PARAMETERS

After all the memory slots in the controller are filled (n > N + 1, the output of the identifier is simply ignored. The system then operates open-loop and the performance as measured by the magnitude of the error signal begins to deteriorate even if the parameters of the plant remain unchanged.

This is

because a growing number of the parameters in G(z) are being approximated by gN- An update scheme is needed for readjusting the values of~l, g2.... gN after the memory storage is filled so that the finite memory controller with the adjusted fgi's may best keep the output error small. The update scheme shown in Fig. 5 is devised heuristically. It is based upon the following reasons: 1. Increasing each term in the convolution sum between ui and ~ in proportion to the output error decreases the output error. 2. In each feedback loop, there must be at least a time delay to account for the computation time. A salient feature of this scheme is that the complicated controller structure of Fig. 5 can be reduced to a simple adjustable-gain controller of Fig. 6, as shown in the following derivation.

300

HSI-HAN YEH ET AL.

Vn+l

n-N+3

1

Un-N+3 Un-1

En T

Cn~

En

c(t)

Unknown System - U 0 5(n)

Fig. 5 Updating Scheme

U 0 5(n)


301

Relating the variables at the adders and subtractors of the diagram of Fig. 5 gives, for 2 < i < N - 1 and n > N

~ i ( n - 1) Un.i+ 1 " gi(n)

~ N ( n - 1)

Un-i+2 = b e n

(18)

fgN(n) Vn = ben

(19)

~'i(n)un+2"i + ~N(n)Vn

(20)

Vn. 1-

N-I rn+2= X 1=1

where 13 and b are design parameters and, ~l(n), ~2(N) ..... ~N(n) now are functions of n as they undergo updating at each sampling instant after the memory storages are filled. Simple algebraic manipulation of these equations give

13r.+l - rn+2 = b(N-1) en + ~l(n-1)Un - ~l(n)un+l

(21)

In view of this equation, it is readily seen that the input-output relation of the system of Fig. 5 is completely determined by the following set of equations:

Cn = Un * gn = Z

giun-i

(22)

i=l

en = rn - ca

en

=

~En.l + b(N-1)% - I]rn+l + rn+2

(23)

(24)

302

HSI-HAN YEH ET AL.

El(n)

= E--an

(26)

Un

~n

(27)

Un+l = ~,l(n)

g

Fig. 6 Reduced Block Diagram of Updating Scheme

These equations are represented by the reduced block diagram of Fig. 6. The stability and convergence of this scheme have not been analyzed. Preliminary simulation results given at the end of this chapter show that the Finite memory adaptive controller with the update scheme performs well on the example.

IV. MODEL REFERENCE CONTROL Fig. 7 shows the configuration of a model reference adaptive control system, in which the unknown plant is denoted by G(z), the model is given by

CONTROLOF UNKNOWNSYSTEMSVIADECONVOLUTION

A.(z) %(z) _

~

303

[ a(z) ~

z

)

Unknown[ H(z) 1-.,, System /

I-" Fig. 7 Model Reference Control Gm(z). The adaptive controller is to be designed so that the plant output follows the model output as closely as possible. The model error Em(z) is given by GeG G Em = 1 + GcGH R + 1 + GcGH A - GmR

(28)

where the z-transform variable is suppressed for brevity's sake. Assume that a Ge is implemented so that the system output would be identical to the model output if A(z) were absent, i.e., GcG = Gm 1 + GcGH

(29)

Then G

Gm

Em=I+GcGH A=~c

A

This means that if Gc can be implemented as a deconvolution given by

(30)

304

HSI-HAN YEH ET AL.

AGm G c - Em

(31)

Then G Em = 1 + GeGH A = (1 - GmH)GA

(32)

I ,z, I zl R(z)

E(~A(z)C~(2) _ z2~(zJ i~(z) ~

A(z) .C(z_ Em(Z) ~(~z)G(z)

i

Fig. 8 Model Reference Control via Deconvolution

When the unknown plant has a convergent unit-sample response, a starter signal A(z) may be chosen so that the model error E m is small and quickly vanishing. The block diagram of the model reference control via deconvolution is given in Fig. 8.

A. REALIZATION OF THE CONTROLLER

The controller in Fig. 8 is drawn on the asstanption that the plant G(z) has one delay, so that the block z -l/Era(z) may be realized as in Fig. 2 with the first non zero signal in Em(z ) arriving at the controller at the end of the first sampling period. The extra delay at the output of the controller is introduced to


305

account for the computational delay required for the digital implementation. As in the self-adaptive control case, this delay may be avoided by using staggered sampling. It is important to note that a correct number of delays must be incorporated into the model Gin(z) in order for the deconvolution controller to be realizable. Assume that the feedback has no delay, and the starter signal A(z) and the reference input R(z) are applied without delay. Then the controller output (before the computation delay) is given by (see Fig, 8) Y(z) =

E(z)A(z)Gm(z)z Em(z)

-

X(z)z -1 Em(z)

I (33)

where E(z) and A(z) have no delay, Era(z) has one delay and the number of delays needed in Gm(z) is to be determined. If Gm(z) has two delays, i.e.,

Gm(z) = gm(2)z "2 + gm(3)z"3 + ...

(34)

X ( z ) -- x 0 4- x 1z- 1 4- ...

(35)

with gm(2) ;~ 0, then

for some x0 ;~ 0. Substituting Eq. (35) and the Taylor series expansion of Em(z) into Eq. (33) gives

Y(z) =

(X o + Xl z ' l + ...)Z -1

1 emlZ" + em2z2 + ...

Therefore at t = 0, the initial signal of Y(z) must be

(36)

306

HSI-HAN YEH ET AL.

Yo= xo

(37)

eml

But eml is the time signal of Era(z) at the end of the first sampling period, i.e., at t=T. Therefore the transfer function from E(z) to Y(z) is anticipatory. It is easily verified that the controller is causal only if the model Gin(z) has three or more delays. If the computation time is accommodate~ by staggered sampling, then one less delay is required for Gin(z). As in the self-adaptive scheme, a starter signal is necessary for the deconvolution controller to operate. The plant also must have minimum phase and have a convergent unit-sample response. After all the memory slots in the controller are filled, the model error is ignored but the system still operates closed-loop. It is possible to adopt an update scheme similar to the self-adaptive case but that has not been attempted in this exploratory study. The following section presents numerical examples for the control schemes developed in this chapter.

V. NUMERICAL EXAMPLES

This section presents the simulation results of applying the adaptive control algorithms developed in this chapter to a generic aircraft model. The plant model is a two-degree-of-freedom short-period approximation to the longitudinal dynamics of a delta-winged configuration. At Mach 0.9 sea level flight condition, the longitudinal dynamics are given by

i (t) =

[ -2.544 1004.4 I x ( t ) + 1"'710"39] -.01545 -3.3270 [-43.534 u(t)

CONTROL OF UNKNOWNSYSTEMSVIA DECONVOLUTION

307

c(0 = [0 1]xCt)

(39)

where c(0 is the pitch rate and u(t) is the control surface deflection. The transfer function of the above state equation is CCs) -4.3543(s + 2.2919) U ( s ) - s 2 + 5.8715s + 23.982

.

.

.

.

.

.

.

.

.

.

.

.

.

X

-!

-

X

-2

o - input x - - output

-3

-4

X

-5

-E;

~ , . ,

I, I ,,, ,

.5

I+ , , , , ,

I

, l.., , , ,

1.5

. , I ,I

2

, , ,,

, I , , ,, ,

2.5

,

.

3

TIME

Fig. 9 Open-Loop Response to Reference Input

3.5

.

308

HSI-HAN YEH ET AL.

The system is descretized with a sampling period of T=.04 second and a zeroorder hold. Fig. 9 shows the response of the uncompensated plant to an input which rises exponentially to a normalized final value after an initial two-period delay, with a time constant equal to 0.28854.

A. SELF-ADAPTIVE CONTROL 1.2

.9 .S .3

xO

x o 0 ( 0

R @

E

-.3

-.6 -.9

o-x u

Input output

1.5

Z

-1.2

-1.s -1.8 95

1

2.5

TIME Fig. 10 Adaptive Control Response

3. S


309

Fig. 10 shows the response of the adaptive system under the same input. The number of memory locations used in the adaptive controller is N=50. The forgetting factor is arbitrarily chosen at f= 1. The starter signal is chosen as A(z) = u0(1-z-l), with u0 = 1. In the first two seconds of operation, the adaptive controller operates in the identification-and-control phase. (The controller in this phase is given by Fig. 4.) At t=2 seconds, the controller is switched over to the update phase which is represented by Fig. 5 with the design parameters chosen to be b = 1/N and [3 = 1. (The simulation is run using Fig. 6, which is equivalent to Fig. 5.) Significant improvement in the output performance due to the adaptive controller is observed by comparing Fig. 10 with Fig. 9. Also, the smooth transition at the switch-over at t=2 as indicated by the smoothness of the output curve in Fig. 10 is worth noting. Since the starter signal A(z) is chosen to be u0(1 - zl), the error signal is given by -uoAgn (See Eq. (17)). In this example, u0 is arbitrarily selected at u0 -1. A much smaller value could have been used. The lower bound of u0 is only limited by the numerical accuracy of the controller implementation (a gain of -l/uo is implemented in the error processor, or identifier, of Fig. 4). Thus, the error signal may be made arbitrarily small as long as the numerical error generated by computations involving u0 and l/u0 is negligible. Note that at t=T, the error signal is exceptionally large (1.62). It is the response of the plant to the starter signal that is arbitrarily chosen. It can be reduced by selecting a smaller U0.

The performance of the self-adaptive control on a time-varying plant has also been simulated. The damping ratio of the plant of Eq. (40) is assumed to vary at a rate of 0.075 r/sec, where r is a random number between 0 and 1. (If r= 1, the damping ratio of the plant would vary from 0.6 to 0.3 in 4 seconds.) Simulation results show that if the plaiat variation starts after the 38th sampling instant (1.5 sec after the starter signal), the adaptive control output follows the

310

H S I - H A N YEH ET AL.

reference trajectory closely with negligible error. The output plot resembles that of Fig. 10 and therefore is not shown.

B. MODEL REFERENCE CONTROL

A simulation run is made for the model reference control system of Fig. 8 using the plant of Eq. (40) for G(s). The system again is discretized with a sampling period of T = 0.04 second and a zero order hold. The model is chosen tobe

Gm(z ) _ (I- ~)z, 3 -

1

~ Z -l

(41)

Its response to a unit-step input is an exponential saturation with three delays, given by Cm(n) = (1 - otn-2) and Cm(0) = Cm(1) = Cm(2) = 0. The starter signal is chosen to be

A(z) = u0 (1

- o~z " l )

(42)

so that (see Fig. 8)

A(z)Gm(z)z 2 = (1 - tz)u0z-I

(43)

The feedback compensator is arbitrarily taken to be H(z) = 1. Fig. 11 shows the model output and the plant output of this system, when the reference input is a unit step, u0 = 0.1 and ot = 0.87055. Fifty memory slots (N=50) are assigned to the convolution controller. The forgetting factor f is arbitrarily chosen to be f--0.5. The numerical result verifies that the model reference control via deconvolution performs well in controlling an unknown plant, as anticipated in the theory.

CONTROL OF UNKNOWN

SYSTEMS VIA DECONVOLUTION

311

- . . . . . . . . . . :- - - - - - ~ - i~ 9- - - ...... ~ _ ~ _ _,_ _ i ~ . . . . . . . . . . . _. . . . . . . . . . . . . . . . . . . . . . . . ~ --~ . . . . . . . . . . . . . . . . . . . ~ - _ ~- _ '

f

....

B

.8

.S

•

o x o

o~model x--system

output output

X 0

.4

X 0 X 0

.2

.it_

.

r -.2

f•

8

'''

i ' I'

.S

.

! I I ' ''

1

.

! ....

I.S

.

I,,

2

.

! s I,,,,

2. S

.

1,

3

,,

.

, I,,

,

3.S

4

TIME

Fig. 11 Response of Model Response Control

VI. COMPARISON WITH MODEL ALGOR/THMIC CONTROL (MAC)

MAC is one of the variants of the heuristic control schemes which f'mds applications in the aerospace industries [2-3, 6-7]. The MAC concept is based on the prediction of the system output using a unit-sample response model of the plant. The MAC system is designed by setting up an algorithm that computes a sequence of future control inputs that will steer the output along a given path.

312

HSI-HAN YEH ET AL.

Since the control sequence is computed in every sampling period, only the nextstep input is used. The deconvolution control shares with MAC the same design philosophy of treating the plant output as convolution of the input and the unitsample response. The main difference between MAC and deconvolution control is that MAC assumes that the unit-sample response of the plant is known, whereas the deconvolution control either identifies the unit-sample response of the plant on-line when it is needed, or generates the desired control sequence via deconvolution without requiring that the plant model be known. The comparison becomes more meaningful when one puts the modelreference deconvolution control of Fig. 8 in the perspective of MAC design. Substitution of Eq. (32) for Em in the system of Fig. 8 gives a single loop system with a controller Gc(z) equals to

Gm(z) Gc(z) = (-I"- Gm(z)H(z))G(zi

(44)

This compares with the unconstrained MAC with closed-loop linear prediction (See Fig. 1) which has a controller 1 - o~

Gc(z) = ~

1

x C,(z)

(45)

where G(z) is the finite unit-sample response model of the plant. Thus, it is seen that the deconvolution controller is able to implement the inverse of the exact plant model in the controller via on-line deconvolution, whereas the MAC system needs to have a unit-sample response model of the plant as a trade-off for the real-time deconvolution, the deconvolution control system must have a starter signal at the input of the plant, and the system model Gm(z) must have


313

sufficient number of delays. The output error of the deconvolution control system is contributed by the starter signal (Eq. (32)) which may be selected to make the error small and quickly vanishing. The output error of the MAC system is due to the error of the plant model G(z). Finally, it is noted that both the deconvolution control and the unconstrained, closed-loop linear MAC are subject to the same restrictions, namely, the plant must be minimum phase and have convergent unit-sample response.

VII. CONCLUSIONS

The model-based controller of the unconstrained model algorithmic control may be implemented by on-line deconvolution, without prior knowledge of the unit-sample response sequence of the plant. On-line deconvolution can also be used to generate control signals that will steer the output of an unknown plant to follow a reference trajectory as closely as the numerical accuracy of the instrumentation allows, provided that the unknown plant is stable and has minimum phase.

VIII. REFERENCES

1. J. Richalaet, A.Rault, J. L. Testud, and J. Papon, "Model Predictive Heuristic Control: Application to Industrial Process Control," Automatica, Vol. 14, 1978, p. 413-419. 2. R. K. Mehra, W. C. Kessel, A. Rault, J. Richalet, and J. Papon, "Model Algorithmic Control Using IDCOM for the F-100 Jet Engine Multivariable

314

HSI-HAN YEH ET AL.

Control Design Problem," in Alternatives for Linear Multivariable Control with

Turbofan Engine Theme Problem, edited by Sain, Peczhowski and Mesa, National Engineering Consortium, Inc. Chicago, 1978, pp. 317-350. 3. R. K. Mehra, R. Rouhani, and L. Praly, "New Theoretical Developments in Multivariable Predictive Algorithmic Control," Proceedings of the 1980 Joint

Automatic Control Conference, pp. 387-392. 4.

J. G. Reid, D. E. Chaffin, and J. T. Silvetthorn, "Output Predictive

Algorithmic Control:

Precision Tracking with Application to Terrain

Following," Journal of Guidance and Control, Vol. 4, No. 5, Sept-Oct 1981, pp. 502-509. 5. M. E. B ise, and J. G. Reid, "Further Application of Output Predictive Algorithmic Control to Terrain Following," Proceedings of the 1984 American

Control Conference, pp. 937-942. 6. W. E. Larimore, and S. Mahmood, "Basic Research on Adaptive Model Algorithmic Control," Technical Report, AFWAL-TR-85-3113, Flight Dynamics Laboratory, Wright-Patterson Air Force Base, Ohio, 1985. 7. J. V. Carroll, and R. F. Gendron, "Vectored Thrust Digital Flight Control For Crew Escape," Technical Report, AFWAL-TR-85-3116, Vols. I-II, Flight Dynamics Laboratory, Wright-Patterson Air Force Base, Ohio, 1985. 8. J. G. Reid, and S. Mahmood, "A State Space Analysis of the Stability and Robusmess Properties of Model Predictive Control," Proceedings of the 1986

American Control Conference, pp. 335-338. 9. J. J. Downs, E. F. Vogel, and V. J. Williams, "Tuning and Sensitivity Study for Model Predictive Control," Proceedings of the 1988 American Control

Conference, pp. 2357-2361. 10. O. A. Asbjorsen, "Stability and Robusmess of the DMC Algorithmic as Compared With the Traditional ARMA Model," Proceedings of the 1988

American Control Conference, pp. 278-283.

INDEX

B-polynomial methods, recursive time delay estimations, sampled-data control systems, 162-166 adaptive dead-time estimator, 164-165 basic extended B-polynomial algorithm, 162-163 other algorithms, 165-166 recursive least squares method, 163-164 simulation experiments, 186-189

Catalytic cracker, discrete, 4 Chemical plant, discrete model, 8-9 Controlled Markov chains, singularly perturbed, s e e Markov chains, singularly perturbed Correlation methods, recursive time delay estimations, sampled-data control systems, 167-171 delay estimation based on correlation analysis, 167-169 fixed model variable regression estimator, 170 other algorithms, 171 simulation experiments, 190-192

Deconvolution, control of unknown systems, 289-314 background, 289-291 conclusions, 313 model algorithmic control versus, 311-313

315

model reference control, 302-306 realization of controller, 304-306 numerical examples, 306--311 model reference control, 310-311 self-adaptive control, 308-310 real,time deconvolution, 291-293 self-adaptive control, 293-302 constraints on unknown plant, 297-298 realization of controller, 296-297 starter signal, 298-299 updating controller parameters, 299-302 Discrete-time systems randomly perturbed, robust stability, 89-120 background, 89-91 colored perturbations, 104-105 discussion, 114-116 jump Markov perturbations, 106-109 white and, 110-114 models and mathematical preliminaries, 91-95 white noise perturbations, 95-104 linear structured, 100-104 unstructured, 98-99 nonlinear, 95-98 stochastic parameters, observer design, 121-158 appendices Lemma 3, 150-151 Lemma 4, 152 Lemma 5, 152-153 discussion, 148-149 full order observation, 124-I 28 Lemma 1,131-133 Lemma 2, 133 Lemma 3, 135 appendix, 150-151

316

INDEX

Lemma 4, 135-136 appendix, 152 Lemma 5, 136 appendix, 152-153 mean square detectability, 130-133 mean square stability, 128-130 notation and mathematical preliminaries, 123-124 reduced order observation, 134-145 linear biased estimation arbitrary order, 137-139 with general observer, 139-141 linear unbiased estimation with general observer, 141-145 robust estimation, 145-148 steady state case, 133-134 Theorem 1,127-128 Theorem 2, 128-129 Theorem 3, 129-130 Theorem 4, 132-133 Theorem 5, 133-134 Theorem 6, 138-139 Theorem 7, 141 Theorem 8, 145-146 Theorem 9, 146-147 weakly coupled, stochastic, 1--40 Lemma 1, 28-31 Lemma 2, 31-32 recursive methods, 2-9 algebraic equation Lyapunov, 2-4 case study: discrete catalytic cracker, 4 Riccati, 4-9 case study: chemical plant model, 8-9 reduced-order controller, 9-18 background, 1-2 case study: distillation column, 17-18 Kalman filter, 13-14 new method, 19-36 filtering problem, 26-32 linear-quadratic control problem, 20-26 optimal, Gaussian, 32-36 linear-quadratic Gaussian optimal control problem, case study: distillation column, 35-36 Theorem t, 4 Theorem 2, 17 Distillation column control problem, 17-18, 35-36

Filtering, see also Kalman filtering discrete-time stochastic systems, 26-32; see also Discrete-time systems, weakly coupled

Jump Markov type parameters, s e e also Discrete-time systems, stochastic parameters, observer design randomly perturbed discrete-time systems, 106-114 K

Kalman filtering stability analysis, 207-236 appendices error representations of floating-point computation, 228-230 proof of Theorem 1,231-233 proof of Theorem 2, 233-235 background, 208-210 conclusions, 227-228 definitions, 214-215 Lemma 1,215-219 Lemma 2, 220 numerical example, 224-227 problem formulation, 210-214 Theorem 1,221-223 proof, 231-233 " Theorem 2, 223-224 proof, 233-235 stochastic systems, noisy and fuzzy measurements, 240-247; see also Stochastic systems, noisy and fuzzy measurements background, 240-241 compression of fuzziness, 245-247 fuzzy parameters, 242-245 review, 240-241 weakly coupled stochastic discrete-time systems, 13-14 L Least squares methods, recursive time delay estimations, sampled-data control systems, 163-164, 171-175

INDEX nonlinear time delay identification, 171-173 simulation experiments, 193-196 estimated model variable regression estimator, 194-196 nonlinear time delay identification, 193-194 Linear-quadratic control problem, weakly coupled discrete systems, 20-26, 32-36; s e e a l s o Discrete-time systems, weakly coupled Linear systems, stochastic, noisy input and output measurements, s e e Stochastic systems, linear, noisy input and output measurements Lyapunov algebraic equation, discrete, parallel algorithm, 2--4; s e e a l s o Discrete-time systems, weakly coupled case study: discrete catalytic cracker, 4

M

Markov chains, singularly perturbed, 257-287 background, 257-258 controlled, 269-284 decentralized algorithms, 274-279 limit Markov control problem, 280-284 aggregation'disaggregation algorithm, 283-284 linear programming algorithm, 283 MIMO, linear stochastic systems, noisy input and output measurements, 51-53 Model algorithmic control, 290 deconvolution and, 311-313 Model reference control, unknown systems, 302-306, 310 realization of controller, 304-306

Noisy input and output measurements, linear stochastic systems, s e e Stochastic systems, linear, noisy input and output measurements

Observer design, discrete-time stochastic parameter systems, s e e Discrete-time systems, stochastic parameters

317

Pad6 approximation, recursive time delay estimations, sampled-data control systems, 176-178, 178-181 simulation experiments, 196-197 Parameter systems, stochastic, discrete-time, s e e Discrete-time systems, stochastic parameters

Rational approximations recursive time delay estimations, sampleddata control systems, 175-182 basic concepts, 175-176 Padr-based delay identification algorithm continuous-time, 178-181 discrete-time, 176--178 simulation experiments, 196-197 sampled-data control systems, recursive time delay estimations, other algorithms, 182 Real-time deconvolution, 291-293 Recursive time delay estimations, sampled-data control systems, s e e Sampled-data control systems, recursive time delay estimations Reduced-order control, stochastic discrete-time weakly coupled large scale systems, s e e Discrete-time systems, weakly coupled Riccati algebraic equation, parallel algorithm, 4-9; s e e a l s o Discrete-time systems, weakly coupled case study: chemical plant model, 8-9 Robust stability, randomly perturbed discretetime systems, s e e Discrete-time systems, randomly perturbed, robust stability

Sampled-data control systems, recursive time delay estimations, 159-206 background, 159-160 conclusions, 201-202 correlation methods, 167-171 9delay estimation, correlation analysis, 167-169 fixed model variable regression estimator, 170 other algorithms, 171 extended B-polynomial methods, 162-166 adaptive dead-time estimator, 164--165 basic extended B-polynomial algorithm, 162-163

318

INDEX

other algorithms, 165-166 recursive least squares method, 163-164 model formulation, 160-161 nonlinear least squares methods, 171-175 estimated model variable regression estimator, 173-175 nonlinear time delay identification, 171-173 rational approximations approach, 175-182 basic concepts, 175-176 continuous-time Pad6-based delay identification algorithm, 178-181 discrete-time Pad6-based delay identifica-, tion algorithm, 176-178 other algorithms, 182 simulation experiments, 185-201 adaptive dead-time estimator, 189-190 algorithm, 197-201 basic extended B-polynomial algorithm, 186-189

correlation analysis, 190-192 correlation methods, 190-192 discrete-time PadE-based delay identification algorithm, 196-197 experimental set-up, 185-186 nonlinear least squares method, 193-196 estimated model variable regression estimator, 194-196 nonlinear time delay identification, 193-194 rational approximations approach, 196-197 variable regressor method, 197-201 variable regressor method, 182-184 simulation experiments, 197-201 time delay estimation algorithm, 182-184 Self-adaptive control, unknown systems, 293-302, 308-310 constraints, 297-298 realization of controller, 296-297 starter signal, 298-299 updating controller parameters, 299-302 Stochastic systems large scale, discrete-time, weakly coupled, s e e Discrete-time systems, weakly coupled linear, noisy input and output measurements, 41-88 appendix: Theorem 5, 85-88 background, 41-45 bispectral approaches, 53-59 consistency, 73-74

integrated, 66-69 linear estimator, 59-63 bispectrum and cross-bispectrum, 60-62 overdetermined, 63 nonlinear estimator, 64--65 conclusions, 80 consistency, 71-75 bispectral approaches, 73-74 integrated polyspectral approaches, 74-75 Lemma 1, 64 Lemma 2, 64 Lemma 3, 64 Lemma 4, 68 Lemma 5, 68-69 Lemma 6, 70 Lemma 7, 70-71 Lemma 8, 71-72 Lemma 9, 85-86 Lemma 10, 86 polyspectral techniques, 53-59 integrated, 65-71 bispectral, 66-69 auto- and cross-bispectrum, 66-67 linear estimator, 66 nonlinear estimator, 68-69 overdetermined linear system of equations, 67-68 consistency, 74-75 trispectral, 69-71 integrated auto- and cross-trispectrum, 69-70 nonlinear estimator, 70-71 simulation examples, 75-79 Theorem l, 50 Theorem 2, 51 Theorem 3, 53 Theorem 4, 73-74 Theorem 5, 74 proof, 86-88 Theorem 6, 74 Theorem 7, 74 Theorem 8, 74-75 Theorem 9, 75 Theorem 10, 75 time-domain technique, 45-53 asymptotic properties, 50-51 fourth cumulant criterion, 48-50 MIMO, 51-53 model assumptions, 45-48 noisy and fuzzy measurements, 237-256 background, 237-241

INDEX fuzzy numbers and fuzzy arithmetic, 238-240 Kalman filter, 240-241 discrete filtering, 247-255 conclusions, 251 example, 251-252, 253-255 Kalman filtering, 240-247 background, 240-241 compression of fuzziness, 245-247 fuzzy parameters, 242-245 review, 240-241

319

Time delay estimations, sampled-data control systems, s e e Sampled-data control systems, recursive time delay estimations W

Weakly coupled large scale systems, discretetime, stochastic, see Discrete-time systems, weakly coupled


Techniques in Discrete-Time Stochastic Control Systems, Volume 73: Advances in Theory and Applications (Advances in Theory & Applications)

Bonivento Advances in Control Theory and Applications

Advances in Control Theory and Applications

Advances in computational intelligence: theory & applications

Recent advances in operator theory and applications

Advances in Computational Intelligence: Theory And Applications

Advances in Queueing Theory and Network Applications

Advances in Computational Intelligence: Theory And Applications

Advances in Queueing Theory and Network Applications

Advances in queueing theory and network applications

Recent Advances in Operator Theory and Applications

Recent Advances in Operator Theory and Applications

Techniques in Discrete and Continuous Robust Systems, Volume 74: Advances in Theory and Applications

Density Functional Theory: Advances in Applications

Advances in Immunology Volume 73

Advances in Genetics, Volume 73

Advances in Immunology Volume 73

Advances in Agronomy, Volume 73

Recent Advances in Operator Theory in Hilbert and Krein Spaces (Operator Theory: Advances and Applications)

Recent Advances in Operator Theory in Hilbert and Krein Spaces (Operator Theory: Advances and Applications)

Multidimensional Systems Signal Processing Algorithms and Application Techniques, Volume 77: Advances in Theory and Applications (Control and Dynamic Systems)

Advances in Ring Theory

New directions and applications in control theory

Advances in moduli theory

Nonlinear Integrals And Their Applications In Data Mining (Advances in Fuzzy Systems-Applications and Theory - Vol. 24) (Advances in Fuzzy Systemss - Applications and Theory)

Advances in Graph Theory

Advances in Moduli Theory

Advances in ring theory

Advances in Ring Theory

Advances in Ring Theory

Recent Advances in Operator Theory and Its Applications: The Israel Gohberg Anniversary Volume (Operator Theory: Advances and Applications)

Techniques in Discrete-Time Stochastic Control Systems, Volume 73: Advances in Theory and Applications (Advances in Theory & Applications)

Bonivento Advances in Control Theory and Applications

Advances in Control Theory and Applications

Advances in computational intelligence: theory & applications

Recent advances in operator theory and applications

Advances in Computational Intelligence: Theory And Applications

Advances in Queueing Theory and Network Applications

Advances in Computational Intelligence: Theory And Applications

Advances in Queueing Theory and Network Applications

Advances in queueing theory and network applications

Recent Advances in Operator Theory and Applications

Recent Advances in Operator Theory and Applications

Techniques in Discrete and Continuous Robust Systems, Volume 74: Advances in Theory and Applications

Density Functional Theory: Advances in Applications

Advances in Immunology Volume 73

Advances in Genetics, Volume 73

Advances in Immunology Volume 73

Advances in Agronomy, Volume 73

Recent Advances in Operator Theory in Hilbert and Krein Spaces (Operator Theory: Advances and Applications)

Recent Advances in Operator Theory in Hilbert and Krein Spaces (Operator Theory: Advances and Applications)

Multidimensional Systems Signal Processing Algorithms and Application Techniques, Volume 77: Advances in Theory and Applications (Control and Dynamic Systems)

Advances in Ring Theory

New directions and applications in control theory

Advances in moduli theory

Nonlinear Integrals And Their Applications In Data Mining (Advances in Fuzzy Systems-Applications and Theory - Vol. 24) (Advances in Fuzzy Systemss - Applications and Theory)

Advances in Graph Theory

Advances in Moduli Theory

Advances in ring theory

Advances in Ring Theory

Advances in Ring Theory

Recent Advances in Operator Theory and Its Applications: The Israel Gohberg Anniversary Volume (Operator Theory: Advances and Applications)

Recommend Documents