New Trends in Optimal Filtering and Control for Polynomial and Time-Delay Systems

Lecture Notes in Control and Information Sciences Editors: M. Thoma, M. Morari

380

Michael Basin

New Trends in Optimal Filtering and Control for Polynomial and Time-Delay Systems

ABC

Series Advisory Board F. Allgöwer, P. Fleming, P. Kokotovic, A.B. Kurzhanski, H. Kwakernaak, A. Rantzer, J.N. Tsitsiklis

Author Michael Basin Dept. of Physical and Mathematical Sciences Autonomous University of Nuevo Leon Av. Universidad s/n. Ciudad Universitaria San Nicolás de los Garza, Nuevo León, C.P. 66451 Mexico E-Mail: [email protected]

ISBN 978-3-540-70802-5

e-ISBN 978-3-540-70803-2

DOI 10.1007/978-3-540-70803-2 Lecture Notes in Control and Information Sciences

ISSN 0170-8643

Library of Congress Control Number: 2008931011 c 2008 

Springer-Verlag Berlin Heidelberg

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed in acid-free paper 543210 springer.com

Preface

0.1 Introduction Although the general optimal solution of the filtering problem for nonlinear state and observation equations confused with white Gaussian noises is given by the Kushner equation for the conditional density of an unobserved state with respect to observations (see [48] or [41], Theorem 6.5, formula (6.79) or [70], Subsection 5.10.5, formula (5.10.23)), there are a very few known examples of nonlinear systems where the Kushner equation can be reduced to a finite-dimensional closed system of filtering equations for a certain number of lower conditional moments. The most famous result, the Kalman-Bucy filter [42], is related to the case of linear state and observation equations, where only two moments, the estimate itself and its variance, form a closed system of filtering equations. However, the optimal nonlinear finite-dimensional filter can be obtained in some other cases, if, for example, the state vector can take only a finite number of admissible states [91] or if the observation equation is linear and the drift term in the state equation satisfies the Riccati equation d f /dx + f 2 = x2 (see [15]). The complete classification of the “general situation” cases (this means that there are no special assumptions on the structure of state and observation equations and the initial conditions), where the optimal nonlinear finite-dimensional filter exists, is given in [95]. There also exists a considerable bibliography on robust filtering for the “general situation” systems (see, for example, [55, 74, 75, 76]). This book presents the technique for designing the optimal finite-dimensional filters for polynomial system states over linear observations. The optimal finite-dimensional filters are consequently obtained for polynomial states, in particular, quadratic, third-order, and fourth-order ones, polynomial state equations with multiplicative noises, polynomial states with partially observed linear parts, and polynomial state equations with multiplicative noises and partially observed linear parts. The optimal filtering problems are treated proceeding from the general expression for the stochastic Ito differential of the optimal estimate and the error variance ([41], Theorem 6.6, or [70], Section 5.10). As the first result, the Ito differentials for the optimal estimate and error variance corresponding to the stated filtering problem are derived. It is then proved that closed finitedimensional systems of the optimal filtering equations with respect to a finite number of

VI

Preface

filtering variables can be obtained for polynomial state equation in each considered problem. The corresponding procedures for designing the optimal filtering equations are established. Finally, the closed system of the optimal finite-dimensional filtering equations with respect to two variables, the optimal estimate and the error variance, are derived in the explicit form in some particular cases, as a rule, for polynomial states of order from one to four and multiplicative polynomial noises of order from one to two. Performance of each designed optimal finite-dimensional filter is demonstrated in a separate illustrative example for, as a rule, quadratic (or bilinear system) states, which are completely observable or have a partially observed linear part, with a quadratic (or bilinear) multiplicative noise over linear observations. The optimal filter performance is verified against a certain optimal filter, based on a reduced system, and a conventional extended Kalman-Bucy filter. In all cases, the simulation results show a definite advantage of the designed optimal filter in regard to proximity of the estimate to the real state value. Moreover, it can frequently be seen that the estimation error produced by the optimal filter rapidly reaches and then maintains the zero mean value even in a close vicinity of the asymptotic time point, although the system state itself is unstable and the quadratic (bilinear) components go to infinity for a finite time. On the contrary, the estimation errors given by other applied filters often diverge to infinity or yield unrealistic estimate values. The obtained optimal filter for a polynomial state equation of degree 3 is applied to solution of the state estimation problem for a nonlinear automotive system [60] whose state equation for car orientation angle is nonlinear (contains tangent). Along with the original state equation, its expansion to Taylor polynomial up to degree 3 is also considered. For both state equations and linear observations, the optimal filtering equations for a polynomial state of third degree are written and then compared to the linear KalmanBucy filter applied to the linearized system. Numerical simulations are conducted for the optimal polynomial filter and also compared to those for the linear Kalman-Bucy filter applied to the linearized system. The simulation results given in the paper show significant advantage of the optimal polynomial filter in comparison to the Kalman-Bucy one. The obtained optimal filter for bilinear system states and linear observations is applied to solution of the terpolymerization process estimation problem in the presence of direct linear observations. The mathematical model of terpolymerization process given in [64] is reduced to ten equations for the concentrations of input reagents, the zeroth live moments of the product molecular weight distribution (MWD), and its first bulk moments. These equations are intrinsically nonlinear (bilinear), so their linearization leads to large deviations from the real system dynamics, as it can be seen from the simulation results. Numerical simulations are conducted for the optimal filter for bilinear system states, the optimal linear filter available for the linearized model, and the mixed filter designed as a combination of those filters. The simulation results show an advantage of the optimal bilinear filter in comparison to the other filters. Based on the obtained optimal filter for polynomial systems with polynomial multiplicative noise, the book presents the optimal finite-dimensional filter for linear system states over polynomial observations. Designing the optimal filter over polynomial observations presents a significant advantage in the filtering theory and practice, since it enables one to address some filtering problems with observation nonlinearities, such as the optimal cubic sensor problem [39]. The optimal filtering problem is again treated

0.1 Introduction

VII

proceeding from the general expression for the stochastic Ito differential of the optimal estimate and the error variance ([41], Theorem 6.6, or [70], Section 5.10). As a result, the Ito differentials for the optimal estimate and error variance corresponding to the stated filtering problem are derived. It is then proved that a closed finite-dimensional system of the optimal filtering equations with respect to a finite number of filtering variables can be obtained for a polynomial observation equation, additionally assuming a conditionally Gaussian initial condition for the third order state. In this case, the corresponding procedure for designing the optimal filtering equations is established. As an illustrative example, the closed system of the optimal filtering equations with respect to two variables, the optimal estimate and the error variance, is derived in the explicit form for the particular case of the third order polynomial observations, assuming a conditionally Gaussian initial condition for the third order state. This filtering problem generalizes the optimal cubic sensor problem stated in [39]. The resulting filter yields a reliable and rapidly converging estimate, in spite of a significant difference in the initial conditions between the state and estimate and very noisy observations, in the situation where the unmeasured state itself is a Wiener process and the extended Kalman filter (EKF) approach fails. The problem of the optimal simultaneous state estimation and parameter identification for stochastic systems with unknown parameters has been receiving systematic treatment beginning from the seminal paper [7]. The optimal result was obtained in [7] for a linear discrete-time system with constant unknown parameters within a finite filtering horizon, using the maximum likelihood principle (see, for example, [71]), in view of a finite set of the state and parameter values at time instants. The application of the maximum likelihood concept was continued for linear discrete-time systems in [28] and linear continuous-time systems in [27]. Nonetheless, the use of the maximum likelihood principle reveals certain limitations in the final result: a. the unknown parameters are assumed constant to avoid complications in the generated optimization problem and b. no direct dynamical (difference) equations can be obtained to track the optimal state and parameter estimates dynamics in the “general situation,” without imposing special assumptions on the system structure. Other approaches are presented by the optimal parameter identification methods without simultaneous state estimation, such as designed in [18, 23, 100], which are also applicable to nonlinear stochastic systems. Another approach, based on the optimization of robust H∞ -filters, has recently been introduced in [76, 78, 92] for linear stochastic systems with bounded uncertainties in coefficients. The overall comment is that, despite a significant number of excellent works in the area of simultaneous state estimation and parameter identification, the optimal state filter and parameter identifier in the form of a closed finite-dimensional system of stochastic ODEs has not yet been obtained even for linear systems. This book presents the optimal filter and parameter identifier for linear stochastic systems with unknown multiplicative and additive parameters over linear observations with an invertible observation matrix, where unknown parameters are considered Wiener processes. The filtering problem is formalized considering the unknown parameters as additional system states satisfying linear stochastic Ito equations with zero drift and unit diffusion. Thus, the problem is reduced to the filtering problem for polynomial (bilinear) system states with partially measured linear part over linear observations,

VIII

Preface

whose solution has already been obtained in a preceding section. The designed optimal filter for the extended state vector also serves as the optimal identifier for the unknown parameters. This presents the optimal algorithm for simultaneous state estimation and parameter identification in linear systems with unknown multiplicative and additive parameters over linear observations. In the illustrative example, performance of the designed optimal filter is verified for a linear system with an unknown multiplicative parameter over linear observations. The simulations are conducted for both, negative and positive, values of the parameter, thus considering stable and unstable linear systems. The simulation results demonstrate reliable performance of the filter: in both cases, the state estimate converges to the real state and the parameter estimate converges to the real parameter value rapidly, in less than 10 time units. The result is viewed even more promising, taking into account large deviations in the initial values for the real state and its estimate and large values of the initial error variances. Although the optimal control (regulator) problem for linear system states was solved, as well as the filtering one, in 1960s [30, 49], the optimal control function for nonlinear systems has to be determined using the general principles of maximum [67] or dynamic programming [14] which do not provide an explicit form for the optimal control in most cases. However, taking into account that the optimal control problem can be solved in the linear case applying the duality principle to the solution of the optimal filtering problem ([49], Section 5.3), the same idea is employed in this book for designing the optimal control in a polynomial system with linear control input, using the optimal filter for polynomial system states over linear observations. This continues a long tradition of the optimal control design for nonlinear systems (see, for example, [1, 37, 51, 53, 73, 90, 96, 101]), but, perhaps, it is the first attempt of using the duality principle to obtain the optimal control for polynomial systems, proceeding from the optimal filter. Based on the polynomial filter for quadratic states and linear observations, obtained in a preceding section, the optimal regulator is obtained for a state of a quadratic system with linear control input and quadratic cost function in a closed form, finding the optimal regulator gain matrix as dual transpose to the optimal filter gain one and constructing the optimal regulator gain equation as dual to the variance equation in the optimal filter. The results obtained by virtue of the duality principle are then rigorously proved through the general equations of the maximum principle [67] applied to this specific quadratic polynomial case, although the physical duality seems obvious: if the optimal filter exists in a closed from, the optimal closed form regulator should also exist, and vice versa. Since the results of the preceding sections yield the possibility to design the optimal filter for any polynomial state over linear observations in a closed form, there consequently exists the possibility to design the optimal regulator for an arbitrary polynomial system with linear control input and quadratic criterion in the same manner: first, deriving the specific form of the optimal control by means of the duality principle and, then, employing the maximum principle for rigorous substantiation. The relatively simple quadratic case is important for practical applications, since a nonlinear state equation can better be approximated by a quadratic polynomial than a linear function, and the control input is, as a rule, linear. Moreover, the optimal control problem for a polynomial quadratic state equation is significant itself, because many, for example, terpolymerization reactor processes are described by quadratic equations

0.1 Introduction

IX

(see [64]). Finally, designed optimal regulator is applied to solution of the optimal control problem for the terpolymerization reactor (see [64] for the model equations) and compared to the best linear regulator available for the linearized model. The simulation results show a definitive advantage of the designed optimal regulator with respect to the the criterion and regulated state values. Based on the obtained polynomial filter of the third degree, the optimal regulator for a polynomial system of degree 3 with linear control input and quadratic cost criterion is obtained in a closed form, finding the optimal regulator gain matrix as dual transpose to the optimal filter gain one and constructing the optimal regulator gain equation as dual to the variance equation in the optimal filter. The results obtained by virtue of the duality principle could be rigorously verified through the general equations of [67] or [14] applied to a specific third degree polynomial case. Finally, the obtained optimal control for a polynomial system of the third degree is applied for regulation of the same automotive system [60], which the optimal filter for third degree polynomial systems was applied to in a preceding section, with the objective of increasing values of the state variables and consuming the minimum control energy. Although the optimal controller problem for linear system states was solved in 1960s, based on the solutions to the optimal filtering [42] and regulator [49, 67] problems, solution to the optimal controller problem for nonlinear (in particular, polynomial) systems has been impossible due to the absence of the solution to the corresponding filtering and control problems for nonlinear systems. Uniting the obtained results in the optimal filtering and control for polynomial systems, the book presents solution to the optimal controller problem for unobservable third degree polynomial system states over linear observations and quadratic criterion. Due to the separation principle for polynomial systems with linear observations and quadratic criterion, which is stated and substantiated analogously to that for linear ones (see [67], Section 5.3), the original controller problem is split into the optimal filtering problem for third degree polynomial system states over linear observations and the optimal control (regulator) problem for observable third degree polynomial system states with quadratic criterion. The case of third degree polynomial systems is even more important for practical applications, since a nonlinear state equation can usually be well approximated by a polynomial of degree 3, the observations are frequently direct, that is linear, and the cost function in the controlling problems, where the desired value of the controlled variable should be maintained or maximized using the minimum control energy, is intrinsically quadratic. Moreover, the controlling problem for a polynomial state equation of lower degree is significant itself, because many, for example, chemical processes are described by quadratic equations (see [51]). The obtained optimal controller for a polynomial state equation of degree 3 is applied to solution of the state controlling problem for a nonlinear automotive system [60] whose state equation for car orientation angle is nonlinear (contains tangent), with the objective of increasing values of the state variables and consuming the minimum control energy. To apply the developed polynomial technique, the original state equation is expanded as a Taylor polynomial, up to degree 3. The optimal controller equations for a polynomial state of third degree are written and then compared to the best linear controller available for the linearized system. Numerical simulations are conducted for

X

Preface

the optimal polynomial controller and also compared to those for the linear one applied to the linearized system. The simulation results show a significant, more than one and half times, advantage of the optimal polynomial controller performance in comparison to the linear one. Recall that the optimal filtering problem for linear system states and observations without delays was solved in 1960s [42], and this closed-form solution is known as the Kalman-Bucy filter. However, the related optimal filtering problem for linear states with delay has not been solved in a closed form, regarding as a closed form solution a closed system of a finite number of ordinary differential equations for any finite filtering horizon. The optimal filtering problem for time delay systems itself did not receive so much attention as its control counterpart, and most of the research was concentrated on the filtering problems with observation delays (the papers [4, 40, 43, 52] could be mentioned to make a reference). A Kalman-like estimator for linear systems with observation delay has recently been designed in [99]. There also exists a considerable bibliography related to the robust filtering problems for time-delay systems (such as [24, 33, 54, 55, 74, 76, 77, 78, 83, 84, 94]). A number of papers, published in 1970s, were dedicated to some particular optimal filtering problems for time-delay systems (see, for example, [45, 50]). Comprehensive reviews of theory and algorithms for time delay systems are given in [16, 21, 24, 35, 46, 47, 56, 62, 72]. This book presents solutions to the optimal filtering problems for linear systems with multiple observation delays, state delay, single state and observation delays, state and multiple observation delays, and multiple state and observation delays. The solutions are obtained proceeding from the general expression for the stochastic Ito differential of the optimal estimate, error variance, and various error covariances ([41], Theorem 6.6, or [70], Section 5.10). As a result, the optimal estimate equation similar to the traditional Kalman-Bucy one is derived in each problem. The optimal filtering equations similar to the traditional Kalman-Bucy ones can be obtained in the case of multiple observation delays, if no state delays are considered. The obtained equations contain specific adjustments in the filter gain matrix and the quadratic term of the variance equation, which are calculated in view of linear functional dependence between the system states taken at different time moments, i.e., using linearity of the state equation. That form of the filtering equations is dual to the Smith predictor [82], commonly used for robust control design in time delay systems. However, for all other filtering problems where state delay exists, it is impossible to obtain a system of the filtering equations, that is closed with respect to the only two variables, the optimal estimate and the error variance, as in the Kalman-Bucy filter. Thus, the resulting system of equations for determining the filter gain matrix consists, in the general case, of an infinite set of equations. It is however demonstrated that a finite set of the filtering equations can be obtained in the particular case of equal or commensurable (τi = qi h) delays in the observation and state equations, where τi are the observation delays, h is the state one, and qi are natural numbers. This finite number of the filtering equations whose number is specified by the ratio between the current filtering horizon and the delay values and increases as the filtering horizon tends to infinity. Performance of each designed optimal filter for a linear system with state and/or observation delays is verified in a separate illustrative example against the best Kalman-Bucy

0.1 Introduction

XI

filter available for linear systems without delays. In all cases, the simulation results show a definite advantage of the designed optimal filter in regard to proximity of the estimate to the real state value and its asymptotic convergence. The book further presents an alternative solution to the optimal filtering problem for linear systems with state delay over linear observations, using the optimal estimate of the state transition matrix from the current time moment to the delayed one. In doing so, the employed method closely resembles the well-known Smith predictor approach [82] (see [58] for more research on this resemblance). As a result, the optimal filter is derived in the form similar to the traditional Kalman-Bucy one, i.e., consists of only two equations, for the optimal estimate and the estimation error variance. This presents a significant advantage in comparison to the previously obtained optimal filter consisting of a variable number of covariance equations, which is specified by the ratio between the current filtering horizon and the delay value in the state equation and unboundedly grows as the filtering horizon tends to infinity. Note that the approach based on the optimal estimation of the state transition matrix would be applicable to any system of state and observation equations with time delays, where the the optimal estimate of the state transition matrix is uncorrelated with the estimation error variance, including certain classes of nonlinear systems. Finally, performance of the designed alternative optimal filter for linear systems with state delay is compared in the illustrative example with the performance of the optimal filter obtained in a preceding section. The simulation results show an insignificant difference in values of the obtained estimates for both filters. The book then presents the optimal filtering problem for nonlinear systems over linear observations with time delay, proceeding from the general expression for the stochastic Ito differential of the optimal estimate and the error variance. The Ito differentials for the optimal estimate and error variance corresponding to the stated filtering problem are derived. It is then proved that a closed finite-dimensional system of the optimal filtering equations with respect to a finite number of filtering variables can be obtained if the state equation is polynomial, the observations are linear, and the observation matrix is invertible. The corresponding procedure for designing the optimal filtering equations is established. Finally, the closed system of the optimal filtering equations with respect to two variables, the optimal estimate and the error variance, is derived in the explicit form for the particular case of a bilinear state equation. In the illustrative example, performance of the designed optimal filter is verified for a quadratic state over linear observations with delay against the best filter available for a quadratic state over linear observations without delay, obtained in a preceding section, and the conventional extended Kalman-Bucy filter. The simulation results show a definite advantage of the designed optimal filter in regard to proximity of the estimate to the real state value. Moreover, it can be seen that the estimate produced by the optimal filter converges to the real values of the reference variables as time tends to the asymptotic time point, although the quadratic system state itself is unstable and even goes to infinity for a finite time. Although the optimal control (regulator) problem for linear systems without delays was solved, as well as the filtering one, in 1960s [30, 49], the optimal control problem for linear systems with delays is still open, depending on the delay type, specific

XII

Preface

system equations, criterion, etc. Discussing the maximum principle [67] or the dynamic programming method [14] for systems with delays, the reference books on time delay systems [16, 21, 46, 47, 56] note that finding an explicit form of a particular optimal control function might still remain difficult. A particular form of the criterion must also be taken into account: the studies mostly focused on the time-optimal criterion (see the paper [65] for linear systems) or the quadratic one [2, 20, 26, 57, 85, 98]. Virtually all studies of the optimal control in time-delay systems are related to systems with delays in the state, although the case of delays in control input is no less challenging, if the control function should be causal, i.e., does not depend on the future values of the state. There also exists a considerable bibliography for the robust control problem for time delay systems (such as [24, 54]). The current state-of-the-art in the time delay system theory has recently been reviewed in two survey papers [35, 72]. In the optimal control area, the book presents the solution of the optimal control problem for a linear system with multiple delays in control input and a quadratic criterion. The solution is obtained as a feedback control law linear in state, whose gain matrix satisfies a Riccati equation. Optimality of the solution is proved in two steps. First, the necessary optimality condition is derived from the maximum principle [44, 67]. Then, the Hamilton-Jabobi-Bellman equation, following from the dynamic programming method [14, 66], is satisfied. The last result implies sufficiency of the necessary optimality condition, robustness of the solution with respect to parameter variations, and its time-consistency. The time-consistency property (see [13]) means that the obtained control, being applied by any participant of a differential game, leads at every moment t to such changes in the game cost function that are considered optimal from the viewpoint of this participant. The obtained optimal regulator makes an advance with respect to general optimality results for time delay systems (such as given in [2, 20, 26, 57, 85]). First, the optimal control law is given explicitly and not as a solution of a system of integro-differential or PDE equations. Second, the Riccati equation for the gain matrix does not contain any time advanced or delayed arguments and does not depend on the state variables. This leads to a conventional two points boundary-valued problem generated in the LQ optimal control problems with finite horizon (see, for example, [49], Chapter 3). Thus, the obtained optimal regulator is realizable using a delay-differential equation for the state and an ordinary differential one for the gain matrix. Taking into account that the state space of a delayed system is infinite-dimensional ([56], Section 2.3), this seems to be a significant advantage. Performance of the obtained optimal regulator is verified in the illustrative example against the optimal LQ regulator for linear systems without delays and some other feasible feedback regulators linear in state. The simulation results show an advantage of the obtained optimal regulator in the criterion value. The book establishes duality between the solutions of the optimal filtering problem for linear systems over observations with multiple delays and the optimal LQ control problem for linear systems with multiple time delays in control input. For this purpose, the optimal filtering equations for a linear state equation over linear observations with multiple time delays, obtained in a preceding section, are briefly reviewed. Then, both results are compared and discussed. It is established that the duality between the

0.1 Introduction

XIII

solutions to both problems really exists: the pairs of dual functions are explicitly indicated and the time inversion between the problems is pointed out. This result naturally generalizes the solution to the optimal control problem for linear systems with one delay in control input, obtained in [11], to the case of multiple delays in control input. It should be also noted that the delay values can also be variable: neither the resulting equations nor the derivation technique would be changed. The book now concentrates on the solution of the optimal control problem for a linear system with equal delays in state and input and a quadratic criterion. Using the maximum principle [44, 67], the solution to the stated optimal control problem is obtained in a closed form, i.e., it is represented as a linear feedback control law, whose gain matrix satisfies an ordinary differential (quasi-Riccati) equation, which does not contain time-advanced or delayed arguments and does not depend on the state variables. The obtained optimal regulator makes an advance with respect to general optimality results for time delay systems (such as given in [2, 46, 47, 56]), since it is realizable using only two equations: a delay-differential equation for the state and an ordinary differential one for the gain matrix. Taking into account that the state space of a delayed system is infinite-dimensional, this should again be a significant advantage. Performance of the obtained optimal control for a linear system with equal delays in state and input and a quadratic criterion is verified in the illustrative example against the best linear regulator available for the linear system without delays. The simulation results show a significant (about twenty times) difference in the values of the cost function in favor of the obtained optimal regulator. Moreover, the obtained optimal regulator is compared to the best linear regulator based on a rational approximation of the original time-delay system. In this case, the simulations show that the approximationbased regulator generates an increase in the problem dimension, producing additional computational difficulties, and yields still unsatisfactory values of the cost function in comparison to the optimal regulator. The book turns to the solution of the optimal control problem for a linear system with multiple state and/or input delays and a quadratic criterion. The original problem is reduced to a conventional linear-quadratic regulator (LQR) problem for systems without delays, using a number of the state and/or control transition matrices from the current time moment to a delayed one for the original time-delay system. In doing so, the employed method closely resembles the well-known Smith predictor approach [82] (see [58] for more research on this resemblance). Since those state and control transition matrices, taken at the current time moment, are known time-dependent functions, the solution is found using the optimal linear-quadratic regulator [49]. As a result, the solution to the original optimal control problem is obtained in a closed form, i.e., it is represented as a linear feedback control law, whose gain matrix satisfies an ordinary differential Riccati equation. The latter does not contain time-advanced arguments, but its coefficients depend on the state variables. The obtained optimal regulator makes an advance with respect to general optimality results for time-delay systems (such as given in [2, 46, 47, 56]), since it is realizable using only two delay-differential coupled equations: one for the state and another for the gain matrix. Taking into account that the state space of a delayed system is infinite-dimensional [56], this again constitutes a significant advantage. Since the obtained Riccati equation does not contain time-advanced

XIV

Preface

arguments, it can be numerically solved using simple methods, such as “shooting,” which consists in varying the initial conditions until the terminal one is satisfied. At the first glance, it may be apprehended that the conventional LQR theory is inapplicable to the reduced optimal control problem, since the obtained state and control transition matrices seem to depend on the system states and may be unbounded. This question is specially addressed and clarified in the corresponding remarks. It should be also noted that the optimal control is indeed obtained as a linear currenttime feedback control, although in some papers (see, for example, [2]) it was derived as an integral of the system state over the delay interval. Comments on this result are also given in the corresponding remarks. Performance of the obtained optimal control for a linear system with multiple state and/or delays and a quadratic criterion is verified in the illustrative example against the best linear regulator available for the linear system without delays. The simulation results show definitive differences in the values of the cost function in favor of the obtained optimal regulator. Moreover, the obtained optimal regulator is compared to the best linear regulators based on two rational approximations, linear and quadratic, of the original time-delay system. In both cases, the simulations show that the approximationbased regulators yield unsatisfactory values of the cost function in comparison to the optimal regulator. In addition, the quadratic approximation generates an increase in the problem dimension, producing additional computational difficulties. Uniting the obtained results in the optimal filtering and control for linear time-delay systems, the book presents solution to the optimal controller problem for unobserved linear system states with input delay, linear observations with delay confused with white Gaussian noises, and a quadratic criterion. Due to the separation principle for linear systems with input and observation delays and a quadratic criterion, which is stated and substantiated analogously to that for linear systems without delays (see [67], Section 5.3), the original controller problem is split into the optimal filtering problem for linear system states over linear observations with delay and the optimal control (regulator) problem for linear system states with input delay. The obtained optimal controller equations contain specific adjustments for time delays in the filter and regulator gain matrices and the quadratic terms of the variance and regulator gain matrix equations, which are calculated in view of linear functional dependence between the system states taken at different time moments, i.e., using linearity of the state equation. That form of the controller equations closely resembles the Smith predictor [82], commonly used for robust control design in time delay systems. It should be also noted that both delay values can also be variable: neither the resulting equations nor the derivation technique would be changed. Performance of the obtained optimal controller for a linear system with input and observation delays and is verified in the illustrative example against the best linear controller available for linear systems without delays. The simulation results show a definite advantage of the obtained optimal controller in the criterion value, the values of the controlled variable and its controlled estimate, and the proximity of the controlled estimate to the real value of the controlled variable. One of the most important estimation and control problems is functioning under heavy uncertainty conditions. Although there are a number of sophisticated methods

0.1 Introduction

XV

like adaptation based on identification and observation, or absolute stability methods, the most obvious way to withstand the uncertainty is to keep some constraints by brutal force. The most simple way to obtain this is to immediately react to any deviation from the real system state and apply sufficient energy to suppress a deviation. Sliding modes as a phenomenon present in dynamic systems lead to ordinary differential equations with discontinuities and, therefore, to systems with variable structure. The proper concept of sliding modes appeared in the context of relay-based control systems. It may happen that the control as a function of the system state switches at high, theoretically infinite frequency, and this motion is called sliding mode. Application of the variable structure systems and sliding mode technique to design of state observers is actively studied nowadays. The first sliding mode observers were designed for linear systems [86]. In recent years, the most attention has been paid to linear and nonlinear uncertain systems with bounded disturbances (see, for example, [3, 25, 59, 81, 89]). The observer design based on higher order sliding modes has been discussed in [31] and references therein. Application of the sliding mode technique to design of filters for stochastic system states was initiated in [22]. Other recent applications of the sliding mode approach to stochastic systems can be found in [5, 63, 68, 79, 93]. The identification and robust control design using the sliding mode approach has been receiving a considerable attention in recent years. A particular interest is given to the integral sliding mode (ISM) technique, which maintains the state trajectory on the sliding manifold from the initial time moment, thus assuring insensitivity against matched uncertainties. The ISM robust control design was initiated in [87] and has been continued for linear deterministic regulators in [10, 80] and linear stochastic filters in [9, 12]. The book designs an integral sliding mode regulator robustifying the obtained optimal controller for linear systems with input and observation delays and a quadratic criterion, which has been obtained in a preceding section. The idea is to add two compensators, one to the optimal control and another to the observation process, to suppress external disturbances in the state and observation equations, respectively, that deteriorate the optimal system behavior. The integral sliding mode compensators are realized as relay controls in a such way that the sliding mode motion starts from the initial moment, thus solving the problem of compensation for external disturbances from the beginning of system functioning, without any transitional phase. This constitutes the crucial advantage of the integral sliding modes in comparison to the conventional ones. Moreover, if certain matching conditions hold, the designed compensator in the state equation can simultaneously suppress observation disturbances, as well as the designed compensator in the observation equation can simultaneously suppress state disturbances. Note that in the framework of this modified (in comparison to [10, 88]) integral sliding mode approach, the optimal control is not required to be differentiable and the sliding mode manifold matrix is always invertible. An example illustrates the quality of simultaneous disturbance suppression in state and observation equations, provided by one robust integral sliding mode control compensator in the observation equation, in comparison to performance the optimal controller in the presence of disturbances. The book then presents a solution to the optimal control problem for a linear system with state delay and a quadratic criterion. Using the maximum principle [44, 67],

XVI

Preface

the solution to the stated optimal control problem is obtained in a closed form, i.e., it is represented as a linear in state control law, whose gain matrix satisfies an ordinary differential (quasi-Riccati) equation, which does not contain time-advanced arguments and does not depend on the state variables. The obtained optimal regulator makes an advance with respect to general optimality results for time delay systems (such as given in [2, 20, 26, 66, 85]), since (a) the optimal control law is given explicitly and not as a solution of a system of integro-differential or PDE equations, and (b) the quasi-Riccati equation for the gain matrix does not contain any time advanced arguments and does not depend on the state variables and, therefore, leads to a conventional two points boundaryvalued problem generated in the optimal control problems with quadratic criterion and finite horizon (see, for example, [49], Chapter 3). Thus, the obtained optimal regulator is realizable using two delay-differential equations. Taking into account that the state space of a delayed system is infinite-dimensional, this constitutes a significant advantage. Performance of the obtained optimal control for a linear system with state delay and a quadratic criterion is verified in the illustrative example against the best linear regulators available for the system without delay and the first-order approximation of the original state-delay system. The simulation results show a definitive advantage of the obtained optimal regulator in the criterion value. Finally, the book designs an integral sliding mode regulator robustifying the optimal regulator for linear systems with state delay and a quadratic criterion. The idea is to add a compensator to the known optimal control to suppress external disturbances deteriorating the optimal system behavior. The integral sliding mode compensator is realized as a relay control in a such way that the sliding mode motion starts from the initial moment, thus eliminating the matched uncertainties from the beginning of system functioning. The proposed solution to the optimal control problem for linear state-delay system assumes that the system state is completely measured. Nonetheless, the obtained result can be readily extended to the case of unmeasured system states, using the results presented previously in this book: the optimal filter for linear systems with state delay and the separation principle for linear time-delay systems. An example illustrates the quality of control provided by the obtained optimal regulator for linear systems with state delay against the best linear regulators available for the system without delay and the first-order approximation of the original state-delay system. Simulation graphs and comparison tables demonstrating better performance of the obtained optimal regulator are included. The example continues illustrating the quality of disturbance suppression provided by the obtained robust integral sliding mode regulator against the optimal regulator under the presence of disturbances. Very satisfactory results are obtained. 0.1.1

Organization of This Book

Chapter 1 designs the finite-dimensional optimal mean-square filters for various classes of polynomial systems over linear observations. The problem statement for polynomial system states over linear observations is declared in Section 1.1. The optimal finite-dimensional filters are consequently obtained for polynomial states, in particular, quadratic, third-order, and fourth-order ones in Section 1.1, polynomial state equations with multiplicative noises in Section 1.2,

0.1 Introduction

XVII

polynomial states with partially observed linear parts in Section 1.3, and polynomial state equations with multiplicative noises and partially observed linear parts in Section 1.4. Each of the obtained filters is verified in a separate illustrative example against some optimal filter, based on a reduced system, and and a conventional extended KalmanBucy filter. The obtained optimal filter for bilinear system states and linear observations is applied to solutions of the state estimation problem for a nonlinear automotive system and the terpolymerization process estimation problem in the presence of linear observations in Section 1.1. The optimal finite-dimensional filter for linear system states over polynomial observations is obtained in Section 1.5. As an illustrative example, the closed system of the optimal filtering equations with respect to two variables, the optimal estimate and the error variance, is derived in the explicit form for the particular case of the third order polynomial observations, assuming a conditionally Gaussian initial condition for the third order state. Chapter 2 presents further results based on the optimal mean-square filtering for polynomial systems: the optimal joint state filter and parameter identifier for linear stochastic systems with unknown multiplicative and additive parameters and the dual optimal regulator and controller for polynomial systems with linear control input. The optimal joint state filtering and parameter identification problem for linear stochastic systems with unknown multiplicative and additive parameters over linear observations with an invertible observation matrix, where unknown parameters are considered Wiener processes, are studied in Section 2.1. In the illustrative example, performance of the designed optimal filter is verified for a linear system with an unknown multiplicative parameter over linear observations. The simulations are conducted for both, negative and positive, values of the parameter, thus considering stable and unstable linear systems. Turning to the optimal control problems for polynomial systems with linear control input, Section 2.2 presents the optimal regulator is obtained for a state of a quadratic system with linear control input and quadratic cost function. The obtained optimal regulator is applied to solution of the optimal control problem for the terpolymerization reactor and compared to the best linear regulator available for the linearized model. The optimal regulator for a polynomial system of degree 3 with linear control input and quadratic cost criterion is obtained and applied for regulation of the automotive system, with the objective of increasing values of the state variables and consuming the minimum control energy. Section 2.3 presents solution to the optimal controller problem for unobservable third degree polynomial system states over linear observations and quadratic criterion. Due to the separation principle for polynomial systems with linear observations and quadratic criterion, the original controller problem is split into the optimal filtering problem for third degree polynomial system states over linear observations and the optimal control (regulator) problem for observable third degree polynomial system states with quadratic criterion. The obtained optimal controller for a polynomial state equation of degree 3 is applied to solution of the state controlling problem for a nonlinear automotive system, with the objective of increasing values of the state variables and consuming the minimum control energy.

XVIII

Preface

Chapter 3 designs the optimal finite-dimensional filters for linear and some nonlinear time-delay system states over linear time-delay observations, assuming single or multiple delays in states and/or observations. This chapter presents solutions to the optimal filtering problems for linear systems with multiple observation delays in Section 3.1, linear systems with a single state delay in Section 3.2, linear systems with single state and observation delays in Section 3.3, linear systems with single state and multiple observation delays in Section 3.4, and linear systems with multiple state and observation delays in Section 3.5. Performance of each designed optimal filter for a linear system with state and/or observation delays is verified in a separate illustrative example against the best Kalman-Bucy filter available for linear systems without delays. An alternative solution to the optimal filtering problem for linear systems with state delay over linear observations is presented in Section 3.6, using the optimal estimate of the state transition matrix from the current time moment to the delayed one. As a result, the optimal filter is derived in the form similar to the traditional Kalman-Bucy one, i.e., consists of only two equations, for the optimal estimate and the estimation error variance. Performance of the designed alternative optimal filter for linear systems with state delay is compared in the illustrative example with the performance of the optimal filter obtained in Section 3.2. The optimal filtering problem for nonlinear systems over linear observations with time delay is presented in Section 3.7, proceeding from the general expression for the stochastic Ito differential of the optimal estimate and the error variance. Performance of the designed optimal filter is verified for a quadratic state over linear observations with delay against the best filter available for a quadratic state over linear observations without delay, obtained in Section 1.1, and the conventional extended Kalman-Bucy filter. Chapter 4 presents the optimal finite-dimensional regulators and controllers for linear time-delay systems, assuming single or multiple delays in states and/or control inputs. This chapter presents the solution of the optimal control problem for a linear system with multiple delays in control input and a quadratic criterion in Section 4.1. Performance of the obtained optimal regulator is verified in the illustrative example against the optimal LQ regulator for linear systems without delays and some other feasible feedback regulators linear in state. This section also establishes duality between the solutions of the optimal filtering problem for linear systems over observations with multiple delays and the optimal LQ control problem for linear systems with multiple time delays in control input. Section 4.2 focuses on the solution of the optimal control problem for a linear system with equal delays in state and input and a quadratic criterion. Performance of the obtained optimal control for a linear system with equal delays in state and input and a quadratic criterion is verified in the illustrative example against the best linear regulator available for the linear system without delays. The solutions to the optimal control problem for a linear system with a) multiple state delays, and b) multiple state and input delays, and a quadratic criterion are presented in Sections 4.3 and 4.4, respectively. In both cases, the original problem is reduced to a conventional linear-quadratic regulator (LQR) problem for systems without delays,

0.1 Introduction

XIX

using a number of the state and/or control transition matrices from the current time moment to a delayed one for the original time-delay system. Performance of the obtained optimal regulators for linear systems with multiple state and/or delays and a quadratic criterion is verified in the illustrative example against the best linear regulators available for the linear system without delays and two rational approximations, linear and quadratic, of the original time-delay system. Uniting the obtained results in the optimal filtering and control for linear time-delay systems, Section 4.5 presents solution to the optimal controller problem for unobserved linear system states with input delay, linear observations with delay confused with white Gaussian noises, and a quadratic criterion. Due to the separation principle for linear systems with input and observation delays and a quadratic criterion, the original controller problem is split into the optimal filtering problem for linear system states over linear observations with delay and the optimal control (regulator) problem for linear system states with input delay. Performance of the obtained optimal controller for a linear system with input and observation delays and is verified in the illustrative example against the best linear controller available for linear systems without delays. Chapter 5 applies the sliding mode control technique to robustification of some of the designed optimal finite-dimensional regulators and controllers for linear time-delay systems, assuming the presence of external disturbances in the system equations. This results in compensating (eliminating) the parasitic disturbances in the sliding mode, which ensures successful realization of the optimal control algorithm. Section 5.1 designs an integral sliding mode regulator robustifying the obtained optimal controller for linear systems with input and observation delays and a quadratic criterion. The idea is to add two compensators, one to the optimal control and another to the observation process, to suppress external disturbances in the state and observation equations, respectively, that deteriorate the optimal system behavior. An example illustrates the quality of simultaneous disturbance suppression in state and observation equations, provided by one robust integral sliding mode control compensator in the observation equation, in comparison to performance the optimal controller in the presence of disturbances. Section 5.2 presents a solution to the optimal control problem for a linear system with state delay and a quadratic criterion. Performance of the obtained optimal control for a linear system with state delay and a quadratic criterion is verified in the illustrative example against the best linear regulators available for the system without delay and the first-order approximation of the original state-delay system. Then, this section designs integral sliding mode regulator robustifying the optimal regulator for linear systems with state delay and a quadratic criterion. The example continues illustrating the quality of disturbance suppression provided by the obtained robust integral sliding mode regulator against the optimal regulator under the presence of disturbances. 0.1.2

References to Theoretical Background

After a double thought, it was decided not to write a special section exposing the theoretical background for this book. Indeed, there are excellent textbooks and original research monographs, which would serve for presenting the state-of-the-art and providing the background much better. It is a real pleasure to refer to them as the theoretical

XX

Preface

sources; multiple references are also given in the main text of the book, directly indicating sections, theorems, or pages to be consulted. The theory of stochastic processes and the general filtering theory are brilliantly exposed in two books, [41], a classical textbook written in the end of 1960s, and [70], which is generalized translated version of a Russian book, originally published in 1984. The main results of Chapters 1 and 2 are derived form the background filtering theory theorems given in those two sources. A minor number of the basic probability theory propositions are referred to another book [69], which is a verbatim translation of a classical Russian probability theory textbook, published in 1978. The background principles of the optimal control theory, dynamic programming and maximum principle, are referred to their original sources, [67] and [14], respectively, or to a comprehensive optimal control textbook [30]. The specific results from the linear optimal control theory are referred to a 1970s milestone book [49] or a later textbook [19]. All those books are well known to control scientists. Some facts from nonlinear mechanics, which are also useful for nonlinear optimal control, are given according to [32], which is a translation of a classical Russian analytical mechanics textbook. The basic concepts from the time-delay systems theory are given according to a textbook [56], published in 1987. Some existence and uniqueness propositions related to differential equations with time-delayed arguments are referred to research monographs [29] and [38], which are also popular among the time-delay systems community. The definitions and methods of the sliding mode control theory are referred to original monographs [86] and [88], whose first author is a worldwide recognized founder of the sliding mode approach. Some recently designed tools, such as integral sliding mode observers, are presented according to a comprehensive survey [31].

Contents

1

Optimal Filtering for Polynomial Systems . . . . . . . . . . . . . . . . . . 1.1 Filtering Problem for Polynomial State over Linear Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.2 Optimal Filter for Polynomial State over Linear Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.3 Optimal Third-Order State Filter for Automotive System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.4 State Estimation of Bilinear Terpolymerization Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Filtering Problem for Polynomial State with Partially Measured Linear Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Optimal Filter for Polynomial State with Partially Measured Linear Part over Linear Observations . . . . . . . . 1.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Filtering Problem for Polynomial State with Multiplicative Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Optimal Filter for Polynomial State with Multiplicative Noise over Linear Observations . . . . . . . . . . . . . . . . . . . . . . . 1.3.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Filtering Problem for Polynomial State with Partially Measured Linear Part and Multiplicative Noise . . . . . . . . . . . . . . . 1.4.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4.2 Optimal Filter for Polynomial State with Partially Measured Linear Part and Polynomial Multiplicative Noise over Linear Observations . . . . . . . . . . . . . . . . . . . . . . . 1.4.3 Cubic Sensor Optimal Filtering Problem . . . . . . . . . . . . . .

1 1 1 2 8 10 17 17 18 21 24 24 25 29 33 33 35 38

XXII

Contents

1.5 Filtering Problem for Linear State over Polynomial Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.2 Optimal Filter for Linear State over Polynomial Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.3 Example: Third-Order Sensor Filtering Problem . . . . . . . . 2

3

Further Results: Optimal Identification and Control Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Optimal Joint State and Parameter Identification Problem for Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Optimal State Filter and Parameter Identifier for Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Dual Optimal Control Problems for Polynomial Systems . . . . . . . 2.2.1 Optimal Control Problem for Bilinear State with Linear Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Optimal Regulator for Terpolymerization Reactor . . . . . . 2.2.3 Optimal Control for Third-Order Polynomial State with Linear Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.4 Optimal Third-Order Polynomial Regulator for Automotive System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Optimal Controller Problem for Third-Order Polynomial Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 Separation Principle for Polynomial Systems . . . . . . . . . . . 2.3.3 Optimal Controller Problem Solution . . . . . . . . . . . . . . . . . 2.3.4 Optimal Third-Order Polynomial Controller for Automotive System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimal Filtering Problems for Time-Delay Systems . . . . . . . . 3.1 Filtering Problem over Observations with Multiple Delays . . . . . 3.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Optimal Filter over Observations with Multiple Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Filtering Problem for Linear Systems with State Delay . . . . . . . . 3.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Optimal Filter for Linear Systems with State Delay . . . . . 3.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Filtering Problem for Linear Systems with State and Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Optimal Filter for Linear Systems with State and Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

40 40 41 43 47 47 47 48 50 53 53 56 61 62 65 65 66 68 69 75 75 75 76 80 84 84 85 89 94 94 95

Contents

3.4

3.5

3.6 3.7

4

XXIII

3.3.3 Optimal Filter for Linear Systems with Commensurable State and Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . 3.3.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Filtering Problem for Linear Systems with State and Multiple Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Optimal Filter for Linear Systems with State and Multiple Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 Optimal Filter for Linear Systems with Commensurable State and Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . 3.4.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Filtering Problem for Linear Systems with Multiple State and Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Optimal Filter for Linear Systems with Multiple State and Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alternative Optimal Filter for Linear State Delay Systems . . . . . 3.6.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Filtering Problem for Nonlinear State over Delayed Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.2 Optimal Filter for Nonlinear State over Delayed Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Optimal Control Problems for Time-Delay Systems . . . . . . . . . 4.1 Optimal Control Problem for Linear Systems with Multiple Input Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 Optimal Control Problem Solution . . . . . . . . . . . . . . . . . . . 4.1.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.4 Proof of Optimal Control Problem Solution . . . . . . . . . . . . 4.1.5 Duality between Filtering and Control Problems for Time-Delay Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Optimal Control Problem for Linear Systems with Equal State and Input Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Optimal Control Problem Solution . . . . . . . . . . . . . . . . . . . 4.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 Proof of Optimal Control Problem Solution . . . . . . . . . . . . 4.3 Optimal Control Problem for Linear Systems with Multiple State Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 Optimal Control Problem Solution . . . . . . . . . . . . . . . . . . . 4.3.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

98 99 103 103 103 105 108 110 113 113 114 118 120 122 122 123 127 131 131 131 132 132 134 138 141 141 141 142 146 148 148 148 150

XXIV

Contents

4.3.4 Proof of Optimal Control Problem Solution . . . . . . . . . . . . 4.4 Optimal Control Problem for Linear Systems with Multiple State and Input Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.2 Optimal Control Problem Solution . . . . . . . . . . . . . . . . . . . 4.4.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.4 Proof of Optimal Control Problem Solution . . . . . . . . . . . . 4.5 Optimal Controller Problem for Linear Systems with Input and Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.2 Separation Principle for Time-Delay Systems . . . . . . . . . . 4.5.3 Optimal Control Problem Solution . . . . . . . . . . . . . . . . . . . 4.5.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Sliding Mode Applications to Optimal Filtering and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Optimal Robust Sliding Mode Controller for Linear Systems with Input and Observation Delays . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 Design Principles for State Disturbance Compensator . . . 5.1.3 Design Principles for Observation Disturbance Compensator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.4 Robust Sliding Mode Controller Design for Linear System with Input and Observation Delays . . . . . . . . . . . . 5.1.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Optimal and Robust Control for Linear State Delay Systems . . . 5.2.1 Optimal Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Optimal Control Problem Solution . . . . . . . . . . . . . . . . . . . 5.2.3 Robust Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.4 Design Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.5 Robust Sliding Mode Control Design for Linear State Delay Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.7 Proof of Optimal Control Problem Solution . . . . . . . . . . . .

153 156 156 157 159 163 166 166 167 168 168 175 175 175 176 177 179 181 185 185 185 186 187 189 189 192

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

1 Optimal Filtering for Polynomial Systems

1.1 Filtering Problem for Polynomial State over Linear Observations 1.1.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ t0 , and let (W1 (t), Ft ,t ≥ t0 ) and (W2 (t), Ft ,t ≥ t0 ) be independent Wiener processes. The Ft -measurable random process (x(t), y(t)) is described by a nonlinear differential equation with polynomial drift for the system state dx(t) = f (x,t)dt + b(t)dW1(t),

x(t0 ) = x0 ,

(1.1)

and a linear differential equation for the observation process dy(t) = (A0 (t) + A(t)x(t))dt + B(t)dW2(t).

(1.2)

Here, x(t) ∈ Rn is the state vector and y(t) ∈ Rn is the linear observation vector, such that the matrix A(t) ∈ Rn×n is invertible. The initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t), and W2 (t) are independent. It is assumed that B(t)BT (t) is a positive definite matrix. All coefficients in (1.1)–(1.2) are deterministic functions of time of appropriate dimensions. The nonlinear function f (x,t) is considered polynomials of n variables, components of the state vector x(t) ∈ Rn , with time-dependent coefficients. Since x(t) ∈ Rn is a vector, this requires a special definition of the polynomial for n > 1. Generally, a pdegree polynomial of a vector x(t) ∈ Rn is regarded as a p-linear form of n components of x(t) f (x,t) = a0 (t) + a1(t)x + a2 (t)xxT + . . . + a p(t)x . . . p times . . . x, (1.3) where a0 (t) is a vector of dimension n, a1 is a matrix of dimension, a2 is a 3D tensor of dimension n × n × n, a p is an (p + 1)D tensor of dimension n × . . .(p+1) times . . . × n, and x × . . . p times . . . × x is a pD tensor of dimension n × . . . p times . . . × n obtained by p times spatial multiplication of the vector x(t) by itself. Such a polynomial can also be expressed in the summation form M. Basin: New Trends in Optimal Filtering, LNCIS 380, pp. 1–45, 2008. c Springer-Verlag Berlin Heidelberg 2008 springerlink.com 

2

1 Optimal Filtering for Polynomial Systems

fk (x,t) = a0 k (t) + ∑ a1 ki (t)xi (t) + ∑ a2 ki j (t)xi (t)x j (t) + . . . i

+

∑

a p ki1 ...i p (t)xi1 (t) . . . xi p (t),

ij

k, i, j, i1 , . . . , i p = 1, . . . , n.

i1 ...i p

The estimation problem is to find the optimal estimate x(t) ˆ of the system state x(t), based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, that minimizes the Euclidean 2-norm ˆ | FtY ] J = E[(x(t) − x(t)) ˆ T (x(t) − x(t)) at every time moment t. Here, E[z(t) | FtY ] means the conditional expectation of a ˆ with respect to the σ - algebra FtY stochastic process z(t) = (x(t) − x(t)) ˆ T (x(t) − x(t)) generated by the observation process Y (t) in the interval [t0 ,t]. As known from ([41], Theorem 5.3 or [70], Subsection 5.10.2), this optimal estimate is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY ) of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As usual, the matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differential of the conditional expectation E(x(t) | FtY ) and its variance P(t) and given in the following subsection. 1.1.2

Optimal Filter for Polynomial State over Linear Observations

The optimal filtering equations could be obtained using the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E( f (x,t) | FtY )dt + E(x[ϕ1 (x,t) − E(ϕ1 (x,t) | FtY )]T | FtY )×  −1 B(t)BT (t) (dy(t) − E(ϕ1(x,t) | FtY )dt), where f (x,t) is the polynomial drift term in the state equation, and ϕ1 (x,t) is the linear drift term in the observation equation equal to ϕ1 (x,t) = A0 (t) + A(t)x(t). Upon performing substitution, the estimate equation takes the form dm(t) = E( f (x,t) | FtY )dt + E(x(t)[A(t)(x(t) − m(t))]T | FtY )× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt) = E( f (x,t) | FtY )dt + E(x(t)(x(t) − m(t))T | FtY )AT (t)× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt) = E( f (x,t) | FtY )dt + P(t)AT (t)(B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt).

(1.4)

1.1 Filtering Problem for Polynomial State over Linear Observations

3

The equation (1.4) should be complemented with the initial condition m(t0 ) = E(x(t0 ) | FtY0 ). Trying to compose a closed system of the filtering equations, the equation (1.4) should be complemented with the equation for the error variance P(t). For this purpose, the formula for the Ito differential of the variance P(t) = E((x(t)− m(t))(x(t)− m(t))T | FtY ) could be used (see ([41], Theorem 6.6, formula (6.101) or [70], Subsection 5.10.9, formula (5.10.42))): dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY ) + E( f (x,t)(x(t) − m(t))T ) | FtY )+ b(t)bT (t) − E(x(t)[ϕ1 (x,t) − E(ϕ1 (x,t) | FtY )]T | FtY )×  −1 B(t)BT (t) E([ϕ1 (x,t) − E(ϕ1 (x) | FtY )]xT (t) | FtY ))dt+ E((x(t) − m(t))(x(t) − m(t))[ϕ1 (x,t) − E(ϕ1 (x,t) | FtY )]T | FtY )×  −1 B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where the last term should be understood as a 3D tensor (under the expectation sign) convoluted with a vector, which yields a matrix. Upon substituting the expressions for ϕ1 , the last formula takes the form dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY ) + E( f (x,t)(x(t) − m(t))T ) | FtY )+ b(t)bT (t) − (E(x(t)(x(t) − m(t))T | FtY )AT (t)× (B(t)BT (t))−1 A(t)E((x(t) − m(t))xT (t)) | FtY ))dt+ E((x(t) − m(t))(x(t) − m(t))(A(t)(x(t) − m(t)))T | FtY )(B(t)BT (t))−1 × (dy(t) − A(t)m(t))dt). Using the variance formula P(t) = E((x(t) − m(t))xT (t)) | FtY ), the last equation can be represented as dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY ) + E( f (x,t)(x(t) − m(t))T ) | FtY ) +

(1.5)

b(t)bT (t) − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt+ E(((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY )× AT (t)(B(t)BT (t))−1 (dy(t) − A(t)m(t − h))dt). The equation (5) should be complemented with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. The equations (1.4) and (1.5) for the optimal estimate m(t) and the error variance P(t) form a non-closed system of the filtering equations for the nonlinear state (1.1) over linear observations (1.2). The non-closeness means that the system (1.4),(1.5) includes terms depending on x, such as E( f (x,t) | FtY ) and E((x(t)− m(t))( f (x,t))T | FtY ), which are not expressed yet as functions of the system variables, m(t) and P(t). Let us prove now that this system becomes a closed system of the filtering equations in view of the polynomial properties of the function f (x,t) in the equation (1.1).

4

1 Optimal Filtering for Polynomial Systems

Indeed, since the observation equation is linear, the innovations process    ν (t) = y(t) − tt0 (A0 (t) + A(t)m(t))dt = tt0 (A0 (t) + A(t)x(t))dt + tt0 B(t)dW2 (t) − tt0 (A0 (t) +   A(t)m(t))dt = tt0 (A(t)(x(t) − m(t)))dt + tt0 B(t)dW2 (t) is a Wiener process with respect to observations (see, for example,[61], formula (31) and the paragraph afterwards). Moreover, the observation noise tt0 B(t)dW2 (t) is considered a Wiener process with respect to observations. Hence, the random variable A(t)(x(t) − m(t)) is conditionally Gaussian with respect to observations for any t ≥ t0 . If the matrix A−1 exists, then the random vector (x(t) − m(t)) is also conditionally Gaussian for any t ≥ t0 ([69], Section 5.3). Thus, the following considerations are applicable to the filtering equations (1.4),(1.5). First, the conditional third moment E((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY ) of x(t) − m(t) with respect to observations, which stands in the last term of the equation (1.5), is equal to zero, because the process x(t) − m(t) is conditionally Gaussian. Thus, the entire last term in (1.5) is vanished and the following variance equation is obtained dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY ) + E( f (x,t)(x(t) − m(t))T ) | FtY ) +

(1.6)

b(t)bT (t) − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt, with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. Second, if the function f (x,t) is a polynomial function of the state x with timedependent coefficients, the expressions of the terms E( f (x,t) | FtY ) in (1.4) and E((x(t) − m(t)) f T (x,t)) | FtY ) in (1.6) would also include only polynomial terms of x. Then, those polynomial terms can be represented as functions of m(t) and P(t) using the following property of Gaussian random variable x(t) − m(t): all its odd conditional moments, m1 = E[(x(t)−m(t)) | Y (t)], m3 = E[(x(t)−m(t)3 | Y (t)], m5 = E[(x(t)−m(t))5 | Y (t)], ... are equal to 0, and all its even conditional moments m2 = E[(x(t) − m(t))2 | Y (t)], m4 = E[(x(t) − m(t))4 | Y (t)], .... can be represented as functions of the variance P(t). For example, m2 = P, m4 = 3P2 , m6 = 15P3, ... etc. After representing all polynomial terms in (1.4) and (1.6), that are generated upon expressing E( f (x,t) | FtY ) and E((x(t) − m(t)) f T (x,t)) | FtY ), as functions of m(t) and P(t), a closed form of the filtering equations would be obtained. The corresponding representations of E( f (x,t) | FtY ) and E((x(t) − m(t))( f (x,t))T | FtY ) will be further derived for certain polynomial functions f (x,t). In the next subsections, a closed form of the filtering equations will be obtained from (1.4) and (1.6) for bilinear, third-order, and fourth-order functions f (x,t) in the equation (1.1). It should be noted, however, that application of the same procedure would result in designing a closed system of the filtering equations for any polynomial function f (x,t) in (1.1). Optimal Filter for Linear State: Kalman-Bucy Filter In a particular case, if the function f (x,t) = a0 (t) + a1(t)x(t) is linear, the representations for E( f (x,t) | FtY ) and E((x(t) − m(t))( f (x,t))T | FtY ) as functions of m(t) and P(t) are derived as follows E( f (x,t) | FtY ) = a0 (t) + a1(t)m(t),

(1.7)

1.1 Filtering Problem for Polynomial State over Linear Observations

5

E( f (x,t)(x(t) − m(t))T ) | FtY ) + E((x(t) − m(t))( f (x,t))T | FtY ) = a1 (t)P(t) + P(t)aT1 (t).

(1.8)

Substituting the expression (1.7) into (1.4) and the expression (1.8) into (1.6), the filtering equations for the optimal estimate m(t) and the error variance P(t) are obtained dm(t) = (a0 (t) + a1(t)m(t))dt +

(1.9)

P(t)AT (t)(B(t)BT (t))−1 [dy(t) − (A0(t) + A(t)m(t))dt], m(t0 ) = E(x(t0 ) | FtY )), dP(t) = (a1 (t)P(t) + P(t)aT1 (t) +

(1.10)

−1

b(t)b (t) − P(t)A (t)(B(t)B (t)) A(t)P(t))dt. T

T

T

P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )). The equations (1.9)–(1.10) form the well-known Kalman-Bucy filter [42]. Note that the observation matrix A(t) should not be necessarily invertible to obtain the filtering equations (1.9)–(1.10), since no polynomial equalities mentioned in the previous subsection are used for the derivation. Optimal Filter for Bilinear State Let the state drift function f (x,t) = a0 (t) + a1(t)x + a2 (t)xxT

(1.11)

be a bilinear polynomial, where x is an n-dimensional vector, a0 (t) is an n-dimensional vector, a1 (t) is an n × n - matrix, a2 (t) is a 3D tensor of dimension n × n × n. In this case, the representations for E( f (x,t) | FtY ) and E((x(t) − m(t))( f (x,t))T | FtY ) as functions of m(t) and P(t) are derived as follows E( f (x,t) | FtY ) = a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t),

(1.12)

E( f (x,t)(x(t) − m(t))T ) | FtY ) + E((x(t) − m(t))( f (x,t))T | FtY ) = a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T .

(1.13)

Substituting the expression (1.12) into (1.4) and the expression (1.13) in (1.6), the filtering equations for the optimal estimate m(t) and the error variance P(t) are obtained dm(t) = (a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t))dt + P(t)AT (t)(B(t)BT (t))−1 [dy(t) − (A0(t) + A(t)m(t))dt],

(1.14)

m(t0 ) = E(x(t0 ) | FtY )),

dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T + b(t)bT (t) − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt. P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )).

(1.15)

6

1 Optimal Filtering for Polynomial Systems

By means of the preceding derivation, the following result is proved. Theorem 1.1. The optimal finite-dimensional filter for the bilinear state with bilinear multiplicative noise (1.1), where the bilinear polynomial f (x,t) is defined by (1.11), over the linear observations (1.2), is given by the equation (1.14) for the optimal estimate m(t) = E(x(t) | FtY ) and the equation (1.15) for the estimation error variance P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ]. Thus, based on the general non-closed system of the filtering equations (1.4),(1.6), it is proved that the closed system of the filtering equations can be obtained for any polynomial state with a polynomial multiplicative noise (1.1) over linear observations (1.2). Furthermore, the specific form (1.14),(1.15) of the closed system of the filtering equations corresponding to a bilinear state is derived. Optimal Filter for Third-Order State Let now the state drift function f (x,t) = a0 (t) + a1(t)x(t) + a2(t)x2 (t) + a3(t)x3 (t)

(1.16)

be a third degree polynomial of vector x ∈ Rn , which is defined differently in comparison to the preceding subsections. Namely, a superior power of a vector is defined as the vector of superior powers of each component of the original vector, i.e., x2 (t) = (x21 (t), x22 (t), . . . , x2n (t)), x3 (t) = (x31 (t), x32 (t), . . . , x3n (t)), etc; ak (t), k = 1, 2, 3, . . ., are square n × n - dimensional matrices. The following notation is introduced. Let m(t) = E(x(t)) | FtY ) ∈ Rn be the optimal estimate vector, m(t) = m1 (t), m2 (t), .., mn (t)), and P(t) ∈ Rn be the error variance matrix. Superior powers of m(t) are defined as m2 (t) = (m21 (t), m22 (t), . . . , m2n (t)), m3 (t) = (m31 (t), m32 (t), . . . , m3n (t)), etc. Let p(t) ∈ Rn be the vector whose components are the error variances of the components of x(t), i.e., the diagonal elements of P(t), P(t)m(t) be the conventional product of a matrix P(t) by a vector m(t), and p(t)∗m(t) be the product of two vectors by components: p(t) ∗ m(t) = [p1 (t)m1 (t), p2 (t)m2 (t), ..., pn (t)mn (t)]. In this case, using the equalities E[(x(t) − m(t))2 | Y (t)] = p(t), E[(x(t) − m(t))3 | Y (t)] = 0, and E[(x(t) − m(t))4 | Y (t)] = 3p2 (t), the representations for E(xn (t) | FtY ), n = 2, 3, 4, are derived as follows E[x2 (t) | Y (t)] = p(t) + m2(t), E[x3 (t) | Y (t)] = 3p(t) ∗ m(t) + m3(t), E[x4 (t) | Y (t)] = 3p2 (t) + 6p(t) ∗ m2(t) + m4(t).

(1.17)

Substituting the obtained expressions (1.17) into the equations (1.4) and (1.6), where the drift function f (x,t) is given by (1.16), the filtering equations for the optimal estimate m(t) and the error variance P(t) are obtained dm(t) = (a0 (t) + a1 (t)m(t) + a2 (t)p(t) + a2 (t)m2 (t) + a3 (t)(3p(t) ∗ m(t) + m3 (t)))dt + (1.18) P(t)AT (t)(B(t)BT (t))−1 [dy(t) − (A0(t) + A(t)m(t))dt], m(t0 ) = E(x(t0 ) | FtY )),

1.1 Filtering Problem for Polynomial State over Linear Observations

dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t) ∗ P(t) +

7

(1.19)

2(P(t) ∗ m (t))aT2 (t) + 3a3(t)(p(t) ∗ P(t)) + 3(p(t) ∗ P(t))T aT3 (t)+ 3a3 (t)(m2 (t) ∗ P(t)) + 3(P(t) ∗ (m2(t))T )aT3 (t)+ T T T −1 T

b(t)b (t) − P(t)A (t)(B(t)B (t)) A(t)P(t))dt,

P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )). Here, the product m(t) ∗ P(t) between a vector m(t) and a matrix P(t) is defined as the matrix whose rows are equal to rows of P(t) multiplied by the same corresponding element of m(t): ⎤ ⎡ m1 (t) P11 (t) P12 (t) · · · P1n (t) ⎥ ⎢ ⎢ m2 (t) P21 (t) P22 (t) · · · P2n (t) ⎥ ⎢ . .. ⎥ .. .. . . ⎥= ⎢ . . ⎣ . . ⎦ . . mn (t) Pn1 (t) Pn2 (t) · · · Pnn (t) ⎤ ⎡ m1 (t)P11 (t) m1 (t)P12 (t) · · · m1 (t)P1n (t) ⎥ ⎢ ⎢ m2 (t)P21 (t) m2 (t)P22 (t) · · · m2 (t)P2n (t) ⎥ ⎥, ⎢ .. .. .. .. ⎥ ⎢ . ⎦ ⎣ . . . mn (t)Pn1 (t) mn (t)Pn2 (t) · · · mn (t)Pnn (t) and the transposed matrix P(t) ∗ mT (t) = (m(t) ∗ P(t))T is defined as the matrix whose columns are equal to columns of P(t) multiplied by the same corresponding element of m(t). Thus, the equation (1.18) for the optimal estimate m(t) and the equation (1.19) for the error variance matrix P(t) form a closed system of filtering equations in the case of a third-order polynomial state equation and linear observations. Optimal Filter for Fourth-Order State Let the state drift function f (x,t) = a0 (t) + a1(t)x(t) + a2(t)x2 (t) + a3(t)x3 (t) + a4(t)x4 (t)

(1.20)

be a fourth degree polynomial of vector x ∈ Rn , which is defined in the same manner as in the preceding subsection. The notation introduced there is also employed. In this case, using the equality E[(x(t) − m(t))5 | Y (t)] = 0, the representation for E(x5 (t) | FtY ) is derived as follows E[x5 (t) | Y (t)] = 15m(t) ∗ p2(t) + 10p(t) ∗ m3(t) + m5(t). Substituting the last expression and the expressions (1.17) obtained in the preceding subsection into the equations (1.4) and (1.6), where the drift function f (x,t) is given by (1.20), the filtering equations for the optimal estimate m(t) and the error variance P(t) are obtained dm(t) = (a0 (t) + a1(t)m(t) + a2(t)p(t) + a2(t)m2 (t) +

(1.21)

8

1 Optimal Filtering for Polynomial Systems

a3 (t)(3p(t) ∗ m(t) + m3(t)) + 3a4(t)p2 (t) + 6a4(t)p(t) ∗ m2(t) + a4(t)m4 (t)))dt+ P(t)AT (t)(B(t)BT (t))−1 [dy(t) − (A0(t) + A(t)m(t))dt],

m(t0 ) = E(x(t0 ) | FtY )),

dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t) ∗ P(t) +

(1.22)

2(P(t) ∗ m (t))aT2 (t) + 3a3(t)(p(t) ∗ P(t)) + 3(p(t) ∗ P(t))T aT3 (t)+ 3a3 (t)(m2 (t) ∗ P(t)) + 3(P(t) ∗ (m2(t))T )aT3 (t)+ 12a4 (t)((m(t) ∗ p(t)) ∗ P(t)) + 12(P(t) ∗ (m(t) ∗ p(t))T )(a4 (t))T + T

4a4(t)(m3 (t) ∗ P(t)) + 4(P(t) ∗ (m3(t))T )(a4 (t))T + b(t)bT (t) − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt, P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )). Remark. If we continue obtaining the filters for polynomial state equations of degrees 5, 6, etc., the corresponding equations for the estimate m(t) and the variance P(t) would contain the terms of the preceding lower degrees, complemented with new terms. In other words, the filtering equations for the quadratic state contain all terms of the linear filtering equations, plus new quadratic terms, the filtering equations for the cubic state contain all terms of the linear and quadratic filtering equations, plus new cubic terms, and so on. 1.1.3

Optimal Third-Order State Filter for Automotive System

This subsection presents application of the obtained filter for a third degree polynomial state over linear observations to estimation of the state variables, orientation and steering angles, in a nonlinear kinematical model of car movement [60] satisfying the following equations dx(t)/dt = v cos φ (t), dy(t)/dt = v sin φ (t), d φ (t)/dt = (v/l) tan δ (t),

(1.23)

d δ (t)/dt = u(t). Here, x(t) and y(t) are Cartesian coordinates of the mass center of the car, φ (t) is the orientation angle, u is the velocity, l is the longitude between the two axes of the car, δ (t) is the steering wheel angle, and u(t) is the control variable (steering angular velocity). The zero initial conditions for all variables are assumed. The problem is to find the optimal estimates for the variables φ (t) and δ (t), using direct linear observations confused with independent and identically distributed disturbances modeled as white Gaussian noises. The corresponding observations equations are (1.24) dzφ (t) = φ (t)dt + dW1 (t), dzδ (t) = δ (t)dt + dW2 (t), where zφ (t) is the observation variable for φ (t), zδ (t) is the observation variable for δ (t), and W1 (t) and W2 (t) are independent standard Wiener process, whose weak mean

1.1 Filtering Problem for Polynomial State over Linear Observations

9

square derivatives are white Gaussian noises (see ([41], Subsection 3.8, or [70], Subsection 4.6.5)). To apply the obtained filtering algorithms to the nonlinear system (1.23) and linear observations (1.24), let us make the Taylor expansion of the two last equations in (1.23) at the origin up to degree 3 (the fourth degree does not appear in the Taylor series for tangent) v v δ 3 (t) ), (1.25) d φ (t)/dt = ( )δ (t) + ( )( l l 3 d δ (t)/dt = u(t). The filtering equations (1.18) and (1.19) for the third degree polynomial state (1.25) over linear observations (1.24) take the form v v dmφ = (( )mδ + ( )(3pδ + m3δ ))dt + pφ φ (dzφ − mφ dt) + pφ δ (dzδ − mδ dt), l 3l dmδ = u(t)dt + pδ φ (dzφ − mφ dt) + pδ δ (dzδ − mδ dt), d pφ φ /dt = (2v/l)pδ φ pδ δ +

2v 2v p + m2φ pδ φ − p2φ φ − p2φ δ , l δφ l

(1.26)

v v d pφ δ /dt = pδ δ + m2δ pδ δ − pφ φ pφ δ − pφ δ pδ δ , l l d pδ δ /dt = −p2δ φ − p2δ δ , where mφ and mδ are the estimates for variables φ and δ , and pφ φ , pφ δ , pδ δ are elements of the symmetric covariance matrix P. The estimates obtained upon solving the equations (1.26) are compared the the conventional Kalman-Bucy estimates satisfies the following Kalman-Bucy filtering equations for a state of the linearized system (only the linear term is present in the Taylor expansion for tangent) over linear observations (1.24) v dmφ = mδ dt + pφ φ (dzφ − mφ dt) + pφ δ (dzδ − mδ dt), l dmδ = u(t)dt + pδ φ (dzφ − mφ dt) + pδ δ (dzδ − mδ dt), d pφ φ /dt =

2v p − p2φ φ − p2φ δ , l δφ

(1.27)

v d pφ δ /dt = pδ δ − pφ φ pφ δ − pφ δ pδ δ , l d pδ δ /dt = −p2δ φ − p2δ δ . Numerical simulation results are obtained solving the systems of filtering equations (1.26) and (1.27). The obtained values of the estimates mφ and mδ are compared to the real values of the variables mφ and mδ in the original system (1.23). Thus, the following graphs are obtained: graphs of variables φ and δ for the original system (1.23) (Figs. 1.1 and 1.2); graphs of the Kalman-Bucy filter estimates mφ and mδ satisfying the equations (1.27) (Fig. 1.1); and graphs of the optimal third degree polynomial filter estimates mφ and mδ satisfies the equations (1.26) (Fig. 1.2).

10

1 Optimal Filtering for Polynomial Systems

For each of the two filters and reference system involved in simulation, the following values of the input variables and initial values are assigned: u = 1, l = 1, u(t) = 0.05, mφ (0) = 10, mδ (0) = 0.1, φ (0) = δ (0) = 0, Pφ φ (0) = 100, Pφ δ (0) = 10, Pδ δ (0) = 1. Gaussian disturbances w1 (t) and w2 (t) in (1.24) are realized as the built-in MatLab white noise functions. The obtained values of the reference variables φ and δ satisfying the polynomial approximation system (1.25) are compared to the Kalman-Bucy filter and optimal third degree polynomial filter estimates mφ and mδ at the terminal time T = 20 in the following table (corresponding to Figs. 1.1 and 1.2). Kalman-Bucy filter Third degree polynomial filter φ (20) = 12.3 φ (20) = 12.5 δ (20) = 1 δ (20) = 1 mφ (20) = 7.35 mφ (20) = 11.83 mδ (20) = 0.61 mδ (20) = 0.905 The simulation results show that the estimates obtained by using the optimal third degree polynomial filter are much closer to the real values of the reference variables than those obtained by using the conventional Kalman-Bucy linear filter. Although this conclusion follows from the developed theory, the numerical comparison serves as a convincing illustration. 1.1.4

State Estimation of Bilinear Terpolymerization Process

The optimal filter for bilinear system states and linear observations, obtained in Subsection 1.1.2.2, is now applied to solution of the state estimation problem for a terpolymerization process in the presence of direct linear observations. The mathematical model of terpolymerization process given in [64] is reduced to ten equations for the concentrations of input reagents, the zeroth live moments of the product molecular weight distribution (MWD), and its first bulk moments. These equations are intrinsically nonlinear (bilinear), so their linearization leads to large deviations from the real system dynamics, as it can be seen from the simulation results. Actually, the assumption that the MWD moments can be measured in the real time is artificial, since this can be done only with large time delays; however, at this step, the objective is to verify the performance of the obtained filtering algorithm for a nonlinear system and compare it to other filtering algorithms based on the linearized model. Let us rewrite the bilinear state equation and the linear observation equation in the component form using index summations dxk (t)/dt =a0k (t)+ ∑ a1ki (t)xi (t)+∑ a2ki j (t)xi (t)x j (t)+ ∑ bki (t)ψ1i (t), i

ij

yk (t) = ∑ Aki (t)xi (t) + ∑ Bki (t)ψ2i (t), i

i

k=1, . . . , n,

i

A0k = 0, k = 1, . . . , n,

1.1 Filtering Problem for Polynomial State over Linear Observations

1.5

10

1

phi

delta

15

5 0

0.5

0

10 time

0

20

10

10 time

20

0

10 time

20

0.5

mdelta

mphi

0

1

5 0 −5

11

0

−0.5 0

10 time

20

−1

Fig. 1.1. Graphs of variables φ (rad.) and δ (rad.) for the original system (1.23) against time (sec.); graphs of the Kalman-Bucy filter estimates mφ (rad.) and mδ (rad.), satisfying the equations (1.27), against time (sec.)

where ψ1 (t) and ψ2 (t) are white Gaussian noises. Then, the filtering equations (1.14),(1.15) can be rewritten in the component form as follows: dmk (t)/dt = a0k (t) + ∑ a1ki (t)mi (t) + ∑ a2ki j (t)mi (t)m j (t) + i

(1.28)

ij

∑ a2ki j (t)Pi j (t))dt + ∑ Pk j (t)ATjl (t)(Bl p (t)B ps(t))−1 [ys (t) − ∑ Asr (t)mr (t)], ij

r

i jl ps

mk (t0 ) = E[xk (t0 ) | Y (t0 )]; dPi j (t)/dt = ∑ a1ik (t)Pk j (t) + ∑ Pki (t)a1 jk (t) + 2 ∑ a2ikl (t)ml (t)Pk j + k

k

2 ∑ a2 jkl (t)ml (t)Pki (t) + ∑ bik (t)bk j (t) − kl

k

(1.29)

kl

∑ Pik (t)ATkl (t)(Bl p (t)B ps(t))−1 Asr (t)Pr j (t),

kl psr

Pi j (t0 ) = E[(xi (t0 ) − mi (t0 ))(x j (t0 ) − m j (t0 ))T | Y (t0 )]. The terpolymerization process model reduced to ten bilinear equations selected from [64] is given by dCm1 /dt = (1/V )(d Δm1 /dt) − ((1/θ ) + KL1C∗ + K11 μPo + K21 μQo + K31 μRo )Cm1 ; (1.30) dCm2 /dt = (1/V )(d Δm2 /dt) − ((1/θ ) + KL2C∗ + K12 μPo + K22 μQo )Cm2 ; dCm3 /dt = (1/V )(d Δm3 /dt) − ((1/θ ) + K13μPo )Cm3 ;

1 Optimal Filtering for Polynomial Systems

1.5

10

1

delta

15

phi

12

5

0

10 time

20

0

15

1

10

0.5

5

mdelta

mphi

0

0.5

0

−0.5

−5

0

10 time

20

0

10 time

20

0

0

10 time

20

−1

Fig. 1.2. Graphs of variables φ (rad.) y δ (rad.) for the original system (1.23) against time (sec.); graphs of the third degree polynomial filter estimates mφ (rad.) y mδ (rad.), satisfying the equations (1.26), against time (sec.)

dC∗ /dt = (1/V )(d Δm∗ /dt) − ((1/θ ) + Kd + KL1Cm1 + KL2Cm2 )C∗ ; d μPo /dt = (−1/θ − Kt1 )μPo + KL1Cm1C∗ − (K12Cm2 + K13Cm3 )μPo + K21Cm1 μQo + K31Cm1 μRo ; d μQo /dt = (−1/θ )μQo + KL2Cm2C∗ − (K21Cm1 + Kt2 )μQo + K12Cm2 μPo ; d μRo /dt = (−1/θ )μRo − (K31Cm1 + Kt3 )μRo + K13Cm3 μPo ; d λ1100 /dt = (−1/θ )λ1100 + KL1Cm1C∗ + KL2Cm2C∗ + K11Cm1 μPo + K21Cm1 μQo + K31Cm1 μRo ; d λ1010 /dt = (−1/θ )λ1010 + KL1Cm1C∗ + KL2Cm2C∗ + K12Cm2 μPo + K22Cm2 μQo ; d λ1001 /dt = (−1/θ )λ1001 + (KL1Cm1 + KL2Cm2 )C∗ + K13Cm3 μPo . Here, the state variables are: Cm1 , Cm2 , and Cm3 are the reagent (monomer) concentrations, C∗ is the active catalyst concentration; μPo , μQo , and μRo are the zeroth live moments of the product MWD, and λ1100 , λ1010 , and λ1001 are its first bulk moments. The reactor volume V and residence time θ , as well as all coefficients K’s, are known parameters, and Δm1 , Δm2 , Δm3 , Δm∗ stand for net molar flows of the reagents and active catalyst into the reactor.

1.1 Filtering Problem for Polynomial State over Linear Observations

13

The estimation (filtering) problem is to find the optimal estimate for the unmeasured states (1.30) assuming that the direct observations yi confused with Gaussian noises ψ2 ’s are provided for each of the ten state components xi yi = xi + ψ2i ,

i = 1, . . . , 10.

Here, x1 denotes Cm1 , x2 denotes Cm2 , and so on up to x10 = λ1001 . In this situation, the bilinear filtering equations (1.28) for the vector of the optimal estimates m(t) take the form dm1 (t)/dt = (1/V )(d Δm1 /dt) − ((1/θ ) + KL1m4 (t) + K11 m5 (t) + K21 m6 (t)

(1.31)

+K31 m7 (t))m1 (t)−KL1 P14 (t)−K11 P15 (t)−K21 P16 (t)−K31 P17 (t)+ ∑ P1 j [y j (t)−m j (t)]; j

dm2 (t)/dt = (1/V )(d Δm2 /dt) − ((1/θ ) + KL2m4 (t) + K12 m5 (t)+ K22 m6 (t))m2 (t) − KL2 P24 (t) − K12P25 (t) − K22 P26 (t) + ∑ P2 j [y j (t) − m j (t)]; j

dm3 (t)/dt = (1/V )(d Δm3 /dt) − ((1/θ ) + K13m5 (t))m3 (t)− K13 P35 (t) + ∑ P3 j [y j (t) − m j (t)]; j

dm4 (t)/dt = (1/V )(d Δm∗ /dt) − ((1/θ ) + Kd + KL1 m1 (t)+ K12 m2 (t))m4 (t) − KL1 P14 (t) − K12P24 (t) + ∑ P4 j [y j (t) − m j (t)]; j

dm5 (t)/dt = (−1/θ − Kt1 )m5 (t) + KL1 m4 (t)m1 (t) − K12 m2 (t)m5 (t) + K21 m6 (t)m1 (t)+ K31 m7 (t)m1 (t) − K13m5 (t)m3 (t) + KL1 P14 (t) + K21 P16 (t) + K31P17 (t) − K12 P25 (t)− K13 P35 (t) + ∑ P5 j [y j (t) − m j (t)]; j

dm6 (t)/dt = (−1/θ − Kt2 − K21 m1 (t))m6 (t) + KL2 m4 (t)m2 (t) + K12 m5 (t)m2 (t) −K21 P16 (t) + KL2 P24(t) + K12 P25 (t) + ∑ P6 j [y j (t) − m j (t)]; j

dm7 (t)/dt = (−1/θ − Kt3 − K31 m1 (t))m7 (t) + K13 m5 (t)m3 (t)− K31 P17 (t) + K13P35 (t) + ∑ P7 j [y j (t) − m j (t)]; j

dm8 (t)/dt = (−1/θ )m8 (t) + (KL1 m4 (t) + K11m5 (t) + K21 m6 (t) + K31m7 (t))m1 (t)+ KL2 m4 (t)m2 (t) + KL1 P14 (t) + K11P15 (t) + K21 P16 (t)+ K31 P17 (t) + KL2 P24 (t) + ∑ P8 j [y j (t) − m j (t)]; j

dm9 (t)/dt = (−1/θ )m9(t) + KL1 m4 (t)m1 (t) + KL2 m4 (t)m2 (t) + K12 m5 (t)m2 (t)+ K22 m6 (t)m2 (t) + KL1 P14 (t) + KL2 P24 (t)K12 P25 (t) + K22 P26 (t) + ∑ P9 j [y j (t) − m j (t)]; j

14

1 Optimal Filtering for Polynomial Systems

dm10 (t)/dt = (−1/θ )m10(t) + KL1 m4 (t)m1 (t) + KL2 m4 (t)m2 (t)+ K13 m5 (t)m3 (t) + KL1 P14 (t) + KL2 P24 (t) + K13P35 (t) + ∑ P10 j [y j (t) − m j (t)]. j

Here, m1 (t) is the optimal estimate for Cm1 , m2 (t) for Cm2 , and so on up to m10 (t). The fifty-five variance component equations are similarly generated by the equations (1.29): ten equations for diagonal elements of P(t) are given below, other forty-five equations for non-diagonal elements are omitted. dP11 (t)/dt = ((−2/θ ) − 4(KL1m4 + K11 m5 + K21 m6 + K31 m7 ))P11 − ∑ P12j ; j

dP22 (t)/dt = ((−2/θ ) − 4(KL2m4 + K12 m5 + K22 m6 ))P22 − ∑ P22j ; j

dP33(t)/dt = (−2/θ − 4(K13 m5 ))P33 − ∑ P32j ; j

dP44 (t)/dt = (−2(1/θ + Kd ))P44 − 4(KL1 m4 P41 + K12 m4 P42 ) − ∑ P42j ; j

dP55 (t)/dt = 2(−1/θ + Kt1 )P55 + 4(KL1 m4 P51 + K21 m6 P51 + K31 m7 P51 − K12 m5 P52 − K13 m5 P53 ) − ∑ P52j ; j

dP66 (t)/dt = 2(−1/θ + Kt2 )P66 + 4(−K21 m6 P61 + KL2 m4 P62 + K12 m5 P27 ) − ∑ P62j ; j

dP77 (t)/dt = 2(−1/θ + Kt3 )P77 + 4(−K31m7 P71 + K13 m5 P73 ) − ∑ P72j ; j

dP88 (t)/dt = 2(−1/θ )P88 + 4(KL1 m4 P81 + K11 m5 P81 + K21m6 P81 + K31 m7 P81 + KL2 m4 P82 ) − ∑ P82j ; j

dP99 (t)/dt = 2(−1/θ )P99 + 4(KL1 m4 P91 + KL2 m4 P92 + K12 m5 P92 + K22 m6 P92 )− ∑ P92j ; j

2 dP1010 (t)/dt = 2(−1/θ )P1010 + 4(KL1 m4 P101 + KL2 m4 P102 + K13 m5 P103) − ∑ P10 j. j

In the simulation process, the initial conditions at t = 0 are equal to zero for the state variables Cm1 , . . . , λ1001 , to 0.5 for the estimates m1 (t), . . . , m10 (t), to 1 for the diagonal entries of the variance matrix, and to zero for its other entries. For the purpose of testing the obtained filter, the system parameter values are all set to 1: V = 1; d Δm1 /dt = 1; KL1 = 1; K11 = 1; K21 = 1; K31 = 1; K32 = 1; d Δm2 /dt = 1; d Δm3 /dt = 1; d Δm∗ /dt = 1; KL2 = 1; KL3 = 1; K12 = 1; K13 = 1; K22 = 1; Kd = 1; Kt1 = 1; Kt2 = 1; Kt3 = 1; θ = 1. The white Gaussian noises ψ2i , i = 1, . . . , 10, in the observation equations are realized as the built-in MatLab white noise functions.

1.1 Filtering Problem for Polynomial State over Linear Observations

15

In Fig. 1.3, the obtained values of the state variables Cm1 , . . . , λ1001 are given by the thin plain line, and the values of the bilinear optimal filter estimates m1 (t), . . . , m10 (t) are depicted by the stroke-marked line. The performance of the optimal bilinear filter (1.28),(1.29) is compared to the performance of the optimal linear Kalman-Bucy filter available for the linearized system. This linear filter consists of only the linear terms and innovations processes in the equations (1.28) (or (1.31)) for the optimal estimates and the Riccati equations for the variance matrix components corresponding to the equations (1.29): dmk (t)/dt = a0k (t) + ∑ a1ki (t)mi (t) +

(1.32)

i

∑ Pk j (t)ATjl (t)(Bl p B ps))−1 (t)[ys (t) − ∑ Asr (t)mr (t)], r

jl ps

mk (t0 ) = E[xk (t0 ) | Y (t0 )]; dPi j (t)/dt = ∑ a1ik (t)Pk j (t) + ∑ Pki (t)a1 jk (t) + k

(1.33)

k

∑ bik (t)bk j (t) − ∑ Pik (t)ATkl (t)(Bl p B ps))−1 Asr Pr j (t), k

kl psr

Pi j (t0 ) = E[(xi (t0 ) − mi (t0 ))(x j (t0 ) − m j (t0 ))T | Y (t0 )]. The graphs of the estimates obtained using this linear Kalman-Bucy filter are shown in Fig. 1.3 by the thick dashed line. Finally, the performance of the optimal bilinear filter (1.28),(1.29) is compared to the performance of the mixed filter designed as follows. The estimate equations in this filter coincide with the bilinear equations (1.28) (or (1.31)) from the optimal bilinear filter, and the variance equations coincide with the Riccati equations (1.33) from the linear Kalman-Bucy filter. The graphs of the estimates obtained using this mixed filter are shown in Fig. 1.3 by the thick dotted line. The initial conditions and white Gaussian noise realizations remain the same for all the filters involved in the simulation. Discussion Upon comparing all simulation results given in Fig. 1.3, it can be concluded that the optimal bilinear filter gives the best estimates in comparison to two other filters. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration. On the other hand, since the Kalman-Bucy estimates obtained for the linearized model do not converge to the real state values, it can be concluded that linearization fails and is not applicable even to simple bilinear systems. It should finally be noted that the results obtained applying the mixed filter are actually very close to (and for the first two variables even better than) the results obtained using the optimal bilinear filter. The advantage of the mixed filter consists in its better realizability, since the matrix P(t) for the mixed filter satisfies the conventional Riccati equation (1.33). Thus, the mixed filter could also be widely used to obtain reasonably good approximations of the optimal estimates for bilinear system states.

16

1 Optimal Filtering for Polynomial Systems

Fig. 1.3. Graphs of the ten state variables (1.30) (thin plain line), the estimates given by the optimal bilinear filter (1.28),(1.29) (stroke-marked line), the estimates given by the linear KalmanBucy filter (1.32),(1.33) (thick dashed line), the estimates given by the mixed filter (1.28),(1.33) (thick dotted line)

1.2 Filtering Problem for Polynomial State with Partially Measured Linear Part

17

1.2 Filtering Problem for Polynomial State with Partially Measured Linear Part 1.2.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ t0 , and let (W1 (t) = [W11 (t),W12 (t)], Ft ,t ≥ t0 ) and (W2 (t) = [W21 (t),W22 (t)], Ft ,t ≥ t0 ) be independent Wiener processes. The Ft -measurable random process (x(t) = [x1 (t), x2 (t)], y(t) = [y1 (t), y2 (t)]) is described by nonlinear differential equations for the system state dx1 (t) = f (x1 , x2 ,t)dt + b11(t)dW11 (t),

x1 (t0 ) = x10 ,

dx2 (t) = (a20 (t) + a21(t)x2 (t))dt + b12(t)dW12 (t),

x2 (t0 ) = x20 ,

(1.34) (1.35)

and linear differential equations for the observation process dy1 (t) = (A01 (t) + A1(t)x1 (t))dt + B1 (t)dW21 (t),

(1.36)

dy2 (t) = (A02 (t) + A2 (t)x2 (t))dt + B2(t)dW22 (t).

(1.37)

Here, x(t) = [x1 (t), x2 (t)] ∈ Rn is the state vector, x1 (t) ∈ Rn1 is the completely measured nonlinear component and x2 (t) ∈ Rn2 is the partially measured linear one, y(t) = [y1 (t), y2 (t)] ∈ Rm is the linear observation vector, such that the component y1 (t) ∈ Rn1 corresponds the completely measured nonlinear state component x1 (t) ∈ Rn1 , i.e., the matrix A1 (t) ∈ Rn1 ×n1 is invertible, and y2 (t) ∈ Rm2 corresponds to the partially measured linear component x2 (t) ∈ Rn2 , m2 ≤ n2 , i.e., the dimension of y2 (t) may be less than that of x2 (t). The initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t) = [W11 (t),W12 (t)], and W2 (t) = [W21 (t),W22 (t)] are independent. It is assumed that B(t)BT (t), where B(t) = diag[B1 (t), B2 (t)], is a positive definite matrix. All coefficients in (1.34)–(1.37) are deterministic functions of time of appropriate dimensions. Without loss of generality, the observation process components y1 (t) and y2 (t) are assumed to be uncorrelated. Indeed, if y1 (t) and y2 (t) are correlated a priori, their mutual correlation can always be set to zero by adjusting terms A01 (t) and A02 (t) in the equations (1.36) and (1.37) (see [69], Section 3.4). The nonlinear function f (x1 , x2 ,t) is considered a polynomial of n variables, components of the state vector x(t) = [x1 (t), x2 (t)] ∈ Rn , with time-dependent coefficients. In this and following sections, the general definition of such a polynomial, given in Subsection 1.1.1, is applied. The estimation problem is to find the optimal estimate x(t) ˆ = [xˆ1 (t), xˆ2 (t)] of the system state x(t) = [x1 (t), x2 (t)], based on the observation process Y (t) = {y(s) = [y1 (s), y2 (s)], 0 ≤ s ≤ t}, that minimizes the Euclidean 2-norm J = E[(x(t) − x(t)) ˆ T (x(t) − x(t)) ˆ | FtY ] at every time moment t. Here, E[z(t) | FtY ] means the conditional expectation of a ˆ with respect to the σ - algebra FtY stochastic process z(t) = (x(t) − x(t)) ˆ T (x(t) − x(t)) generated by the observation process Y (t) in the interval [t0 ,t]. As known from ([41],

18

1 Optimal Filtering for Polynomial Systems

Theorem 5.3 or [70], Subsection 5.10.2), this optimal estimate is given by the conditional expectation x(t) ˆ = [xˆ1 (t), xˆ2 (t)] = m(t) = [m1 (t), m2 (t)] = E(x(t) | FtY ) of the system state x(t) = [x1 (t), x2 (t)] with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. The matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differential of the conditional expectation E(x(t) | FtY ) and its variance P(t) and given in the following subsection. 1.2.2

Optimal Filter for Polynomial State with Partially Measured Linear Part over Linear Observations

The optimal filtering equations could be obtained using the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E( f¯(x,t) | FtY )dt + E(x[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY )× −1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where f¯(x,t) = [ f (x,t), a20 (t) + a21(t)x2 (t)] is the polynomial drift term in the entire state equation, f (x,t) is the polynomial drift term in the equation (1), and ϕ1 (x) is the linear drift term in the entire observation equation equal to ϕ1 (x) = A0 (t) + A(t)x(t), where A0 (t) = [A01 (t), A02 (t)] and A(t) = diag[A1 (t), A2 (t)]. Upon performing substitution, the estimate equation takes the form dm(t) = E( f¯(x,t) | FtY )dt + E(x(t)[A(t)(x(t) − m(t))]T | FtY )× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt) = E( f¯(x,t) | FtY )dt + E(x(t)(x(t) − m(t))T | FtY )AT (t)× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt) = E( f¯(x,t) | FtY )dt + P(t)AT (t)(B(t)BT (t))−1 (dy(t) − (A0 (t) + A(t)m(t))dt).

(1.38)

The equation (1.38) should be complemented with the initial condition m(t0 ) = E(x(t0 ) | FtY0 ). To compose a closed system of the filtering equations, the equation (1.38) should be complemented with the equation for the error variance P(t). For this purpose, the formula for the Ito differential of the variance P(t) = E((x(t) − m(t))(x(t) − m(t))T | FtY ) could be used (see ([41], Theorem 6.6, formula (6.101) or [70], Subsection 5.10.9, formula (5.10.42))):

1.2 Filtering Problem for Polynomial State with Partially Measured Linear Part

19

dP(t) = (E((x(t) − m(t))( f¯(x,t))T | FtY ) + E( f¯(x,t)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)−  −1 E(x(t)[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY ) B(t)BT (t) × E([ϕ1 (x) − E(ϕ1 (x) | FtY )]xT (t) | FtY ))dt+ E((x(t) − m(t))(x(t) − m(t))[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY )× −1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where the last term should be understood as a 3D tensor (under the expectation sign) convoluted with a vector, which yields a matrix. Upon substituting the expressions for ϕ1 and using the variance formula P(t) = E((x(t) − m(t))xT (t)) | FtY ), the last equation can be represented as dP(t) = (E((x(t) − m(t))( f¯(x,t))T | FtY ) + E( f¯(x,t)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)− P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt +

(1.39)

E(((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY )× AT (t)(B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt). The equation (7) should be complemented with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. The equations (1.38) and (1.39) for the optimal estimate m(t) and the error variance P(t) form a non-closed system of the filtering equations for the nonlinear state (1.34),(1.35) over linear observations (1.36),(1.37). Let us prove now that this system becomes a closed system of the filtering equations in view of the polynomial properties of the function f (x,t) in the equation (1.34). As shown in Subsection 1.1.2, a closed system of the filtering equations for a polynomial state over linear observations can be obtained if the observation matrix A(t) is invertible for any t ≥ t0 . This condition implies that the random variable x(t) − m(t) is conditionally Gaussian with respect to the observation process y(t) for any t ≥ t0 . In the considered observation equations (1.36),(1.37), only the matrix A1 (t) in (1.36) is invertible, whereas the matrix A2 (t) in (4) is not. Nonetheless, the error variable components x1 (t) − m1 (t), m1 (t) = E[x1 (t) | FtY ], corresponding to A1 (t) in (1.36), form a conditionally Gaussian vector with respect to the entire observation process y (t) and y2 (t) are uncory(t) = [y1 (t), y2 (t)], since the observation process components  1 related (by assumption) and the innovations process y1 (t) − tt0 (A01 (s) + A1 (s)m1 (s))ds is conditionally Gaussian with respect to y1 (t) (as shown in Subsection 1.1.2). The error variable components x2 (t) − m2 (t), m2 (t) = E[x2 (t) | FtY ], corresponding to A2 (t) in (1.37) also form a conditionally Gaussian vector with respect to the entire observation process y(t) = [y1 (t), y2 (t)], since x2 (t) is Gaussian and the observation process components y1 (t) and y2 (t) are uncorrelated. Thus, the entire vector x(t) − m(t) = [x1 (t) − m1(t), x2 (t) − m2 (t)] is conditionally Gaussian with respect to the entire observation process y(t) = [y1 (t), y2 (t)] (see [69], Section 5.3), and, therefore, the following considerations are applicable.

20

1 Optimal Filtering for Polynomial Systems

First, since the random variable x(t)− m(t) is conditionally Gaussian, the conditional third moment E((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY ) of x(t) − m(t) with respect to observations, which stands in the last term of the equation (1.39), is equal to zero, because the process x(t) − m(t) is conditionally Gaussian. Thus, the entire last term in (1.39) is vanished and the following variance equation is obtained dP(t) = (E((x(t) − m(t))( f¯(x,t))T | FtY ) + E( f¯(x,t)(x(t) − m(t))T ) | FtY ) +

(1.40)

b(t)bT (t) − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt, with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. Second, if the function f¯(x,t) is a polynomial function of the state x with timedependent coefficients, the expressions of the terms E( f¯(x,t) | FtY ) in (1.38) and E((x(t)−m(t)) ( f¯(x,t))T | FtY ) in (1.40) would also include only polynomial terms of x. Then, those polynomial terms can be represented as functions of m(t) and P(t) using the following property of Gaussian random variable x(t) − m(t): all its odd conditional moments, m1 = E[(x(t)−m(t)) | Y (t)], m3 = E[(x(t)−m(t)3 | Y (t)], m5 = E[(x(t)−m(t))5 | Y (t)], ... are equal to 0, and all its even conditional moments m2 = E[(x(t) − m(t))2 | Y (t)], m4 = E[(x(t) − m(t))4 | Y (t)], .... can be represented as functions of the variance P(t). For example, m2 = P, m4 = 3P2 , m6 = 15P3, ... etc. After representing all polynomial terms in (1.38) and (1.40), that are generated upon expressing E( f¯(x,t) | FtY ) and E((x(t)− m(t))( f¯(x,t))T | FtY ), as functions of m(t) and P(t), a closed form of the filtering equations would be obtained. The corresponding representations of E( f (x,t) | FtY ) and E((x(t) − m(t))( f (x,t))T | FtY ) have been derived in Section 1.1 for certain polynomial functions f (x,t). In the next subsection, a closed form of the filtering equations will be obtained from (1.38) and (1.40) for a bilinear function f (x,t) in the equation (1.34). It should be noted, however, that application of the same procedure would result in designing a closed system of the filtering equations for any polynomial function f (x,t) in (1.34). Optimal Filter for Bilinear State with Partially Measured Linear Part over Linear Observations In a particular case, if the function f (x,t) = a10 (t) + a11(t)x + a12(t)xxT

(1.41)

is a bilinear polynomial, where x is an n-dimensional vector, a10 (t) is an n1 -dimensional vector, a11 is an n1 × n - matrix, and a12 is a 3D tensor of dimension n1 × n × n, the representations for E( f¯(x,t) | FtY ) and E((x(t) − m(t))( f¯(x,t))T | FtY ) as functions of m(t) and P(t) are derived as follows E( f¯(x,t) | FtY ) = a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t),

(1.42)

E( f¯(x,t)(x(t) − m(t))T ) | FtY ) + E((x(t) − m(t))( f¯(x,t))T | FtY ) = a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T .

(1.43)

1.2 Filtering Problem for Polynomial State with Partially Measured Linear Part

21

where a0 (t) = [a10 (t), a20 (t)], a1 (t) = diag[a11(t), a21 (t)], and a2 is a 3D tensor of dimension n × n × n defined as a2 ki j = a12 ki j , if k ≤ n1 , and a2 ki j = 0, otherwise. Substituting the expression (1.42) in (1.38) and the expression (1.43) in (1.40), the filtering equations for the optimal estimate m(t) = [m1 (t), m2 (t)] of the bilinear state x(t) = [x1 (t), x2 (t)] and the error variance P(t) are obtained dm(t) = (a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t))dt + P(t)AT (t)(B(t)BT (t))−1 [dy(t) − (A0(t) + A(t)m(t))dt],

(1.44)

m(t0 ) = E(x(t0 ) | FtY )),

dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T +

(1.45)

−1

b(t)b (t))dt − P(t)A (t)(B(t)B (t)) A(t)P(t)dt. T

T

T

P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )), where A0 (t) = [A01 (t), A02 (t)], A(t) = diag[A1 (t), A2 (t)], and B(t) = diag[B1 (t), B2 (t)]. By means of the preceding derivation, the following result is proved. Theorem 1.2. The optimal finite-dimensional filter for the bilinear state (1.34),(1.35) with partially unmeasured linear part (1.35), where the quadratic polynomial f (x,t) is defined by (1.41), over the linear observations (1.36),(1.37) is given by the equation (1.44) for the optimal estimate m(t) = E(x(t) | FtY ) and the equation (1.45) for the estimation error variance P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ]. Thus, based on the general non-closed system of the filtering equations (1.38),(1.39), it is proved that the closed system of the filtering equations (1.38),(1.40) can be obtained for any polynomial state (1.34),(1.35) with partially measured linear part (1.35) over linear observations (1.36),(1.37). Furthermore, the specific form (1.44),(1.45) of the closed system of the filtering equations corresponding to a bilinear state is derived. In the next subsection, performance of the designed optimal filter for a bilinear state with partially measured linear part over linear observations is verified against a conventional extended Kalman-Bucy filter. 1.2.3

Example

This subsection presents an example of designing the optimal filter for a quadraticlinear state with unmeasured linear part over linear observations and comparing it to a conventional extended Kalman-Bucy filter. Let the unmeasured state x(t) = [x1 (t), x2 (t)] satisfy the quadratic and linear equations (1.46) x˙1 (t) = 0.1(x21 (t) + x2(t)), x1 (0) = x10 , x˙2 (t) = 0.1x2 (t),

x2 (0) = x20 ,

(1.47)

and the observation process be given by the linear equation y(t) = x1 (t) + ψ (t),

(1.48)

where ψ (t) is a white Gaussian noise, which is the weak mean square derivative of a standard Wiener process (see ([41], Subsection 3.8, or [70], Subsection 4.6.5)). The

22

1 Optimal Filtering for Polynomial Systems

equations (1.46)–(1.48) present the conventional form for the equations (1.34)–(1.36), which is actually used in practice [6]. Note that only the state quadratic part (1.46) is measured, whereas the linear part (1.47) is not measured at all. The filtering problem is to find the optimal estimate for the quadratic-linear state (1.46),(1.47), using linear observations (1.48) confused with independent and identically distributed disturbances modeled as white Gaussian noises. The filtering horizon is set to T = 6.5. The filtering equations (1.38),(1.40) take the following particular form for the system (1.46)–(1.48) m˙ 1 (t) = 0.1(m21 (t) + P11(t) + m2(t)) + P11(t)[y(t) − m1(t)],

(1.49)

m˙ 2 (t) = 0.1m2 (t) + P12(t)[y(t) − m1(t)],

(1.50)

with the initial conditions m1 (0) = [E(x1 (0) | y(0)) = m10 and m2 (0) = [E(x2 (0) | y(0)) = m20 , 2 (t), (1.51) P˙11 (t) = 0.4(P11(t)m1 (t)) + 0.2P12(t) − P11 P˙12 (t) = 0.2(P12(t)m1 (t)) + 0.1(P12(t) + P22(t)) − P11(t)P12 (t), 2 P˙22 (t) = 0.2P22(t) − P12 (t),

with the initial condition P(0) = E((x(0) − m(0))(x(0) − m(0))T | y(0)) = P0 . The estimates obtained upon solving the equations (1.49)–(1.51) are compared to the estimates satisfying the following extended Kalman-Bucy filtering equations for the quadratic-linear state (1.46),(1.47) over the linear observations (1.48), obtained using the direct copy of the state dynamics (1.46),(1.47) in the estimate equation and assigning the filter gain as the solution of the Riccati equation: m˙ K1 (t) = 0.1(m2K1 (t) + mK2 (t)) + PK11(t)[y(t) − mK1 (t)],

(1.52)

m˙ K2 (t) = 0.1mK2 (t) + PK12(t)[y(t) − mK1 (t)],

(1.53)

with the initial conditions mK1 (0) = [E(x1 (0) | y(0)) = m10 and mK2 (0) = [E(x2 (0) | y(0)) = m20 , 2 (t), (1.54) P˙K11 (t) = 0.2PK12(t) − PK11 P˙K12 (t) = 0.1(PK12 (t) + PK22(t)) − PK11(t)PK12 (t), 2 P˙K22 (t) = 0.2PK22(t) − PK12 (t),

with the initial condition PK (0) = E((x(0) − m(0))(x(0) − m(0))T | y(0)) = P0 . Numerical simulation results are obtained solving the systems of filtering equations (1.49)–(1.51) and (1.52)–(1.54). The obtained values of the estimates m(t) and mK (t) satisfying the equations (1.49),(1.50) and (1.52),(1.53), respectively, are compared to the real values of the state variables x(t) in (1.46),(1.47). For each of the two filters (1.49)–(1.51) and (1.52)–(1.54) and the reference system (1.46),(1.47) involved in simulation, the following initial values are assigned: x10 = x20 = 1.1, m10 = m20 = 0.1, P011 = P012 = P022 = 1. Gaussian disturbance ψ (t) in (1.48) is realized using the built-in MatLab white noise function.

1.2 Filtering Problem for Polynomial State with Partially Measured Linear Part

100

23

2.5

State 2

State 1

80 60 40

2 1.5

20 1

2

3

4 time

5

1

6 6.5

80 60 40 20 0

1

2

3

4 time

5

1

2

3

4 time

5

6 6.5

0

1

2

3

4 time

5

6 6.5

0

1

2

3

4 time

5

6 6.5

2 1.5 1 0.5 0

6 6.5

30

8

25 20 15 10 5 0

0

2.5 Optimal estimate 2

100

0

Kalman Estimate 1

0

Kalman Estimate 2

Optimal estimate 1

0

0

1

2

3

4 time

5

6 6.5

6 4 2 0

Fig. 1.4. Graphs of the reference state variables x1 (t) (State 1) and x2 (State 2) satisfying the equations (1.46),(1.47), graphs of the optimal filter estimates m1 (t) (Optimal estimate 1) and m2 (t) (Optimal estimate 2) satisfying the equations (1.49)–(1.51), and graphs of the estimates mK1 (t) (Kalman estimate 1) and mK2 (t) (Kalman estimate 2) satisfying the equations (1.52)– (1.54), on the entire simulation interval [0, 6.5]

24

1 Optimal Filtering for Polynomial Systems

The following graphs are obtained: graphs of the reference state variables x1 (t) and x2 (t) for the system (1.46),(1.47); graphs of the optimal filter estimates m1 (t) and m2 (t) satisfying the equations (1.49)–(1.51); graphs of the estimates mK1 (t) and mK2 satisfying the equations (1.52)–(1.54). The graphs of all those variables are shown on the entire simulation interval from t0 = 0 to T = 6.5 (Fig. 1.4). Note that the gain matrix entry P11 (t) does not converge to zero as time tends to infinity, since the polynomial dynamics of third order is stronger than the quadratic Riccati terms in the right-hand sides of the equation (1.51). The following values of the reference state variables x1 (t), x2 (t) and the estimates m1 (t), m2 (t), mK1 (t), mK2 (t) are obtained at the reference time points T = 5, 6, 6.5: for T = 5, x1 (5) = 6.05, m1 (5) = 5.69, mk1 (5) = 4.40, x2 (5) = 1.81, m2 (5) = 1.75, mK2 (5) = 2.35; for T = 6, x1 (6) = 16.05, m1 (6) = 16.42, mk1 (6) = 9.99, x2 (6) = 2.00, m2 (6) = 1.97, mK2 (6) = 3.47; for T = 6.5, x1 (6.5) = 96.99, m1 (6.5) = 96.68, mk1 (6.5) = 25.33, x2 (6.5) = 2.11, m2 (6.5) = 2.09, mK2 (6.5) = 6.26. Thus, it can be concluded that the obtained optimal filter (1.49)–(1.51) for a quadratic-linear state with unmeasured linear part over linear observations yield definitely better estimates than the conventional extended Kalman-Bucy filter (1.52)–(1.54). Discussion The simulation results show that the values of the estimate calculated by using the obtained optimal filter for a quadratic-linear state with unmeasured linear part over linear observations are noticeably closer to the real values of the reference variable than the values of the estimate given by the conventional extended Kalman-Bucy filter. Moreover, it can be seen that the estimate produced by the optimal filter asymptotically converges to the real values of the reference variables as time tends to infinity, although the reference system (1.46),(1.47) itself is unstable and the nonlinear component x1 (t) goes to infinity for a finite time. On the contrary, the conventionally designed extended Kalman-Bucy estimates diverge from the real values. This significant improvement in the estimate behavior is obtained due to the more careful selection of the filter gain matrix in the equations (1.49)–(1.51), as it should be in the optimal filter. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration.

1.3 Filtering Problem for Polynomial State with Multiplicative Noise 1.3.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ t0 , and let (W1 (t), Ft ,t ≥ t0 ) and (W2 (t), Ft ,t ≥ t0 ) be independent Wiener processes. The Ft -measurable random process (x(t), y(t)) is described by a nonlinear differential equation with both polynomial drift and diffusion terms for the system state dx(t) = f (x,t)dt + g(x,t)dW1 (t), x(t0 ) = x0 , (1.55)

1.3 Filtering Problem for Polynomial State with Multiplicative Noise

25

and a linear differential equation for the observation process dy(t) = (A0 (t) + A(t)x(t))dt + B(t)dW2(t).

(1.56)

Here, x(t) ∈ Rn is the state vector and y(t) ∈ Rn is the linear observation vector, such that the matrix A(t) ∈ Rn×n is invertible. The initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t), and W2 (t) are independent. It is assumed that B(t)BT (t) is a positive definite matrix. All coefficients in (1.55),(1.56) are deterministic functions of time of appropriate dimensions. The nonlinear diffusion function g(x,t) forms a statedependent multiplicative noise in the state equation (1.55). The nonlinear functions f (x,t) and g(x,t) are considered polynomials of n variables, components of the state vector x(t) ∈ Rn , with time-dependent coefficients (see Subsection 1.1.1 for definition). The estimation problem is stated identically to the one in Subsection 1.1.1: to find the optimal estimate x(t) ˆ of the system state x(t), based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, that is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY ) of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As usual, the matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differential of the conditional expectation E(x(t) | FtY ) and its variance P(t) and given in the following subsection. 1.3.2

Optimal Filter for Polynomial State with Multiplicative Noise over Linear Observations

The optimal filtering equations could be obtained using the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E( f (x,t) | FtY )dt + E(x[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY )× −1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where f (x,t) is the polynomial drift term in the state equation, and ϕ1 (x) is the linear drift term in the observation equation equal to ϕ1 (x) = A0 (t) + A(t)x(t). Upon performing substitution, the estimate equation takes the form dm(t) = E( f (x,t) | FtY )dt + E(x(t)[A(t)(x(t) − m(t))]T | FtY )× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt) = E( f (x,t) | FtY )dt + E(x(t)(x(t) − m(t))T | FtY )AT (t)×

26

1 Optimal Filtering for Polynomial Systems

(B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt) = E( f (x,t) | FtY )dt + P(t)AT (t)(B(t)BT (t))−1 (dy(t) − (A0 (t) + A(t)m(t))dt).

(1.57)

The equation (1.57) should be complemented with the initial condition m(t0 ) = E(x(t0 ) | FtY0 ). Trying to compose a closed system of the filtering equations, the equation (1.57) should be complemented with the equation for the error variance P(t). For this purpose, the formula for the Ito differential of the variance P(t) = E((x(t)− m(t))(x(t)− m(t))T | FtY ) could be used (see ([41], Theorem 6.6, formula (6.101) or [70], Subsection 5.10.9, formula (5.10.42))): dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY ) + E( f (x,t)(x(t) − m(t))T ) | FtY )+ E(g(x,t)gT (x,t) | FtY ) − E(x(t)[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY )×  −1 B(t)BT (t) E([ϕ1 (x) − E(ϕ1 (x) | FtY )]xT (t) | FtY ))dt+ E((x(t) − m(t))(x(t) − m(t))[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY )× −1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where the last term should be understood as a 3D tensor (under the expectation sign) convoluted with a vector, which yields a matrix. Upon substituting the expressions for ϕ1 and using the variance formula P(t) = E((x(t) − m(t))xT (t)) | FtY ), the last equation can be represented as dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY ) + E( f (x,t)(x(t) − m(t))T ) | FtY ) +

(1.58)

E(g(x,t)gT (x,t) | FtY ) − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt+ E(((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY )× AT (t)(B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt). The equation (5) should be complemented with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. The equations (1.57) and (1.58) for the optimal estimate m(t) and the error variance P(t) form a non-closed system of the filtering equations for the nonlinear state (1.55) over linear observations (1.56). Let us prove now that this system becomes a closed system of the filtering equations in view of the polynomial properties of the functions f (x,t) and g(x,t) in the equation (1.55). As shown in Subsection 1.1.2, a closed system of the filtering equations for a system state with polynomial drift and state-independent diffusion over linear observations can be obtained if the observation matrix A(t) is invertible for any t ≥ t0 . The last condition, also assumed for the observation process (1.56), implies that the random variable x(t) − m(t) is conditionally Gaussian with respect to the observation process y(t) for any t ≥ t0 . Hence, the following considerations outlined in Section 1.1.2 are applicable to the filtering equations (1.57),(1.58).

1.3 Filtering Problem for Polynomial State with Multiplicative Noise

27

First, the conditional third moment E((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY ) of x(t) − m(t) with respect to observations, which stands in the last term of the equation (1.58), is equal to zero, and the following variance equation is obtained dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY ) + E( f (x,t)(x(t) − m(t))T ) | FtY ) +

(1.59)

E(g(x,t)gT (x,t) | FtY ) − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt, with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. Second, if the functions f (x,t) and g(x,t) are polynomial functions of the state x with time-dependent coefficients, the expressions of the terms E( f (x,t) | FtY ) in (1.57) and E((x(t) − m(t)) f T (x,t)) | FtY ) and E(g(x,t)gT (x,t) | FtY ) in (1.59) would also include only polynomial terms of x. Then, those polynomial terms can be represented as functions of m(t) and P(t) using the expressions for superior conditional moments of Gaussian random variable x(t) − m(t), which are pointed out in Subsection 1.1.2. After representing all polynomial terms in (1.57) and (1.59), that are generated upon expressing E( f (x,t) | FtY ), E((x(t) − m(t)) f T (x,t)) | FtY ), and E(g(x,t)gT (x,t) | FtY ), as functions of m(t) and P(t), a closed form of the filtering equations would be obtained. The corresponding representations of E( f (x,t) | FtY ), E((x(t) − m(t))( f (x,t))T | FtY ), and E(g(x,t)gT (x,t) | FtY ) will be further shown for certain polynomial functions f (x,t). In the next subsections, a closed form of the filtering equations will be obtained from (1.57) and (1.59) for linear and bilinear functions f (x,t) and g(x,t) in the equation (1.55). It should be noted, however, that application of the same procedure would result in designing a closed system of the filtering equations for any polynomial functions f (x,t) and g(x,t) in (1.55). Optimal Filter for Linear State with Linear Multiplicative Noise In a particular case, if the functions f (x,t) = a0 (t) + a1 (t)x(t) and g(x,t) = b0 (t) + b1 (t)x(t) are linear, where b1 is a 3D tensor of dimension n × n × n, the representations for E( f (x,t) | FtY ), E((x(t) − m(t))( f (x,t))T | FtY ), and E(g(x,t)gT (x,t) | FtY ) as functions of m(t) and P(t) are derived as follows E( f (x,t) | FtY ) = a0 (t) + a1(t)m(t), E( f (x,t)(x(t) − m(t))T ) | FtY ) + E((x(t) − m(t))( f (x,t))T | FtY ) =

(1.60) (1.61)

a1 (t)P(t) + P(t)aT1 (t). E(g(x,t)gT (x,t) | FtY ) = b0 (t)bT0 (t) + b0(t)(b1 (t)m(t))T +

(1.62)

(b1 (t)m(t))bT0 (t) + b1(t)P(t)bT1 (t) + b1(t)m(t)mT (t)bT1 (t), where bT1 (t) denotes the tensor obtained from b1 (t) by transposing its two rightmost indices. Substituting the expression (1.60) in (1.57) and the expressions (1.61),(1.62) in (1.59), the filtering equations for the optimal estimate m(t) and the error variance P(t) are obtained (1.63) dm(t) = (a0 (t) + a1(t)m(t))dt +

28

1 Optimal Filtering for Polynomial Systems

P(t)AT (t)(B(t)BT (t))−1 [dy(t) − (A0(t) + A(t)m(t))dt],

m(t0 ) = E(x(t0 ) | FtY )),

dP(t) = (a1 (t)P(t) + P(t)aT1 (t) +

(1.64)

b0 (t)bT0 (t) + b0(t)(b1 (t)m(t))T + (b1(t)m(t))bT0 (t) + b1(t)P(t)bT1 (t)+ b1 (t)m(t)mT (t)bT1 (t))dt − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t)dt. P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )). Note that the observation matrix A(t) should not even be necessarily invertible to obtain the filtering equations (1.63)–(1.64). Indeed, the only used polynomial equality, E(x(t)xT (t) | FtY ) = P(t) + m(t)mT (t), is valid for any random variable with finite second moments, not only Gaussian. Optimal Filter for Bilinear State with Bilinear Multiplicative Noise Let the functions f (x,t) = a0 (t) + a1(t)x + a2 (t)xxT

(1.65)

g(x,t) = b0 (t) + b1(t)x + b2 (t)xxT

(1.66)

and be bilinear polynomials, where x is an n-dimensional vector, a0 (t) is an n-dimensional vector, a1 (t) and b0 (t) are n × n - matrices, a2 (t) and b1 (t) are 3D tensors of dimension n × n × n, and b2 (t) is a 4D tensor of dimension n × n × n × n. In this case, the representations for E( f (x,t) | FtY ), E((x(t) − m(t))( f (x,t))T | FtY ), and E(g(x,t)gT (x,t) | FtY ) as functions of m(t) and P(t) are derived as follows E( f (x,t) | FtY ) = a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t),

(1.67)

E( f (x,t)(x(t) − m(t))T ) | FtY ) + E((x(t) − m(t))( f (x,t))T | FtY ) = a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T .

(1.68)

= b0 (t)bT0 (t) + b0(t)(b1 (t)m(t))T + E(g(x,t)g (x,t) | (b1 (t)m(t))bT0 (t) + b1(t)P(t)bT1 (t) + b1(t)m(t)mT (t)bT1 (t)+ T

FtY )

b0 (t)(P(t) + m(t)mT (t))bT2 (t) + b2(t)(P(t) + m(t)mT (t))bT0 (t) +

(1.69)

T

b1 (t)(3m(t)P(t) + m(t)(m(t)m (t)))bT2 (t)+ b2 (t)(3P(t)mT (t) + (m(t)mT (t))mT (t))bT1 (t)+ 3b2 (t)P2 (t)bT2 (t) + 3b2(t)(P(t)m(t)mT (t)+ m(t)mT (t)P(t))bT2 (t) + b2(t)(m(t)mT (t))2 bT2 (t). where bT2 (t) denotes the tensor obtained from b2 (t) by transposing its two rightmost indices. Substituting the expression (1.67) in (1.57) and the expressions (1.68),(1.69) in (1.59), the filtering equations for the optimal estimate m(t) and the error variance P(t) are obtained dm(t) = (a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t))dt +

(1.70)

1.3 Filtering Problem for Polynomial State with Multiplicative Noise

P(t)AT (t)(B(t)BT (t))−1 [dy(t) − (A0(t) + A(t)m(t))dt],

m(t0 ) = E(x(t0 ) | FtY )),

dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T + b0 (t)bT0 (t) + b0(t)(b1 (t)m(t))T

29

(1.71)

+ (b1(t)m(t))bT0 (t) + b1(t)P(t)bT1 (t)+

b1 (t)m(t)mT (t)bT1 (t) + b0(t)(P(t) + m(t)mT (t))bT2 (t)+ b2 (t)(P(t) + m(t)mT (t))bT0 (t) + b1(t)(3m(t)P(t) + m(t)(m(t)mT (t)))bT2 (t)+ b2 (t)(3P(t)mT (t) + (m(t)mT (t))mT (t))bT1 (t) + 3b2(t)P2 (t)bT2 (t)+ 3b2 (t)(P(t)m(t)mT (t) + m(t)mT (t)P(t))bT2 (t)+ b2 (t)(m(t)mT (t))2 bT2 (t))dt − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t)dt. P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )). By means of the preceding derivation, the following result is proved. Theorem 1.3. The optimal finite-dimensional filter for the bilinear state with bilinear multiplicative noise (1.55), where the bilinear polynomials f (x,t) and g(x,t) are defined by (1.65),(1.66), over the linear observations (1.56), is given by the equation (1.70) for the optimal estimate m(t) = E(x(t) | FtY ) and the equation (1.71) for the estimation error variance P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ]. Thus, based on the general non-closed system of the filtering equations (1.57),(1.59), it is proved that the closed system of the filtering equations can be obtained for any polynomial state with a polynomial multiplicative noise (1.55) over linear observations (1.56). Furthermore, the specific form (1.70),(1.71) of the closed system of the filtering equations corresponding to a bilinear state with a bilinear multiplicative noise is derived. In the next subsection, performance of the designed optimal filter for a bilinear state with a bilinear multiplicative noise over linear observations is verified against the optimal filter for a bilinear state with a state-independent noise and a conventional extended Kalman-Bucy filter. 1.3.3

Example

This subsection presents an example of designing the optimal filter for a quadratic state with a quadratic multiplicative noise over linear observations and comparing it to the optimal filter for a quadratic state with a state-independent noise and a conventional extended Kalman-Bucy filter. Let the real state x(t) satisfy the quadratic equation x(t) ˙ = 0.1x2 (t) + 0.1x2(t)ψ1 (t),

x(0) = x0 ,

(1.72)

and the observation process be given by the linear equation y(t) = x(t) + ψ2 (t),

(1.73)

where ψ1 (t) and ψ2 (t) are white Gaussian noises, which are the weak mean square derivatives of standard Wiener processes (see ([41], Subsection 3.8, or [70], Subsection

30

1 Optimal Filtering for Polynomial Systems

4.6.5)). The equations (1.72),(1.73) present the conventional form for the equations (1.55),(1.56), which is actually used in practice [6]. The filtering problem is to find the optimal estimate for the quadratic state with quadratic noise (1.72), using linear observations (1.73) confused with independent and identically distributed disturbances modeled as white Gaussian noises. The filtering horizon is set to T = 9.2. The filtering equations (1.70),(1.71) take the following particular form for the system (1.72),(1.73) (1.74) m(t) ˙ = 0.1(m2 (t) + P(t)) + P(t)[y(t) − m(t)], with the initial conditions m(0) = E(x(0) | y(0)) = m0 , ˙ = 0.4P(t)m(t) + 0.03P2(t) + 0.06P(t)m2(t) + 0.01m4(t) − P2(t), P(t)

(1.75)

with the initial condition P(0) = E((x(0) − m(0))(x(0) − m(0))T | y(0)) = P0 . The estimates obtained upon solving the equations (1.74)–(1.75) are compared first to the estimates satisfying the optimal filtering equations for a quadratic state with a state-independent noise (see Subsection 1.1.2.2), based on the system (1.72) where the quadratic multiplicative noise x2 (t)ψ1 (t) is replaced by the standard additive noise ψ1 (t). The corresponding filtering equations are given by m˙ 1 (t) = 0.1(m21 (t) + P1(t)) + P1(t)[y(t) − m1(t)],

(1.76)

with the initial conditions m(0) = E(x(0) | y(0)) = m0 , P˙1 (t) = 0.4P1(t)m(t) + 0.01 − P12(t),

(1.77)

with the initial condition P(0) = E((x(0) − m(0))(x(0) − m(0))T | y(0)) = P0 . The estimates obtained upon solving the equations (1.74)–(1.75) are also compared to the estimates satisfying the following extended Kalman-Bucy filtering equations for the quadratic state (1.72) over the linear observations (1.73), obtained by replacing the quadratic multiplicative noise x2 (t)ψ1 (t) by the standard additive noise ψ1 (t), using the direct copy of the state dynamics (1.72) in the estimate equation, and assigning the filter gain as the solution of the Riccati equation: m˙ K (t) = 0.1m2K (t) + PK (t)[y(t) − mK (t)],

(1.78)

with the initial conditions mK (0) = E(x(0) | y(0)) = m0 , P˙K (t) = 0.4PK (t) + 0.01 − PK2 (t),

(1.79)

with the initial condition PK (0) = E((x(0) − m(0))(x(0) − m(0))T | y(0)) = P0 . Numerical simulation results are obtained solving the systems of filtering equations (1.74)–(1.75), (1.76)–(1.77), and (1.78)–(1.79). The obtained values of the estimates m(t), m1 (t), and mK (t) satisfying the equations (1.74), (1.76), and (1.78), respectively, are compared to the real values of the state variables x(t) in (1.72). For each of the three filters (1.74)–(1.75), (1.76)–(1.77), and (1.78)–(1.79) and the reference system (1.72)–(1.73) involved in simulation, the following initial values are

1.3 Filtering Problem for Polynomial State with Multiplicative Noise

31

Optimal estimate error

1

0.5

0

−0.5

−1

0

1

2

3

4

5

6

7 7.3

4

5

6

7 7.3

4

5

6

7 7.3

time

1

Estimate 1 error

0.5

0

−0.5

−1

0

1

2

3 time

Estimate 2 error

1

0.5

0

−0.5

−1

0

1

2

3 time

Fig. 1.5. Graph of the error between the real state x(t) satisfying the equation (1.72) and the optimal filter estimate m(t) satisfying the equation (1.74) (Optimal estimate error), graph of the error between the real state x(t) satisfying the equation (1.72) and the estimate m1 (t) satisfying the equation (1.76) (Estimate 1 error), graph of the error between the real state x(t) satisfying the equation (1.72) and the estimate mK (t) satisfying the equation (1.78) (Estimate 2 error), on the simulation interval [0, 7.3]

32

1 Optimal Filtering for Polynomial Systems

Optimal estimate error

3 2 1 0 −1 −2 −3

0

1

2

3

4

5

6

7

8

9.2

5

6

7

8

9.2

5

6

7

8

9.2

time

Estimate 1 error

300

200

100

0

−100

0

1

2

3

4 time

Estimate 2 error

800 600 400 200 0 −100

0

1

2

3

4 time

Fig. 1.6. Graph of the error between the real state x(t) satisfying the equation (1.72) and the optimal filter estimate m(t) satisfying the equation (1.74) (Optimal estimate error), graph of the error between the real state x(t) satisfying the equation (1.72) and the estimate m1 (t) satisfying the equation (1.76) (Estimate 1 error), graph of the error between the real state x(t) satisfying the equation (1.72) and the estimate mK (t) satisfying the equation (1.78) (Estimate 2 error), on the entire simulation interval [0, 9.2]

1.4 Filtering Problem for Polynomial State with Partially Measured Linear Part

33

assigned: x0 = 1.1, m0 = 0.1, P0 = 1. Gaussian disturbances ψ1 (t) in (1.72) and ψ2 (t) in (1.73) are realized using the built-in MatLab white noise function. The following graphs are obtained: graphs of the error between the reference state variable x(t) satisfying the equation (1.72) and the optimal filter estimate m(t) satisfying the equation (1.74); graph of the error between the reference state variable x(t) satisfying the equation (1.72) and the estimate m1 (t) satisfying the equation (1.76); graph of the error between reference state variable x(t) satisfying the equation (1.72) and the estimate mK (t) satisfying the equation (1.78). The graphs of all estimation errors are shown on the simulation interval from t0 = 0 to T = 7.3 (Fig. 1.5) and the entire simulation interval from t0 = 0 to T = 9.2 (Fig. 1.6). It can be observed that the error given by the optimal filter estimate (1.74) rapidly reaches and then maintains the zero mean value even in a close vicinity of the asymptotic time point T = 9.205, where the reference quadratic state variable (1.72) goes to infinity. On the contrary, the errors given by the other considered filters reach zero more slowly or do not reach it at all, have systematic (biased) deviations from zero, and clearly diverge to infinity near the asymptotic time point. Note that the optimal filtering error variance P(t) does not converge to zero as time tends to the asymptotic time point, since the polynomial dynamics of fourth order is stronger than the quadratic Riccati terms in the right-hand side of the equation (1.75). Thus, it can be concluded that the obtained optimal filter (1.74)–(1.75) for a quadratic state with a quadratic multiplicative noise over linear observations yield definitely better estimates than the optimal filter for a quadratic state with a state-independent noise or a conventional extended Kalman-Bucy filter. Discussion The simulation results show that the values of the estimate calculated by using the obtained optimal filter for a quadratic state with a quadratic multiplicative noise over linear observations are noticeably closer to the real values of the reference variable than the values of the estimates given by the optimal filter for a quadratic state with a stateindependent noise or a conventional extended Kalman-Bucy filter. Moreover, it can be seen that the estimation error produced by the optimal filter rapidly reaches and then maintains the zero mean value even in a close vicinity of the asymptotic time point, where the reference quadratic state variable (1.72) goes to infinity for a finite time. On the contrary, the estimation errors given by the other two applied filters diverge to infinity near the asymptotic time point. This significant improvement in the estimate behavior is obtained due to the more careful selection of the filter gain matrix in the equations (1.74)–(1.75), as it should be in the optimal filter.

1.4 Filtering Problem for Polynomial State with Partially Measured Linear Part and Multiplicative Noise 1.4.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ t0 , and let (W1 (t) = [W11 (t),W12 (t)], Ft ,t ≥ t0 ) and (W2 (t) = [W21 (t),W22 (t)], Ft ,t ≥ t0 ) be independent Wiener processes. The Ft -measurable

34

1 Optimal Filtering for Polynomial Systems

random process (x(t) = [x1 (t), x2 (t)], y(t) = [y1 (t), y2 (t)]) is described by a nonlinear differential equation with both polynomial drift and diffusion terms for the system state and a linear equation with state-independent coefficients dx1 (t) = f (x1 , x2 ,t)dt + g(x1, x2 ,t)dW11 (t),

x1 (t0 ) = x10 ,

(1.80)

dx2 (t) = (a20 (t) + a21(t)x2 (t))dt + b21(t)dW12 (t),

x2 (t0 ) = x20 ,

(1.81)

and linear differential equations for the observation process dy1 (t) = (A01 (t) + A1(t)x1 (t))dt + B1 (t)dW21 (t),

(1.82)

dy2 (t) = (A02 (t) + A2 (t)x2 (t))dt + B2(t)dW22 (t).

(1.83)

Here, x(t) = [x1 (t), x2 (t)] ∈ Rn is the state vector, x1 (t) ∈ Rn1 is the completely measured nonlinear component and x2 (t) ∈ Rn2 is the partially measured linear one, y(t) = [y1 (t), y2 (t)] ∈ Rm is the linear observation vector, such that the component y1 (t) ∈ Rn1 corresponds the completely measured nonlinear state component x1 (t) ∈ Rn1 , i.e., the matrix A1 (t) ∈ Rn1 ×n1 is invertible, and y2 (t) ∈ Rm2 corresponds to the partially measured linear component x2 (t) ∈ Rn2 , m2 ≤ n2 , i.e., the dimension of y2 (t) may be less than that of x2 (t). The initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t) = [W11 (t),W12 (t)], and W2 (t) = [W21 (t),W22 (t)] are independent. It is assumed that B(t)BT (t), where B(t) = diag[B1 (t), B2 (t)], is a positive definite matrix. All coefficients in (1.80)–(1.83) are deterministic functions of time of appropriate dimensions. Without loss of generality, the observation process components y1 (t) and y2 (t) are assumed to be uncorrelated (see Section 1.2). The nonlinear functions f (x1 , x2 ,t) and g(x1 , x2 ,t) are considered polynomials of n variables, components of the state vector x(t) = [x1 (t), x2 (t)] ∈ Rn , with time-dependent coefficients (see Subsection 1.1.1 for definition). The estimation problem is stated identically to the one in Subsection 1.2.1: to find the optimal estimate x(t) ˆ = [xˆ1 (t), xˆ2 (t)] of the system state x(t) = [x1 (t), x2 (t)], based on the observation process Y (t) = {y(s) = [y1 (s), y2 (s)], 0 ≤ s ≤ t}, that is given by the conditional expectation x(t) ˆ = [xˆ1 (t), xˆ2 (t)] = m(t) = [m1 (t), m2 (t)] = E(x(t) | FtY ) of the system state x(t) = [x1 (t), x2 (t)] with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As usual, the matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differential of the conditional expectation E(x(t) | FtY ) and its variance P(t) and given in the following subsection.

1.4 Filtering Problem for Polynomial State with Partially Measured Linear Part

1.4.2

35

Optimal Filter for Polynomial State with Partially Measured Linear Part and Polynomial Multiplicative Noise over Linear Observations

The optimal filtering equations could be obtained using the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E( f¯(x,t) | FtY )dt + E(x[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY )× −1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where f¯(x,t) = [ f (x,t), a20 (t) + a21(t)x2 (t)] is the polynomial drift term in the entire state equation, f (x,t) is the polynomial drift term in the equation (1), and ϕ1 (x) is the linear drift term in the entire observation equation equal to ϕ1 (x) = A0 (t) + A(t)x(t), where A0 (t) = [A01 (t), A02 (t)] and A(t) = diag[A1 (t), A2 (t)]. Upon performing substitution, the estimate equation takes the form dm(t) = E( f¯(x,t) | FtY )dt + E(x(t)[A(t)(x(t) − m(t))]T | FtY )× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt) = E( f¯(x,t) | FtY )dt + E(x(t)(x(t) − m(t))T | FtY )AT (t)× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt) = E( f¯(x,t) | FtY )dt + P(t)AT (t)(B(t)BT (t))−1 (dy(t) − (A0 (t) + A(t)m(t))dt).

(1.84)

The equation (1.84) should be complemented with the initial condition m(t0 ) = E(x(t0 ) | FtY0 ). To compose a closed system of the filtering equations, the equation (6) should be complemented with the equation for the error variance P(t). For this purpose, the formula for the Ito differential of the variance P(t) = E((x(t) − m(t))(x(t) − m(t))T | FtY ) could be used (see ([41], Theorem 6.6, formula (6.101) or [70], Subsection 5.10.9, formula (5.10.42))): dP(t) = (E((x(t) − m(t))( f¯(x,t))T | FtY )+ E( f¯(x,t)(x(t) − m(t))T ) | FtY ) + E(g(x,t) ¯ g¯T (x,t) | FtY )−  −1 E(x(t)[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY ) B(t)BT (t) × E([ϕ1 (x) − E(ϕ1 (x) | FtY )]xT (t) | FtY ))dt+ E((x(t) − m(t))(x(t) − m(t))[ϕ1 (x) − E(ϕ1 (x) | FtY )]T | FtY )× −1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where g(x,t) ¯ = [g(x,t), b21 (t)] is the polynomial diffusion (multiplicative noise) term in the entire state equation, g(x,t) is the polynomial diffusion term in the equation (1.80), and the last term should be understood as a 3D tensor (under the expectation sign) convoluted with a vector, which yields a matrix. Upon substituting the expressions for

36

1 Optimal Filtering for Polynomial Systems

ϕ1 and using the variance formula P(t) = E((x(t) − m(t))xT (t)) | FtY ), the last equation can be represented as dP(t) = (E((x(t) − m(t))( f¯(x,t))T | FtY ) +

(1.85)

¯ g¯T (x,t) | FtY )− E( f¯(x,t)(x(t) − m(t))T ) | FtY ) + E(g(x,t) P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt+ E(((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY )× AT (t)(B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t))dt). The equation (1.85) should be complemented with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. The equations (1.84) and (1.85) for the optimal estimate m(t) and the error variance P(t) form a non-closed system of the filtering equations for the nonlinear state (1.80),(1.81) over linear observations (1.82),(1.83). Let us prove now that this system becomes a closed system of the filtering equations in view of the polynomial properties of the functions f (x,t) and g(x,t) in the equation (1.80). Indeed, as shown in Subsection 1.2.2, the entire vector x(t) − m(t) = [x1 (t) − m1 (t), x2 (t) − m2 (t)] is conditionally Gaussian with respect to the entire observation process y(t) = [y1 (t), y2 (t)]. Hence, the following considerations outlined in Subsection 1.2.2 are applicable to the filtering equations (1.84),(1.85). First, the conditional third moment E((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY ) of x(t) − m(t) with respect to observations, which stands in the last term of the equation (1.85), is equal to zero, and the following variance equation is obtained dP(t) = (E((x(t) − m(t))( f¯(x,t))T | FtY ) + E( f¯(x,t)(x(t) − m(t))T ) | FtY ) +

(1.86)

+E(g(x,t) ¯ g¯T (x,t) | FtY ) − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt, with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. Second, if the functions f¯(x,t) and g(x,t) ¯ are polynomial functions of the state x with time-dependent coefficients, the expressions of the terms E( f¯(x,t) | FtY ) in (1.83) and E((x(t) − m(t)) f¯T (x,t)) | FtY ) and E(g(x,t) ¯ g¯T (x,t) | FtY ) in (1.84) would also include only polynomial terms of x. Then, those polynomial terms can be represented as functions of m(t) and P(t) using the expressions for superior conditional moments of Gaussian random variable x(t) − m(t), which are pointed out in Subsection 1.2.2. After representing all polynomial terms in (1.84) and (1.86), that are generated upon express¯ g¯T (x,t) | FtY ), as funcing E( f¯(x,t) | FtY ), E((x(t) − m(t)) f¯T (x,t)) | FtY ), and E(g(x,t) tions of m(t) and P(t), a closed form of the filtering equations would be obtained. The corresponding representations of E( f¯(x,t) | FtY ), E((x(t) − m(t)) f¯T (x,t)) | FtY ), and E(g(x,t) ¯ g¯T (x,t) | FtY ) will be further shown for certain polynomial functions f¯(x,t) and g(x,t). ¯ In the next subsections, a closed form of the filtering equations will be obtained from (1.84) and (1.86) for bilinear functions f (x,t) and g(x,t) in the equation (1.80). It should be noted, however, that application of the same procedure would result in designing a closed system of the filtering equations for any polynomial functions f (x,t) and g(x,t) in (1.80).

1.4 Filtering Problem for Polynomial State with Partially Measured Linear Part

37

Optimal Filter for Bilinear State with Partially Measured Linear Part and Bilinear Multiplicative Noise over Linear Observations Let the functions f (x,t) = a10 (t) + a11(t)x + a12(t)xxT

(1.87)

g(x,t) = b10 (t) + b11(t)x + b12(t)xxT

(1.88)

and be bilinear polynomials, where x is an n-dimensional vector, a10 (t) is an n-dimensional vector, a11 (t) and b10 (t) are n × n - matrices, a12 (t) and b11 (t) are 3D tensors of dimension n × n × n, and b12 (t) is a 4D tensor of dimension n × n × n × n. In this case, the rep¯ g¯T (x,t) | resentations for E( f¯(x,t) | FtY ), E((x(t) − m(t))( f¯(x,t))T | FtY ), and E(g(x,t) Y Ft ) as functions of m(t) and P(t) are derived as follows E( f¯(x,t) | FtY ) = a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t),

(1.89)

E( f¯(x,t)(x(t) − m(t))T ) | FtY ) + E((x(t) − m(t))( f¯(x,t))T | FtY ) =

(1.90)

a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T . E(g(x,t) ¯ g¯T (x,t) | FtY ) = b0 (t)bT0 (t) + b0(t)(b1 (t)m(t))T +

(1.91)

(b1 (t)m(t))bT0 (t) + b1(t)P(t)bT1 (t) + b1(t)m(t)mT (t)bT1 (t)+ b0 (t)(P(t) + m(t)mT (t))bT2 (t) + b2(t)(P(t) + m(t)mT (t))bT0 (t)+ b1 (t)(3m(t)P(t) + m(t)(m(t)mT (t)))bT2 (t)+ b2 (t)(3P(t)mT (t) + (m(t)mT (t))mT (t))bT1 (t)+ 3b2 (t)P2 (t)bT2 (t) + 3b2(t)(P(t)m(t)mT (t)+ m(t)mT (t)P(t))bT2 (t) + b2(t)(m(t)mT (t))2 bT2 (t). where a0 (t) = [a10 (t), a20 (t)], a1 (t) = diag[a11 (t), a21 (t)], b0 (t) = diag[b10(t), b21 (t)]; a2 is a 3D tensor of dimension n × n × n defined as a2 ki j = a12 ki j , if k ≤ n1 , and a2 ki j = 0, otherwise; b1 is a 3D tensor of dimension n × n × n defined as b1 ki j = b11 ki j , if k ≤ n1 , and a2 ki j = 0, otherwise; b2 is a 3D tensor of dimension n × n × n × n defined as b2 mki j = b12 mki j , if m ≤ n1 , and a2 mki j = 0, otherwise; bT1 (t) denotes the tensor obtained from b1 (t) by transposing its two rightmost indices. Substituting the expression (1.89) in (1.84) and the expressions (1.90),(1.91) in (1.86), the filtering equations for the optimal estimate m(t) = [m1 (t), m2 (t)] of the bilinear state x(t) = [x1 (t), x2 (t)] and the error variance P(t) are obtained dm(t) = (a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t))dt + P(t)AT (t)(B(t)BT (t))−1 [dy(t) − (A0(t) + A(t)m(t))dt],

(1.92)

m(t0 ) = E(x(t0 ) | FtY )),

dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T + b0 (t)bT0 (t) + b0(t)(b1 (t)m(t))T + (b1(t)m(t))bT0 (t) + b1(t)P(t)bT1 (t)+ b1 (t)m(t)mT (t)bT1 (t) + b0(t)(P(t) + m(t)mT (t))bT2 (t)+

(1.93)

38

1 Optimal Filtering for Polynomial Systems

b2 (t)(P(t) + m(t)mT (t))bT0 (t) + b1(t)(3m(t)P(t) + m(t)(m(t)mT (t)))bT2 (t)+ b2 (t)(3P(t)mT (t) + (m(t)mT (t))mT (t))bT1 (t) + 3(b2(t)P(t))(b2 (t)P(t))T + 3b2 (t)(P(t)m(t)mT (t) + m(t)mT (t)P(t))bT2 (t)+ (b2 (t)(m(t)mT (t)))(b2 (t)(m(t)mT (t)))T )dt − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t)dt. P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )). By means of the preceding derivation, the following result is proved. Theorem 1.4. The optimal finite-dimensional filter for the bilinear state (1.80),(1.81) with partially unmeasured linear part (1.81), where the quadratic polynomials f (x,t) and g(x,t) are defined by (1.87),(1.88), over the linear observations (1.82),(1.83) is given by the equation (1.92) for the optimal estimate m(t) = E(x(t) | FtY ) and the equation (1.93) for the estimation error variance P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ]. Thus, based on the general non-closed system of the filtering equations (1.84),(1.85), it is proved that the closed system of the filtering equations (1.84),(1.86) can be obtained for any polynomial state (1.80),(1.81) with partially measured linear part (1.81) and multiplicative polynomial noise over linear observations (1.82),(1.83). Furthermore, the specific form (1.92),(1.93) of the closed system of the filtering equations corresponding to a bilinear state is derived. In the next subsection, as an example, the designed optimal filter for a bilinear state with partially measured linear part and polynomial multiplicative noise over linear observations is applied to solution of the cubic sensor optimal filtering problem, stated in [39], additionally assuming a Gaussian initial condition for the augmented system state. 1.4.3

Cubic Sensor Optimal Filtering Problem

This subsection presents an example of designing the optimal filter for the cubic sensor filtering problem, stated in [39], where a Gaussian state initial condition is additionally assumed, reducing it to the optimal filtering problem for a bilinear state with partially measured linear part and polynomial multiplicative noise over linear observations. Let the unmeasured scalar state x(t) satisfy the trivial linear equation dx(t) = dw1 (t),

x(0) = x0 ,

(1.94)

and the observation process be given by the scalar cubic sensor equation dy(t) = x3 (t)dt + dw2 (t),

(1.95)

where w1 (t) and w2 (t) are standard Wiener processes independent of each other and of a Gaussian random variable x0 serving as the initial condition in (1.94). The filtering problem is to find the optimal estimate for the linear state (1.94), using the cubic sensor observations (1.95). Let us reformulate the problem, introducing the stochastic process z(t) = x3 (t). Using the Ito formula (see ([41], Lemma 4.2, formula (4.55) or [70], Theorem 5.1.1, formula

1.4 Filtering Problem for Polynomial State with Partially Measured Linear Part

39

(5.1.7)) for the stochastic differential of the cubic function x3 (t), where x(t) satisfies the equation (1.94), the following equation is obtained for z(t) dz(t) = 3x(t)dt + 3x2(t)dt + 3x2 (t)dw1 (t),

z(0) = x30 .

(1.96)

Note that the addition 3x(t)dt appears in view of the second derivative in x in the Ito formula. The initial condition z(0) = x30 is considered a Gaussian random vector. This assumption is quite admissible in the filtering framework, since the real distributions of x(t) and z(t) are actually unknown. In terms of the process z(t), the observation equation (1.95) takes the form (1.97) dy(t) = z(t)dt + dw2 (t). The obtained filtering system includes two equations, (1.96) and (1.94), for the partially measured state [z(t), x(t)] and an equation (1.97) for the observations y(t), where z(t) is a completely measured quadratic state with multiplicative quadratic noise, x(t) is an unmeasured linear state, and y(t) is a linear observation process directly measuring the state z(t). Hence, the designed optimal filter can be applied for solving this problem. The filtering equations (1.92),(1.93) take the following particular form for the system (1.96),(1.94),(1.97) dm1 (t) = (3m2 (t) + 3m22(t) + 3P22(t))dt + P11(t)[dy(t) − m1(t)dt], dm2 (t) = P12 (t)[dy(t) − m1(t)dt], with the initial conditions m1 (0) = E(x0 | y(0)) = m10 and

m2 (0) = E(x30

(1.98) (1.99)

| y(0)) = m20 ,

2 2 P˙11 (t) = 12(P12 (t)m2 (t)) + 6P12(t) + 27P22 (t) + 27P22(t)m22 (t) + 9m42(t) − P11 (t), (1.100) P˙12 (t) = 6(P22(t)m2 (t)) + 3P22(t) + 3(m22 (t) + P22(t)) − P11(t)P12 (t), (1.101) 2 P˙22 (t) = 1 − P12 (t),

(1.102)

with the initial condition P(0) = E(([x0 , x30 ]T − m(0))([x0 , x30 ]T − m(0))T | y(0)) = P0 . Here, m1 (t) is the optimal estimate for the state z(t) = x3 (t) and m2 (t) is the optimal estimate for the state x(t). Numerical simulation results are obtained solving the systems of filtering equations (1.98)–(1.102). The obtained values of the state estimate m2 (t) satisfying the equation (1.99) are compared to the real values of the state variable x(t) in (1.94). For the filter (1.98)–(1.102) and the reference system (1.96),(1.94),(1.97) involved in simulation, the following initial values are assigned: x0 = 0, m20 = 10, m10 = 1000, P011 = 15, P012 = 3, P022 = 1. Gaussian disturbances dw1 (t) and dw2 (t) are realized using the built-in MatLab white noise functions. The simulation interval is [0, 0.05]. Figure 1.7 shows the graphs of the reference state variable x(t) (1.94) and its optimal estimate m2 (t) (1.99), as well as the observation process y(t) (1.95), in the entire simulation interval from t0 = 0 to T = 0.05. It can be observed that the optimal estimate given by (1.98)–(1.102) converges to the real state (1.94) very rapidly, in spite of a considerable error in the initial conditions, m20 − x0 = 10, m10 − z(0) = 1000, and very noisy observations which do not even reproduce the shape of z(t) = x3 (t). Moreover, the estimated signal x(t) itself is a Wiener process, i.e., the integral of a white Gaussian

40

1 Optimal Filtering for Polynomial Systems

noise, which makes the filtering problem even more difficult. It should also be noted that the extended Kalman filter (EKF) approach fails for the system (1.94),(1.95), since the linearized value of x3 (t) at zero is the zero-valued constant, therefore, the observation process would consist of pure noise. Thus, it can be concluded that the obtained optimal filter (1.98)–(1.102) solves the optimal cubic sensor filtering problem ([39]) and yields a really good estimate of the unmeasured state in presence of quite complicated observation conditions. Discussion The optimal filter is obtained for polynomial system states with polynomial multiplicative noise and partially measured linear part over linear observations. It is shown that the optimal filter can be obtained in a closed form for any polynomial state and any polynomial multiplicative noise; for bilinear state and multiplicative noise, the optimal closed-form filter is explicitly derived in Theorem 1.4. Based on the optimal filter for a bilinear state, the optimal solution is obtained for the cubic sensor filtering problem, additionally assuming a Gaussian initial condition for the augmented system state. The resulting filter yields a reliable and rapidly converging estimate, in spite of a significant difference in the initial conditions between the state and estimate and very noisy observations, in the situation where the unmeasured state itself is a Wiener process and the extended Kalman filter (EKF) approach fails. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration.

1.5 Filtering Problem for Linear State over Polynomial Observations 1.5.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ t0 , and let (W1 (t), Ft ,t ≥ t0 ) and (W2 (t), Ft ,t ≥ t0 ) be independent Wiener processes. The Ft -measurable random process (x(t), y(t)) is described by a linear differential equation for the system state dx(t) = (a0 (t) + a(t)x(t))dt + b(t)dW1(t)dt,

x(t0 ) = x0 ,

(1.103)

and a nonlinear polynomial differential equation for the observation process dy(t) = h(x,t)dt + B(t)dW2(t).

(1.104)

Here, x(t) ∈ Rn is the state vector and y(t) ∈ Rn is the observation vector of the same dimension. The initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t), and W2 (t) are independent. It is assumed that B(t)BT (t) is a positive definite matrix. All coefficients in (1.103)–(1.104) are deterministic functions of time of appropriate dimensions. The nonlinear function h(x,t) forms the drift in the observation equation (1.104). The nonlinear function h(x,t) is considered a polynomial of n variables, components of the state vector x(t) ∈ Rn , with time-dependent coefficients (see Subsection 1.1.1 for definition).

1.5 Filtering Problem for Linear State over Polynomial Observations

41

The estimation problem is to find the optimal estimate x(t) ˆ of the system state x(t), based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, that is given by the conditional expectation x(t) ˆ = mx (t) = E(x(t) | FtY ) of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As usual, the matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differential of the conditional expectation E(x(t) | FtY ) and its variance P(t) and given in the following subsection. 1.5.2

Optimal Filter for Linear State over Polynomial Observations

Let us reformulate the problem, introducing the stochastic process z(t) = h(x,t). Using the Ito formula (see ([41], Lemma 4.2, formula (4.55) or [70], Theorem 5.1.1, formula (5.1.7)) for the stochastic differential of the nonlinear function h(x,t), where x(t) satisfies the equation (1.103), the following equation is obtained for z(t) dz(t) =

∂ h(x,t) ∂ h(x,t) (a0 (t) + a(t)x(t))dt + dt+ ∂x ∂t

∂ h(x,t) 1 ∂ h2 (x,t) b(t)dW1 (t), b(t)bT (t)dt + 2 2 ∂x ∂x

z(0) = z0 .

(1.105)

T Note that the addition 12 ∂ h∂ x(x,t) 2 b(t)b (t)dt appears in view of the second derivative in x in the Ito formula. The initial condition z0 ∈ Rn is considered a conditionally Gaussian random vector. This assumption is quite admissible in the filtering framework, since the real distributions of x(t) and z(t) are actually unknown. A key point for further derivations is that the right-hand side of the equation (1.105) is a polynomial in x. Indeed, since h(x,t) is a polynomial in x, the functions ∂ h(x,t) ∂x , 2

∂ h(x,t) ∂ h(x,t) ∂ x x(t), ∂t ,

∂ h2 (x,t)

and ∂ x2 are also polynomial in x. Thus, the equation (1.105) is a polynomial state equation with a polynomial multiplicative noise. It can be written in the compact form dz(t) = f (x,t)dt + g(x,t)dW1(t),

z(t0 ) = z0 ,

(1.106)

where

∂ h(x,t) ∂ h(x,t) (a0 (t) + a(t)x(t))dt + dt+ ∂x ∂t 1 ∂ h2 (x,t) ∂ h(x,t) b(t). b(t)bT (t)dt, g(x,t) = 2 2 ∂x ∂x In terms of the process z(t), the observation equation (1.104) takes the form f (x,t) =

dy(t) = z(t)dt + B(t)dW2 (t).

(1.107)

42

1 Optimal Filtering for Polynomial Systems

The reformulated estimation problem is now to find the optimal estimate [mz (t), mx (t)] of the system state [z(t), x(t)], based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}. This optimal estimate is given by the conditional expectation m(t) = [mz (t), mx (t)] = [E(z(t) | FtY ), E(x(t) | FtY )] of the system state [z(t), x(t)] with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. The matrix function P(t) = E[([z(t), x(t)] − [mz (t), mx (t)])([z(t), x(t)] − [mz (t), mx (t)])T | FtY ] is the estimation error variance for this reformulated problem. The obtained filtering system includes two equations, (1.105) (or (1.106)) and (1.103), for the partially measured state [z(t), x(t)] and an equation (1.107) for the observations y(t), where z(t) is a completely measured polynomial state with polynomial multiplicative noise, x(t) is an unmeasured linear state, and y(t) is a linear observation process directly measuring the state z(t). Hence, the optimal filter for the polynomial system states with unmeasured linear part and polynomial multiplicative noise over linear observations, obtained in Section 1.4, can be applied to solving this problem. The filtering equations take the following particular form for the system (1.106), (1.103), (1.107) dm(t) = E( f¯(x,t) | FtY )dt + P(t)[I, 0]T (B(t)BT (t))−1 (dy(t) − mz (t)dt). dP(t) = (E((x(t) − m(t))( f¯(x,t))T | FtY ) +

(1.108) (1.109)

¯ g¯T (x,t) | FtY )− E( f¯(x,t)(x(t) − m(t))T ) | FtY ) + E(g(x,t) P(t)[I, 0]T (B(t)BT (t))−1 [I, 0]P(t))dt, where f¯(x,t) = [ f (x,t), a0 (t) + a(t)x(t)] is the polynomial drift term and g(x,t) ¯ = [g(x,t), b(t)] is the polynomial diffusion (multiplicative noise) term in the entire system of the state equations (1.105), (1.103). The matrix [I, 0] is the n × 2n matrix composed of the n-dimensional identity matrix and n-dimensional zero matrix. The equations (1.108), (1.109) should be complemented with the initial conditions m(t0 ) = [mz (t0 ), mx (t0 )] = E([z0 , x0 ] | FtY0 ) and P(t0 ) = E[([z0 , x0 ] − m(t0 )([z0 , x0 ] − m(t0 )T | FtY0 ]. The result given in Section 1.4 claims that a closed system of the filtering equations can be obtained for the state [z(t), x(t)] over the observations y(t), in view of the polynomial properties of the functions in the right-hand side of the equation (1.105). Indeed, since the observation matrix in (1.107) is the identity one, i.e., invertible, and the initial condition z0 is assumed conditionally Gaussian, the random variable z(t) − mz (t) is conditionally Gaussian with respect to the observation process y(t) for any t ≥ t0 . Moreover, the random variable x(t) − mx (t) is also conditionally Gaussian with respect to the observation process y(t) for any t ≥ t0 , because x(t) is Gaussian, in view of (1), y(t) depends only on z(t), in view of (1.107), and the assumed conditional Gaussianity of the initial random vector z0 . Hence, the entire random vector [z(t), x(t)] − m(t) is conditionally Gaussian with respect to the observation process y(t) for any t ≥ t0 , and the following considerations are applicable.

1.5 Filtering Problem for Linear State over Polynomial Observations

43

If the functions f¯(x,t) and g(x,t) ¯ are polynomial functions of the state x with timedependent coefficients, the expressions of the terms E( f¯(x,t) | FtY ) in (1.108) and E((x(t) − m(t)) f¯T (x,t)) | FtY ) and E(g(x,t) ¯ g¯T (x,t) | FtY ) in (1.109), which should be calculated to obtain a closed system of filtering equations, would also include only polynomial terms of x. Then, those polynomial terms can be represented as functions of m(t) and P(t) using the expressions for superior conditional moments of random variable x(t) − m(t), which are pointed out in Subsection 1.2.2. After representing all polynomial terms in (1.108) and (1.109), that are generated upon expressing E( f¯(x,t) | FtY ), E((x(t)− m(t)) f¯T (x,t)) | FtY ), and E(g(x,t) ¯ g¯T (x,t) | FtY ), as functions of m(t) and P(t), a closed form of the filtering equations would be obtained. The corresponding repre¯ g¯T (x,t) | FtY ) sentations of E( f¯(x,t) | FtY ), E((x(t) − m(t)) f¯T (x,t)) | FtY ), and E(g(x,t) have been derived in for certain polynomial functions f (x,t) and g(x,t) in Sections 1.3 and 1.4. In the next subsection, a closed form of the filtering equations will be obtained for a particular case of a scalar third degree monomial function h(x,t) in the equation (1.104). It should be noted, however, that application of the same procedure would result in designing a closed system of the filtering equations for any polynomial function h(x,t) ∈ Rn in (1.104). 1.5.3

Example: Third-Order Sensor Filtering Problem

This subsection presents an example of designing the optimal filter for a linear state over third-order polynomial observations, reducing it to the optimal filtering problem for a quadratic polynomial state with partially measured linear part and quadratic polynomial multiplicative noise over linear observations, where a conditionally Gaussian state initial condition is additionally assumed. Let the unmeasured scalar state x(t) satisfy the trivial linear equation dx(t) = dw1 (t),

x(0) = x0 ,

(1.110)

and the observation process be given by the scalar third degree sensor equation dy(t) = (x3 (t) + x(t))dt + dw2 (t),

(1.111)

where w1 (t) and w2 (t) are standard Wiener processes independent of each other and of a Gaussian random variable x0 serving as the initial condition in (1.110). The filtering problem is to find the optimal estimate for the linear state (1.110), using the third-order sensor observations (1.111). Let us reformulate the problem, introducing the stochastic process z(t) = h(x,t) = x3 (t) + x(t). Using the Ito formula (see ([41], Lemma 4.2, formula (4.55) or [70], Theorem 5.1.1, formula (5.1.7)) for the stochastic differential of the cubic function h(x,t) = x3 (t) + x(t), where x(t) satisfies the equation (1.110), the following equation is obtained for z(t) dz(t) = (1 + 3x(t) + 3x2(t))dt + (3x2(t) + 1)dw1 (t),

z(0) = z0 .

(1.112)

Here, ∂ h(x,t) = 3x2 (t) + 1, 12 ∂ h∂ x(x,t) = 3x(t), and ∂ h(x,t) = 0; therefore, f (x,t) = 2 ∂x ∂t 2 2 1 + 3x(t) + 3x (t) and g(x,t) = 3x (t) + 1. The initial condition z0 is considered a 2

44

1 Optimal Filtering for Polynomial Systems

conditionally Gaussian random variable. In terms of the process z(t), the observation equation (1.111) takes the form dy(t) = z(t)dt + dw2 (t).

(1.113)

The obtained filtering system includes two equations, (1.112) and (1.110), for the partially measured state [z(t), x(t)] and an equation (1.113) for the observations y(t), where z(t) is a completely measured quadratic state with multiplicative quadratic noise, x(t) is an unmeasured linear state, and y(t) is a linear observation process directly measuring the state z(t). Hence, the designed optimal filter can be applied for solving this problem. The filtering equations (1.108),(1.109) take the following particular form for the system (1.112),(1.110),(1.113) dm1 (t) = (1 + 3m2(t) + 3m22(t) + 3P22(t))dt + P11(t)[dy(t) − m1(t)dt], dm2 (t) = P12 (t)[dy(t) − m1(t)dt],

(1.114) (1.115)

with the initial conditions m1 (0) = E(x0 | y(0)) = m10 and m2 (0) = E(x30 | y(0)) = m20 , 2 P˙11 (t) = 12(P12 (t)m2 (t)) + 6P12(t) + 27P22 (t) + 27P22(t)m22 (t) +

(1.116)

2 9m42 (t) + 6P22(t) + 6m22 + 1 − P11 (t),

P˙12 (t) = 6(P22(t)m2 (t)) + 3P22(t) + 3(m22 (t) + P22(t)) − P11(t)P12 (t), 2 P˙22 (t) = 1 − P12 (t),

(1.117) (1.118)

with the initial condition P(0) = E(([x0 , z0 ]T − m(0))([x0 , z0 ]T − m(0))T | y(0)) = P0 . Here, m1 (t) is the optimal estimate for the state z(t) = x3 (t) + x(t) and m2 (t) is the optimal estimate for the state x(t). Numerical simulation results are obtained solving the systems of filtering equations (1.114)–(1.118). The obtained values of the state estimate m2 (t) satisfying the equation (1.115) are compared to the real values of the state variable x(t) in (1.110). For the filter (1.114)–(1.118) and the reference system (1.112),(1.110),(1.113) involved in simulation, the following initial values are assigned: x0 = z0 = 0, m20 = 10, m10 = 1000, P011 = 15, P012 = 3, P022 = 1. Gaussian disturbances dw1 (t) and dw2 (t) are realized using the built-in MatLab white noise functions. The simulation interval is [0, 0.05]. The obtained simulation results are very much similar to those shown in Fig. 1.7 from the preceding section. The optimal estimate given by (1.114)–(1.118) converges to the real state (1.110) very rapidly, in spite of a considerable error in the initial conditions, m20 − x0 = 10, m10 − z(0) = 1000, and very noisy observations which do not even reproduce the shape of z(t) = x3 (t) + x(t). Moreover, the estimated signal x(t) itself is a Wiener process, i.e., the integral of a white Gaussian noise, which makes the filtering problem even more difficult. It should also be noted that the extended Kalman filter (EKF) approach fails for the system (1.116),(1.117), since the linearized value 3x2 (t) + 1 at zero is the unit-valued constant, therefore, the observation process would consist of pure noise.

1.5 Filtering Problem for Linear State over Polynomial Observations

45

1500

observations

1000 500 0 -500 -1000 -1500

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0.05

0.03

0.035

0.04

0.045

0.05

time

real state and optimal estimate

10

5

0

-5

0

0.005

0.01

0.015

0.02

0.025 time

Fig. 1.7. Above. Graph of the observation process y(t) in the interval [0, 0.05]. Below. Graphs of the real state x(t) (solid line) and its optimal estimate m2 (t) (dashed line) in the interval [0, 0.05].

Thus, it can be concluded that the obtained optimal filter (1.114)–(1.118) solves the optimal third-order sensor filtering problem for the system (1.110),(1.111) and yields a really good estimate of the unmeasured state in presence of quite complicated observation conditions. Discussion The optimal filter is obtained for linear system states over polynomial observations. It is shown that the optimal filter can be obtained in a closed form for any polynomial function in the observation equation. Based on the optimal filter for a quadratic state with partially measured linear part and quadratic multiplicative noise, the optimal solution is obtained for the third-order sensor filtering problem, additionally assuming a conditionally Gaussian initial condition for the augmented system state. The resulting filter yields a reliable and rapidly converging estimate, in spite of a significant difference in the initial conditions between the state and estimate and very noisy observations, in the situation where the unmeasured state itself is a Wiener process and the extended Kalman filter (EKF) approach fails.

2 Further Results: Optimal Identification and Control Problems

2.1 Optimal Joint State and Parameter Identification Problem for Linear Systems 2.1.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ t0 , and let (W1 (t), Ft ,t ≥ t0 ) and (W2 (t), Ft ,t ≥ t0 ) be independent Wiener processes. The Ft -measurable random process (x(t), y(t)) is described by a linear differential equation with unknown vector parameter θ for the system state dx(t) = (a0 (θ ,t) + a(θ ,t)x(t))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(2.1)

and a linear differential equation for the observation process dy(t) = (A0 (t) + A(t)x(t))dt + B(t)dW2(t).

(2.2)

Here, x(t) ∈ Rn is the state vector, y(t) ∈ Rn is the linear observation vector, such that the observation matrix A(t) ∈ Rn×n is invertible, and θ (t) ∈ R p , p ≤ n × n + n, is the vector of unknown entries of matrix a(θ ,t) and unknown components of vector a0 (θ ,t). The latter means that both structures contain unknown components a0i (t) = θk (t), k = 1, . . . , p1 ≤ n and ai j (t) = θk (t), k = p1 + 1, . . ., p ≤ n × n + n, as well as known components a0i (t) and ai j (t), whose values are known functions of time. The initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t), and W2 (t) are independent. It is assumed that B(t)BT (t) is a positive definite matrix. All coefficients in (2.1)–(2.2) are deterministic functions of time of appropriate dimensions. It is considered that there is no useful information on values of the unknown parameters θk , k = 1, . . . , p, and this uncertainty even grows as time tends to infinity. In other words, the unknown parameters can be modeled as Ft -measurable Wiener processes d θ (t) = dW3 (t),

(2.3)

with unknown initial conditions θ (t0 ) = θ0 ∈ R p , where (W3 (t), Ft ,t ≥ t0 ) is a Wiener process independent of x0 , W1 (t), and W2 (t). M. Basin: New Trends in Optimal Filtering, LNCIS 380, pp. 47–74, 2008. c Springer-Verlag Berlin Heidelberg 2008 springerlink.com 

48

2 Further Results: Optimal Identification and Control Problems

The estimation problem is to find the optimal estimate zˆ(t) = [x(t), ˆ θˆ (t)] of the combined vector of the system states and unknown parameters z(t) = [x(t), θ (t)], based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, that minimizes the Euclidean 2-norm J = E[(z(t) − zˆ(t))T (z(t) − zˆ(t)) | FtY ] at every time moment t. Here, E[ξ (t) | FtY ] means the conditional expectation of a stochastic process ξ (t) = (z(t) − zˆ(t))T (z(t) − zˆ(t)) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As known, this optimal estimate is given by the conditional expectation zˆ(t) = m(t) = E(z(t) | FtY ) of the system state z(t) = [x(t), θ (t)] with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. The matrix function P(t) = E[(z(t) − m(t))(z(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the optimal filtering equations for polynomial-linear states with partially measured linear part over linear observations (Subsection 1.2.2). 2.1.2

Optimal State Filter and Parameter Identifier for Linear Systems

To apply the optimal filtering equations (1.44),(1.45) obtained in Subsection 1.2.2 to the state vector z(t) = [x(t), θ (t)], governed by the equations (2.1) and (2.3), over the linear observations (2.2), the state equation (2.1) should be written in the polynomial form. For this purpose, matrix a1 (t) ∈ R(n+p)×(n+p), a cubic tensor a2 (t) ∈ R(n+p)×(n+p)×(n+p), and a vector c0 (t) ∈ R(n+p) are introduced as follows. The equation for the i-th component of the state vector is given by n

n

j=1

j=1

dxi (t) = (a0i (t) + ∑ ai j (t)x j (t))dt + ∑ bi j (t)dW1 j (t),

xi (t0 ) = x0i .

Then: 1. If the variable a0i (t) is a known function, then the i-th component of the vector c0 (t) is set to this function, c0i (t) = a0i (t); otherwise, if the variable a0i (t) is an unknown function, then the (i, n + i)-th entry of the matrix a1 (t) is set to 1. 2. If the variable ai j (t) is a known function, then the (i, j)-th component of the matrix a1 (t) is set to this function, a1i j (t) = ai j (t); otherwise, if the variable ai j (t) is an unknown function, then the (i, p1 + k, j)-th entry of the cubic tensor a2 (t) is set to 1, where k is the number of this current unknown entry in the matrix ai j (t), counting the unknown entries consequently by rows from the first to n-th entry in each row. 3. All other unassigned entries of the matrix a1 (t), cubic tensor a2 (t), and vector c0 (t) are set to 0.

2.1 Optimal Joint State and Parameter Identification Problem for Linear Systems

49

Using the introduced notation, the state equations (2.1),(2.3) for the vector z(t) = [x(t), θ (t)] ∈ Rn+p can be rewritten as dz(t) = (c0 (t) + a1(t)z(t) + a2(t)z(t)zT (t))dt + diag[b(t), I p×p]d[W1T (t),W3T (t)]T , (2.4) z(t0 ) = [x0 , θ0 ], where the matrix a1 (t), cubic tensor a2 (t), and vector c0 (t) have already been defined, and I p×p is the p × p - dimensional identity matrix. The equation (2.4) is bilinear with respect to the extended state vector z(t) = [x(t), θ (t)]. Thus, the estimation problem is now reformulated as to find the optimal estimate zˆ(t) = m(t) = [x(t), ˆ θˆ (t)] for the state vector z(t) = [x(t), θ (t)], governed by the bilinear equation (2.4), based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, satisfying the equation (2.2). The solution of this problem is obtained using the optimal filtering equations (1.44),(1.45) for bilinear-linear states with partially measured linear part over linear observations, obtained in Subsection 1.2.2, and given by dm(t) = (c0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t))dt +

(2.5)

P(t)[A(t), 0m×p ]T (B(t)BT (t))−1 [dy(t) − [A(t), 0m×p]m(t)dt], m(t0 ) = [E(x(t0 ) | FtY ), E(θ (t0 ) | FtY )], dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T +

(2.6)

(diag[b(t), I p])(diag[b(t), I p]T ))dt− P(t)[A(t), 0m×p ]T (B(t)BT (t))−1 [A(t), 0m×p ]P(t)dt, P(t0 ) = E((z(t0 ) − m(t0 ))(z(t0 ) − m(t0 ))T | FtY ), where 0m×p is the m × p - dimensional zero matrix; P(t) is the conditional variance of the estimation error z(t) − m(t) with respect to the observations Y (t). By means of the preceding derivation, the following result is proved. Theorem 2.1. The optimal finite-dimensional filter for the extended state vector z(t) = [x(t), θ (t)], governed by the equation (2.4), over the linear observations (2) is given by the equation (2.5) for the optimal estimate zˆ(t) = m(t) = [x(t), ˆ θˆ (t)] = E([x(t), θ (t)] | FtY ) and the equation (2.6) for the estimation error variance P(t) = E[(z(t) − m(t))(z(t) − m(t))T | FtY ]. This filter, applied to the subvector θ (t), also serves as the optimal identifier for the vector of unknown parameters θ (t) in the equation (1), yielding the estimate subvector θˆ (t) as the optimal parameter estimate. Thus, based on the general optimal filtering equations for bilinear-linear states with partially measured linear part over linear observations, the optimal state filter and parameter identifier is obtained for the linear system (2.1) with unknown parameters, based on the linear state observations (2.2). In the next section, performance of the designed optimal state filter and parameter identifier is verified in an illustrative example.

50

2.1.3

2 Further Results: Optimal Identification and Control Problems

Example

This section presents an example of designing the optimal filter and identifier for a linear system state with an unknown multiplicative parameter, based on linear state measurements. Let the system state x(t) satisfy the linear equation with unknown parameter θ x(t) ˙ = θ x(t) + ψ1(t),

x(0) = x0 ,

(2.7)

and the observation process be given by the linear equation y(t) = x(t) + ψ2 (t),

(2.8)

where ψ1 (t) and ψ2 (t) are white Gaussian noises, which are the weak mean square derivative of standard Wiener processes. The equations (2.7)–(2.8) present the conventional form for the equations (2.1)–(2.2), which is actually used in practice [6]. The parameter θ is modelled as a standard Wiener process, i.e., satisfies the equation d θ (t) = dW3 (t),

θ (0) = θ0 ,

which can also be written as

θ˙ (t) = ψ3 (t),

θ (0) = θ0 ,

(2.9)

where ψ3 (t) is a white Gaussian noise. The filtering problem is to find the optimal estimate for the bilinear-linear state (2.7),(2.9), using linear observations (2.8) confused with independent and identically distributed disturbances modelled as white Gaussian noises. Let us set the filtering horizon time to T = 10. The filtering equations (2.5),(2.6) take the following particular form for the system (2.7)–(2.9) (2.10) m˙ 1 (t) = m1 (t)m2 (t) + P12(t) + P11(t)[y(t) − m1(t)], m˙ 2 (t) = P12 (t)[y(t) − m1(t)],

(2.11)

with the initial conditions m1 (0) = E(x0 | y(0)) = m10 and m2 (0) = E(θ0 | y(0)) = m20 , 2 (t), P˙11 (t) = 1 + 2P11(t)m2 (t) − P11

(2.12)

P˙12 (t) = 1 + 2P12(t)m2 (t) − P11(t)P12 (t), 2 (t), P˙22 (t) = 1 − P12

with the initial condition P(0) = E(([x0 , θ0 ] − m(0))([x0, θ0 ] − m(0))T | y(0)) = P0 . Numerical simulation results are obtained solving the system of filtering equations (2.10)–(2.12). The obtained values of the estimates m1 (t), estimate for x(t), and m2 (t), estimate for θ (t), are compared to the real values of the state variable x(t) and parameter θ in (2.7)–(2.9). For the filter (2.10)–(2.12) and the reference system (2.7)–(2.9) involved in simulation, the following initial values are assigned: x0 = 1000, m10 = 0.1, m20 = 0, P110 = P120 = P220 = 100. The unknown parameter θ is assigned as θ = 0.1 in the first

Real state value, Optimal state estimate

2.1 Optimal Joint State and Parameter Identification Problem for Linear Systems

51

3000 2500 2000 1500 1000 500 0

0

1

2

3

4

5

6

7

8

9

10

6

7

8

9

10

time

Optimal parameter estimate

5 4 3 2 1 0.1 −1 −2

0

1

2

3

4

5 time

Fig. 2.1. Graphs of the reference state variable x(t) (thick line above), the optimal state estimate m1 (t) (thin line above), and graph of the optimal parameter estimate m2 (t) (thick line below) for the unstable system (2.7) on the simulation interval [0, 10]

simulation and as θ = −0.1 in the second one, thus considering the system (2.7) unstable and stable, respectively. Gaussian disturbances ψ1 (t) and ψ2 (t) in (2.9) is realized using the built-in MatLab white noise function. The following graphs are obtained: graphs of the reference state variable x(t), the optimal state estimate m1 (t), and the optimal state estimate m2 (t) for the unstable system (2.7) (θ = 0.1); graphs of the reference state variable x(t), the optimal state estimate m1 (t), and the optimal state estimate m2 (t) for the stable system (2.7) (θ = −0.1). The graphs of all those variables are shown on the entire simulation interval from t0 = 0 to T = 10 in Figs. 2.1 and 2.2, respectively. It can be observed that, in both cases, the state estimate m1 (t) converges to the real state x(t) and the parameter estimate m2 (t) converges to the real value (0.1 or -0.1) of the unknown parameter θ (t). Note that the gain matrix entries P11 (t) and P12 (t) converge

Real state value, Optimal state estimate

52

2 Further Results: Optimal Identification and Control Problems

1500

1000

500

0

0

1

2

3

4

5

6

7

8

9

10

6

7

8

9

10

time

Optimal parameter estimate

4 3 2 1 −0.1 −1 −2

0

1

2

3

4

5 time

Fig. 2.2. Graphs of the reference state variable x(t) (thick line above), the optimal state estimate m1 (t) (thin line above), and graph of the optimal parameter estimate m2 (t) (thick line below) for the stable system (2.7) on the simulation interval [0, 10]

to finite values close to 1 as time tends to infinity, which are expected. Thus, it can be concluded that, in both cases, the designed optimal state filter and parameter identifier (2.5)–(2.6) yields reliable estimates of the unobserved system state and the unknown parameter value. Discussion The simulation results show that the state and parameter estimates calculated using the obtained optimal filter and parameter identifier for linear systems with unknown parameters converge to the real state and parameter values rapidly, in less than 10 time units. This behavior can be classified as very reliable, especially taking into taking into account large deviations in the initial values for the real state and its estimate and large

2.2 Dual Optimal Control Problems for Polynomial Systems

53

values of the initial error variances. Another advantage to be mentioned is that the designed filter and parameter identifier works equally well for stable and unstable systems, which correspond to operation of linear systems in nominal conditions and under persistent external disturbances, respectively. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration.

2.2 Dual Optimal Control Problems for Polynomial Systems 2.2.1

Optimal Control Problem for Bilinear State with Linear Input

Problem Statement Consider the polynomial bilinear system x(t) ˙ = a1 (t)x(t) + a2(t)x(t)xT (t) + G(t)u(t),

x(t0 ) = x0 ,

(2.13)

where x is the n-dimensional state vector, a1 is an n × n - matrix, a2 is a 3D tensor of dimension n × n × n, which is symmetric over its rightmost indices: (a2 )ki j (t) = (a2 )k ji (t), k, i, j = 1, . . . , n, and u(t) is the control variable. The quadratic cost function to be minimized is defined as follows: 1 1 J = xT (T )ψ x(T ) + 2 2

 T t0

uT (s)R(s)u(s)ds +

1 2

 T

xT (s)L(s)x(s)ds,

(2.14)

t0

where R is positive and ψ , L are nonnegative definite symmetric matrices, and T > t0 is a certain time moment. The optimal control problem is to find the control u(t), t ∈ [t0 , T ], that minimizes the criterion J along with the trajectory x∗ (t), t ∈ [t0 , T ], generated upon substituting u∗ (t) into the state equation (2.13). To find the solution to this optimal control problem, the duality principle [49] can be used. If the optimal control exists in the optimal control problem for a linear system with the quadratic cost function J, the optimal filter exists for the dual linear system with Gaussian disturbances and can be found from the optimal control problem solution, using simple algebraic transformations (duality between the gain matrices and between the gain matrix and variance equations), and vice versa (see [49]). Taking into account the physical duality of the filtering and control problems, the last conjecture should be valid for all cases where the optimal control (or, vice versa, the optimal filter) exists in a closed finite-dimensional form [30]. This proposition is now applied to the optimal filtering problem for bilinear system states over linear observations, which is dual to the stated optimal control problem (2.13),(2.14), and where the optimal filter has already been obtained (see Subsection 1.3). The derived result is then rigorously proved using the maximum principle [67] in Subsection 2.2.4. Optimal Filter for Bilinear State and Linear Observations In this section, the optimal filtering equations for a bilinear state equation over linear observations (obtained in Subsection 1.3) are briefly reminded for reference purposes. Let an unobservable random process x(t) satisfy a bilinear equation dx(t) = (a1 (t)x(t) + a2(t)x(t)xT (t))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(2.15)

54

2 Further Results: Optimal Identification and Control Problems

and linear observations are given by: dy(t) = (A0 (t) + A(t)x(t))dt + B(t)dW2 (t),

(2.16)

where x(t) ∈ Rn is the unobservable state vector and y(t) ∈ Rm is the observation process. W1 (t) and W2 (t) are Wiener processes, whose weak derivatives are Gaussian white noises and which are assumed independent of each other and of the Gaussian initial value x0 . The filtering problem is to find dynamical equations for the best estimate for the real process x(t) at time t, based in the observations Y (t) = [y(s) | t0 ≤ s ≤ t], that is the conditional expectation m(t) = E[x(t) | Y (t)] of the real process x(t) with respect to the observations Y (t). Let P(t) = E[(x(t) − m(t))(x(t) − m(t))T | Y (t)] be the estimate covariance (correlation function). The solution to the stated problem is given by the following system of filtering equations, which is closed with respect to the introduced variables, m(t) and P(t): dm(t) = (a1 (t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t))dt + P(t)AT (t)(B(t)BT (t))−1 [dy(t) − A(t)m(t)dt],

(2.17)

m(t0 ) = E(x(t0 ) | Y (t0 )), dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2(a2(t)m(t))P(t) + 2P(t)(a2(t)m(t))T + (2.18) b(t)bT (t))dt − P(t)AT (t)(B(t)BT (t))−1 A(t)P(t)dt, P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | Y (t0 )) Optimal Control Problem Solution Let us return to the optimal control problem for the bilinear state state (2.13) and the cost function (2.14). This problem is dual to the filtering problem for the bilinear state (2.15) and linear observations with delay (2.16). Since the optimal filter gain matrix in (2.17) is equal to K f = P(t)AT (t)(B(t)BT (t))−1 , the gain matrix in the optimal control problem takes the form of its dual transpose Kc = Kc = (R(t))−1 GT (t)Q(t), and the optimal control law is given by u∗ (t) = Kc x = (R(t))−1 GT (t)Q(t)x(t),

(2.19)

where the matrix function Q(t) is the solution of the following equation dual to the variance equation (2.18) ˙ Q(t) = L(t) − aT1 (t)Q(t) − Q(t)a1(t) − 2(a2(t)x(t))T Q(t) − −1

2Q(t)(a2 (t)x(t)) − Q(t)G(t)R (t)G (t)Q(t), with the terminal condition Q(T ) = ψ .

T

(2.20)

2.2 Dual Optimal Control Problems for Polynomial Systems

55

Upon substituting the optimal control (2.19) into the state equation (2.13), the optimally controlled state equation is obtained x(t) ˙ = a0 (t) + a(t)x(t) + a2(t)x(t)xT (t) + G(t)(R(t))−1 GT (t)Q(t)x(t),

(2.21)

x(t0 ) = x0 .

The results obtained in this section by virtue of the duality principle are proved in the next section, using the general equations of the Pontryagin maximum principle [67]. (Bellman dynamic programming [14] could serve as an alternative verifying approach). It should be noted, however, that application of the maximum principle to the present case gives one only a system of state and co-state equations and does not provide the explicit form of the optimal control or co-state vector. So, the duality principle approach actually provides one with the explicit form of the optimal control and co-state vector, which should then be substituted into the equations given by the rigorous optimality tools and thereby verified. Finally, it should be noted that since the system state (2.13) is nonlinear, it is hard to say something definitive about sufficiency of the obtained necessary optimality conditions, as well as about existence of the optimal control. Equally, boundedness of the optimal solution of the equation (2.13) cannot always be guaranteed for any finite time interval, and existence of solutions to the two-point boundary problem for the equations (2.20),(2.21) needs to be additionally investigated for any particular case of system coefficients. Summing up: the obtained solution presents, in case of a nonlinear system, only some necessary optimality conditions, not all of them, whereas all questions about sufficiency, existence, and solvability remain subject to more extensive research. Proof of Optimal Control Problem Solution Define the Hamiltonian function [67] for the optimal control problem (2.13),(2.14) as H(x, u, q,t) =

1 T (u R(t)u + xT L(t)x) + qT [a0 (t) + 2 a1 (t)x + a2(t)xxT + B(t)u].

(2.22)

Applying the maximum principle condition ∂ H/∂ u = 0 to this specific Hamiltonian function (2.22) yields

∂ H/∂ u = 0 ⇒ R(t)u(t) + GT (t)q(t) = 0, and the optimal control law is obtained as u∗ (t) = −R−1 (t)GT (t)q(t). Taking linearity of the control input in (2.13) into account, let us seek q(t) as a linear function in x(t) q(t) = −Q(t)x(t),

(2.23)

where Q(t) is a square symmetric matrix of dimension n. This yields the complete form of the optimal control u∗ (t) = R−1 (t)GT (t)Q(t)x(t).

(2.24)

56

2 Further Results: Optimal Identification and Control Problems

Note that the transversality condition [67] for q(T ) implies that q(T ) = −∂ J/∂ x(T ) = −ψ x(T ) and, therefore, Q(T ) = ψ . Taking into account that the Hamiltonian is constant in t at the optimal values of its arguments, H(x∗ , u∗ , q∗ ,t) = const, (see [67]) and using the co-state equation dq(t)/dt = −∂ H/∂ x, which gives − dq(t)/dt = L(t)x(t) + aT1 (t)q(t) + (a2(t)x(t))T q(t),

(2.25)

and then substituting (2.23) into (2.25), we obtain ˙ Q(t)x(t) + Q(t)d(x(t))/dt = L(t)x(t) − aT1 (t)Q(t)x(t) − (a2(t)x(t))T Q(t)x(t).

(2.26)

Substituting the expression for x(t) ˙ from the state equation (2.13) into (2.26) and taking into account the optimal control (2.24) yields ˙ Q(t)x(t) + Q(t)(a1 (t)x(t) + a2(t)x(t)xT (t) − G(t)R−1(t)GT (t)Q(t)x(t)) = L(t)x(t) − aT1 (t)Q(t)x(t) − (a2 (t)x(t))T Q(t)x(t).

(2.27)

Upon differentiating the equation (2.27) in x, it is finally transformed into the equation (2.20): ˙ = L(t) − Q(t)a1 (t) − aT1 (t)Q(t) − 2Q(t)(a2(t)x(t)) Q(t) −2(a2(t)x(t))T Q(t) − Q(t)G(t)R−1(t)GT (t)Q(t).

(2.28)

The optimal control problem solution is proved. In the next section, performance of the obtained optimal regulator for bilinear systems is verified against the best regulator available for linearized systems in the optimal regulation problem for a terpolymerization reactor. 2.2.2

Optimal Regulator for Terpolymerization Reactor

The obtained optimal regulator for bilinear system states and linear observations is applied to solution of the optimal control problem for a terpolymerization reactor. The mathematical model of terpolymerization process given by Ogunnaike ([64]) is reduced to ten equations for the concentrations for input reagents, the zeroth live moments of the product molecular weight distribution (MWD), and its first bulk moments. This equations are intrinsically nonlinear (bilinear), so use of their linearization in forming a control law leads to worse performance with respect to the criterion and regulated state values, as it could be seen from the simulations results. Let us rewrite the bilinear state equations (2.13) in the component form using index summations dxk (t)/dt = ∑ a1ki (t)xi (t) + ∑ a2ki j (t)xi (t)x j (t) + ∑ cki (t)ui (t), i

ij

ui (t) = Ri j (t)GTjk (t)Qkl (t)xl (t),

i

k = 1, ..., n,

2.2 Dual Optimal Control Problems for Polynomial Systems

57

where ui (t) is the input control. Then, the control law (2.19) is given by ui (t) = ∑ Ri j (t)GTjk (t)Qkl (t)xl (t),

k = 1, ..., n,

i jkl

and the gain matrix equation (2.20) can be rewritten in the component form as dQi j (t)/dt = Li j (t) − ∑ a1ki (t)Qk j (t) − ∑ Qki (t)a1ik (t) − i

k

2 ∑ a2i jk (t)Qkl xl (t) − 2 ∑ a2 jik (t)Qkl (t)xl (t) − kl

∑

kl

T Qik (t)Gkl (t)R−1 lm (t)Gmn (t)Qn j (t),

klmn

with the final condition Qi j (T ) = ψi j . The optimally controlled state equation (2.21) takes the component form dxk (t)/dt = ∑ a1ki (t)xi (t) + ∑ a2ki j (t)xi (t)x j (t) + ∑ cki (t)Ri j (t)GTjm (t)Qml (t)xl (t), i

ij

i jml

where k = 1, ..., n. The terpolimerization process model reduced to 10 bilinear equations selected from [64] is given by dCm1 (t)/dt = [(1/V )(d Δm1 /dt) − ((1/θ ) + KL1C∗ + K11 μPo

(2.29)

+K21 μQo + K31 μRo )Cm1 ;

dCm2 (t)/dt = (1/V )(d Δm2 /dt) − ((1/θ ) + KL2C∗ + K12 μPo + K22 μQo )Cm2 ); dCm3 (t)/dt = (1/V )(d Δm3 /dt) − ((1/θ ) + K13μPo )Cm3 ; dCm∗ (t)/dt = (1/V )(d Δm∗ /dt) − ((1/θ ) + Kd + KL1Cm1 + KL2Cm2 )C∗ ; d μPo (t)/dt = (−1/θ − Kt1 )μPo + KL1Cm1C∗

−(K12Cm2 + K13Cm3 )μPo + K21Cm1 μQo + K31Cm1 μRo ; d μQo (t)/dt = (−1/θ )μQo + KL2Cm2C∗ − (K21Cm1 + Kt2 )μQo + K12Cm2 μPo ; d μRo (t)/dt = (−1/θ )μRo − (K31Cm1 + Kt3 )μRo + K13Cm3 μPo ; d λ1100(t)/dt = (−1/θ )λ1100 + KL1Cm1C∗ + KL2Cm2C∗ + K11Cm1 μPo +K21Cm1 μQo + K31Cm1 μRo ; d λ1010(t)/dt = (−1/θ )λ1010 + KL1Cm1C∗ + KL2Cm2C∗ + K12Cm2 μPo + K22Cm2 μQo ; d λ1001(t)/dt = (−1/θ )λ1001 + (KL1Cm1 + KL2Cm2 )C∗ + K13Cm3 μPo . Here, the state variables Cm1 ,Cm2 , and Cm3 are the reagent (monomer) concentrations, C∗ is the active catalyst concentration; μPo , μQo , and μRo are the zeroth live moments of the product MWD, and λ1100 , λ1010 , and λ1001 are its first bulk moments. The reactor volume V and residence time θ as well as all coefficients K  s, are known parameters, and Δm1 , Δm2 , Δm3 , Δm∗ stand for the net molar flows of the reagents and active catalyst into the reactor. In this case, only the first four equations contain control inputs: Δm1 = u1 (t), Δm2 = u2 (t), Δm3 = u3 (t), Δm∗ = u4 (t).

58

2 Further Results: Optimal Identification and Control Problems

The quadratic cost function to be minimized takes the form: 2 ) + 1/2 J = 1/2(μRo2 + Cm1

 0.2



+u23 (s) + u24(s))ds +

0

0 0.2

(u21 (s) + u22 (s)

(2.30)

2 (μRo2 (s) + Cm1 (s))ds,

where μRo2 and Cm1 (t) are the variables to be minimized. In other words, the optimal control problem consists in minimizing a certain reaction product moment and the corresponding reagent concentration, using the minimum control energy. In this application, the optimal control law is given by u∗ (t) = (u∗1 (t), u∗2 (t), u∗3 (t), u∗4 (t)), u∗1 (t) = ∑ Q1i (t)xi (t), u∗2 (t) = ∑ Q2i (t)xi (t), i

u∗3 (t)

=

∑

(2.31)

i

Q3i (t)xi (t), u∗4 (t)

i

= ∑ Q4i (t)xi (t), i

where Qi j (t) are the solutions of the system of differential equations dQ11 (t)/dt = 1 + (2/θ )Q11(t) − Q211 (t) − Q212(t) − Q213(t) − Q214(t); dQ22 (t)/dt = (2/θ )Q24(t) − Q212 (t) − Q222(t) − Q223(t) − Q224(t);

(2.32)

dQ33 (t)/dt = (2/θ )Q33(t) − Q213 (t) − Q223(t) − Q233(t) − Q234(t); dQ44 (t)/dt = 2(1/θ + Kd )Q44 (t) + 4(KL1 Q14 (t)(Cm1 (t) + Cm∗ (t) − μ po(t) −λ1100(t) − λ1010(t) − λ1001(t)) + KL2 Q24 (t)(Cm2 (t) + Cm∗ (t) −μQo (t) − λ1100(t) − λ1010(t) − λ1001(t))) − Q214(t) − Q224 (t) − Q234(t) − Q244(t); dQ55 (t)/dt = 2(1/θ − Kt1 )Q55 (t) + 4(K11 Q15 (t)(Cm1 (t) + λ1100(t)) +K12 Q25 (t)(Cm2 (t) + μ po (t) − μQo (t) − λ1010(t)) +K13 Q35 (t)(x(3) + μ po(t) − μRo (t) − λ1001(t))) −Q215 (t) − Q225(t) − Q235(t) − Q245(t); dQ66 (t)/dt = −2(−1/θ + Kt2 )Q66 (t) +4(K21 Q16 (t)(Cm1 (t) − μ po(t) + μQo (t) − λ1100) +K22 Q26(t) (Cm2 (t) − λ1010(t))) − Q216 (t) − Q226(t) − Q236(t) − Q246(t); dQ77 (t)/dt = 1 − 2(−1/θ + Kt3 )Q77 +4(K31 Q17 (Cm1 (t) − μ po(t) + μRo (t) − λ1100(t))) −Q217 (t) − Q227(t) − Q237(t) − Q247(t); dQ88 (t)/dt = (2/θ )Q88(t) − Q218 (t) − Q228(t) − Q238(t) − Q248(t); dQ99 (t)/dt = (2/θ )Q99(t) − Q219 (t) − Q229(t) − Q239(t) − Q249(t); dQ1010 (t)/dt = (2/θ )Q1010(t) − Q2110(t) − Q2210(t) − Q2310(t) − Q2410(t); dQ17 (t)/dt = 1 + (2(1/θ ) − Kt3)Q17 (t) +2(K31 Q11 (t)(Cm1 (t) − μ po(t) + μRo (t) − λ1100(t)))) −Q11 (t)Q17 (t) − Q12 (t)Q27 (t) − Q13(t)Q37 (t) − Q14(t)Q47 (t).

2.2 Dual Optimal Control Problems for Polynomial Systems

59

Upon substituting the optimal control law (2.31) into the state equations (2.29), the optimally controlled state equations are obtained. dCm1 (t)/dt = [(1/V )(du∗1 (t)/dt) − ((1/θ ) + KL1C∗ +K11 μPo + K21 μQo + K31 μRo )Cm1 ;

(2.33)

dCm2 (t)/dt = (1/V )(du∗2 (t)/dt) − ((1/θ ) + KL2C∗ + K12 μPo + K22 μQo )Cm2 ; dCm3 (t)/dt = (1/V )(du∗3 (t)/dt) − ((1/θ ) + K13μPo )Cm3 ; dCm∗ (t)/dt = (1/V )(du∗4 (t)/dt) − ((1/θ ) + Kd + KL1Cm1 + KL2Cm2 )C∗ ;

d μPo (t)/dt = (−1/θ − Kt1 )μPo + KL1Cm1C∗ − (K12Cm2 + K13Cm3 )μPo + K21Cm1 μQo + K31Cm1 μRo ;

d μQo (t)/dt = (−1/θ )μQo + KL2Cm2C∗ − (K21Cm1 + Kt2 )μQo + K12Cm2 μPo ; d μRo (t)/dt = (−1/θ )μRo − (K31Cm1 + Kt3 )μRo + K13Cm3 μPo ; 100(t)

d λ1

/dt = (−1/θ )λ1100 + KL1Cm1C∗ + KL2Cm2C∗ + K11Cm1 μPo +

K21Cm1 μQo + K31Cm1 μRo ; 010(t)

d λ1

/dt = (−1/θ )λ1010 + KL1Cm1C∗ + KL2Cm2C∗ + K12Cm2 μPo + K22Cm2 μQo ;

d λ1001 (t)/dt = (−1/θ )λ1001 + (KL1Cm1 + KL2Cm2 )C∗ + K13Cm3 μPo . For numerical simulation purposes, the system parameter values are all set to 1: V = 1; d Δm1 /dt = 1; KL1 = 1; K11 = 1; K21 = 1; K31 = 1; K32 = 1; d Δm2 /dt = 1; d Δm3 /dt = 1; d Δm∗ /dt = 1; KL2 = 1; KL3 = 1; K12 = 1; K13 = 1; K22 = 1; Kd = 1; Kt1 = 1; Kt2 = 1; Kt3 = 1; θ = 1. The terminal time is set to T = 0.2. For the state variables, the initial conditions are set to 0, except for Cm1 (0) = μRo (0) = 0.5. According to the the gain matrix equation (2.29) and the specific criterion form (2.30), the terminal conditions for (2.32) are given by Qi j (T ) = 0 for all i, j = 1, ..., 10, except for Q17 (T ) = Q77 (T ) = Q11 (T ) = 1. These terminal conditions have been satisfied with the initial conditions Qi j (0) = 0 for all i, j = 1, ..., 10, except for Q17 (0) = 0.49, Q77 (0) = 0.397, and Q11 (0) = 0.61. The rest of the Qi j equations are homogeneous and their solutions are equal to 0. Performance of the obtained optimal regulator for bilinear systems is verified against the best regulator available for the linearized system (2.29), where the optimal control law keeps the form (2.31), but the gain matrix equations (2.32) does not include the bilinear terms, thus forming the classical system of Riccati equations. The optimally controlled state equations keep the form (2.33). For the numerical simulation in the linear case, the system parameters, terminal time, and initial state conditions are assigned the same values. The terminal conditions are also given by Qi j (T ) = 0 for all i, j = 1, ..., 10, except for Q17 (T ) = Q77 (T ) = Q11 (T ) = 1. These terminal conditions have been satisfied with the initial conditions Qi j (0) = 0 for all i, j = 1, ..., 10, except for Q17 (0) = 0.765, Q77 (0) = 0.958, and Q11 (0) = 0.61. The rest of the Qi j equations are homogeneous and their solutions are equal to 0. In Figure 2.3, the simulation graphs are presented for the regulated state variables Cm1 (t) and μRo (t), the criterion J, and the control law u(t) for both examined regulators. The graphs corresponding to the optimal regulator for bilinear systems are depicted by thick continuous lines. The obtained final values are J(0.2) = 0.1467 for the criterion and Cm1 (0.2) = 0.635 and μRo (0.2) = 0.298 for the regulated state variables. The graphs

60

2 Further Results: Optimal Identification and Control Problems 0.7

1.4 1.2

Cm

1

Control

0.65

0.6

0.55

0.5

0

0.05

0.1 time

0.15

0.4

0.2

0

0.05

0.1 time

0.15

0.2

0

0.05

0.1 time

0.15

0.2

0.2

0.45 Criterion

0.15

0.4

r

Miu0

0.8 0.6

0.5

0.35

0.1

0.05

0.3 0.25

1

0

0.05

0.1 time

0.15

0.2

0

Fig. 2.3. Graphs of the state variables Cm1 (t), μRo (t), control law u(t), and the criterion J for both examined regulators. The continuous line corresponds to the designed optimal regulator, and the dotted line corresponds to the linear regulator.

corresponding to the linear regulator are depicted by dotted lines, and the obtained final values are J(0.2) = 0.16 for the criterion and Cm1 (0.2) = 0.662 and μRo (0.2) = 0.299 for the regulated state variables. Note that whereas the thick continuous line for μRo (t) corresponding to the optimal bilinear regulator covers and overshadows the dotted line for μRo (t) corresponding to the linear regulator, the graphs do not coincide although are very close, as it follows from the difference of their final values. Thus, the obtained optimal regulator for bilinear systems shows definitively better performance with respect to the criterion and regulated state values, in comparison to the best regulator available for the linearized system. Discussion The paper has presented the optimal control regulator for a polynomial bilinear system with linear control input and quadratic criterion. The obtained result has been first derived by means of the duality principle, based on the previously designed optimal filter for a bilinear polynomial system over linear observations, and then proved using the maximum principle. Since the previously obtained results yielded the possibility to design the optimal filter for any polynomial state over linear observations in a closed form, there has consequently realized the possibility to design the optimal regulator for an

2.2 Dual Optimal Control Problems for Polynomial Systems

61

arbitrary polynomial system with linear control input and quadratic criterion in the same manner: first deriving the specific form of the optimal control by means of the duality principle and then exploiting the maximum principle for rigorous substantiation. Upon observing the simulation results, it can be concluded that the obtained optimal regulator for bilinear systems shows better performance with respect to the criterion and regulated state values, in comparison to the best regulator available for the linearized system. 2.2.3

Optimal Control for Third-Order Polynomial State with Linear Input

Consider the polynomial system dx(t) = (a0 (t) + a1(t)x(t) + a2(t)x2 (t) + a3 (t)x3 (t))dt + B(t)u(t)dt,

(2.34)

x(t0 ) = x0 . where x(t) ∈ Rn is the system state, whose superior powers are defined as in Subsection 1.3.2: x(t) = (x1 (t), x2 (t), . . . , xn (t)) in Rn , x2 (t) = (x21 (t), x22 (t), . . . , x2n (t)), x3 (t) = (x31 (t), x32 (t), . . . , x3n (t)), and u(t) is the control variable. The quadratic cost function to be minimized is defined as follows: 1 1 J = xT (T )Ψ x(T ) + 2 2

 T t0

uT (s)R(s)u(s)ds +

1 2

 T

xT (s)L(s)x(s)ds,

(2.35)

t0

where x1 is a given vector, R is positive and Ψ , L are nonnegative definite symmetric matrices, and T > t0 is a certain time moment. The optimal control problem is to find the control u∗ (t), t ∈ [t0 , T ], that minimizes the criterion J along with the trajectory x∗ (t),t ∈ [t0 , T ], generated upon substituting u∗ (t) into the state equation (2.34). To find the solution to this optimal control problem, the duality principle [49] could be used. For linear systems, if the optimal control exists in the optimal control problem for a linear system with the quadratic cost function J, the optimal filter exists for the dual linear system with Gaussian disturbances and can be found from the optimal control problem solution, using simple algebraic transformations (duality between the gain matrices and the matrix and variance equations), and vice versa. Taking into account the physical duality of the filtering and control problems, the last conjecture should be valid for all cases where the optimal control (or, vice versa, the optimal filter) exists in a closed finite-dimensional form. This proposition is now applied to a third order polynomial system, for which the optimal filter has already been obtained (see Subsection 1.3.2). Let us return to the optimal control problem for the polynomial state (2.34) with linear control input and the cost function (2.35). This problem is dual to the filtering problem for the polynomial state (4) of degree 3 and the linear observations (2). Since the optimal polynomial filter gain matrix in (2.32) is equal to K f = P(t)AT (t)(B(t)BT (t))−1 , the gain matrix in the optimal control problem takes the form of its dual transpose Kc = (R(t))−1 B(t)Q(t),

62

2 Further Results: Optimal Identification and Control Problems

and the optimal control law is given by u∗ (t) = Kc x = (R(t))−1 B(t)Q(t)x(t),

(2.36)

where the matrix function is the solution of the following equation dual to the variance equation dQ(t) = (−aT1 (t)Q(t) − Q(t)aT1 (t) − 2aT2 (t)Q(t) ∗ xT (t) −

(2.37)

2x(t) ∗ Q(t)a2(t) − 3aT3 (t)Q(t) ∗ qT (t) − 3q(t) ∗ Q(t)a3(t) − 3aT3 (t)Q(t) ∗ ((x2 )T (t)) − 3(x2 (t) ∗ Q(t))a3(t) + L(t) − Q(t)B(t)R−1(t)BT (t)Q(t))dt, with the terminal condition Q(T ) = Ψ . The binary operation ∗ has been introduced in Section 4, and q(t) = (q1 (t), q2 (t), ..., qn (t)) denotes the vector consisting of the diagonal elements of Q(t). Upon substituting the optimal control (2.36) into the state equation (2.34), the optimally controlled state equation is obtained dx(t) = (a0 (t) + a1(t)x(t) + a2(t)x2 (t) + a3(t)x3 (t))dt + B(t)(R(t))−1 BT (t)Q(t)x(t)dt, x(t0 ) = x0 , Note that if the real state vector x(t) is unknown (unobservable), the optimal controller uniting the obtained optimal filter and regulator equations, can be constructed using the separation principle [7] for polynomial systems, which should also be valid if solutions of the optimal filtering and control problems exist in a closed finite-dimensional form. The results obtained in this section by virtue of the duality principle could be rigorously verified through the general equations of the Pontryagin maximum principle [11] or Bellman dynamic programming [2]. 2.2.4

Optimal Third-Order Polynomial Regulator for Automotive System

This section presents application of the obtained optimal regulator for a polynomial system of degree 3 with linear control input and quadratic criterion to controlling the state variables, orientation and steering angles, in the nonlinear kinematical model of car movement [9] given by the nonlinear equations (1.23) from Subsection 1.1.3. The optimal control problem is to maximize the orientation angle using the minimum energy of control u. The corresponding criterion J to be minimized takes the form J = [φ (t) − φ ∗ ]2 +

 T

u2 (t)dt,

0

where T = 0.3, and φ ∗ = 1 is a large value of φ (t) unreachable for time T . Since R = 1 and BT = [0, 1], the optimal control u∗ (t) = (R(t))−1 BT (t)Q(t)x(t) takes the

2.2 Dual Optimal Control Problems for Polynomial Systems

63

form u∗ (t) = q21 (t)φ (t) + q22 (t)δ (t), where the elements q11 (t), q21 (t), q22 (t) of the symmetric matrix Q(t) satisfy the equations dq11 (t) = (−q221 (t))dt, v v v dq12 (t) = (− q211 − q12q22 − q11 − φ 2 q11 )dt, l l l 2v 2v 2v 2 dq22 (t) = (− q12 − q12 q22 − δ q12 − q222)dt. l l l

(2.38)

The system composed of the two last equations of (1.23) and the equations (2.38) should be solved with initial conditions φ (0) = 0.1, δ (0) = 0.1 and terminal conditions q11 (T ) = 1, q12(T ) = 0, q22 (T ) = 0. This boundary problem is solved numerically using the iterative method of direct and reverse passing as follows. The first initial conditions for q’s are guessed, and the system is solved in direct time with the initial conditions at t = 0, thus obtaining certain values for φ and δ at the terminal point T = 0.3. Then, the system is solved in reverse time, taking the obtained terminal values for φ and δ in direct time as the initial values in reverse time, thus obtaining certain values for q’s at the initial point t = 0, which are taken as the initial values for the passing in direct time, and so on. The given initial conditions φ (0) = 0.1, δ (0) = 0.1 are kept fixed for any direct passing, and the given terminal conditions q11 (T ) = 1, q12 (T ) = 0, q22 (T ) = 0 are used as the fixed initial conditions for any reverse passing. The algorithm stops when the system arrives at values q11 (T ) = 1, q12 (T ) = 0, q22 (T ) = 0 after direct passing and at values φ (0) = 0.1, δ (0) = 0.1 after reverse passing. The obtained simulation graphs for φ , δ , and the criterion J are given in Fig. 2.4. These results for polynomial regulator of degree 3 are then compared to the results obtained using the optimal linear regulator, whose matrix Q(t) elements satisfy the Riccati equations dq11 (t) = (−q212 (t))dt, v dq12 (t) = (−q12 q22 − q11 )dt, l 2v dq22 (t) = (− q12 − q222)dt. l

(2.39)

Note that in the linear case the only reverse passing for q’s is necessary, because the system (2.39) does not depend of φ and δ , and the initial values for q’s at t = 0 are obtained after single reverse passing. The simulation graphs for the linear case are given in Fig. 2.5. Thus, two sets of graphs are obtained. 1. Graphs of variables φ and δ satisfying the original system (1.23) and controlled using the optimal linear regulator defined by (2.39); graphs of the corresponding values of the criterion J (Fig. 2.4). 2. Graphs of variables φ and δ satisfying the original system (1.23) and controlled using the optimal third order polynomial regulator defined by (2.38); graphs of the corresponding values of the criterion J (Fig. 2.5). The obtained values of the controlled variables φ and δ and the criterion J are compared for the optimal third order polynomial and linear regulators at the terminal time T = 0.3 in the following table (corresponding to Figs. 2.4 and 2.5).

64

2 Further Results: Optimal Identification and Control Problems

Lineal regulator Third degree polynomial regulator φ (0.3) = 0.132 φ (0.3) = 0.136 δ (0.3) = 0.106 δ (0.3) = 0.127 J = 0.758 J = 0.751 The simulation results show that the values of the controlled variables φ and δ at the terminal point T = 0.3 are greater for the third order regulator than for the linear one (although only the variable φ is maximized) and the criterion value at the terminal point is less for the third order regulator as well. Thus, the third order polynomial regulator controls the system variables better than the linear one from both points of view, thus illustrating, as well as for the filtering problem, the theoretical conclusion. 0.14 0.135 0.13 phi

0.12 0.11 0.1

0

0.05

0.1

0.15 time

0.2

0.25

0.3

0

0.05

0.1

0.15 time

0.2

0.25

0.3

0

0.05

0.1

0.15 time

0.2

0.25

0.3

0.13

delta

0.12 0.11 0.1 0.85

criterion

0.825 0.8

0.775 0.755 0.75

Fig. 2.4. Graphs of variables φ (rad.) and δ (rad.), satisfying the original system (1.23) and controlled using the optimal linear regulator defined by (2.39), against time (sec.); graphs of the corresponding values of the criterion J (rad2 .) against time (sec.)

2.3 Optimal Controller Problem for Third-Order Polynomial Systems

65

phi

0.14 0.135 0.13 0.12 0.11 0.1

0

0.05

0.1

0.15 time

0.2

0.25

0.3

0

0.05

0.1

0.15 time

0.2

0.25

0.3

0

0.05

0.1

0.15 time

0.2

0.25

0.3

delta

0.13 0.12 0.11 0.1 0.85 criterion

0.825 0.8 0.775 0.755 0.75

Fig. 2.5. Graphs of variables φ (rad.) and δ (rad.), satisfying the original system (1.23) and controlled using the optimal third order polynomial regulator defined by (2.38), against time (sec.); graphs of the corresponding values of the criterion J (rad2 .) against time (sec.)

2.3 Optimal Controller Problem for Third-Order Polynomial Systems 2.3.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ 0, and let (W1 (t), Ft ,t ≥ 0) and (W2 (t), Ft , t ≥ 0) be Ft -adapted Wiener processes. Let us consider the unobservable Ft -measurable random process x(t) governed by the third degree polynomial state equation dx(t) = (a0 (t) + a1(t)x(t) + a2(t)x2 (t) + a3(t)x3 (t))dt + G(t)u(t)dt + b(t)dW1(t), (2.40)

66

2 Further Results: Optimal Identification and Control Problems

x(t0 ) = x0 , and the linear output (observation) process dy(t) = (A0 (t) + A(t)x(t))dt + B(t)dW2(t).

(2.41)

Here, x(t) ∈ Rn is the unobservable state vector, whose second and third degrees are defined in the componentwise sense as in Subsection 1.3.2: x2 (t) = (x21 (t), x22 (t), . . . , x2n (t)), x3 (t) = (x31 (t), x32 (t), . . . , x3n (t)), u(t) ∈ R p is the control variable, y(t) ∈ Rm is the observation process, and the independent Wiener processes W1 (t) and W2 (t) represent random disturbances in state and observation equations, which are also independent of an initial Gaussian vector x0 . Let A(t) be a nonzero matrix and B(t)BT (t) be a positive definite matrix. In addition, the quadratic cost function J to be minimized is defined as follows 1 1 J = E[ xT (T )Φ x(T ) + 2 2

 T t0

uT (s)K(s)u(s)ds +

1 2

 T

xT (s)L(s)x(s)ds],

(2.42)

t0

where K is positive definite and Φ , L are nonnegative definite symmetric matrices, T > t0 is a certain time moment, the symbol E[ f (x)] means the expectation (mean) of a function f of a random variable x, and aT denotes transpose to a vector (matrix) a. The optimal control problem is to find the control u∗ (t), t ∈ [t0 , T ], that minimizes the criterion J along with the trajectory x∗ (t), t ∈ [t0 , T ], generated upon substituting u∗ (t) into the state equation (2.40). 2.3.2

Separation Principle for Polynomial Systems

As well as for a linear stochastic system, the separation principle remains valid for a stochastic system given by a third order polynomial equation, linear observations, and a quadratic criterion. Indeed, let us replace the unobservable system state x(t) by its optimal estimate m(t) given by the equation (see Subsection 1.3.2 for statement and derivation) dm(t) = (a0 (t) + a1(t)m(t) + a2(t)p(t) + a2(t)m2 (t) + (2.43) a3 (t)(3p(t) ∗ m(t) + m3(t))dt + G(t)u(t)+ PT (t)AT (t)(B(t)BT (t))−1 (dy − (A0 (t) + A(t)m(t))dt), with the initial condition m(t0 ) = E(x(t0 ) | FtY0 ). Here, m(t) is the best estimate for the unobservable process x(t) at time t based on the observation process Y (t) = {y(s),t0 ≤ s ≤ t}, that is the conditional expectation m(t) = E(x(t) | FtY ), m(t) = [m1 (t), m2 (t), . . . , mn (t)]; P(t) = E[(x(t) − m(t))(x(t) − m(t))T | Y (t)] ∈ Rn is the error covariance matrix; p(t) ∈ Rn is the vector whose components are the variances of the components of x(t) − m(t), i.e., the diagonal elements of P(t); m2 (t) and m3 (t) are defined as the vectors of squares and cubes of the components of m(t): m2 (t) = (m21 (t), m22 (t), . . . , m2n (t), m3 (t) = (m31 (t), m32 (t), . . . , m3n (t); P(t)m(t) is the conventional product of a matrix P(t) by a vector m(t); and p(t) ∗ m(t) is the product of two vectors by components: p(t) ∗ m(t) = [p1 (t)m1 (t), p2 (t)m2 (t), . . . , pn (t)mn (t)]. The best estimate m(t) minimizes the criterion J = E[(x(t) − x(t)) ˆ T (x(t) − x(t)) ˆ | FtY ],

(2.44)

2.3 Optimal Controller Problem for Third-Order Polynomial Systems

67

with respect to selection of the estimate m as a function of observations y(t), at every time moment t. The complementary equation for the covariance matrix P(t) takes the form (see Subsection 2.1 for derivation) dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t) ∗ P(t) +

(2.45)

2(P(t) ∗ mT (t))aT2 (t) + 3a3(t)(p(t) ∗ P(t))+ 3(p(t) ∗ P(t))T aT3 (t) + 3a3(t)(m2 (t) ∗ P(t))+ 3(P(t) ∗ (m2(t))T )aT3 (t) + (b(t)bT (t))− P(t)AT (t)(B(t)BT (t))−1 A(t)P(t))dt, with the initial condition P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | y(t0 )), where the product m(t) ∗ P(t) between a vector m(t) and a matrix P(t) is defined as the matrix whose rows are equal to rows of P(t) multiplied by the same corresponding element of m(t): ⎡

m1 (t) P11 (t) P12 (t) ⎢ ⎢ m2 (t) P21 (t) P22 (t) ⎢ . .. .. ⎢ . ⎣ . . . mn (t) Pn1 (t) Pn2 (t)

⎡

m1 (t)P11 (t) m1 (t)P12 (t) ⎢ ⎢ m2 (t)P21 (t) m2 (t)P22 (t) ⎢ .. .. ⎢ ⎣ . . mn (t)Pn1 (t) mn (t)Pn2 (t)

··· ··· .. . ···

⎤ · · · P1n (t) ⎥ · · · P2n (t) ⎥ . ⎥ .. ⎥= . .. ⎦ · · · Pnn (t) ⎤ m1 (t)P1n (t) ⎥ m2 (t)P2n (t) ⎥ ⎥. .. ⎥ ⎦ . mn (t)Pnn (t)

It is readily verified (see [49], Section 5.3) that the optimal control problem for the system state (2.40) and cost function (2.42) is equivalent to the optimal control problem for the estimate (2.43) and the cost function J represented as 1 J = E{ mT (T )Φ m(T ) + 2 1 2

 T

uT (s)K(s)u(s)ds +

t0

+

1 2

 T t0

1 2

 T

(2.46)

mT (s)L(s)m(s)ds

t0

tr[P(s)L(s)]ds + tr[P(T )Φ ]},

where tr[A] denotes trace of a matrix A. Since the latter part of J is independent of control u(t) or state x(t), the reduced effective cost function M to be minimized takes the form 1 M = E{ mT (T )Φ m(T ) + (2.47) 2

68

2 Further Results: Optimal Identification and Control Problems

1 2

 T t0

uT (s)K(s)u(s)ds +

1 2

 T

mT (s)L(s)m(s)ds}.

t0

Thus, the solution for the optimal control problem specified by (2.40),(2.42) can be found solving the optimal control problem given by (2.43),(2.47). However, the minimal value of the criterion J should be determined using (2.46). This conclusion presents the separation principle in third order polynomial systems. 2.3.3

Optimal Controller Problem Solution

Based on the solution of the optimal control problem obtained in Section 2.2 in the case of an observable system state governed by a third order polynomial equation, the following results are valid for the optimal control problem (2.43),(2.47), where the system state (the estimate m(t)) is completely available and, therefore, observable. The optimal control law is given by u∗ (t) = K −1 (t)GT (t)Q(t)m(t),

(2.48)

where the matrix function is the solution of the following equation dual to the variance equation dQ(t) = (−aT1 (t)Q(t) − Q(t)aT1 (t) − 2aT2 (t)Q(t) ∗ mT (t)− 2m(t) ∗ Q(t)a2(t) − 3aT3 (t)Q(t) ∗ qT (t) −

(2.49)

3q(t) ∗ Q(t)a3(t) − 3aT3 (t)Q(t) ∗ ((m2 )T (t))− 3(m2 (t) ∗ Q(t))a3(t) + L(t) − Q(t)G(t)K −1(t)GT (t)Q(t))dt, with the terminal condition Q(T ) = Φ . The binary operation ∗ has been introduced in preceding subsection, and q(t) = (q1 (t), q2 (t), ..., qn (t)) denotes the vector consisting of the diagonal elements of Q(t). In the process of derivation of the equation (2.49), it has been taken into account that the last term in the equation (2.43), PT (t)AT (t)× ×(B(t)BT (t))−1 (dy − (A0(t) + A(t)m(t))dt), is a Gaussian white noise. Upon substituting the optimal control (2.48) into the equation (2.43) for the reconstructed system state m(t), the following optimally controlled state estimate equation is obtained (2.50) dm(t) = (a0 (t) + a1(t)m(t) + a2(t)p(t) + a2(t)m2 (t) + a3 (t)(3p(t) ∗ m(t) + m3(t))dt + G(t)(K(t))−1 GT (t)Q(t)m(t)dt+ PT (t)AT (t)(B(t)BT (t))−1 (dy − (A0 (t) + A(t)m(t))dt), m(t0 ) = E(x(t0 ) | FtY0 ). Thus, the optimally controlled state estimate equation (2.50), the gain matrix constituent equation (2.49), the optimal control law (2.48), and the variance equation (2.45) give the complete solution to the optimal controller problem for unobservable states of third degree polynomial systems.

2.3 Optimal Controller Problem for Third-Order Polynomial Systems

2.3.4

69

Optimal Third-Order Polynomial Controller for Automotive System

This section presents application of the obtained controller for a polynomial state of degree 3 over linear observations and a quadratic cost function to controlling the unobservable state variables, orientation and steering angles, in a nonlinear kinematical model of car movement [60] satisfying the equations (cf. Example in Subsection 1.1.3): dx(t) = v cos φ (t)dt,

(2.51)

dy(t) = v sin φ (t)dt, d φ (t) = (v/l) tan δ (t)dt, d δ (t) = u(t)dt. Here, x(t) and y(t) are Cartesian coordinates of the mass center of the car, φ (t) is the orientation angle, u is the velocity, l is the longitude between the two axes of the car, δ (t) is the steering wheel angle, and u(t) is the control variable (steering angular velocity). The zero initial conditions for all variables are assumed. The observation process for the unobservable variables φ (t) and δ (t) is given by direct linear observations confused with independent and identically distributed disturbances modelled as white Gaussian noises. The corresponding observations equations are dzφ (t) = φ (t)dt + w1 (t)dt,

(2.52)

dzδ (t) = δ (t)dt + w2 (t)dt, where zφ (t) is the observation variable for φ (t), zδ (t) is the observation variable for δ (t), and w1 (t) and w2 (t) are white Gaussian noises independent of each other. The examined values of the velocity and longitude are v = 17, l = 2, which correspond to the idle engine mode of a full-size car. In other words, the problem is to make the maximum turn of the running wheels from their initial position, using the minimum steering energy. For the reasons of economizing the fuel and reducing air pollution, the weight of the control term in the criterion is assigned ten times more than the weight of the state terminal term. The corresponding criterion J to be minimized takes the form J = [φ (t) − φ ∗ ]2 + 10

 T

u2 (t)dt,

(2.53)

0

where T = 0.3, and φ ∗ = 10 is a large value of φ (t) unreachable for time T . To apply the obtained optimal controller algorithms to the nonlinear system (2.51), linear observations (2.52), and the quadratic criterion (2.53), let us make the Taylor expansion of the two last equations in (2.51) at the origin up to degree 3 (the fourth degree does not appear in the Taylor series for tangent) v v δ 3 (t) ))dt, d φ (t) = (( )δ (t) + ( )( l l 3 d δ (t) = u(t)dt.

(2.54)

70

2 Further Results: Optimal Identification and Control Problems

The solution to the stated optimal controller problem is given as follows. Since K = 1 and GT = [0, 1] in (2.53) and (2.54), the optimal control u∗ (t) = (K(t))−1 GT (t)Q(t)m(t) takes the form u∗ (t) = q21 (t)mφ (t) + q22(t)mδ (t),

(2.55)

and the following optimal controller equations (2.48)–(2.50) and (2.45) for the third degree polynomial state (2.54) over the linear observations (2.52) and the quadratic criterion (2.53) are obtained v v dmφ = (( )mδ + ( )(3pδ + m3δ ))dt + pφ φ (dzφ − mφ dt) + pφ δ (dzδ − mδ dt), (2.56) l 3l dmδ = u∗ (t)dt + pδ φ (dzφ − mφ dt) + pδ δ (dzδ − mδ dt), d pφ φ = ((2v/l)pδ φ pδ δ +

2v 2v p + m2φ pδ φ − p2φ φ − p2φ δ )dt, l δφ l

v v d pφ δ = ( pδ δ + m2δ pδ δ − pφ φ pφ δ − pφ δ pδ δ )dt, l l d pδ δ = (−p2δ φ − p2δ δ )dt, dq11 (t) = (−q221 (t))dt, v v v dq12 (t) = (− q211 − q12q22 − q11 − m2φ q11 )dt, l l l 2v 2v 2v dq22 (t) = (− q12 − q12 q22 − m2δ q12 − q222 )dt. l l l Here, mφ and mδ are the estimates for variables φ and δ ; pφ φ , pφ δ , pδ δ are elements of the symmetric covariance matrix P; and q11 (t), q21 (t), q22 (t) are elements of the symmetric matrix Q(t) forming the optimal control (2.55). The following values of the input variables and initial values are assigned: v = 17, l = 2, mφ (0) = 1, mδ (0) = 0.1, φ (0) = δ (0) = 0, Pφ φ (0) = 10, Pφ δ (0) = 1, Pδ δ (0) = 1. Gaussian disturbances w1 (t) and w2 (t) in (2.52) are realized as sinusoidal signals: w1 (t) = w2 (t) = sint. The terminal conditions for the matrix Q elements are: q11 (T ) = 0.1, q12(T ) = 0, q22 (T ) = 0, where the final time T = 0.3. Thus, the system composed of the two last equations of (2.51) and the equations (2.56) should be solved with initial conditions mφ (0) = 1, mδ (0) = 0.1, φ (0) = δ (0) = 0, Pφ φ (0) = 10, Pφ δ (0) = 1, Pδ δ (0) = 1 and terminal conditions q11 (T ) = 0.1, q12 (T ) = 0, q22 (T ) = 0. This boundary problem is solved numerically using the iterative method of direct and reverse passing as follows. The first initial conditions for q’s are guessed, and the system is solved in direct time with the initial conditions at t = 0, thus obtaining certain values for the other listed variables at the terminal point T = 0.3. Then, the system is solved in reverse time, taking the obtained terminal values for the other variables in direct time as their initial values in reverse time, thus obtaining certain values for q’s at the initial point t = 0, which are taken as their initial values for the passing in direct time, and so on. The given initial conditions mφ (0) = 1, mδ (0) = 0.1, φ (0) = δ (0) = 0, Pφ φ (0) = 10, Pφ δ (0) = 1, Pδ δ (0) = 1 are kept fixed for any direct passing, and the given terminal conditions q11 (T ) = 0.1, q12 (T ) = 0, q22 (T ) = 0 are used as

0.06

0.03

0.05

0.025

0.04

0.02 delta

phi

2.3 Optimal Controller Problem for Third-Order Polynomial Systems

0.03

0.015

0.02

0.01

0.01

0.005

0 0

0.1

0.2

0 0

0.3

0.1

1

0.1

0.9

0.09

0.3

0.2

0.3

0.08

0.8

0.07 0.7

Mdelta

Mphi

0.2 time

time

0.6

0.4 0

0.06 0.05

0.5

0.04 0.1

0.2

0.03 0

0.3

0.1 time

time 100.4

0.35

100.2

0.3

100

0.25

99.8

0.2

99.6

0.15

control

criterion

71

99.4

0.1

99.2

0.05

99

0

98.8 0

0.1

0.2 time

0.3

time

−0.05 0

0.1

0.2

0.3

time

Fig. 2.6. Graphs of the variables φ (rad.) and δ (rad.), satisfying the polynomial system (2.54) and controlled using the optimal linear regulator defined by (2.55),(2.57), against time (sec.); graphs of the estimates mφ (rad.) and mδ (rad.), satisfying the system (2.57) and controlled using the optimal linear regulator defined by (2.55), (2.57), against time (sec.); graphs of the corresponding values of the criterion J (rad2 .) against time (sec.); graphs of the corresponding values of the optimal control u∗ (rad./sec.) against time (sec.)

72

2 Further Results: Optimal Identification and Control Problems

the fixed initial conditions for any reverse passing. The algorithm stops when the system arrives at values q11 (T ) = 0.1, q12 (T ) = 0, q22 (T ) = 0 after direct passing and at values mφ (0) = 1, mδ (0) = 0.1, φ (0) = δ (0) = 0, Pφ φ (0) = 10, Pφ δ (0) = 1, Pδ δ (0) = 1 after reverse passing. The obtained simulation graphs for φ , δ , mφ , mδ , the criterion J, and the optimal control u∗ (t) are given in Fig. 2.6. These results for the polynomial controller of degree 3 are then compared to the results obtained using the best linear controller available for the linearized model (only the linear term is present in the Taylor expansion for tangent). The optimal control law in this linear controller is the same as in (2.55) and the optimal linear controller equations are given by v dmφ = mδ dt + pφ φ (dzφ − mφ dt) + pφ δ (dzδ − mδ dt), l

(2.57)

dmδ = u∗ (t)dt + pδ φ (dzφ − mφ dt) + pδ δ (dzδ − mδ dt), d pφ φ = (

2v p − p2φ φ − p2φ δ )dt, l δφ

v d pφ δ = ( pδ δ − pφ φ pφ δ − pφ δ pδ δ )dt, l d pδ δ = (−p2δ φ − p2δ δ )dt. dq11 (t) = (−q221 (t))dt, v dq12 (t) = (−q12 q22 − q11 )dt, l 2v q12 − q222)dt. l Note that in the linear case the only reverse passing for q’s is necessary, because the the equations for q’s in (2.57) do not depend of φ , δ , mφ , or mδ , and the initial values for q’s at t = 0 can be obtained after single reverse passing. The simulation graphs for the linear case are given in Fig. 2.7. Thus, two sets of graphs are obtained. 1. Graphs of the variables φ and δ satisfying the polynomial system (2.56) and controlled using the optimal linear regulator defined by (2.55), (2.57); graphs of the estimates mφ and mδ satisfying the system (2.57) and controlled using the optimal linear regulator defined by (2.55), (2.57); graphs of the corresponding values of the criterion J; graphs of the corresponding values of the optimal control u∗ (Fig. 2.6). 2. Graphs of the variables φ and δ satisfying the polynomial system (2.54) and controlled using the optimal third order polynomial controller defined by (2.55), (2.56); graphs of the estimates mφ and mδ satisfying the system (2.56) and controlled using the optimal third order polynomial controller defined by (2.55), (2.56); graphs of the corresponding values of the criterion J; graphs of the corresponding values of the optimal control u∗ (Fig. 2.7). The obtained values of the controlled variable φ and the criterion J are compared for the optimal third order polynomial and linear controllers at the terminal time T = 0.3 in the following table (corresponding to Figs. 2.6 and 2.7). dq22 (t) = (−

2.3 Optimal Controller Problem for Third-Order Polynomial Systems

0.05

0.08

0.04

0.06

0.03

phi

delta

0.1

0.04

0.02

0.02

0.01

0 0

0.05

0.1

0.15

0.2

0.25

0 0

0.3

0.05

0.1

time

0.15

73

0.2

0.25

0.3

0.2

0.25

0.3

0.2

0.25

0.3

time

1

0.1

0.9

0.09

Mdelta

Mphi

0.8 0.7

0.08 0.07

0.6 0.06

0.5 0.4

0

0.05

0.1

0.15

0.2

0.25

0.05 0

0.3

0.05

0.1

0.15 time

time 100.5

0.7 0.6

100

control

criterion

0.5 99.5 99

0.4 0.3 0.2

98.5 0.1 98

0

0.05

0.1

0.15 time

0.2

0.25

0.3

0 0

0.05

0.1

0.15 time

Fig. 2.7. Graphs of the variables φ (rad.) and δ (rad.), satisfying the polynomial system (2.54) and controlled using the optimal third order polynomial controller defined by (2.55),(2.56), against time (sec.); graphs of the estimates mφ (rad.) and mδ (rad.), satisfying the system (2.56) and controlled using the optimal third order polynomial controller defined by (2.55), (2.56), against time (sec.); graphs of the corresponding values of the criterion J (rad2 .) against time (sec.); graphs of the corresponding values of the optimal control u∗ (rad./sec.) against time (sec.)

74

2 Further Results: Optimal Identification and Control Problems

Lineal controller Third degree polynomial controller φ (0.3) = 0.0545 φ (0.3) = 0.0876 J = 98.9625 J = 98.3884 The simulation results show that the value of the controlled variable φ at the terminal point T = 0.3 is more than one and half times greater for the third order polynomial controller than for the linear one, and the difference between the initial and final criterion values is more than one and half times greater for the third order polynomial controller as well. Thus, the third order polynomial controller regulates the system variables better than the linear one from both points of view, thus illustrating the theoretical conclusion. Discussion The optimal nonlinear controller for a stochastic system state given by a polynomial equation of degree 3, linear observations confused with white Gaussian noises, and a quadratic criterion has been obtained. The optimal polynomial controller of degree 3 has been then applied to solution of the controlling problem for state variables, orientation and steering angles, of a nonlinear automotive system describing kinematics of car movement. Application of the obtained controller to the nonlinear automotive system has yielded more than one and half times better values of the criterion and greater values of the controlled variable in comparison to the best linear controller available for the linearized model. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration.

3 Optimal Filtering Problems for Time-Delay Systems

3.1 Filtering Problem over Observations with Multiple Delays 3.1.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ 0, and let (W1 (t), Ft ,t ≥ 0) and (W2 (t), Ft ,t ≥ 0) be independent Wiener processes. The partially observed Ft -measurable random process (x(t), y(t)) is described by an ordinary differential equation for the dynamic system state dx(t) = (a0 (t) + a(t)x(t))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(3.1)

and a differential equation with multiple delays for the observation process: p

dy(t) = (A0 (t) + A(t)x(t) + ∑ Ai (t)x(t − hi))dt + B(t)dW2(t),

(3.2)

i=1

where x(t) ∈ Rn is the state vector, y(t) ∈ Rm is the observation process, the initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t), W2 (t) are independent. The observation process y(t) depends on delayed states x(t − hi ), i = 1, . . . , p, where hi > 0 are positive delay shifts, as well as non-delayed state x(t), which assumes that collection of information on the system state for the observation purposes is made not only at the current time but also after certain time lags hi > 0, i = 1, . . . , p. The vector-valued function a0 (s) describes the effect of system inputs (controls and disturbances). It is assumed that A(t) is a nonzero matrix and B(t)BT (t) is a positive definite matrix. All coefficients in (3.1),(3.2) are deterministic functions of appropriate dimensions. The estimation problem is to find the optimal estimate x(t) ˆ of the system state x(t), based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, that minimizes the Euclidean 2-norm ˆ | FtY ] J = E[(x(t) − x(t)) ˆ T (x(t) − x(t)) at every time moment t. Here, E[z(t) | FtY ] means the conditional expectation of a ˆ with respect to the σ - algebra FtY stochastic process z(t) = (x(t) − x(t)) ˆ T (x(t) − x(t)) generated by the observation process Y (t) in the interval [t0 ,t]. As known from ([41], M. Basin: New Trends in Optimal Filtering, LNCIS 380, pp. 75–130, 2008. c Springer-Verlag Berlin Heidelberg 2008 springerlink.com 

76

3 Optimal Filtering Problems for Time-Delay Systems

Theorem 5.3 or [70], Subsection 5.10.2), this optimal estimate is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY ) of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As usual, the matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differential of the conditional expectation E(x(t) | FtY ) and its variance P(t) and given in the following section. 3.1.2

Optimal Filter over Observations with Multiple Delays

In the situation of multiple delays, the optimal filtering equations could be obtained directly from the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E(ϕ (x(t),t) | FtY )dt + E(x[ϕ1 (x(t), x(t − h1), . . . , x(t − h p),t)−

(3.3)

E(ϕ1 (x(t), x(t − h1 ), . . . , x(t − h p),t) | FtY )]T | FtY )×  −1 B(t)BT (t) (dy(t) − E(ϕ1 (x(t), x(t − h1), . . . , x(t − h p),t) | FtY )dt), where ϕ (x(t),t) : Rn × R → Rn is the drift term in the state equation equal to ϕ (x(t),t) = a0 (t) + a(t)x(t) and ϕ1 (x,t) : Rn×(p+1) × R → Rn is the drift term in the observation p

equation equal to ϕ1 (x(t), x(t − h1 ), . . . , x(t − h p),t) = A0 (t) + A(t)x(t) + ∑ Ai (t)x(t − i=1

hi ). Note that the conditional expectation equalities E(x(t − hi ) | FtY ) = E(x(t − hi ) | Y ) = m(t − h ) are valid for any h > 0, since, in view of positive delay shifts h > 0, Ft−h i i i i the treated problem (3.1),(3.2) is a filtering problem, not a smoothing one, and, therefore, the formula (3.3) yields the optimal estimate m(s) for any time s, t0 < s ≤ t, if the observations (3.2) are obtained until the current moment t (see ([41], Theorem 6.6, or [70], Subsection 5.10.7)). Upon performing substitution of the expressions for ϕ and ϕ1 into (3.3) and taking into account the conditional expectation equalities, the estimate equation takes the form dm(t) = (a0 (t) + a(t)m(t))dt + E(x(t)[A(t)(x(t) − m(t))+  −1 B(t)BT (t) ×

p

∑ Ai (t)(x(t − hi) − m(t − hi))]T | FtY )

i=1

p

(dy(t) − (A0(t) + A(t)m(t) + ∑ Ai (t)m(t − hi))dt) = i=1

(a0 (t) + a(t)m(t))dt + [E(x(t)(x(t) − m(t))T | FtY )AT (t)+

3.1 Filtering Problem over Observations with Multiple Delays

77

 −1 B(t)BT (t) ×

p

∑ E(x(t)(x(t − hi) − m(t − hi))T | FtY )ATi (t)]

i=1

p

(dy(t) − (A0(t) + A(t)m(t) + ∑ Ai (t)m(t − hi))dt). i=1

The obtained form of the optimal estimate equation is similar to the Kalman filp

ter one, except the term E(x(t)(x(t) − m(t))T | FtY )AT (t) + ∑ E(x(t)(x(t − hi ) − m(t − hi

))T

|

FtY )ATi (t)

=

P(t)AT (t) +

i=1

p

∑ E(x(t)(x(t − hi ) − m(t − hi ))T | FtY )ATi (t) standing

i=1

instead of P(t)AT (t) = E((x(t) − m(t))(x(t) − m(t))T | FtY )AT (t). However, the former term can be expressed as a function of the variance, using the Cauchy formula for x(t) as the solution of the linear equation (3.1) and m(t) as its conditional expectation. Indeed, x(t) = Φ (t,t − h)x(t − h) +  t t−h

t t−h

Φ (t, τ )a0 (τ )d τ +

(3.4)

Φ (t, τ )b(τ )dW1 (τ ),

where Φ (t, τ ) is the matrix of fundamental solutions of the homogeneous equation (3.1), that is solution of the matrix equation d Φ (t, τ ) = a(t)Φ (t, τ ), dt

Φ (t,t) = I,

where I is the identity matrix. In other words, Φ (t,t − h) = exp delayed term in the estimate equation is equal to

t

t−h a(s)ds.

Thus, the

p

∑ E(x(t)(x(t − hi) − m(t − hi))T | FtY )ATi (t) =

i=1 p

∑ E(x(t)(x(t) − m(t))T | FtY ) exp (−

i=1

p

∑ P(t) exp (−

 t

i=1

t−hi

 t t−hi

aT (s)ds)ATi (t) =

aT (s)ds)ATi (t),

and the entire equation takes the form p

dm(t) = (a0 (t) + a(t)m(t))dt + P(t)[AT (t) + ∑ exp(− i=1

 t t−hi

aT (s)ds)ATi (t)]×

p  −1 B(t)BT (t) (dy(t) − (A0 (t) + A(t)m(t) + ∑ Ai (t)m(t − hi ))dt).

(3.5)

i=1

So far, the optimal estimate equation, similarly to the classic Kalman-Bucy case, p  −1 t aT (s)ds)ATi (t)] B(t)BT (t) deincludes the gain matrix P(t)[AT (t) + ∑ exp (− t−h i i=1

pending on the estimate variance, but now also depending on the delay adjustment

78

3 Optimal Filtering Problems for Time-Delay Systems p

[AT (t) + ∑ exp (− i=1

t

t−hi a

T (s)ds)AT (t)]. i

The problem now is to find the equation for

P(t) in a closed form. For this purpose, the formula for the Ito differential of the conditional expectation variance P(t) = E((x(t) − m(t))(x(t) − m(t))T | FtY ) could be used (see ([41], Theorem 6.6, formula (6.101) or [70], Subsection 5.10.9, formula (5.10.42))): dP(t) = (E((x(t) − m(t))ϕ T (x(t),t) | FtY )+ E(ϕ (x(t),t)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)− E(x(t)[ϕ1 (x(t), x(t − h1 ), . . . , x(t − h p),t)− E(ϕ1 (x(t), x(t − h1 ), . . . , x(t − h p),t) | FtY )]T | FtY )×  −1 B(t)BT (t) E([ϕ1 (x(t), x(t − h1 ), . . . , x(t − h p),t)− E(ϕ1 (x(t), x(t − h1), . . . , x(t − h p),t) | FtY )]xT (t) | FtY ))dt+ E((x(t) − m(t))(x(t) − m(t))[ϕ1(x(t), x(t − h1 ), . . . , x(t − h p),t)− E(ϕ1 (x(t), x(t − h1 ), . . . , x(t − h p),t) | FtY )]T | FtY )×  −1 B(t)BT (t) (dy(t) − E(ϕ1 (x(t), x(t − h1), . . . , x(t − h p),t) | FtY )dt), where the last term should be understood as a 3D tensor (under the expectation sign) convoluted with a vector, which yields a matrix. Upon substituting the expressions for ϕ and ϕ1 , the last formula takes the form dP(t) = (E((x(t) − m(t))xT (t)aT (t) | FtY )+ E(a(t)x(t)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)− p

[E(x(t)(x(t) − m(t))T | FtY )AT (t) + ∑ E(x(t)(x(t − hi ) − m(t − hi ))T | FtY )ATi (t)]× i=1

 −1 B(t)BT (t) [A(t)E((x(t) − m(t))xT (t)) | FtY )+ p

∑ Ai (t)E((x(t − hi) − m(t − hi))xT (t)) | FtY )])dt + E((x(t) − m(t))(x(t) − m(t))×

i=1

p

([A(t)(x(t) − m(t)) + ∑ Ai (t)(x(t − hi ) − m(t − hi ))]T ) | FtY )× i=1

−1  B(t)BT (t) (dy(t) − E(ϕ1 (x(t), x(t − h1), . . . , x(t − h p),t) | FtY )dt). Using again the formula (3.4) for delayed values of the state and considering that p

E(x(t)(x(t) − m(t))T | FtY )AT (t) + ∑ E(x(t)(x(t − hi ) − m(t − hi ))T | FtY )ATi (t) = i=1

p

P(t)AT (t) + ∑ E(x(t)(x(t) − m(t))T | FtY ) exp (− i=1

 t t−hi

aT (s)ds)ATi (t) =

3.1 Filtering Problem over Observations with Multiple Delays p

P(t)AT (t) + ∑ P(t) exp (− i=1

 t t−hi

79

aT (s)ds)ATi (t),

the equation for P(t) is reduced to dP(t) = (P(t)aT (t) + a(t)P(t) + b(t)bT (t)− p

[P(t)AT (t) + ∑ P(t) exp (−

 t

i=1

t−hi

aT (s)ds)ATi (t)]×

 p  −1 B(t)BT (t) [A(t)P(t) + ∑ Ai (t) exp (−

t

a(s)ds)P(t)])dt+

t−hi

i=1

p

E((x(t) − m(t))(x(t) − m(t))(x(t) − m(t) | FtY )[AT (t) + ∑ exp(− i=1

 t

aT (s)ds)]×

t−hi

 −1 ATi (t) B(t)BT (t) (dy(t) − E(ϕ1 (x(t), x(t − h1), . . . , x(t − h p),t) | FtY )dt). The last term in this formula contains the conditional third central moment E((x(t) − m(t))(x(t)−m(t))(x(t)−m(t)) | FtY ) of x(t) with respect to observations, which is equal to zero, because x(t) is conditionally Gaussian, in view of Gaussianity of the noises and the initial condition and linearity of the state and observation equations. Thus, the entire last term is vanished and the following variance equation is obtained dP(t) = (P(t)aT (t) + a(t)P(t) + b(t)bT (t)− p

P(t)[AT (t) + ∑ exp (− i=1

 t t−hi

aT (s)ds)ATi (t)]×

 p  −1 B(t)BT (t) [A(t) + ∑ Ai (t) exp (− i=1

(3.6)

t

a(s)ds)]P(t))dt.

t−hi

The obtained system of filtering equations (3.5) and (3.6) should be complemented with the initial conditions m(t0 ) = E[x(t0 ) | FtY0 ] and P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. As noted, this system is similar to the conventional Kalman-Bucy filter, except the adjustments for delays in the estimate and variance equations, calculated due to the Cauchy formula for the state equation. It closely resembles the Smith predictor [82] commonly used for robust control design in time delay systems. Nevertheless, the obtained filter is optimal with respect to the introduced form of the observation process, since it is obtained from the exact Ito differential for the conditional expectation and variance. In the case of a constant matrix the optimal filter takes the  t aTin the state equation, a ds) = exp(−aT h)) especially simple form (exp(− t−h p

dm(t) = (a0 (t) + am(t))dt + P(t)[AT (t) + ∑ exp (−aT hi )ATi (t)]× i=1

p  −1 B(t)BT (t) (dy(t) − (A0 (t) + A(t)m(t) + ∑ Ai (t)m(t − hi ))dt), i=1

(3.7)

80

3 Optimal Filtering Problems for Time-Delay Systems

dP(t) = (P(t)aT + aP(t) + b(t)bT (t)−

(3.8)

p

P(t)[AT (t) + ∑ exp (−aT hi )ATi (t)]× i=1

p −1  B(t)BT (t) [A(t) + ∑ Ai (t) exp (−ahi )]P(t))dt. i=1

Remark. The convergence properties of the obtained optimal estimate (3.5) coincide with the convergence properties of the conventional Kalman-Bucy filter estimate, since the asymptotic convergence to zero for the retarded error e(t − h) = x(t − h)− m(t − h) is equivalent to asymptotic convergence to zero for the current error e(t) = x(t)−m(t), and vice versa. Thus, the asymptotic convergence properties of the obtained filter are given by the standard convergence theorem (see, for example, [41], Theorem 7.5 and Section 7.7): if in the system (3.1),(3.2) the pair (a(t), b(t)) is uniformly completely controllable and the pair (a(t), A(t) + A1 (t)Φ (t − h1 ,t) + . . . + A p (t)Φ (t − h p ,t)) is uniformly completely observable, where Φ (t, τ ) is the state transition matrix for the equation (3.1), then the error of the obtained optimal filter (3.5),(3.6) is uniformly asymptotically stable. As usual, the uniform complete controllability condition is required for assuring non-negativeness of the variance matrix P(t) (6) and may be omitted, if the matrix P(t) is non-negative in view of its intrinsic properties. 3.1.3

Example

This section presents an example of applying the obtained filter over linear observations with multiple delays to estimation of the state variable, provided that there are two observation channels with different delay shifts. This situation is very common and can be frequently encountered in various applications (see, for instance, a mixing tank example in Chapter 6 of [49]). Let the unobservable state x(t) be given by x(t) ˙ = 0.1x(t),

x(0) = x0 ,

(3.9)

and there are two observation devices measuring the system state with different delay shifts y1 (t) = x(t − 2) + ψ1(t), (3.10) y2 (t) = x(t − 20) + ψ2(t). where ψ1 (t) and ψ2 (t) are white Gaussian noises which are weak mean square derivatives of standard Wiener processes (see ([41], Subsection 3.8, or [70], Subsection 4.6.5)). Thus, the equations (3.9) and (3.10) present the conventional form for the equations (3.1) and (3.2), which is actually used in practice [6]. The estimation problem is to find the optimal estimate for the variable x(t) using direct linear observations with delays (3.10) confused with independent and identically distributed disturbances modeled as white Gaussian noises. As noted, the delay shifts are different in each observation channel, so the filtering equations (3.6),(3.7) should be employed.

3.1 Filtering Problem over Observations with Multiple Delays

81

4

2.5

x 10

state

2 1.5 1 0.5 0

Kalman estimate

2.5

10

20

30

40

50

0 4 x 10

10

20

30

40

50

0

10

time

60

70

80

90

100

60

70

80

90

100

60

70

80

90

100

2 1.5 1 0.5 0 2.5

Optimal estimate

0 4 x 10

time

2 1.5 1 0.5 0

20

30

40

50

time

Fig. 3.1. Graphs of the reference state variable x(t) and the estimates m1 (t) and m(t) on the entire simulation interval [0, 100]

Since a=0.1 in (3.9) and AT1 =[1 0] and AT2 = [0 1] in (3.10), the equations (3.7),(3.8) take the following particular form m(t) ˙ = 0.1m(t) + P(t)[exp(−0.2)(y1 (t) − m(t − 2))+ exp (−2)(y2 (t) − m(t − 20))],

m(0) = m0 ,

˙ = 0.2P(t) − [exp (−0.2) + exp (−2)]P2 (t), P(t) 2

(3.11)

2

P(0) = P0 .

The estimates obtained upon solving the equations (3.11) are compared to the conventional Kalman-Bucy estimates satisfying the following filtering equations for the unobservable state (3.9) over linear observations with multiple delays (3.10), provided  −1 that the filter gain matrix is considered equal to P(t)AT (t) B(t)BT (t) , as in the standard Kalman-Bucy filter, without the delay adjustment term p

[ ∑ exp(−aT hi )ATi (t)]: i=1

m˙ 1 (t) = 0.1m1 (t) + P1(t)[(y1 (t) − m1(t − 2))+ (y2 (t) − m1(t − 20))], P˙1 (t) = 0.2P1(t) − P12(t),

(3.12)

m1 (0) = m10 , P1 (0) = P10 .

Numerical simulation results are obtained solving the systems of filtering equations (3.11) and (3.12). The obtained values of the estimates m(t) and m1 (t) satisfying (3.11) and (3.12) are compared to the real values of the state variable x(t) in (3.9).

82

3 Optimal Filtering Problems for Time-Delay Systems

409

state

408 407 406 405 404 59.99

59.992

59.994

59.996

59.998

60 60.002 time

60.004

60.006

60.008

60.01

59.992

59.994

59.996

59.998

60 60.002 time

60.004

60.006

60.008

60.01

59.992

59.994

59.996

59.998

60 60.002 time

60.004

60.006

60.008

60.01

Kalman estimate

409 408 407 406 405 404 59.99

Optimal estimate

409 408 407 406 405 404 59.99

Fig. 3.2. Graphs of the reference state variable x(t) and the estimates m1 (t) and m(t) around the reference time point T = 60

For each of the two filters (3.11) and (3.12) and the reference system (3.9) involved in simulation, the following initial values are assigned: x0 = 1, m0 = m10 = 10, P0 = P10 = 100. Gaussian disturbances ψ1 (t) and ψ2 (t) in (10) are realized as sinusoidal signals: ψ1 (t) = ψ2 (t) = sint. The following graphs are obtained: graphs of the reference state variable x(t) for the system (3.9); graphs of the Kalman-Bucy filter estimate m1 (t) satisfying the equations (3.12); and graphs of the optimal linear filter with multiple delays estimate m(t) satisfying the equations (3.11). The graphs of all those variables are shown on the entire simulation interval from T = 0 to T = 100 (Fig. 3.1), and around the reference time points: T = 60 (Fig. 3.2), T = 80 (Fig. 3.3), and T = 100 (Fig. 3.4). It can also be noted that the error variance P(t) converges to zero very rapidly, since the optimal estimate (3.11) converges to the real state (3.9). The following values of the reference state variable x(t) and the estimates m(t) and m1 (t) are obtained at the reference time points: for T = 60, x(60) = 405.30, m(60) = 405.83, m1 (60) = 408.01; for T = 80, x(80) = 2994.80, m(80) = 2994.82, m1 (80) = 2985.88; for T = 100, x(100) = 22026, m(100) = 22026, m1 (100) = 22053. Thus, it can be concluded that the optimal filter for linear systems over observations with multiple delays (3.11) yield definitely better estimates than the conventional Kalman-Bucy filter. The simulations have also been made for the Kalman filter with the gain matrix P2 (t) satisfying the equation P˙2 (t) = 0.2P2(t) − 2P22(t),

P2 (0) = 100,

3.1 Filtering Problem over Observations with Multiple Delays

83

3000

state

2995 2990 2985 2980 79.99

79.992

79.994

79.996

79.998

80 80.002 time

80.004

80.006

80.008

80.01

79.992

79.994

79.996

79.998

80 80.002 time

80.004

80.006

80.008

80.01

79.992

79.994

79.996

79.998

80 80.002 time

80.004

80.006

80.008

80.01

Kalman estimate

3000 2995 2990 2985 2980 79.99

Optimal estimate

3000 2995 2990 2985 2980 79.99

Fig. 3.3. Graphs of the reference state variable x(t) and the estimates m1 (t) and m(t) around the reference time point T = 80

considering that AT A = [1 1][1 1]T = 2, and the estimate m2 (t) given by the first equation in (3.12) m˙ 2 (t) = 0.1m2 (t) + P2(t)[(y1 (t) − m2(t − 2))+ (y2 (t) − m2(t − 20))],

m1 (0) = 10.

However, the results obtained in this case occur to be even worse than for the Kalman filter (3.12). The values of the estimate m2 (t) in the reference time points are: m2 (60) = 461.80, m2 (80) = 3070.66, m2 (100) = 22140. Discussion The simulation results show that the values of the estimate calculated by using the obtained optimal filter over observations with multiple delays are noticeably closer to the real values of the reference variable than the values of the Kalman-Bucy estimates. Moreover, it can be seen that the estimate produced by the optimal filter over observations with multiple delays asymptotically converges to the real values of the reference variable as time tends to infinity, although the reference system (3.9) itself is unstable. On the contrary, the conventionally designed (non-optimal) Kalman-Bucy estimates without delay adjustment diverge from the real values. This significant improvement in the estimate behavior is obtained due to the more careful selection of the filter gain matrix using the delay adjustment term, which compensates for unstable dynamics of the reference system, as it should be in the optimal filter. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration.

84

3 Optimal Filtering Problems for Time-Delay Systems

4

state

2.206

x 10

2.204 2.202

Kalman estimate

2.2 99.99 99.991 4 x 10 2.206

99.993

99.994

99.995 99.996 time

99.997

99.998

99.999

100

99.992

99.993

99.994

99.995 99.996 time

99.997

99.998

99.999

100

99.992

99.993

99.994

99.995 99.996 time

99.997

99.998

99.999

100

2.204 2.202 2.2 99.99 99.991 4 x 10 2.206

Optimal estimate

99.992

2.204 2.202 2.2 99.99

99.991

Fig. 3.4. Graphs of the reference state variable x(t) and the estimates m1 (t) and m(t) around the reference time point T = 100

3.2 Filtering Problem for Linear Systems with State Delay 3.2.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ 0, and let (W1 (t), Ft ,t ≥ 0) and (W2 (t), Ft ,t ≥ 0) be independent Wiener processes. The partially observed Ft -measurable random process (x(t), y(t)) is described by a delay differential equation for the system state dx(t) = (a0 (t) + a(t)x(t − h))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(3.13)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], and a differential equation for the observation process: dy(t) = (A0 (t) + A(t)x(t))dt + B(t)dW2(t),

(3.14)

where x(t) ∈ Rn is the state vector, y(t) ∈ Rm is the observation process, φ (s) is a mean square piecewise-continuous Gaussian stochastic process (see [41], Section 3.6, or [70], Subsection 2.7.5, for definition) given in the interval [t0 − h,t0 ] such that φ (s), W1 (t), and W2 (t) are independent. The system state x(t) dynamics y(t) depends on the delayed state x(t − h), where h is the delay shift, which actually makes the system state space infinite-dimensional (see, for example, [56], Section 2.3). The vector-valued function a0 (s) describes the effect of system inputs (controls and disturbances). It is assumed that A(t) is a nonzero matrix and B(t)BT (t) is a positive definite matrix. All coefficients in (3.13),(3.14) are deterministic functions of appropriate dimensions.

3.2 Filtering Problem for Linear Systems with State Delay

85

The estimation problem is to find the optimal estimate x(t) ˆ of the system state x(t), based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, that minimizes the Euclidean 2-norm ˆ | FtY ] J = E[(x(t) − x(t)) ˆ T (x(t) − x(t)) at every time moment t. Here, E[z(t) | FtY ] means the conditional expectation of a stochastic process z(t) = (x(t) − x(t)) ˆ T (x(t) − x(t)) ˆ with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As known, this optimal estimate is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY ) of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As usual, the matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differentials of the conditional expectation m(t) = E(x(t) | FtY ), the error variance P(t), and other bilinear functions of x(t) − m(t) and given in the following section. 3.2.2

Optimal Filter for Linear Systems with State Delay

In the situation of a state delay, the optimal filtering equations could be obtained using from the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E(ϕ (x) | FtY )dt + E(x[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY )×

(3.15)

−1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where ϕ (x) is the drift term in the state equation equal to ϕ (x) = a0 (t) + a(t)x(t − h) and ϕ1 (x) is the drift term in the observation equation equal to ϕ1 (x) = A0 (t) + A(t)x(t). Y )= Note that the conditional expectation equalities E(x(t −h) | FtY )=E(x(t −h) | Ft−h m(t − h) are valid for any h > 0, since, in view of positive delay shift h > 0, the treated problem (3.13),(3.14) is a filtering problem, not a smoothing one, and, therefore, the formula (3.15) yields the optimal estimate m(s) for any time s, t0 < s ≤ t, if the observations (3.14) are obtained until the current moment t (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) Upon performing substitution for ϕ (x) and ϕ1 (x) and taking into account E(x(t − h) | Y ) = m(t − h) for any h > 0, the estimate equation takes the form FtY ) = E(x(t − h) | Ft−h  −1 dm(t) = (a0 (t) + a(t)m(t − h))dt + E(x(t)[A(t)(x(t) − m(t))]T | FtY ) B(t)BT (t) × (3.16)

86

3 Optimal Filtering Problems for Time-Delay Systems

(dy(t) − (A0(t) + A(t)m(t)dt) =  −1 (a0 (t) + a(t)m(t − h))dt + P(t)AT (t) B(t)BT (t) (dy(t) − (A0(t) + A(t)m(t))dt). The obtained form of the optimal estimate equation is similar to the Kalman filter one, except for the term a(t)m(t − h). To compose a closed system of the filtering equations, the equation for the variance matrix P(t) can be obtained using the formula for the Ito differential of the variance P(t) = E((x(t) − m(t))(x(t) − m(t))T | FtY ) (see ([41], Theorem 6.6, formula (6.101) or [70], Subsection 5.10.9, formula (5.10.42))): dP(t) = (E((x(t) − m(t))ϕ T (x) | FtY ) + E(ϕ (x)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)−  −1 E(x(t)[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY ) B(t)BT (t) × E([ϕ1 − E(ϕ1 (x) | FtY )]xT (t) | FtY ))dt+ E((x(t) − m(t))(x(t) − m(t))[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY )× −1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt). Here, the last term should be understood as a 3D tensor (under the expectation sign) convoluted with a vector, which yields a matrix. Upon substituting the expressions for ϕ and ϕ1 , the last formula takes the form dP(t) = (E((x(t) − m(t))xT (t − h)aT (t) | FtY )+ E(a(t)x(t − h)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)−  −1 E(x(t)(x(t) − m(t))T | FtY )AT (t) B(t)BT (t) [A(t)E((x(t) − m(t))xT (t)) | FtY ))dt+ +E((x(t) − m(t))(x(t) − m(t))(A(t)(x(t) − m(t)))T | FtY )×  −1 B(t)BT (t) (dy(t) − (A0(t) + A(t)m(t))dt). Taking into account that the matrix P1 (t) = E((x(t) − m(t))(x(t − h))T | FtY ) in the first two right-hand side terms of the last formula is not equal to P(t) = E(x(t)(x(t) − m(t))T | FtY ), the equation for P(t) should be represented as  −1 dP(t) = (P1 (t)aT (t) + a(t)P1T (t) + b(t)bT (t) − P(t)AT (t) B(t)BT (t) A(t)P(t))dt+ E((x(t) − m(t))(x(t) − m(t))(x(t) − m(t)) | FtY )AT (t)× −1  B(t)BT (t) (dy(t) − (A0(t) + A(t)m(t))dt). The last term in this formula contains the conditional third central moment E((x(t) − m(t))(x(t)−m(t))(x(t)−m(t)) | FtY ) of x(t) with respect to observations, which is equal to zero, because x(t) is conditionally Gaussian, in view of Gaussianity of the noises and the initial condition and linearity of the state and observation equations. Thus, the entire last term is vanished and the following variance equation is obtained  −1 dP(t) = (P1 (t)aT (t) + a(t)P1T (t) + b(t)bT (t) − P(t)AT (t) B(t)BT (t) A(t)P(t))dt. (3.17)

3.2 Filtering Problem for Linear Systems with State Delay

87

However, the obtained system (3.16),(3.17) is not yet a closed system with respect to the variables m(t) and P(t), since the variance equation (3.17) depends on the unknown matrix P1 (t). Thus, the equation for the matrix P1 (t) should be obtained proceeding from its definition as E((x(t) − m(t))(x(t − h))T | FtY ), which implies that P1 (t) = E(x(t)(x(t − h))T | FtY )− m(t)(m(t − h))T . Based on the equation (3.13) for x(t) (and, therefore, for x(t − h)) and the equation (3.17) for m(t) (and, therefore, m(t − h)) the following formula is obtained for the Ito differential of P1 (t) dP1 (t) = E((a0 (t) + a(t)x(t − h))(x(t − h))T | FtY )dt− (a0 (t)(m(t − h))T + a(t)m(t − h)(m(t − h))T )dt+ E(x(t)(a0 (t − h) + a(t − h)x(t − 2h))T | FtY )dt− 1 (m(t)aT0 (t − h) + m(t)(a(t − h)m(t − 2h))T )dt + (b(t)bT (t − h) + b(t − h)bT (t))dt− 2 1 (P(t)AT (t)(B(t)BT (t))−1 (B(t)BT (t − h))(B(t − h)BT (t − h))−1 A(t − h)P(t − h)+ 2 P(t − h)AT (t − h)(B(t − h)BT (t − h))−1 (B(t − h)BT (t))(B(t)BT (t))−1 A(t)P(t))dt, where the third order term, which is equal to zero in view of conditional Gaussianity of x(t) (as well as in the equation (3.17) for P(t)), is omitted. Upon denoting P2 (t) = E((x(t) − m(t))(x(t − 2h))T | FtY ), the last equation takes the form 1 dP1 (t) = (a(t)P(t − h) + P2(t)aT (t − h))dt + (b(t)bT (t − h) + b(t − h)bT (t))dt− 2 (3.18) 1 T T −1 T T −1 (P(t)A (t)(B(t)B (t)) (B(t)B (t − h))(B(t − h)B (t − h)) A(t − h)P(t − h)+ 2 P(t − h)AT (t − h)(B(t − h)BT (t − h))−1 (B(t − h)BT (t))(B(t)BT (t))−1 A(t)P(t))dt. Adding the equation (3.18) to the system (3.16),(3.17) does not yet result in a closed system of the filtering equations, since the equation (3.18) depends on the unknown matrix P2 (t). The equation for the matrix P2 (t) is obtained directly form the equation (3.18) by changing h to 2h in the definition of P1 (t): 1 dP2 (t) = (a(t)P1 (t − h) + P3 (t)aT (t − 2h))dt + (b(t)bT (t − 2h) + b(t − 2h)bT (t))dt− 2 (3.19) 1 T T −1 T T −1 (P(t)A (t)(B(t)B (t)) (B(t)B (t − 2h))(B(t − 2h)B (t − 2h)) × 2 A(t − 2h)P(t − 2h) + P(t − 2h)AT (t − 2h)(B(t − 2h)BT (t − 2h))−1× (B(t − 2h)BT (t))(B(t)BT (t))−1 A(t)P(t))dt. The equation (3.19) for P2 (t) depends on the unknown matrix P3 (t) = E((x(t) − m(t))(x(t − 3h))T | FtY ), the equation for P3 (t) will depend on P4 (t) = E((x(t) − m(t))(x(t − 4h))T | FtY ), and so on. Thus, to obtain a closed system of the filtering

88

3 Optimal Filtering Problems for Time-Delay Systems

equations for the state (3.13) over the observations (3.14), the following equations for matrices Pi (t) = E((x(t) − m(t))(x(t − ih))T | FtY ), i ≥ 1, should be included dPi (t) = (a(t)Pi−1 (t − h) + Pi+1(t)aT (t − ih))dt+

(3.20)

1 1 (b(t)bT (t − ih) + b(t − ih)bT (t))dt − (P(t)AT (t)(B(t)BT (t))−1 × 2 2 (B(t)BT (t − ih))(B(t − ih)BT (t − ih))−1A(t − ih)P(t − ih) + P(t − ih)AT (t − ih)× (B(t − ih)BT (t − ih))−1(B(t − ih)BT (t))(B(t)BT (t))−1 A(t)P(t))dt. It should be noted that, for every fixed t, the number of equations in (3.20), that should be taken into account to obtain a closed system of the filtering equations, is not equal to infinity, since the matrices a(t), b(t), A(t), and B(t) are not defined for t < t0 . Therefore, if the current time moment t belongs to the semi-open interval (kh, (k + 1)h], where h is the delay value in the equation (3.13), the number of equations in (3.20) is equal to k. The last step is to establish the initial conditions for the system of equations (3.16),(3.17),(3.20). The initial conditions for (3.16) and (3.17) are stated as m(s) = E(φ (s)), s ∈ [t0 − h,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 ,

(3.21)

and P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 ))T | FtY0 ].

(3.22)

The initial conditions for matrices Pi (t) = E((x(t) − m(t))(x(t − ih))T | FtY ) should be stated as functions in the intervals [t0 + (i − 1)h,t0 + ih], since the ith of the equations (3.20) depends on functions with the arguments delayed by ih and the definition of Pi (t) itself assumes dependence on x(t − ih). Thus, the initial conditions for the matrices Pi (t) in (8) are stated as Pi (s) = E((x(s)− m(s))(x(s− ih)− m(s− ih))T | FsY ), s ∈ [t0 + (i− 1)h,t0 + ih]. (3.23) By means of the preceding derivation, the following result is proved. Theorem 3.1. The optimal filter for the unobserved state (3.13) over the observations (3.14) is given by the equation (3.16) for the optimal estimate m(t) = E(x(t) | FtY ), with the initial condition (3.21), the equation (3.17) for the estimation error variance P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ], with the initial condition (3.22), and the system of the equations (3.20) for the estimation error covariances Pi (t) = E((x(t) − m(t))(x(t − ih))T | FtY ), with the initial conditions (3.23). The number of equations in (3.20) is equal to the integer part of the ratio T /h, where h is the state delay in (3.13) and T is the current filtering horizon. A considerable advantage of the designed filter is a finite number of the filtering equations for any fixed filtering horizon, although the state space of the delayed system (3.13) is infinite-dimensional ([56], Section 2.3). Remark 1. The convergence properties of the obtained optimal estimate (3.16) are given by the standard convergence theorem (see, for example, [41], Theorem 7.5 and

3.2 Filtering Problem for Linear Systems with State Delay

89

Section 7.7): if in the system (3.13),(3.14) the pair (a(t), b(t)) is uniformly completely controllable and the pair (a(t), A(t)) is uniformly completely observable, then the error of the obtained optimal filter (3.16),(3.17),(3.20) is uniformly asymptotically stable. As usual, the uniform complete controllability condition is required for assuring nonnegativeness of the variance matrix P(t) (3.17) and may be omitted, if the matrix P(t) is non-negative in view of its intrinsic properties. The uniform complete controllability and observability conditions for a linear system with delay (3.13) and observations (3.14) can be found in ([56], Chapter 5). Remark 2. The increasing with the filtering horizon number of the equations in (3.20) is actually not an obstacle for finding the solution of the optimal filtering problem. Indeed, first, none of the equations (3.17),(3.20) depend on the estimate m(t) and, therefore, all the equations (3.17),(3.20) can be solved off-line, before starting the observation process, similarly to the variance equation in the conventional Kalman-Bucy filter. Second, all the equations (3.17),(3.20) are considered with certain initial conditions, i.e., the Cauchy problem is stated for a system of ordinary matrix differential equations. This problem can be numerically solved (for example, in MatLab) in the forward time, in the same manner as the matrix variance equation in the conventional Kalman-Bucy filter, i.e., no special computational algorithms should be developed. Third, the designed filtering procedure can be re-initialized at any moment, i.e., any time moment t1 can be assigned as the initial one for the system (3.17),(3.20), thus reducing the number of equations in (3.17),(3.20) to one. Of course, this re-initialized filter would no longer be optimal, since the initial conditions for Pi (t) should be recalculated using the previous observations, but still close to optimal. Moreover, the re-initialized filter would be convergent if the original optimal filter is convergent (see Remark 1) and, therefore, the re-initialization does not affect the asymptotic convergence properties of the estimation error. Thus, the obtained filter (3.16),(3.17),(3.20) provides not only the optimal theoretical result but also a practical real-time estimation procedure for linear systems with state delay. 3.2.3

Example

This section presents an example of designing the optimal filter for a linear state with delay over linear observations and comparing it to the best filter available for a linear state without delay, that is the Kalman-Bucy filter [42]. Let the unobserved state x(t) with delay be given by x(t) ˙ = x(t − 5),

x(s) = φ (s), s ∈ [−5, 0],

(3.24)

where φ (s) = N(0, 1) for s ≤ 0, and N(0, 1) is a Gaussian random variable with zero mean and unit variance. The observation process is given by y(t) = x(t) + ψ (t),

(3.25)

where ψ (t) is a white Gaussian noise, which is the weak mean square derivative of a standard Wiener process. The equations (3.24) and (3.25) present the conventional form for the equations (3.13) and (3.14), which is actually used in practice [6].

90

3 Optimal Filtering Problems for Time-Delay Systems

The filtering problem is to find the optimal estimate for the linear state with delay (3.24), using direct linear observations (3.25) confused with independent and identically distributed disturbances modeled as white Gaussian noises. Let us set the filtering horizon time to T = 10. Since 10 ∈ [1 × 5, 2 × 5], where 5 is the delay value in the state equation (3.24), the only first of the equations (3.20), along with the equations (3.16) and (3.17), should be employed. The filtering equations (3.16),(3.17), and the first of the equations (3.20) take the following particular form for the system (3.24),(3.25) m(t) ˙ = m(t − 5) + P(t)[y(t) − m(t)],

(3.26)

with the initial condition m(s) = E(φ (s)) = 0, s ∈ [−5, 0) and m(0) = E(φ (0) | y(0)) = m0 , s = 0; ˙ = 2P1 (t) − P2(t), P(t) (3.27) with the initial condition P(0) = E((x(0) − m(0))2 | y(0)) = P0 ; and P˙1 (t) = 2P(t − 5) + P2(t) − P(t)P(t − 5),

(3.28)

with the initial condition P1 (s) = E((x(s) − m(s))(x(s − 5) − m(s − 5)) | FsY ), s ∈ [0, 5]; finally, P2 (s) = E((x(s) − m(s))(x(s − 10) − m(s − 10)) | FsY ), s ∈ [5, 10). The particular forms of the equations (3.24) and (3.26) and the initial condition for x(t) imply that P1 (s) = P0 for s ∈ [0, 5] and P2 (s) = P0 for s ∈ [5, 10). The estimates obtained upon solving the equations (3.26)–(3.28) are compared to the conventional Kalman-Bucy estimates satisfying the following filtering equations for the linear state with delay (3.24) over linear observations (3.25), where the variance equation is a Riccati one and the equations for matrices Pi (t), i ≥ 1, are not employed: m˙K (t) = mK (t − 5) + PK (t)[y(t) − mK (t)],

(3.29)

with the initial condition mK (s) = E(φ (s)) = 0, s ∈ [−5, 0) and mK (0) = E(φ (0) | y(0)) = m0 , s = 0; P˙K (t) = 2PK (t) − PK2 (t), (3.30) with the initial condition PK (0) = E((x(0) − m(0))2 | y(0)) = P0 . Numerical simulation results are obtained solving the systems of filtering equations (3.26)–(3.28) and (3.29),(3.30). The obtained values of the estimates m(t) and mK (t) satisfying (3.26) and (3.29) respectively are compared to the real values of the state variable x(t) in (3.24). For each of the two filters (3.26)–(3.28) and (3.29),(3.30) and the reference system (3.24) involved in simulation, the following initial values are assigned: x0 = 2, m0 = 10, P0 = 100. Simulation results are obtained on the basis of a stochastic run using realizations of the Gaussian disturbances ψ1 (t) and ψ2 (t) in (3.25) generated by the built-in MatLab white noise function. The following graphs are obtained: graphs of the reference state variable x(t) for the system (3.24); graphs of the Kalman-Bucy filter estimate mK (t) satisfying the equations (3.29),(3.30); graphs of the optimal delayed state filter estimate m(t) satisfying the equations (3.26)–(3.28). The graphs of all those variables are shown on the entire simulation interval from t0 = 0 to T = 10 (Fig. 3.5), and around the reference time points:

3.2 Filtering Problem for Linear Systems with State Delay

91

state

30 20 10 0

0

1

2

3

4

5

6

7

8

9

10

6

7

8

9

10

6

7

8

9

10

time

Kalman estimate

30 20 10 0

0

1

2

3

4

5 time

Optimal estimate

30 20 10 0

0

1

2

3

4

5 time

Fig. 3.5. Graphs of the reference state variable x(t) and the estimates mK (t) and m(t) on the entire simulation interval [0, 10]

T = 8 (Fig. 3.6), T = 9 (Fig. 3.7), and T = 10 (Fig. 3.8). It can also be noted that the error variance P(t) converges to zero, since the optimal estimate (3.26) converges to the real state (3.24). The following values of the reference state variable x(t) and the estimates m(t) and mK (t) are obtained at the reference time points: for T = 8, x(8) = 17.5, m(8) = 17.55, mK (8) = 17.76; for T = 9, x(9) = 23.0, m(9) = 23.02, mK (9) = 23.24; for T = 10, x(10) = 29.5, m(10) = 29.5, mK (10) = 29.75. The behavior of the optimal estimate m(t) in Fig. 3.5 is more oscillatory in the first part of the time interval than that of the Kalman-Bucy one. This phenomenon is readily explained by the fact that the optimal estimate provides a closer approximation of the real state values than the Kalman-Bucy one. In doing so, the optimal estimate must be more sensitive to the incoming observations, which results in its more oscillatory behavior. Note, however, that the optimal estimation error is Lyapunov stable, since the optimal estimate does not leave a certain vicinity of the real state, after the reaching phase ends. Taking into account that all unbiased filters for the linear time-delay system (3.24) are gain-dependent modifications of the extended Kalman-Bucy (EKF) filter for the state (3.24) over the observations (3.25) (see [70]), the obtained optimal filter (3.26)– (3.28) is additionally compared to two frequently encountered EKF versions, the steady state filter [34] and the Smith prediction filter [82]. For all EKF filters, the estimate equation takes the form m(t) ˙ = m(t − 5) + P(t)[y(t) − m(t)],

92

3 Optimal Filtering Problems for Time-Delay Systems

17.55

state

17.5 17.45 17.4 7.99

7.992

7.994

7.996

7.998

8

8.002

8.004

8.006

8.008

8.01

8.002

8.004

8.006

8.008

8.01

8.002

8.004

8.006

8.008

8.01

time

Kalman estimate

17.85 17.8 17.75 17.7 17.65 7.99

7.992

7.994

7.996

7.998

8

Optimal estimate

time 17.8 17.7 17.6 17.5 17.4 7.99

7.992

7.994

7.996

7.998

8 time

Fig. 3.6. Graphs of the reference state variable x(t) and the estimates mK (t) and m(t) around the reference time point T = 8

where P(t) is a gain matrix to be chosen, with the same initial conditions as in (3.26). The gain equation is formed as 0 = 2P(t) − P2(t), yielding P(t) = 2, ˙ = 0 and P1 (t) = P(t) in (3.27)), and as for the steady state filter (assuming P(t) ˙ = 2P(t)Ψˆ (t − 5,t) − P2(t) P(t) for the Smith prediction filter, where Ψˆ (t, τ ) = y(t)/y(τ ) is the approximation of the state transition matrix for the time-delay system (3.24), calculated in view of the observations (3.25). The initial conditions for both gain equations are the same as in (3.27). The corresponding simulation results are m(8) = 17.83, m(9) = 23.26, m(10) = 29.76 for the steady state filter and m(8) = 17.68, m(9) = 24.2, m(10) = 29.81 for the Smith prediction filter. The simulation graphs themselves are thereby not illustrative and not included in the paper. Thus, it can be concluded that the obtained optimal filter for a linear state with delay over linear observations (3.26)–(3.28) yield definitely better estimates than the conventional Kalman-Bucy filter and two extended EKF filters. Discussion The simulation results show that the values of the estimate calculated by using the obtained optimal filter for a linear state with delay over linear observations are noticeably

3.2 Filtering Problem for Linear Systems with State Delay

93

23.1

state

23.05 23 22.95 22.9 8.99

8.992

8.994

8.996

8.998

9

9.002

9.004

9.006

9.008

9.01

9.002

9.004

9.006

9.008

9.01

9.002

9.004

9.006

9.008

9.01

time

Kalman estimate

23.35 23.3 23.25 23.2 8.99

8.992

8.994

8.996

8.998

9

Optimal estimate

time 23.08 23.06 23.04 23.02 23 8.99

8.992

8.994

8.996

8.998

9 time

Fig. 3.7. Graphs of the reference state variable x(t) and the estimates mK (t) and m(t) around the reference time point T = 9

29.5

state

29.48 29.46 29.44 29.42 9.99

9.991

9.992

9.993

9.994

9.995 time

9.996

9.997

9.998

9.999

10

9.991

9.992

9.993

9.994

9.995 time

9.996

9.997

9.998

9.999

10

9.991

9.992

9.993

9.994

9.995 time

9.996

9.997

9.998

9.999

10

Kalman estimate

29.76 29.74 29.72

Optimal estimate

29.7 9.99 29.502 29.5 29.498 29.496 29.494 9.99

Fig. 3.8. Graphs of the reference state variable x(t) and the estimates mK (t) and m(t) around the reference time point T = 10

94

3 Optimal Filtering Problems for Time-Delay Systems

closer to the real values of the reference variable than the values of the Kalman-Bucy or EKF estimates. Moreover, it can be seen that the estimate produced by the optimal filter for a linear state with delay over linear observations asymptotically converges to the real values of the reference variable as time tends to infinity, although the reference system (3.24) itself is unstable. On the contrary, the conventionally designed (non-optimal) Kalman-Bucy estimates and EKF estimates do not converge to the real values. This significant improvement in the estimate behavior is obtained due to the more careful selection of the filter gain matrix using the multi-equational system (3.27),(3.28), which compensates for unstable dynamics of the reference system, as it should be in the optimal filter. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration.

3.3 Filtering Problem for Linear Systems with State and Observation Delays 3.3.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ 0, and let (W1 (t), Ft ,t ≥ 0) and (W2 (t), Ft ,t ≥ 0) be independent Wiener processes. The partially observed Ft -measurable random process (x(t), y(t)) is described by a delay differential equation for the system state dx(t) = (a0 (t) + a(t)x(t − h))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(3.31)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], and a delay differential equation for the observation process: dy(t) = (A0 (t) + A(t)x(t − τ ))dt + B(t)dW2 (t),

(3.32)

where x(t) ∈ Rn is the state vector, y(t) ∈ Rm is the observation process, φ (s) is a mean square piecewise-continuous Gaussian stochastic process (see [41], Section 3.6, or [70], Subsection 2.7.5, for definition) given in the interval [t0 − h,t0 ] such that φ (s), W1 (t), and W2 (t) are independent. The system state x(t) dynamics depends on a delayed state x(t − h) and the observations y(t) are collected depending on another delayed state x(t − τ ), which actually make the system state space infinite-dimensional (see, for example, [56], Section 2.3). The vector-valued function a0 (s) describes the effect of system inputs (controls and disturbances). It is assumed that A(t) is a nonzero matrix and B(t)BT (t) is a positive definite matrix. All coefficients in (3.31),(3.32) are deterministic functions of appropriate dimensions. The estimation problem is to find the best estimate of the system state x(t) based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, which minimizes the Euclidean 2-norm ˆ J = E[(x(t) − x(t)) ˆ T (x(t) − x(t))] at each time moment t. Here, E[x(t)] means the expectation of a stochastic process x(t). As known, this optimal estimate is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY )

3.3 Filtering Problem for Linear Systems with State and Observation Delays

95

of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. The matrix functions P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ], that is the estimation error variance, and P(t,t − t1 ) = E[(x(t) − m(t))(x(t − t1 ) − m(t − t1 ))T | FtY ], that is the covariance between the estimation error values at different time moments, P(t,t) = P(t), are used to obtain a system of filtering equations. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differentials of the conditional expectation m(t) = E(x(t) | FtY ), the error variance P(t), and other bilinear functions of x(t) − m(t) and given in the following section. 3.3.2

Optimal Filter for Linear Systems with State and Observation Delays

The optimal filtering equations can be obtained using the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E(ϕ (x) | FtY )dt + E(x[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY )×

(3.33)

 −1 B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where ϕ (x) is the drift term in the state equation equal to ϕ (x) = a0 (t) + a(t)x(t − h) and ϕ1 (x) is the drift term in the observation equation equal to ϕ1 (x) = A0 (t) + A(t)x(t − τ ). Note that the conditional expectation equality E(x(t − h) | FtY ) = E(x(t − Y ) = m(t − h) is valid for any h > 0, since, in view of a positive delay shift h) | Ft−h h > 0, the treated problem (3.31),(3.32) is a filtering problem, not a smoothing one, and, therefore, the formula (3.33) yields the optimal estimate m(s) for any time s, t0 < s ≤ t, if the observations (3.32) are obtained until the current moment t (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)). Upon performing substitution of the expressions for ϕ and ϕ1 into (3.33) and taking into account the conditional expectation equality, the estimate equation takes the form dm(t) = (a0 (t) + a(t)m(t − h))dt+ E(x(t)[A(t)(x(t − τ ) − m(t − τ ))]T | FtY )× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t − τ )dt) = = (a0 (t) + a(t)m(t − h))dt+ E([x(t) − m(t)][x(t − τ ) − m(t − τ )]T | FtY )AT (t)× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t − τ )dt) = (a0 (t) + a(t)m(t − h))dt + P(t,t − τ )AT (t)×

(3.34)

96

3 Optimal Filtering Problems for Time-Delay Systems

(B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t − τ )dt) To compose a system of the filtering equations, the equation for the conditional expectation E([x(t) − m(t)][(x(t − τ ) − m(t − τ ))]T | FtY ) should be obtained. This can be done using the equation (3.31) for the state x(t), the equation (3.34) for the estimate m(t), and the formula for the Ito differential of a product of two processes satisfying Ito differential equations (see ([41], Corollary 2 to Lemma 4.2, or [70], formula (5.1.13)): d(z1 zT2 ) = z1 dzT2 + (z2 dzT1 )T + (1/2)[y1ν yT2 + y2 ν yT1 ]dt.

(3.35)

Here, the stochastic process z1 satisfies the equation dz1 = x1 dt + y1 dw1 , the stochastic process z2 satisfies the equation dz2 = x2 dt + y2 dw2 , and ν is the covariance intensity matrix of the Wiener vector [w1 w2 ]T . Let us obtain the formula for the Ito differential of the general expression P(t,t − t1 ) = E([x(t) − m(t)][x(t − t1 ) − m(t − t1 )]T | FtY ), where t1 > 0 is an arbitrary delay, not necessarily equal to τ . Upon representing P(t,t − t1 ) as P(t,t − t1 ) = E([x(t)(x(t − t1 ))T ] | FtY ) − m(t)m(t − t1 ), using first x(t) as z1 and x(t − t1 ) as z2 and then m(t) as z1 and m(t − t1 ) as z2 in the formula (3.35), taking into account independence of the Wiener processes W1 and W2 in the equations (3.31) and (3.32), and finally subtracting the second derived equation from the first one, the following formula is obtained dP(t,t − t1 )/dt = a(t)P(t − h,t − t1) + P(t,t − t1 − h)aT (t − t1 )+

(3.36)

(1/2)[b(t)bT (t − t1 ) + b(t − t1 )bT (t)]− (1/2)[P(t,t − τ )AT (t)(B(t)BT (t))−1 B(t)× BT (t − t1 )(B(t − t1 )BT (t − t1 ))−1 A(t − t1 )PT (t − t1 ,t − t1 − τ )− P(t − t1 ,t − t1 − τ )AT (t − t1 )(B(t − t1 )BT (t − t1 ))−1 B(t − t1 )× BT (t)(B(t)BT (t))−1 A(t)PT (t,t − τ )]. Analysis of the formula (3.36) in the case t1 = τ implies that the equation for P(t,t − τ ) includes variables P(t,t − τ − h), P(t − h,t − τ ) and the same P(t,t − τ ) in its righthand side. Taking into account that P(t − h,t − τ ) is represented as P(t,t − τ + h) with the arguments delayed by h, the new variables involved in the equations for P(t,t − τ ) are P(t,t − τ − h) and P(t,t − τ + h). This structure is repeated in the equations for P(t,t − τ − h), P(t,t − τ + h), etc. Hence, the system of the optimal filtering equations for the state (3.31), whose proper dynamics is delayed by h, over the delayed by τ observations (3.32) is the infinitedimensional system composed by the equation (3.34) for the optimal estimate and the equations (3.36) for the covariances P(t,t − τ + kh), where k = . . . , −2, −1, 0, 1, 2, . . . is an arbitrary integer number.

3.3 Filtering Problem for Linear Systems with State and Observation Delays

97

Using the notation Pk (t) = P(t,t − τ − kh), the equation (3.34) can be rewritten as dm(t) = (a0 (t) + a(t)m(t − h))dt + P0(t)AT (t)×

(3.37)

(B(t)BT (t))−1 (dy(t) − (A0 (t) + A(t)m(t − τ )dt), and the system (3.36) can be represented in the following form dPk (t)/dt = a(t)Pk−1 (t − h) + Pk+1(t)aT (t − τ − kh)+

(3.38)

(1/2)[b(t)bT (t − τ − kh) + b(t − τ − kh)bT (t)]− (1/2)[P0(t)AT (t)(B(t)BT (t))−1 B(t)BT (t − τ − kh)× (B(t − τ − kh)BT (t − τ − kh))−1 A(t − τ − kh)P0T (t − τ − kh)− P0 (t − τ − kh)AT (t − τ − kh)(B(t − τ − kh)BT (t − τ − kh))−1 × B(t − τ − kh)BT (t)(B(t)BT (t))−1 A(t)P0T (t)]. Thus, the preceding conclusion can be formulated in the final form: the system of the optimal filtering equations for the state (3.31), whose proper dynamics is delayed by h, over the delayed by τ observations (3.32) is the infinite system composed by the equation (3.37) for the optimal estimate and the equations (3.38) for the covariances Pk (t) = P(t,t − τ − kh), where k = . . . , −2, −1, 0, 1, 2, . . . is an arbitrary integer number. The last step is to establish the initial conditions for the system of equations (3.37),(3.38). The initial conditions for (3.37) are stated as m(s) = E(φ (s)), s ∈ [t0 − h,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 ,

(3.39)

The initial conditions for matrices Pk (t) = E((x(t) − m(t))(x(t − τ − kh))T | FtY ) should be stated as functions in the intervals [max{t0 − h,t0 + τ + (k − 1)h}, max{t0 + τ + kh,t0 }], since the equations (3.38) corresponding to non-negative k depend on coefficients with arguments delayed by τ + kh, which are not defined for t < t0 . Thus, the initial conditions for the matrices Pk (t) are stated as Pk (s) = E((x(s) − m(s))(x(s − τ − kh) − m(s − τ − kh))T | FsY ),

(3.40)

s ∈ [max{t0 − h,t0 + τ + (k − 1)h}, max{t0 + τ + kh,t0 }]. Unfortunately, the system (3.37),(3.38) cannot be reduced to a finite system for any fixed filtering horizon t, as it can be done in the case of only state delay in the equations (3.31),(3.32) (see the preceding section), since the infinite number of the equations (3.38) for Pk (t) with negative k are always needed to compose a closed system for any time t. However, this reduction is possible for some particular cases, for example, in the case of equal, τ = h, (or commensurable, τ = qh, q is natural) delays in the equations (3.31),(3.32), which is considered in details in the next subsection.

98

3 Optimal Filtering Problems for Time-Delay Systems

3.3.3

Optimal Filter for Linear Systems with Commensurable State and Observation Delays

An important and frequently encountered in practical applications particular case of commensurable delays in state and observation equations is recovered assuming τ = qh, q = 1, 2, . . . is a natural number. In doing so, the state and observation equations (3.31),(3.32) take the form dx(t) = (a0 (t) + a(t)x(t − h))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(3.41)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], dy(t) = (A0 (t) + A(t)x(t − qh))dt + B(t)dW2 (t).

(3.42)

Accordingly, the optimal filtering equation (3.37) for the optimal estimate m(t) turns to dm(t) = (a0 (t) + a(t)m(t − h))dt + P0(t)AT (t)×

(3.43)

−1

(B(t)B (t)) (dy(t) − (A0(t) + A(t)m(t − qh)dt), T

and the system (3.38) is given by dPk (t)/dt = a(t)Pk−1 (t − h) + Pk+1(t)aT (t − (q + k)h)+

(3.44)

(1/2)[b(t)bT (t − (q + k)h) + b(t − (q + k)h)bT (t)]− (1/2)[P0(t)AT (t)(B(t)BT (t))−1 B(t)BT (t − (q + k)h)× (B(t − (q + k)h)BT (t − (q + k)h))−1A(t − (q + k)h)P0T (t − (q + k)h)− P0 (t − (q + k)h)AT (t − (q + k)h)(B(t − (q + k)h)BT (t − (q + k)h))−1× B(t − (q + k)h)BT (t)(B(t)BT (t))−1 A(t)P0T (t)]. Using the equality P−q−1(t − h) = P(t − h,t − (−q − 1)h − qh − h) = T (t), P(t − h,t) = PT (t,t − h) = P−q+1

the equation for P−q in (3.44) can be rewritten as T (t) + P−q+1(t)aT (t)+ dP−q (t)/dt = a(t)P−q+1

(3.45)

b(t)bT (t) − P0(t)AT (t)(B(t)BT (t)−1 A(t)P0T (t). Note that P−q (t) = E((x(t) − m(t))(x(t))T | FtY ) is the estimation error variance. If q = 1, 2, . . ., the right-hand side of (3.45) does not include variables Pk corresponding to negative k < −q. Hence, a closed system of the filtering equations is formed by the equations (3.43),(3.45) and the equations (3.44) with k ≥ −q only. This enables one to obtain a finite system of the filtering equations for any fixed filtering horizon t, as follows.

3.3 Filtering Problem for Linear Systems with State and Observation Delays

99

Namely, for every fixed t, the number of equations corresponding to k ≥ −q in (3.44), that should be taken into account to obtain a closed system of the filtering equations, is not equal to infinity, since the matrices a(t), b(t), A(t), and B(t) are not defined for t < t0 . Therefore, if the current time moment t belongs to the semi-open interval (t0 + (k + q)h,t0 + (k + q + 1)h], where h is the delay value in the equations (3.31),(3.32), the number of equations in (3.44) is equal to k + q. The last step is to establish the initial conditions for the system of equations (3.43),(3.45),(3.44). The initial conditions for (3.43) and (3.45) are stated as m(s) = E(φ (s)), s ∈ [t0 − τ ,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 ,

(3.46)

and P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 ))T | FtY0 ].

(3.47)

− (q + k)h))T

The initial conditions for matrices Pk (t) = E((x(t) − m(t))(x(t | FtY ) should be stated as functions in the intervals [t0 + (k + q − 1)h,t0 + (k + q)h], since the kth of the equations (3.44) depends on functions with the arguments delayed by (k + q)h and the definition of Pk (t) itself assumes dependence on x(t − (k + q)h). Thus, the initial conditions for the matrices Pk (t) in (3.44) are stated as Pk (s) = E((x(s) − m(s))(x(s − (q + k)h) − m(s − (q + k)h))T | FsY ), s ∈ [t0 + (q + k − 1)h,t0 + (q + k)h].

(3.48)

The obtained system of the filtering equations (3.43),(3.45),(3.44) with the initial conditions (3.46)–(3.48) presents the optimal solution to the filtering problem for the linear state with delay (3.41) over the linear observations (3.42). A considerable advantage of the designed filter is a finite number of the filtering equations for any fixed filtering horizon, although the state space of the delayed system (3.41) is infinitedimensional. Remark. The convergence properties of the obtained optimal estimate (3.37) are given by the standard convergence theorem (see, for example, [41], Theorem 7.5 and Section 7.7)): if in the system (3.31),(3.32) the pair (a(t)Ψ (t − h,t), b(t)) is uniformly completely controllable and the pair (a(t)Ψ (t − h,t), A(t)Ψ (t − τ ,t) is uniformly completely observable, where Ψ (t, τ ) is the state transition matrix for the equation (3.31) (see [56], Section 3.2, for definition of matrix Ψ ), then the error of the obtained optimal filter (3.37),(3.38) is uniformly asymptotically stable. As usual, the uniform complete controllability condition is required for assuring non-negativeness of the error variance matrix P−q (t) and may be omitted, if the matrix P−q(t) is non-negative in view of its intrinsic properties. The uniform complete controllability and observability conditions for a linear system with delay (3.31) and observations (3.32) can be found in ([56], Chapter 5). 3.3.4

Example

This section presents an example of designing the optimal filter for linear systems with state and observation delays and comparing it to the best filter available for linear systems without delay, that is the Kalman-Bucy filter [42].

100

3 Optimal Filtering Problems for Time-Delay Systems

state

250 200 150 100 50

Kalman estimate

0

0

10

20

30

40

0

10

20

30

40

0

10

20

30

40

time

50

60

70

80

50

60

70

80

50

60

70

80

250 200 150 100 50 0

time

Optimal estimate

250 200 150 100 50 0 −20

time

Fig. 3.9. Graphs of the reference state variable x(t) and the estimates mK (t) and m(t) on the entire simulation interval [0, 80]

Let the unobserved state x(t) with delay be given by x(t) ˙ = x(t − 5),

x(s) = φ (s), s ∈ [−5, 0],

(3.49)

where φ (s) = N(0, 1) for s ≤ 0, and N(0, 1) is a Gaussian random variable with zero mean and unit variance. The observation process is given by y(t) = x(t − 5) + ψ (t),

(3.50)

where ψ (t) is a white Gaussian noise, which is the weak mean square derivative of a standard Wiener process. The equations (3.49) and (3.50) present the conventional form for the equations (3.31) and (3.32), which is actually used in practice [6]. Since the observation delay is equal to the state one, the system (3.49),(3.50) satisfies the conditions of Subsection 3.3.3 with m = 1. The filtering problem is to find the optimal estimate for the linear state with delay (3.49), using the linear observations with delay (3.50) confused with independent and identically distributed disturbances modeled as white Gaussian noises. Let us set the filtering horizon time to T = 80. Since 80 ∈ (15×5, 16×5], where 5 is the delay value in the equations (3.49),(3.50), the first 15 of the equations (3.44), along with the equations (3.43) and (3.45), should be employed. The filtering equations (3.43),(3.45), and the first 15 of the equations (3.44) take the following particular form for the system (3.49),(3.50) m(t) ˙ = m(t − 5) + P0(t)[y(t) − m(t − 5)],

(3.51)

3.3 Filtering Problem for Linear Systems with State and Observation Delays

101

state

12.56 12.55 12.54 39.99

39.992

39.994

39.996

39.998

40

40.002 time

40.004

40.006

40.008

40.01

39.992

39.994

39.996

39.998

40

40.002 time

40.004

40.006

40.008

40.01

39.992

39.994

39.996

39.998

40

40.002 time

40.004

40.006

40.008

40.01

Kalman estimate

12.765 12.76 12.755 12.75 12.745 39.99

Optimal estimate

12.64 12.63 12.62 12.61 39.99

Fig. 3.10. Graphs of the reference state variable x(t) and the estimates mK (t) and m(t) around the reference time point T = 40

with the initial condition m(s) = E(φ (s)) = 0, s ∈ [−5, 0) and m(0) = E(φ (0) | y(0)) = m0 , s = 0; P˙i (t) = Pi−1 (t − 5) + Pi+1(t) − P0(t)P0 (t − 5(i + 1)), (3.52) with the initial condition Pi (0) = E((x(s) − m(s))(x(s − 5(i + 1)) − m(s − 5(i + 1))) | FsY ), s ∈ [5i, 5(i + 1)], i = 0, . . . , 14; and P˙−1 (t) = 2P0 (t) − P02(t),

(3.53)

with the initial condition P−1 (0) = E((x(s) − m(s))2 | y(0)) = R0 ; note that P−1 (t) is the error variance. The particular forms of the equations (3.49) and (3.51) and the initial condition for x(t) imply that Pi (s) = R0 , i = 0, . . . , 15, for s ∈ [5i, 5(i + 1)]. The estimates obtained upon solving the equations (3.51)–(3.53) are compared to the conventional Kalman-Bucy estimates satisfying the following filtering equations for the linear state with delay (3.49) over linear observations with delay (3.50), where the variance equation is a Riccati one and the equation for matrix P0 (t) is not employed: m˙K (t) = mK (t − 5) + PK (t)[y(t) − mK (t − 5)],

(3.54)

with the initial condition mK (s) = E(φ (s)) = 0, s ∈ [−5, 0) and mK (0) = E(φ (0) | y(0)) = m0 , s = 0; P˙K (t) = 2PK (t) − PK2 (t), (3.55) with the initial condition PK (0) = E((x(0) − m(0))2 | y(0)) = R0 .

102

3 Optimal Filtering Problems for Time-Delay Systems

state

51.65 51.6 51.55 51.5 59.99

59.992

59.994

59.996

59.998

60

60.002 time

60.004

60.006

60.008

60.01

59.992

59.994

59.996

59.998

60

60.002 time

60.004

60.006

60.008

60.01

59.992

59.994

59.996

59.998

60

60.002 time

60.004

60.006

60.008

60.01

Kalman estimate

52.16 52.14 52.12 52.1 52.08 59.99

Optimal estimate

51.54 51.52 51.5 51.48 51.46 59.99

Fig. 3.11. Graphs of the reference state variable x(t) and the estimates mK (t) and m(t) around the reference time point T = 60

Numerical simulation results are obtained solving the systems of filtering equations (3.51)–(3.53) and (3.54),(3.55). The obtained values of the estimates m(t) and mK (t) satisfying (3.51) and (3.54) respectively are compared to the real values of the state variable x(t) in (3.49). For each of the two filters (3.51)–(3.53) and (3.54),(3.55) and the reference system (3.49) involved in simulation, the following initial values are assigned: x0 = 1, m0 = 10, R0 = 100. Gaussian disturbance ψ (t) in (3.50) is realized using the built-in MatLab white noise function. The following graphs are obtained: graphs of the reference state variable x(t) for the system (3.49); graphs of the Kalman-Bucy filter estimate mK (t) satisfying the equations (3.54),(3.55); graphs of the optimal filter estimate for linear systems with state and observation delays m(t) satisfying the equations (3.51)–(3.53). The graphs of all those variables are shown on the entire simulation interval from T = 0 to T = 80 (Fig. 3.9), and around the reference time points: T = 40 (Fig. 3.10), T = 60 (Fig. 3.11), and T = 80 (Fig. 3.12). It can also be noted that the error variance P(t) converges to zero, given that the optimal estimate (3.51) converges to the real state (3.49). The following values of the reference state variable x(t) and the estimates m(t) and mK (t) are obtained at the reference time points: for T = 40, x(40) = 12.55, m(40) = 12.62, mK (40) = 12.75; for T = 60, x(60) = 51.56, m(60) = 51.50, mK (60) = 52.12; for T = 80, x(80) = 211.92, m(80) = 211.96, mK (80) = 214.08. Thus, it can be concluded that the obtained optimal filter for a linear systems with state delay and over linear observations with delay (3.51)–(3.53) yield better estimates than the conventional Kalman-Bucy filter.

3.4 Filtering Problem for Linear Systems with State and Multiple Observation Delays

103

state

212.1 212 211.9 211.8 211.7 79.99

79.992

79.994

79.996

79.998

80

80.002 time

80.004

80.006

80.008

80.01

79.992

79.994

79.996

79.998

80

80.002 time

80.004

80.006

80.008

80.01

79.992

79.994

79.996

79.998

80

80.002 time

80.004

80.006

80.008

80.01

Kalman estimate

214.4 214.2 214

Optimal estimate

213.8 79.99

212.2 212.1 212 211.9 211.8 79.99

Fig. 3.12. Graphs of the reference state variable x(t) and the estimates mK (t) and m(t) around the reference time point T = 80

3.3.5

Discussion

The simulation results show that the values of the estimate calculated by using the obtained optimal filter for a linear state with delay over linear observations with delay are noticeably closer to the real values of the reference variable than the values of the Kalman-Bucy estimates. Moreover, it can be seen that the estimate produced by the optimal filter for a linear state with delay over linear observations asymptotically converges to the real values of the reference variable as time tends to infinity, although the reference system (3.49) itself is unstable. On the contrary, the conventionally designed (non-optimal) Kalman-Bucy estimates do not converge to the real values. This significant improvement in the estimate behavior is obtained due to the more careful selection of the filter gain matrix using the multi-equational system (3.51)–(3.53), which compensates for unstable dynamics of the reference system, as it should be in the optimal filter. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration.

3.4 Filtering Problem for Linear Systems with State and Multiple Observation Delays 3.4.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ 0, and let (W1 (t), Ft ,t ≥ 0) and (W2 (t), Ft ,t ≥ 0) be independent

104

3 Optimal Filtering Problems for Time-Delay Systems

Wiener processes. The partially observed Ft -measurable random process (x(t), y(t)) is described by a delay differential equation for the system state dx(t) = (a0 (t) + a(t)x(t − h))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(3.56)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], and a delay differential equation for the observation process: p

dy(t) = (A(t) + ∑ Ai (t)x(t − τi ))dt + B(t)dW2 (t),

(3.57)

i=0

where x(t) ∈ Rn is the state vector, y(t) ∈ Rm is the observation process, τ0 = 0, and φ (s) is a mean square piecewise-continuous Gaussian stochastic process given in the interval [t0 − h,t0 ] such that φ (s), W1 (t), and W2 (t) are independent. The system state x(t) dynamics depends on a delayed state x(t − h) and the observations y(t) are collected depending on the current state x(t), as well as a bunch of delayed states x(t − τi ), τi > 0, i = 1, . . . , p, which actually make the system state space infinite-dimensional (see, for example, [56], Section 2.3). The vector-valued function a0 (s) describes the effect of system inputs (controls and disturbances). It is assumed that at least one of the matrices A j (t) is not zero matrix, and B(t)BT (t) is a positive definite matrix. All coefficients in (3.56),(3.57) are deterministic functions of appropriate dimensions. The estimation problem is to find the best estimate of the system state x(t) based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, which minimizes the Euclidean 2-norm J = E[(x(t) − x(t)) ˆ T (x(t) − x(t))] ˆ at each time moment t. Here, E[x(t)] means the expectation of a stochastic process x(t). As known, this optimal estimate is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY ) of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. The matrix functions P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ], that is the estimation error variance, and P(t,t − t1 ) = E[(x(t) − m(t))(x(t − t1 ) − m(t − t1 ))T | FtY ], that is the covariance between the estimation error values at different time moments, P(t,t) = P(t), are used to obtain a system of filtering equations. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differentials of the conditional expectation m(t) = E(x(t) | FtY ), the error variance P(t), and other bilinear functions of x(t) − m(t) and given in the following section.

3.4 Filtering Problem for Linear Systems with State and Multiple Observation Delays

3.4.2

105

Optimal Filter for Linear Systems with State and Multiple Observation Delays

The optimal filtering equations can be obtained using the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E(ϕ (x) | FtY )dt + E(x[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY )× −1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt),

(3.58)

where ϕ (x) is the drift term in the state equation equal to ϕ (x) = a0 (t) + a(t)x(t − h) and ϕ1 (x) is the drift term in the observation equation equal to ϕ1 (x) = A(t) + p ∑i=0 Ai (t)x(t − τi ). Note that the conditional expectation equality E(x(t − h) | FtY ) = Y ) = m(t − h) is valid for any h > 0, since, in view of a positive delay E(x(t − h) | Ft−h shift h > 0, the treated problem (3.56),(3.57) is a filtering problem, not a smoothing one, and, therefore, the formula (3.58) yields the optimal estimate m(s) for any time s, t0 < s ≤ t, if the observations (3.57) are obtained until the current moment t (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)). Upon performing substitution of the expressions for ϕ and ϕ1 into (3.58) and taking into account the conditional expectation equality, the estimate equation takes the form p

dm(t) = (a0 (t) + a(t)m(t − h))dt + E(x(t)[ ∑ Ai (t)(x(t − τi ) − m(t − τi ))]T | FtY )× i=0

(3.59) p

(B(t)BT (t))−1 (dy(t) − (A(t) + ∑ Ai (t)m(t − τi )dt) = i=0

p

(a0 (t) + a(t)m(t − h))dt + ∑ E([x(t) − m(t)][x(t − τi ) − m(t − τi )]T | FtY )ATi (t)× i=0

p

(B(t)BT (t))−1 (dy(t) − (A(t) + ∑ Ai (t)m(t − τi )dt) = (a0 (t) + a(t)m(t − h))dt+ i=0

p

p

∑ P(t,t − τi )ATi (t)(B(t)BT (t))−1 (dy(t) − (A(t) + ∑ Ai (t)m(t − τi )dt),

i=0

i=0

| FtY ]. where P(t,t − τi ) = E[(x(t) − m(t))(x(t − τi ) − m(t − τi To compose a system of the filtering equations, the equations for the conditional expectations E([x(t) − m(t)][(x(t − τi ) − m(t − τi ))]T | FtY ), i = 0, . . . , p, should be obtained. This can be done using the equation (1) for the state x(t), the equation (3.59) for the estimate m(t), and the formula for the Ito differential of a product of two processes satisfying Ito differential equations (see ([41], Corollary 2 to Lemma 4.2, or [70], formula (5.1.13)): ))T

d(z1 zT2 ) = z1 dzT2 + (z2 dzT1 )T + (1/2)[y1ν yT2 + y2 ν yT1 ]dt. Here, the stochastic process z1 satisfies the equation dz1 = x1 dt + y1 dw1 ,

(3.60)

106

3 Optimal Filtering Problems for Time-Delay Systems

the stochastic process z2 satisfies the equation dz2 = x2 dt + y2 dw2 , and ν is the covariance intensity matrix of the Wiener vector [w1 w2 ]T . Let us obtain the formula for the Ito differential of the general expression P(t,t − t1 ) = E([x(t) − m(t)][x(t − t1 ) − m(t − t1 )]T | FtY ), where t1 > 0 is an arbitrary delay, not necessarily equal to τ . Upon representing P(t,t − t1 ) as P(t,t − t1 ) = E([x(t)(x(t − t1 ))T ] | FtY ) − m(t)m(t − t1 ), using first x(t) as z1 and x(t − t1 ) as z2 and then m(t) as z1 and m(t − t1 ) as z2 in the formula (3.60), taking into account independence of the Wiener processes W1 and W2 in the equations (3.56) and (3.57), and finally subtracting the second derived equation from the first one, the following formula is obtained dP(t,t − t1 )/dt = a(t)P(t − h,t − t1) + P(t,t − t1 − h)aT (t − t1 )+

(3.61)

p

(1/2)[b(t)bT (t − t1 ) + b(t − t1 )bT (t)] − (1/2)[( ∑ P(t,t − τ j )ATj (t))(B(t)BT (t))−1 × j=0

p

B(t)BT (t − t1 )(B(t − t1 )BT (t − t1 ))−1 ( ∑ A j (t − t1 )PT (t − t1 ,t − t1 − τ j ))− j=0

p

( ∑ P(t − t1,t − t1 − τ j )ATj (t − t1 ))(B(t − t1 )BT (t − t1 ))−1 B(t − t1 )× j=0

p

BT (t)(B(t)BT (t))−1 ( ∑ A j (t)PT (t,t − τ j ))]. j=0

Analysis of the formula (3.61) in the case t1 = τi implies that the equation for P(t,t − τi ) includes variables P(t,t − τi − h), P(t − h,t − τi ), P(t,t − τ j ), and P(t − τi ,t − τi − τ j ), j = 0, . . . , p, in its right-hand side. Taking into account that P(t − h,t − τ ) is represented as P(t,t − τ + h) with the arguments delayed by h, and P(t − τi ,t − τi − τ j ) is represented as P(t,t − τ j ) with the arguments delayed by τi , the new variables involved in the equations for P(t,t − τi ) are P(t,t − τi − h), P(t,t − τi + h), and P(t,t − τ j ), j = 0, . . . , p. This structure is repeated in the equations for P(t,t − τi − h), P(t,t − τi + h), etc. Hence, the system of the optimal filtering equations for the state (3.56), whose proper dynamics is delayed by h, over the observations (3.57) delayed by τi , i = 0, . . . , p, is the infinite-dimensional system composed by the equation (3.60) for the optimal estimate and the equations (3.61) for the covariances P(t,t − τi + kh), where i = 0, . . . , p and k = . . . , −2, −1, 0, 1, 2, . . . is an arbitrary integer number. Using the notation Pik (t) = P(t,t − τi − kh), the equation (3.59) can be rewritten as p

dm(t) = (a0 (t) + a(t)m(t − h))dt + ( ∑ Pi0 (t)ATi (t))× i=0

p

(B(t)BT (t))−1 (dy(t) − (A(t) + ( ∑ Ai (t)m(t − τi ))dt), i=0

(3.62)

3.4 Filtering Problem for Linear Systems with State and Multiple Observation Delays

107

and the system (3.61) can be represented in the following form dPik (t)/dt = a(t)Pi,k−1 (t − h) + Pi,k+1(t)aT (t − τi − kh)+

(3.63)

(1/2)[b(t)bT (t − τi − kh) + b(t − τi − kh)bT (t)]− p

(1/2)[( ∑ Pj0 (t)ATj (t))(B(t)BT (t))−1 B(t)× j=0

p

T BT (t − τi − kh)(B(t − τi − kh)BT (t − τi − kh))−1 ( ∑ A j (t − τi − kh)Pj0 (t − τi − kh))− j=0

p

( ∑ Pj0 (t − τi − kh)ATj (t − τi − kh))× j=0

p

T (t))]. (B(t − τi − kh)BT (t − τi − kh))−1 B(t − τi − kh)BT (t)(B(t)BT (t))−1 ( ∑ A j (t)Pj0 j=0

Thus, the preceding conclusion can be formulated in the final form: the system of the optimal filtering equations for the state (3.56), whose proper dynamics is delayed by h, over over the observations (3.57) delayed by τi , i = 0, . . . , p, is the infinite system composed by the equation (3.62) for the optimal estimate and the equations (3.63) for the covariances Pik (t) = P(t,t − τi − kh), where i = 0, . . . , p and k = . . . , −2, −1, 0, 1, 2, . . . is an arbitrary integer number. The last step is to establish the initial conditions for the system of equations (3.62),(3.63). The initial conditions for (3.62) are stated as m(s) = E(φ (s)), s ∈ [t0 − h,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 ,

(3.64)

The initial conditions for matrices Pik (t) = E((x(t)− m(t))(x(t − τi − kh))T | FtY ) should be stated as functions in the intervals [max{t0 − h,t0 + τi + (k − 1)h}, max{t0,t0 + τi + kh}], since the equations (3.63) corresponding to non-negative k depend on coefficients with arguments delayed by τi + kh, which are not defined for t < t0 . Thus, the initial conditions for the matrices Pik (t) are stated as Pik (s) = E((x(s) − m(s))(x(s − τi − kh) − m(s − τi − kh))T | FsY ),

(3.65)

s ∈ [max{t0 − h,t0 + τi + (k − 1)h}, max{t0 + τi + kh,t0 }]. Unfortunately, the system (3.62),(3.63) cannot be reduced to a finite system for any fixed filtering horizon t, as it can be done in the case of only state delay in the equations (3.56),(3.57) (see Subsection 3.2), since the infinite number of the equations (3.63) for Pik (t) with negative k are always needed to compose a closed system for any time t. However, this reduction is possible for some particular cases, for example, in the case of commensurable (τi = qi h, qi = 0, 1, 2, . . .) delays in the equations (3.56),(3.57), which is considered in details in the next subsection.

108

3 Optimal Filtering Problems for Time-Delay Systems

3.4.3

Optimal Filter for Linear Systems with Commensurable State and Observation Delays

An important and frequently encountered in practical applications particular case of commensurable delays in state and observation equations is recovered assuming τi = qi h, qi = 0, 1, 2, . . . are natural numbers. In doing so, the state and observation equations (3.56),(3.57) take the form dx(t) = (a0 (t) + a(t)x(t − h))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(3.66)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], p

dy(t) = (A(t) + ∑ Ai (t)x(t − qi h))dt + B(t)dW2 (t).

(3.67)

i=0

Accordingly, the optimal filtering equation (3.62) for the optimal estimate m(t) turns to

p

dm(t) = (a0 (t) + a(t)m(t − h))dt + ( ∑ Pi0 (t)ATi (t))×

(3.68)

i=0

p

(B(t)BT (t))−1 (dy(t) − (A(t) + ( ∑ Ai (t)m(t − qih))dt), i=0

and the system (3.63) is given by dPik (t)/dt = a(t)Pi,k−1 (t − h) + Pi,k+1(t)aT (t − (qi + k)h)+

(3.69)

(1/2)[b(t)bT (t − (qi + k)h) + b(t − (qi + k)h)bT (t)]− p

(1/2)[( ∑ Pj0 (t)ATj (t))(B(t)BT (t))−1 B(t)BT (t − (qi + k)h)× j=0

p

T (B(t − (qi + k)h)BT (t − (qi + k)h))−1 ( ∑ A j (t − (qi + k)h)Pj0 (t − (qi + k)h))+ j=0

p

( ∑ Pj0 (t − (qi + k)h)ATj (t − (qi + k)h))(B(t − (qi + k)h)BT (t − (qi + k)h))−1 × j=0

p

T B(t − (qi + k)h)BT (t)(B(t)BT (t))−1 ( ∑ A j (t)Pj0 (t))]. j=0

Using the equality Pi,−qi−1 (t − h) = P(t − h,t − (−qi − 1)h − qih − h) = T (t), P(t − h,t) = PT (t,t − h) = Pi,−q i +1

the equation for Pi,−qi in (3.69) (with k = −qi ) can be rewritten as T (t) + Pi,−qi+1 (t)aT (t)+ dPi,−qi (t)/dt = a(t)Pi,−q i +1

(3.70)

3.4 Filtering Problem for Linear Systems with State and Multiple Observation Delays p

p

j=0

j=0

109

T b(t)bT (t) − ( ∑ Pj0 (t)ATj (t))(B(t)BT (t)−1 ( ∑ A j (t)Pj0 (t)).

Note that Pi,−qi (t) = E((x(t) − m(t))(x(t))T | FtY ) is the estimation error variance. If qi = 0, 1, 2, . . ., the right-hand side of (3.70) does not include variables Pik corresponding to negative k < −qi . Hence, a closed system of the filtering equations is formed by the equations (3.68),(3.70) and the equations (3.69) with k ≥ −qi only. This enables one to obtain a finite system of the filtering equations for any fixed filtering horizon t, as follows. Namely, for every fixed t, the number of equations corresponding to k ≥ −qi in (3.69), that should be taken into account to obtain a closed system of the filtering equations, is not equal to infinity, since the matrices a(t), b(t), A(t), Ai (t), and B(t) are not defined for t < t0 . Therefore, if the current time moment t belongs to the semi-open interval (t0 + (ki + qi )h,t0 + (ki + qi + 1)h], where h is the delay value in the equations (3.66),(3.67), the number of equations corresponding to the first index i of P(t) in (3.69) is equal to ki + qi , and the first index has the finite range i = 0, 1, . . . , p. Thus, the total number of the equations in (3.69), which should be employed at a moment t ∈ (t0 + (ki + qi )h,t0 + (ki + qi + 1)h], is equal to (ki + qi )(p + 1), since the quantity (ki + qi) is the same for any i = 0, 1, . . . , p. The last step is to establish the initial conditions for the system of equations (3.68),(3.70),(3.69). The initial conditions for (3.68) and (3.70) are stated as m(s) = E(φ (s)), s ∈ [t0 − h,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 ,

(3.71)

and P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 ))T | FtY0 ].

(3.72)

The initial conditions for matrices Pik (t) = E((x(t) − m(t))(x(t − (qi + k)h))T | FtY ) should be stated as functions in the intervals [t0 + (k + qi − 1)h,t0 + (k + qi )h], since the equation for Pik (t) in (3.69) depends on functions with the arguments delayed by (k + qi )h and the definition of Pik (t) itself assumes dependence on x(t − (k + qi )h). Thus, the initial conditions for the matrices Pik (t) in (3.69) are stated as Pik (s) = E((x(s) − m(s))(x(s − (qi + k)h) − m(s − (qi + k)h))T | FsY ), s ∈ [t0 + (qi + k − 1)h,t0 + (qi + k)h].

(3.73)

The obtained system of the filtering equations (3.68),(3.70),(3.69) with the initial conditions (3.71)–(3.73) presents the optimal solution to the filtering problem for the linear state with delay (3.66) over the linear observations (3.67). A considerable advantage of the designed filter is a finite number of the filtering equations for any fixed filtering horizon, although the state space of the delayed system (3.66) is infinitedimensional. Remark. The convergence properties of the obtained optimal estimate (3.62) are given by the standard convergence theorem (see, for example, [41], Theorem 7.5 and Section 7.7)): if in the system (3.56),(3.57) the pair (a(t)Ψ (t − h,t), b(t)) is uniformly comp pletely controllable and the pair (a(t)Ψ (t − h,t), ∑i=0 Ai (t)Ψ (t − τi ,t)) is uniformly

110

3 Optimal Filtering Problems for Time-Delay Systems

completely observable, where Ψ (t, τ ) is the state transition matrix for the equation (3.56) (see [56], Section 3.2, for definition of matrix Ψ ), then the error of the obtained optimal filter (3.62),(3.63) is uniformly asymptotically stable. As usual, the uniform complete controllability condition is required for assuring non-negativeness of the error variance matrix Pi,−qi (t) and may be omitted, if the matrix Pi,−qi (t) is non-negative in view of its intrinsic properties. The uniform complete controllability and observability conditions for a linear system with delay (3.56) and observations (3.57) can be found in ([56], Chapter 5). 3.4.4

Example

This section presents an example of designing the optimal filter for linear systems with state and multiple observation delays and comparing it to the best filter available for linear systems without delay, that is the Kalman-Bucy filter [42]. Let the unobserved state x(t) with delay be given by x(t) ˙ = x(t − 20),

x(s) = φ (s), s ∈ [−20, 0],

(3.74)

where φ (s) = N(0, 1) for s ≤ 0, and N(0, 1) is a Gaussian random variable with zero mean and unit variance. The observation process is given by y(t) = x(t) + x(t − 20) + ψ (t),

(3.75)

where ψ (t) is a white Gaussian noise, which is the weak mean square derivative of a standard Wiener process. The equations (3.74) and (3.75) present the conventional form for the equations (3.56) and (3.57), which is actually used in practice [6]. Since the observation delay is equal to the state one, the system (3.74),(3.75) satisfies the conditions of Subsection 3.4.3 with q0 = 0 and q1 = 1. The filtering problem is to find the optimal estimate for the linear state with delay (3.74), using the linear observations with multiple delays (3.75) confused with independent and identically distributed disturbances modeled as white Gaussian noises. Let us set the filtering horizon time to T = 35. Since 35 ∈ (1 × 20, 2 × 20], where 20 is the delay value in the equations (3.74),(3.75), only one of the equations (3.69), along with the equations (3.68) and (3.70), should be employed. Indeed, for qi = 0, k = 0, 1, and for qi = 1, k = −1, 0. Taking into account that Pi1 ,k1 (t) = Pi2 ,k2 , if i1 + k1 = i2 + k2 , only the equations for P00 (t), which is the error variance, and P10 (t) are different from each other. The filtering equations (3.68),(3.70), and the equation (3.69) for P10 (t) take the following particular form for the system (3.74),(3.75) m(t) ˙ = m(t − 20) + (P0,0 + P1,0)[y(t) − m(t) − m(t − 20)],

(3.76)

with the initial condition m(s) = E(φ (s)) = 0, s ∈ [−5, 0) and m(0) = E(φ (0) | y(0)) = m0 , s = 0; T P˙0,0 (t) = P1,0 (t) + P1,0(t) − (P0,0(t) + P1,0)2 , (3.77) P˙1,0 (t) = P0,0 (t − 20) + P1,1(t) − (P0,0(t) + P1,0 (t))(P0,0 (t − 20) + P1,0(t − 20)), (3.78)

3.4 Filtering Problem for Linear Systems with State and Multiple Observation Delays

111

with the initial condition P0,0 (0) = E((x(s) − m(s))2 | y(0)) = R0 . The initial conditions for the covariances P1k (s), k = 0, 1, in the intervals s ∈ [20k, 20(k + 1)], are assigned as P10 (s) = R0 /2 and P11 (s) = 0. The estimates obtained upon solving the equations (3.76)–(3.78) are compared to the conventional Kalman-Bucy estimates satisfying the following filtering equations for the linear state with delay (3.74) over linear observations with delay (3.75), where the variance equation is a Riccati one and the equation for matrix P0 (t) is not employed: m˙K (t) = mK (t − 20) + PK (t)[y(t) − mK (t − 20)],

(3.79)

with the initial condition mK (s) = E(φ (s)) = 0, s ∈ [−20, 0) and mK (0) = E(φ (0) | y(0)) = m0 , s = 0; P˙K (t) = 2PK (t) − PK2 (t), (3.80) with the initial condition PK (0) = E((x(0) − m(0))2 | y(0)) = R0 . Numerical simulation results are obtained solving the systems of filtering equations (3.76)–(3.78) and (3.79),(3.80). The obtained values of the estimates m(t) and mK (t) satisfying (3.76) and (3.79) respectively are compared to the real values of the state variable x(t) in (3.74). For each of the two filters (3.76)–(3.78) and (3.79),(3.80) and the reference system (3.74) involved in simulation, the following initial values are assigned: x0 = 1, m0 = 10, R0 = 100. Gaussian disturbance ψ (t) in (3.75) is realized using the built-in MatLab white noise function. The following graphs are obtained: graphs of the reference state variable x(t) for the system (3.74); graphs of the Kalman-Bucy filter estimate mK (t) satisfying the equations (3.79),(3.80); graphs of the optimal filter estimate for linear systems with state and multiple observation delays m(t) satisfying the equations (3.76)–(3.78). The graphs of all those variables are shown on the entire simulation time interval [0, 35] (Fig. 3.13) and the post-delay time interval [20, 35] (Fig. 3.14), where the graphs of x(t), m(t), and mK (t) are given by dotted, dashed, and solid lines, respectively. It can also be noted that the error variance P(t) converges to zero, given that the optimal estimate (3.76) converges to the real state (3.74). The following values of the reference state variable x(t) and the estimates m(t) and mK (t) are obtained at the reference time points: for T = 25, x(25) = 331, m(25) = 342.5, mK (25) = 316.4; for T = 30, x(30) = 711, m(30) = 719.7, mK (30) = 692; for T = 35, x(35) = 1341, m(35) = 1340.6, mK (35) = 1321. Thus, it can be concluded that the obtained optimal filter for a linear systems with state delay and over linear observations with multiple delays (3.76)–(3.78) yield better estimates than the conventional Kalman-Bucy filter. Discussion The simulation results show that the values of the estimate calculated by using the obtained optimal filter for a linear state with delay over linear observations with multiple delays are noticeably closer to the real values of the reference variable than the values of the Kalman-Bucy estimates. Moreover, it can be seen that the estimate produced by the optimal filter for a linear state with delay over linear observations with multiple delays

112

3 Optimal Filtering Problems for Time-Delay Systems

1400

State, Kalman estimate, Optimal estimate

1200

1000

800

600

400

200

0

0

5

10

15

20

25

30

35

Time

Fig. 3.13. Graphs of the reference state variable x(t) (dotted line) and the estimates mK (t) (solid line) and m(t) (dashed line) on the entire simulation time interval [0, 35] 1400

State, Kalman estimate, Optimal estimate

1200

1000

800

600

400

200

0 20

25

30

35

Time

Fig. 3.14. Graphs of the reference state variable x(t) (dotted line) and the estimates mK (t) (solid line) and m(t) (dashed line) on the post-delay time interval [20, 35]

3.5 Filtering Problem for Linear Systems with Multiple State and Observation Delays

113

asymptotically converges to the real values of the reference variable as time tends to infinity, although the reference system (3.74) itself is unstable. On the contrary, the conventionally designed (non-optimal) Kalman-Bucy estimates do not converge to the real values. This significant improvement in the estimate behavior is obtained due to the more careful selection of the filter gain matrix using the multi-equational system (3.76)–(3.78), which compensates for unstable dynamics of the reference system, as it should be in the optimal filter. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration.

3.5 Filtering Problem for Linear Systems with Multiple State and Observation Delays 3.5.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ 0, and let (W1 (t), Ft ,t ≥ 0) and (W2 (t), Ft ,t ≥ 0) be independent Wiener processes. The partially observed Ft -measurable random process (x(t), y(t)) is described by a delay differential equation for the system state p

dx(t) = (a0 (t) + ∑ ai (t)x(t − hi))dt + b(t)dW1(t), x(t0 ) = x0 ,

(3.81)

i=0

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], h = max (h1 , . . . , h p ), and a delay differential equation for the observation process q

dy(t) = (A(t) + ∑ A j (t)x(t − τ j ))dt + B(t)dW2 (t),

(3.82)

j=0

where x(t) ∈ Rn is the state vector, y(t) ∈ Rm is the observation process, h0 = τ0 = 0, and φ (s) is a mean square piecewise-continuous Gaussian stochastic process given in the interval [t0 − h,t0 ] such that φ (s), W1 (t), and W2 (t) are independent. The system state x(t) dynamics and the observations y(t) depend on the current state x(t), as well as on a bunch of delayed states x(t − hi ) and x(t − τi ), hi > 0, τi > 0, i = 1, . . . , p, j = 1, . . . , q, which actually make the system state space infinite-dimensional (see, for example, [56], Section 2.3). The vector-valued function a0 (s) describes the effect of system inputs (controls and disturbances). It is assumed that at least one of the matrices A j (t) is not zero matrix, and B(t)BT (t) is a positive definite matrix. All coefficients in (3.81),3.82) are deterministic functions of appropriate dimensions. The estimation problem is to find the best estimate of the system state x(t) based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, that minimizes the Euclidean 2-norm ˆ | FtY ] J = E[(x(t) − x(t)) ˆ T (x(t) − x(t)) at every time moment t. Here, E[z(t) | FtY ] means the conditional expectation of a stochastic process z(t) = (x(t) − x(t)) ˆ T (x(t) − x(t)) ˆ with respect to the σ - algebra FtY

114

3 Optimal Filtering Problems for Time-Delay Systems

generated by the observation process Y (t) in the interval [t0 ,t]. As known, this optimal estimate is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY ) of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. The matrix functions P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ], that is the estimation error variance, and P(t,t − t1 ) = E[(x(t) − m(t))(x(t − t1 ) − m(t − t1 ))T | FtY ], that is the covariance between the estimation error values at different time moments, P(t,t) = P(t), are used to obtain a system of filtering equations. 3.5.2

Optimal Filter for Linear Systems with Multiple State and Observation Delays

The optimal filtering equations can be obtained using the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7) dm(t) = E(ϕ (x) | FtY )dt + E(x[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY )×

(3.83)

−1  B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), p where ϕ (x) is the drift term in the state equation equal to ϕ (x) = a0 (t) + ∑i=0 ai (t)x(t − hi ) and ϕ1 (x) is the drift term in the observation equation equal to ϕ1 (x) = A(t) + p ∑i=0 Ai (t)x(t − τi ). Note that the conditional expectation equality E(x(t − h) | FtY ) = Y E(x(t − h) | Ft−h ) = m(t − h) is valid for any h > 0, since, in view of a positive delay shift h > 0, the treated problem (3.81),(3.82) is a filtering problem, not a smoothing one, and, therefore, the formula (3.83) yields the optimal estimate m(s) for any time s, t0 < s ≤ t, if the observations (3.82) are obtained until the current moment t ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)). Upon performing substitution of the expressions for ϕ and ϕ1 into (3.83) and taking into account the conditional expectation equality, the estimate equation takes the form p

q

i=0

j=0

dm(t) = (a0 (t) + ∑ ai (t)m(t − hi ))dt + E(x(t)[ ∑ Ai (t)(x(t − τi ) − m(t − τi ))]T | FtY )× (3.84) q

(B(t)B (t)) (dy(t) − (A(t) + ∑ Ai (t)m(t − τi )dt) = T

−1

j=0

p

q

i=0

j=0

(a0 (t) + ∑ ai (t)m(t − hi ))dt + ∑ E([x(t) − m(t)][x(t − τi ) − m(t − τi )]T | FtY )ATi (t)×

3.5 Filtering Problem for Linear Systems with Multiple State and Observation Delays q

115

p

(B(t)BT (t))−1 (dy(t) − (A(t) + ∑ Ai (t)m(t − τi )dt) = (a0 (t) + ∑ ai (t)m(t − hi ))dt+ j=0

i=0

q

q

j=0

j=0

∑ P(t,t − τi )ATi (t)(B(t)BT (t))−1 (dy(t) − (A(t) + ∑ Ai (t)m(t − τi )dt),

where P(t,t − τi ) = E[(x(t) − m(t))(x(t − τi ) − m(t − τi ))T | FtY ]. To compose a system of the filtering equations, the equations for the conditional expectations E([x(t) − m(t)][(x(t − τi ) − m(t − τi ))]T | FtY ), i = 0, . . . , p, should be obtained. This can be done using the equation (3.81) for the state x(t), the equation (3.84) for the estimate m(t), and the formula for the Ito differential of a product of two processes satisfying Ito differential equations (see ([41], Corollary 2 to Lemma 4.2, or [70], formula (5.1.13)): d(z1 zT2 ) = z1 dzT2 + (z2 dzT1 )T + (1/2)[y1ν yT2 + y2 ν yT1 ]dt.

(3.85)

Here, the stochastic process z1 satisfies the equation dz1 = x1 dt + y1 dw1 , the stochastic process z2 satisfies the equation dz2 = x2 dt + y2 dw2 , and ν is the covariance intensity matrix of the Wiener vector [w1 w2 ]T . Let us obtain the formula for the Ito differential of the general expression P(t,t − t1 ) = E([x(t) − m(t)][x(t − t1 ) − m(t − t1 )]T | FtY ), where t1 > 0 is an arbitrary delay, not necessarily equal to τ . Upon representing P(t,t − t1 ) as P(t,t − t1 ) = E([x(t)(x(t − t1 ))T ] | FtY ) − m(t)m(t − t1 ), using first x(t) as z1 and x(t − t1 ) as z2 and then m(t) as z1 and m(t − t1 ) as z2 in the formula (3.85), taking into account independence of the Wiener processes W1 and W2 in the equations (3.81) and (3.82), and finally subtracting the second derived equation from the first one, the following formula is obtained p

p

i=0

i=0

dP(t,t − t1 )/dt = ∑ a(t)P(t − hi ,t − t1) + ∑ P(t,t − t1 − hi )aT (t − t1 )+ (1/2)[b(t)bT (t − t1 ) + b(t − t1 )bT (t)]− p

(1/2)[( ∑ P(t,t − τ j )ATj (t))(B(t)BT (t))−1 B(t)× j=0

p

BT (t − t1 )(B(t − t1 )BT (t − t1 ))−1 ( ∑ A j (t − t1 )PT (t − t1 ,t − t1 − τ j ))− j=0

p

( ∑ P(t − t1,t − t1 − τ j )ATj (t − t1 ))(B(t − t1 )BT (t − t1 ))−1 B(t − t1 )× j=0

p

BT (t)(B(t)BT (t))−1 ( ∑ A j (t)PT (t,t − τ j ))]. j=0

(3.86)

116

3 Optimal Filtering Problems for Time-Delay Systems

Analysis of the formula (3.86) in the case t1 = τk , k = 0, . . . , p, implies that the equation for P(t,t − τk ) includes variables P(t,t − τk − hi ), P(t − hi ,t − τk ), P(t,t − τ j ), and P(t − τk ,t − τk − τ j ), i = 0, . . . , p, j = 0, . . . , p, k = 0, . . . , p, in its right-hand side. Taking into account that P(t − hi,t − τ ) is represented as P(t,t − τ + hi) with the arguments delayed by hi , and P(t − τk ,t − τk − τ j ) is represented as P(t,t − τ j ) with the arguments delayed by τk , the new variables involved in the equations for P(t,t − τ j ) are P(t,t − τ j − hi), P(t,t − τ j + hi ), and P(t,t − τ j ), i = 0, . . . , p, j = 0, . . . , p. This structure is repeated in the equations for P(t,t − τ j − hi ), P(t,t − τ j + hi ), etc. Hence, the system of the optimal filtering equations for the state (3.81), whose proper dynamics is delayed by hi , i = 0, . . . , p, over the observations (3.82) delayed by τ j , j = 0, . . . , q, is the infinite-dimensional system composed by the equation (3.84) for the optimal estimate and the equations (3.86) for the covariances P(t,t − τ j + ki hi ), where j = 0, . . . , q, i = 0, . . . , p and ki = . . . , −2, −1, 0, 1, 2, . . . are arbitrary integer numbers. p Using the notation Pj,k1 ,...,k p (t) = P(t,t − τ j − ∑i=1 ki hi ), the equation for the optimal estimate can be written as p

q

i=0

j=0

dm(t) = (a0 (t) + ∑ ai (t)m(t − hi ))dt + ( ∑ Pj,0,...,0 (t)ATj (t))×

(3.87)

q

(B(t)BT (t))−1 (dy(t) − (A(t) + ( ∑ A j (t)m(t − τ j ))dt), j=0

and the equations for the covariances can be represented in the following form (assump ing t1 = t j,k1 ,...,k p = τ j + ∑i=1 ki h i ) p

dPj,k1 ,...,k p (t)/dt = ∑ [ai (t)Pj,k1 ,...,ki −1,...,k p (t − hi )+ i=0

Pj,k1 ,...,ki +1,...,k p (t)aTi (t − t j,k1 ,...,k p ]+ (1/2)[b(t)bT (t − t j,k1 ,...,k p ) + b(t − t j,k1 ,...,k p )bT (t)]− q

(1/2)[( ∑ Pj,0,...,0 (t)ATj (t))(B(t)BT (t))−1 B(t)× j=0

BT (t − t j,k1 ,...,k p )(B(t − t j,k1 ,...,k p )BT (t − t j,k1 ,...,k p ))−1 × p

T ( ∑ A j (t − t j,k1 ,...,k p )Pj,0,...,0 (t − t j,k1 ,...,k p ))− j=0 q

( ∑ Pj,0,...,0 (t − t j,k1 ,...,k p )ATj (t − t j,k1 ,...,k p ))× j=0

(B(t − t j,k1 ,...,k p )BT (t − t j,k1 ,...,k p ))−1 B(t − t j,k1 ,...,k p )× q

T (t))]. BT (t)(B(t)BT (t))−1 ( ∑ A j (t)Pj,0,...,0 j=0

(3.88)

3.5 Filtering Problem for Linear Systems with Multiple State and Observation Delays

117

Thus, the system of the optimal filtering equations for the state (3.81), whose proper dynamics is delayed by hi , i = 0, . . . , p, over the observations (3.82) delayed by τ j , j = 0, . . . , q, is the infinite system composed by the equation (3.87) for the optimal estimate p and the equations (3.88) for the covariances Pj,k1 ,...,k p (t) = P(t,t − τ j − ∑i=1 ki hi ), where i = 1, . . . , p, j = 0, . . . , q, and ki = . . . , −2, −1, 0, 1, 2, . . . are arbitrary integer numbers. The last step is to establish the initial conditions for the system of equations (3.87),(3.88). The initial conditions for (3.87) are stated as m(s) = E(φ (s)), s ∈ [t0 − h,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 . p The initial conditions for matrices Pj,k1 ,...,k p (t) = P(t,t − τ j − ∑i=1 ki hi ) should be p stated as functions in the intervals [max{t0 − h,t0 − h + τ j + ∑i=1 ki hi }, max{t0 ,t0 + τ j + p ∑i=1 ki hi ], since the equations (3.88) corresponding to non-negative k depend on coeffip cients with arguments delayed by τ j + ∑i=1 ki hi , which are not defined for t < t0 . Thus, the initial conditions for the matrices Pj,k1 ,...,k p (t) are stated as

Pj,k1 ,...,k p (s) = E((x(s) − m(s))× p

p

i=1

i=1

(x(s − τ j − ∑ ki hi ) − m(s − τ j − ∑ ki hi ))T | FsY ), s ∈ [max{t0 − h,t0 − h + t j,k1,...,k p }, max{t0 ,t0 + t j,k1 ,...,k p }]. Unfortunately, the system (3.87),(3.88) cannot be reduced to a finite system for any fixed filtering horizon t, as it can be done in the case of only state delay in the equations (3.81),(3.82) (see Subsection 3.2), since the infinite number of the equations (3.88) for Pj,k1 ,...,k p (t) with negative k1 , . . . , k p are always needed to compose a closed system for any time t. However, this reduction is possible for some particular cases, for example, in the case of commensurable (hi = h, i = 1, . . . , p, a0 (t) = 0, τ j = q j h, j = 0, . . . , q, q j = 0, 1, 2, . . .) delays in the equations (3.81),(3.82), which has been considered in details in Subsection 3.4.3. Remark. The convergence properties of the obtained optimal estimate (3.87) are given by the standard convergence theorem (see, for example, (see, for example, [41], Thep ai (t)Ψ (t − orem 7.5 and Section 7.7): if in the system (3.81),(3.82) the pair (∑i=0 p hi ,t), b(t)) is uniformly completely controllable and the pair (∑i=0 ai (t)Ψ (t − hi ,t), p ∑ j=0 A j (t)Ψ (t − τ j ,t)) is uniformly completely observable, where Ψ (t, τ ) is the state transition matrix for the equation (3.81) (see [56], Section 3.2, for definition of matrix Ψ ), then the error of the obtained optimal filter (3.87),(3.88) is uniformly asymptotically stable. As usual, the uniform complete controllability condition is required for assuring non-negativeness of the error variance matrix Pj,−q j (t) and may be omitted, if the matrix Pj,−q j (t) is non-negative in view of its intrinsic properties. The uniform complete controllability and observability conditions for a linear system with delay (3.81) and observations (3.82) can be found in ([56], Chapter 5).

118

3 Optimal Filtering Problems for Time-Delay Systems

3.6 Alternative Optimal Filter for Linear State Delay Systems Consider again the filtering problem for state delay system of Section 3.2, where the partially observed Ft -measurable random process (x(t), y(t)) is described by a delay differential equation for the system state dx(t) = (a0 (t) + a(t)x(t − h))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(3.89)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], and a differential equation for the observation process: dy(t) = (A0 (t) + A(t)x(t))dt + B(t)dW2(t),

(3.90)

where x(t) ∈ Rn is the state vector, y(t) ∈ Rm is the observation process, φ (s) is a mean square piecewise-continuous Gaussian stochastic process given in the interval [t0 − h,t0 ] such that φ (s), W1 (t), and W2 (t) are independent. Resume the treatment of this problem beginning from the filtering equations obtained in Section 3.2: dm(t) = (a0 (t) + a(t)m(t − h))dt+  −1 P(t)AT (t) B(t)BT (t) (dy(t) − (A0 (t) + A(t)m(t))dt). (3.91) dP(t) = (E((x(t) − m(t))xT (t − h)aT (t) | FtY ) + E(a(t)x(t − h)(x(t) − m(t))T ) | FtY )+  −1 b(t)bT (t) − P(t)AT (t) B(t)BT (t) A(t)P(t))dt. (3.92) Transform the term E(x(t)(x(t − h) − m(t − h))T | FtY ) in the last equation. Denote as x1 (t) the solution of the equation x˙1 (t) = a0 (t) + a(t)x1(t − h) with the initial condition x1 (t0 ) = x0 . Then, the solution x(t) of the equation (3.89) can be represented in the form t

x(t) = x1 (t) +

t0

b(s)dW1 (s).

(3.93)

Let us now introduce the matrix Φ (τ ,t), which would serve as a nonlinear analog of the state transition matrix in the inverse time. Indeed, define Φ (τ ,t) as a such matrix that the equality Φ (τ ,t)x1 (t) = x1 (τ ), τ ≤ t, holds for any t, τ ≥ t0 and τ ≤ t. Naturally, Φ (τ ,t) can be defined as the diagonal matrix with elements equal to x1i (τ )/x1i (t), where x1i (t) are components of the vector x1 (t), if x1i (t) = 0 almost surely. The definition of Φ (τ ,t) for the case of x1i (t) = 0 will be separately considered below. Hence, using the representation (3.93) and the notion of the matrix Φ (τ ,t), the term E(x(t)(x(t − h) − m(t − h))T | FtY ) can be transformed as follows E(x(t)(x(t − h) − m(t − h))T | FtY ) = E((x(t) − m(t))(x(t − h))T | FtY ) = E((x(t) − m(t))(x1 (t − h) +

 t−h t0

b(s)dW1 (s))T | FtY ) =

E((x(t) − m(t))(x1 (t − h))T | FtY ) = E((x(t) − m(t))(Φ (t − h,t)x1(t))T | FtY ) = E((x(t) − m(t))(x1 (t))T | FtY )(Φ ∗ (t − h,t))T =

3.6 Alternative Optimal Filter for Linear State Delay Systems

E((x(t) − m(t))(x1 (t) +

 t−h t0

119

b(s)dW1 (s))T | FtY )(Φ ∗ (t − h,t))T =

E((x(t) − m(t))(x(t))T | FtY )(Φ ∗ (t − h,t))T = P(t)(Φ ∗ (t − h,t))T ,

(3.94)

where P(t) = E((x(t) − m(t))(x(t) − m(t))T | FtY ) is the error variance and Φ ∗ (t − h,t) is the state transition matrix in the inverse time for the process x∗1 (t), that is the solution of the equation x˙∗1 (t) = a0 (t) + a(t)x∗1 (t − h) with the initial condition x∗1 (t0 ) = m0 = E(x(t0 ) | FtY0 ). Note that the transition from the third to fourth line in (3.94) is valid in view of independence of the error variance P(t) from both, x(t) and m(t), as it follows from the filtering equations (3.17),(3.20) in Section 3.2. This is the same situation that takes place in the Kalman-Bucy filter [42]. Let us now define the matrix Φ (t − h,t) in the case of x1i (t) = 0 almost surely for one of the components of x1 (t). Then, the corresponding diagonal entry of Φii (t − h,t) can be set to 0 for any h > 0, because, for the component xi (t),  t

E(xi (t)(x j (t − h) − m j (t − h)) | FtY ) = E((x1i (t) + (

t0

b(s)dW1 (s))i )×

(x j (t − h) − m j (t − h)) | FtY ) = E(x1i (t)(x j (t − h) − m j (t − h)) | FtY ) = 0, almost surely for any j = 1, . . . , m. Hence, the definition Φii (τ ,t) = 0 for any τ < t, if x1i (t) = 0, leads to the same result as in the equation (3.94) and can be employed. The diagonal element Φii∗ (τ ,t) of the matrix Φ ∗ (τ ,t) is defined accordingly and set to 0 for any τ < t, if the corresponding component of the process x∗1 (t) is equal to zero at the moment t, x∗1i (t) = 0. Thus, in view of the transformation (3.94), the equation (3.91) for the optimal estimate takes the form dm(t) = (a0 (t) + a(t)m(t − h))dt + P(t)AT (t)×

(3.95)

(B(t)BT (t))−1 (dy(t) − (A0 (t) + A(t)m(t − h))dt), with the initial condition m(t0 ) = E(x(t0 ) | FtY0 ). To compose a system of the filtering equations, the equation (3.95) should be complemented with the equation for the error variance P(t). In view of the transformation (3.94), the equation (3.92) for the error variance takes the form dP(t) = P(t)(Φ ∗ (t − h,t))T aT (t) + a(t)(Φ ∗ (t − h,t))P(t)+

(3.96)

 −1 b(t)bT (t) − P(t)AT (t) B(t)BT (t) A(t)P(t))dt. The equation (3.96) should be complemented with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. By means of the preceding derivation, the following result is proved. Theorem 3.2. The optimal finite-dimensional filter for the linear state with delay (3.89) over the linear observations (3.90) is given by the equation (3.95) for the optimal estimate m(t) = E(x(t) | FtY ) and the equation (3.96) for the estimation error variance P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ].

120

3 Optimal Filtering Problems for Time-Delay Systems

In the next section, performance of the designed optimal filter is verified against the optimal filter for linear state with delay, that was obtained in Section 3.2 in the form of a set of equations for the optimal state estimate and error covariances, whose number grows as the current filtering horizon tends to infinity. 3.6.1

Example

This section presents an example of designing the alternative optimal filter for a linear state with delay over linear observations and comparing it to the optimal filter for linear state with delay, obtained in Section 3.2. Let the unobserved state x(t) with delay be given by x(t) ˙ = x(t − 5),

x(s) = φ (s), s ∈ [−5, 0],

(3.97)

where φ (s) = N(0, 1) for s ≤ 0, and N(0, 1) is a Gaussian random variable with zero mean and unit variance. The observation process is given by y(t) = x(t) + ψ (t),

(3.98)

where ψ (t) is a white Gaussian noise, which is the weak mean square derivative of a standard Wiener process. The equations (3.97) and (3.98) present the conventional form for the equations (3.89) and (3.90), which is actually used in practice [6]. The filtering problem is to find the optimal estimate for the linear state with delay (3.97), using direct linear observations (3.98) confused with independent and identically distributed disturbances modeled as white Gaussian noises. Let us set the filtering horizon time to T = 10. The filtering equations (3.95),(3.96) take the following particular form for the system (3.97),(3.98) m(t) ˙ = m(t − 5) + P(t)[y(t) − m(t − 5)], (3.99) with the initial condition m(s) = E(φ (s)) = 0, s ∈ [−5, 0) and m(0) = E(φ (0) | y(0)) = m0 , s = 0; ˙ = 2P(t) − P2(t)(Φ ∗ (t − 5,t))2 , P(t) (3.100) with the initial condition P(0) = E((x(0) − m(0))2 | y(0)) = P0 . The auxiliary variable Φ ∗ (t − 5,t) is equal to Φ ∗ (t − 5,t) = x∗ (t − 5)/x∗(t), where x∗1 (t) is the solution of the equation x˙∗1 (t) = x∗1 (t − 5), with the initial condition x∗1 (0) = m0 . The estimates obtained upon solving the equations (3.99),(3.100) are compared to the estimates satisfying the optimal filtering equations for linear state with delay over the linear observations (see Section 3.2), which take the following particular form for the system (3.97),(3.98) m˙A (t) = mA (t − 5) + PA(t)[y(t) − mA (t)],

(3.101)

with the initial condition mA (s) = E(φ (s)) = 0, s ∈ [−5, 0) and mA (0) = E(φ (0) | y(0)) = m0 , s = 0; P˙A (t) = 2P1(t) − P2 (t), (3.102)

3.6 Alternative Optimal Filter for Linear State Delay Systems

121

with the initial condition P(0) = E((x(0) − m(0))2 | y(0)) = P0 ; and P˙1 (t) = 2PA(t − 5) + P2(t) − PA(t)PA (t − 5),

(3.103)

with the initial condition P1 (s) = E((x(s) − m(s))(x(s − 5) − m(s − 5)) | FsY ), s ∈ [0, 5]; finally, P2 (s) = E((x(s) − m(s))(x(s − 10) − m(s − 10)) | FsY ), s ∈ [5, 10). The particular forms of the equations (3.99) and (3.101) and the initial condition for x(t) imply that P1 (s) = P0 for s ∈ [0, 5] and P2 (s) = P0 for s ∈ [5, 10). Numerical simulation results are obtained solving the systems of filtering equations (3.99),(3.100) and (3.101)–(3.103). The obtained values of the estimates m(t) and mA (t) satisfying the equations (3.99) and (3.101), respectively, are compared to the real values of the state variable x(t) in (3.97). For each of the two filters (3.99),(3.100) and (3.101)–(3.103) and the reference system (3.97) involved in simulation, the following initial values are assigned: x0 = 2, m0 = 10, P0 = 100. Simulation results are obtained on the basis of a stochastic run using realizations of the Gaussian disturbances ψ (t) in (3.98) generated by the built-in MatLab white noise function. The following values of the reference state variable x(t) and the estimates m(t) and mA (t) are obtained and compared at the reference time points T = 8, 9, 10: for T = 8, x(8) = 17.5, m(8) = 17.64, mA (8) = 17.55; for T = 9, x(9) = 23.0, m(9) = 23.08, mA (9) = 23.02; for T = 10, x(10) = 29.5, m(10) = 29.5, mA (10) = 29.5. Thus, it can be concluded that the obtained alternative optimal filter (3.99),(3.100) yields insignificantly different values of the estimate in comparison to the optimal filter (3.101)–(3.103) of Section 3.2. Moreover, this small difference arises due to different realizations of the white Gaussian noise in (3.98), in both simulations, and eventually disappears. The conducted simulation provides only numerical comparison between two different forms of the optimal filter for the system (3.97),(3.98), whereas the comparison of the optimal filter to some approximate filters, such as extended Kalman filters (EKF), and the corresponding graphic representation, revealing a better performance of the optimal filter, can be found in Section 3.2. Note that the alternative optimal filter consists of only two equations, whose number and structure do not change as the filtering horizon tends to infinity. Discussion The simulation results show that the values of the estimate calculated by using the obtained alternative optimal filter for linear system with state delay are only insignificantly different from the estimate values provided by the optimal filter previously obtained in Section 3.2. Moreover, the estimates produced by both optimal filters asymptotically converge to the real values of the system state as time tends to infinity. The significant advantage of the alternative filter is that it consists of only two equations, for the optimal estimate and the estimation error variance, whose number and structure do not change as the filtering horizon tends to infinity. On the contrary, the previously obtained optimal filter of Section 3.2 includes a variable number of covariance equations, which unboundedly grows as the filtering horizon tends to infinity, and the structure of the covariance equations also varies with the number. The obtained alternative filter is free from those complications and provides the equally good quality of the state estimation.

122

3 Optimal Filtering Problems for Time-Delay Systems

3.7 Filtering Problem for Nonlinear State over Delayed Observations 3.7.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ 0, and let (W1 (t), Ft ,t ≥ 0) and (W2 (t), Ft ,t ≥ 0) be independent Wiener processes. The partially observed Ft -measurable random process (x(t), y(t)) is described by a nonlinear differential equation for the system state dx(t) = f (x,t)dt + b(t)dW1(t),

x(t0 ) = x0 ,

(3.104)

and a linear delay differential equation for the observation process: dy(t) = (A0 (t) + A(t)x(t − h) + B(t)dW2(t),

(3.105)

where x(t) ∈ Rn is the state vector, y(t) ∈ Rm is the observation process, the initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t), W2 (t) are independent. The observation process y(t) depends on delayed state x(t − h), where h is a delay shift, which assumes that collection of information on the system state for the observation purposes is possible only after a certain time lag h. The nonlinear function f (x,t) is supposed to be measurable in t and Lipshitzian in x, which ensures [29] existence and uniqueness of the solution of the equation x(t) ˙ = f (x,t). It is assumed that A(t) is a nonzero matrix and B(t)BT (t) is a positive definite matrix. All coefficients in (3.104),(3.105) are deterministic functions of time of appropriate dimensions. The estimation problem is to find the best estimate of the system state x(t) based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, which minimizes the Euclidean 2-norm ˆ | FtY ] J = E[(x(t) − x(t)) ˆ T (x(t) − x(t)) at every time moment t. Here, E[z(t) | FtY ] means the conditional expectation of a ˆ with respect to the σ - algebra FtY stochastic process z(t) = (x(t) − x(t)) ˆ T (x(t) − x(t)) generated by the observation process Y (t) in the interval [t0 ,t]. As known, this optimal estimate is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY ) of the system state x(t) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. The matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The proposed solution to this optimal filtering problem is based on the formulas for the Ito differential of the conditional expectation E(x(t) | FtY ) and its variance P(t) and given in the following section.

3.7 Filtering Problem for Nonlinear State over Delayed Observations

3.7.2

123

Optimal Filter for Nonlinear State over Delayed Observations

The optimal filtering equations could be obtained using the formula for the Ito differential of the conditional expectation m(t) = E(x(t) | FtY ) (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)) dm(t) = E( f (x,t) | FtY )dt + E(x[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY )×  −1 B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where f (x) is the nonlinear drift term in the state equation and ϕ1 (x) is the drift term in the observation equation equal to ϕ1 (x) = A0 (t) + A(t)x(t − h). Upon performing Y ) = m(t − h) for any substitution and noticing that E(x(t − h) | FtY ) = E(x(t − h) | Ft−h h > 0 (see ([41], Theorem 6.6, formula (6.100) or [70], Subsection 5.10.7)), the estimate equation takes the form dm(t) = E( f (x,t) | FtY )dt + E(x(t)[A(t)(x(t − h) − m(t − h))]T | FtY )× (B(t)BT (t))−1 (dy(t) − (A0(t) + A(t)m(t − h)) = E( f (x,t) | FtY )dt + E(x(t)(x(t − h) − m(t − h))T | FtY )AT (t)×

(3.106)

(B(t)BT (t))−1 (dy(t) − (A0 (t) + A(t)m(t − h))dt). The equation (3) should be complemented with the initial condition m(t0 ) = E(x(t0 ) | FtY0 ). Let us transform the term E(x(t)(x(t − h) − m(t − h))T | FtY ) in the last equation. Denote as x1 (t) the solution of the equation x˙1 (t) = f (x1 ,t) with the initial condition x1 (t0 ) = x0 . Then, the solution x(t) of the equation (3.104) can be represented in the form  t

x(t) = x1 (t) +

t0

b(s)dW1 (s).

(3.107)

Let us now introduce the matrix Φ (τ ,t), which would serve as a nonlinear analog of the state transition matrix in the inverse time. Indeed, define Φ (τ ,t) as a such matrix that the equality Φ (τ ,t)x1 (t) = x1 (τ ), τ ≤ t, holds for any t, τ ≥ t0 and τ ≤ t. Naturally, Φ (τ ,t) can be defined as the diagonal matrix with elements equal to x1i (τ )/x1i (t), where x1i (t) are components of the vector x1 (t), if x1i (t) = 0 almost surely. The definition of Φ (τ ,t) for the case of x1i (t) = 0 will be separately considered below. Hence, using the representation (3.107) and the notion of the matrix Φ (τ ,t), the term E(x(t)(x(t − h) − m(t − h))T | FtY ) can be transformed as follows E(x(t)(x(t − h) − m(t − h))T | FtY ) = E((x(t) − m(t))(x(t − h))T | FtY ) = E((x(t) − m(t))(x1 (t − h) +

 t−h t0

b(s)dW1 (s))T | FtY ) =

E((x(t) − m(t))(x1 (t − h))T | FtY ) = E((x(t) − m(t))(Φ (t − h,t)x1(t))T | FtY ) = E((x(t) − m(t))(x1 (t))T | FtY )(Φ ∗ (t − h,t))T =

124

3 Optimal Filtering Problems for Time-Delay Systems

E((x(t) − m(t))(x1 (t) +

 t−h t0

b(s)dW1 (s))T | FtY )(Φ ∗ (t − h,t))T =

E((x(t) − m(t))(x(t))T | FtY )(Φ ∗ (t − h,t))T = P(t)(Φ ∗ (t − h,t))T , = E((x(t) − m(t))(x(t) − m(t))T

(3.108)

Φ ∗ (t − h,t)

| FtY )

where P(t) is the error variance and is the state transition matrix in the inverse time for the process x∗1 (t), that is the solution of the equation x˙∗1 (t) = f (x∗1 ,t) with the initial condition x∗1 (t0 ) = m0 = E(x(t0 ) | FtY0 ). Thus, in view of the transformation (3.108), the equation (3.106) for the optimal estimate takes the form dm(t) = E( f (x,t) | FtY )dt + P(t)(Φ ∗ (t − h,t))T AT (t)×

(3.109)

(B(t)BT (t))−1 (dy(t) − (A0 (t) + A(t)m(t − h))dt), with the initial condition m(t0 ) = E(x(t0 ) | FtY0 ). Let us now define the matrix Φ (t − h,t) in the case of x1i (t) = 0 almost surely for one of the components of x1 (t). Then, the corresponding diagonal entry of Φii (t − h,t) can be set to 0 for any h > 0, because, for the component xi (t),  t

E(xi (t)(x j (t − h) − m j (t − h)) | FtY ) = E((x1i (t) + (

t0

b(s)dW1 (s))i )×

(x j (t − h) − m j (t − h)) | FtY ) = E(x1i (t)(x j (t − h) − m j (t − h)) | FtY ) = 0, almost surely for any j = 1, . . . , m. Hence, the definition Φii (τ ,t) = 0 for any τ < t, if x1i (t) = 0, leads to the same result as in the equation (5) and can be employed. The diagonal element Φii∗ (τ ,t) of the matrix Φ ∗ (τ ,t) is defined accordingly and set to 0 for any τ < t, if the corresponding component of the process x∗1 (t) is equal to zero at the moment t, x∗1i (t) = 0. Thus, the equation (3.109) for the optimal estimate m(t) also holds for the complete definition of the matrix Φ ∗ (τ ,t). To compose a system of the filtering equations, the equation (3.109) should be complemented with the equation for the error variance P(t). For this purpose, the formula for the Ito differential of the variance P(t) = E((x(t) − m(t))(x(t) − m(t))T | FtY ) could be used (see ([41], Theorem 6.6, formula (6.101) or [70], Subsection 5.10.9, formula (5.10.42))): dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY )+ E( f (x,t)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)−  −1 E(x(t)[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY ) B(t)BT (t) × E([ϕ1 − E(ϕ1 (x) | FtY )]xT (t) | FtY ))dt+ E((x(t) − m(t))(x(t) − m(t))[ϕ1 − E(ϕ1 (x) | FtY )]T | FtY )×  −1 B(t)BT (t) (dy(t) − E(ϕ1 (x) | FtY )dt), where the last term should be understood as a 3D tensor (under the expectation sign) convoluted with a vector, which yields a matrix. Upon substituting the expressions for ϕ1 , the last formula takes the form dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY )+

3.7 Filtering Problem for Nonlinear State over Delayed Observations

125

E( f (x,t)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)− (E(x(t)(x(t − h) − m(t − h))T | FtY )AT (t)(B(t)BT (t))−1 × A(t)E((x(t − h) − m(t − h))xT (t)) | FtY ))dt + E((x(t) − m(t))(x(t) − m(t))× (A(t)(x(t − h) − m(t − h)))T | FtY )(B(t)BT (t))−1 (dy(t) − A(t)m(t − h))dt). Using the formula (3.108) for the term E((x(t − h) − m(t − h))xT (t)) | FtY ), the last equation can be represented as dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY )+

(3.110)

E( f (x,t)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)− P(t)(Φ ∗ (t − h,t))T AT (t)(B(t)BT (t))−1 A(t)(Φ ∗ (t − h,t))P(t))dt+ E(((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY )× (Φ ∗ (t − h,t))T AT (t)(B(t)BT (t))−1 (dy(t) − A(t)m(t − h))dt). The equation (3.110) should be complemented with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. Thus, the equations (3.109) and (3.110) for the optimal estimate m(t) and the error variance P(t) form a non-closed system of the filtering equations for the nonlinear state (3.104) over linear observations with delay (3.105). In the next subsection, it will be shown that this system becomes a closed system of the filtering equations for some nonlinear functions f (x,t) in the equation (3.104), in particular, for polynomial functions, if the observation matrix A(t) in (3.105) is invertible. Optimal Filter for Polynomial State over Delayed Observations Let us make for the system (3.104),(3.105) the same assumptions that were made in Subsection 1.1.2 to obtain a closed system of the filtering equations for a polynomial state over linear non-delayed observations. Namely, assume that 1) the nonlinear function f (x,t) is a polynomial function of the state x with time-dependent coefficients (since x(t) ∈ Rn is a vector, this requires a special definition of the polynomial for n > 1 (see Section 1.1.1), and 2) the matrix A(t) in the observation equation (3.105) is invertible for any t ≥ t0 . As shown in Section 1.1.2, under these assumptions, the terms E( f (x,t) | FtY ) in (3.109) and E((x(t) − m(t))( f (x,t))T | FtY ) in (3.110) can be expressed as functions of m(t) and P(t) and, thereby, a closed system of the filtering equations can be obtained proceeding from (3.109) and (3.110). The basic property established in Section 1.1.2 claims that, if the matrix A(t) in the linear observation equation (3.105) is invertible, the random variable x(t) − m(t) is conditionally Gaussian for any t ≥ t0 . Two following conclusions can be made at this point. First, since the random variable x(t)− m(t) is conditionally Gaussian, the conditional third moment E((x(t) − m(t))(x(t) − m(t))(x(t) − m(t))T | FtY ) of x(t) − m(t) with respect to observations, which stands in the last term of the equation (3.110), is equal to

126

3 Optimal Filtering Problems for Time-Delay Systems

zero, because the process x(t) − m(t) is conditionally Gaussian. Thus, the entire last term in (3.110) is vanished and the following variance equation is obtained dP(t) = (E((x(t) − m(t))( f (x,t))T | FtY )+

(3.111)

E( f (x,t)(x(t) − m(t))T ) | FtY ) + b(t)bT (t)− P(t)(Φ ∗ (t − h,t))T AT (t)(B(t)BT (t))−1 A(t)(Φ ∗ (t − h,t))P(t))dt, with the initial condition P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. Second, if the nonlinear function f (x,t) is a polynomial function of the state x with time-dependent coefficients, the expressions of the terms E( f (x,t) | FtY ) in (3.109) and E((x(t) − m(t)) ( f (x,t))T | FtY ) in (3.111) would also include only polynomial terms of x. Then, those polynomial terms can be represented as functions of m(t) and P(t) using the following property of Gaussian random variable x(t) − m(t): all its odd conditional moments, m1 = E[(x(t) − m(t)) | Y (t)], m3 = E[(x(t) − m(t)3 | Y (t)], m5 = E[(x(t) − m(t))5 | Y (t)], ... are equal to 0, and all its even conditional moments m2 = E[(x(t)−m(t))2 | Y (t)], m4 = E[(x(t)−m(t))4 | Y (t)], .... can be represented as functions of the variance P(t). For example, m2 = P, m4 = 3P2 , m6 = 15P3 , ... etc. After representing all polynomial terms in (3.109) and (3.111), that are generated upon expressing E( f (x,t) | FtY ) and E((x(t) − m(t))( f (x,t))T | FtY ), as functions of m(t) and P(t), a closed form of the filtering equations would be obtained. The corresponding representations of E( f (x,t) | FtY ) and E((x(t) − m(t))( f (x,t))T | FtY ) have been derived in Subection 1.1.2 for certain polynomial functions f (x,t). In the next subsection, a closed form of the filtering equations will be obtained from (3.109) and (3.111) for a bilinear function f (x,t) in the equation (3.104). It should be noted, however, that application of the same procedure would result in designing a closed system of the filtering equations for any polynomial function f (x,t) in the state equation (3.104). Optimal Filter for Bilinear State over Delayed Observations Since the obtained equations (3.109),(3.111) for m(t) and P(t) do not form a closed system of filtering equations for any nonlinear drift term f (x,t) in the state equation (3.104), some particular cases of f (x,t) should be considered to obtain an optimal on-line filter operating similarly to the Kalman-Bucy one (see [42]). This subsection presents the optimal on-line filter corresponding to the case of a bilinear polynomial drift in (3.104). Indeed, if the function f (x,t) = a0 (t) + a1(t)x + a2 (t)xxT

(3.112)

is a bilinear polynomial, where x is now an n-dimensional vector, a1 is an n × n - matrix, and a2 is a 3D tensor of dimension n × n × n, the representations for E( f (x,t) | FtY ) and E((x(t) − m(t))( f (x,t))T | FtY ) as functions of m(t) and P(t) are derived as follows (see Subsection 1.1.2) (3.113) E( f (x,t) | FtY ) = a0 (t) + a1(t)m(t)+ a2 (t)m(t)mT (t) + a2(t)P(t),

3.7 Filtering Problem for Nonlinear State over Delayed Observations

E( f (x,t)(x(t) − m(t))T ) | FtY ) + E((x(t) − m(t))( f (x,t))T | FtY ) =

127

(3.114)

a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T . Substituting the expression (3.113) in (3.109) and the expression (3.114) in (3.111), the filtering equations for the optimal estimate m(t) of the bilinear state x(t) and the error variance P(t) are obtained dm(t) = (a0 (t) + a1(t)m(t) + a2(t)m(t)mT (t) + a2(t)P(t))dt+

(3.115)

P(t)(Φ ∗ (t − h,t))T AT (t)(B(t)BT (t))−1 [dy(t) − A(t)m(t)dt], m(t0 ) = E(x(t0 ) | FtY )), dP(t) = (a1 (t)P(t) + P(t)aT1 (t) + 2a2(t)m(t)P(t) + 2(a2(t)m(t)P(t))T + ∗

−1

(3.116)

∗

b(t)b (t))dt − P(t)(Φ (t − h,t)) A (t)(B(t)B (t)) A(t)(Φ (t − h,t))P(t)dt. T

T

T

T

P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | FtY )), where Φ ∗ (t − h,t) is such a matrix that Φ ∗ (t − h,t)x∗1 (t) = x∗1 (t − h), h > 0, where x∗1 (t) is the solution of the equation x˙∗1 (t) = f (x∗1 ,t) with the initial condition x∗1 (t0 ) = m0 = E(x(t0 ) | FtY0 ), if x∗1 (t) = 0, and Φ ∗ (t − h,t) = 0, if x∗1 (t) = 0. One of possible ways to define such a matrix Φ ∗ (t − h,t) by components is described in the paragraphs after formulas (3.107) and (3.109). By means of the preceding derivation, the following result is proved. Theorem 3.3. The optimal finite-dimensional filter for the bilinear state (3.104), where the quadratic polynomial f (x,t) is defined by (3.112), over the linear observations with delay (3.105) is given by the equation (3.115) for the optimal estimate m(t) = E(x(t) | FtY ) and the equation (3.116) for the estimation error variance P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ]. Thus, based on the general non-closed system of the filtering equations (3.109), (3.110) for a nonlinear state (3.104) over linear delayed observations (3.105), it is proved that the closed system of the filtering equations (3.109), (3.111) can be obtained for any polynomial state over linear delayed observations with an invertible observation matrix. Furthermore, the specific form (3.115),(3.116) of the closed system of the filtering equations corresponding to a bilinear state is obtained. In the next section, performance of the designed optimal filter is verified for a quadratic state over delayed observations against the optimal bilinear filter for linear observations without delays, obtained in Subsection 1.1.2, and the conventional extended Kalman-Bucy filter. 3.7.3

Example

This section presents an example of designing the optimal filter for a quadratic state over linear observations with delay and comparing it to the best filter available for a quadratic state without delay (see Subsection 1.1.2), and to the conventional extended Kalman-Bucy filter. Let the unobserved state x(t) satisfies the quadratic equation x(t) ˙ = 0.09x2 (t),

x(0) = 1,

(3.117)

128

3 Optimal Filtering Problems for Time-Delay Systems

and the observation process is given by the linear delay-differential equation y(t) = x(t − 0.8) + ψ (t),

(3.118)

where ψ (t) is a white Gaussian noise, which is the weak mean square derivative of a standard Wiener process. The equations (3.117) and (3.118) present the conventional form for the equations (3.104) and (3.105), which is actually used in practice [6]. The filtering problem is to find the optimal estimate for the quadratic state (3.117), using delayed linear observations (3.118) confused with independent and identically distributed disturbances modelled as white Gaussian noises. Let us set the filtering horizon time to T = 4. The filtering equations (3.109),(3.111) take the following particular form for the system (3.117),(3.118) m(t) ˙ = 0.09(m2 (t) + P(t)) + P(t)Φ ∗(t − 0.8,t)[y(t) − m(t − 0.8)],

(3.119)

with the initial condition m(0) = E(x(0) | y(0)) = m0 , ˙ = 0.36P(t)m(t) − P2(t)(Φ ∗ (t − 0.8,t))2, P(t)

(3.120)

with the initial condition P(0) = E((x(0) − m(0))2 | y(0)) = P0 . The auxiliary variable Φ ∗ (t − 0.8,t) is equal to Φ ∗ (t − 0.8,t) = x∗1 (t − 0.8)/x∗1(t) for t ≥ 0.8, and Φ ∗ (t − 0.8,t) = 0 for t < 0.8, where x∗1 (t) is the solution of the equation x˙∗1 (t) = 0.09(x∗1(t))2 , with the initial condition x∗1 (0) = m0 . The estimates obtained upon solving the equations (3.119),(3.120) are compared to the estimates satisfying the following filtering equations for the quadratic state (3.117) over the linear observations (3.118) (obtained in Subsection 1.1.2), without taking into account the gain adjustment provided by Φ ∗ (t − 0.8,t): m˙ 1 (t) = 0.09(m21(t) + P1(t)) + P1(t)[y(t) − m1 (t − 0.8)],

(3.121)

with the initial condition m1 (0) = E(x(0) | y(0)) = m0 , P˙1 (t) = 0.36P1(t)m1 (t) − P12(t),

(3.122)

with the initial condition P1 (0) = E((x(0) − m(0))2 | y(0)) = P0 . Moreover, the estimates obtained upon solving the equations (3.119),(3.120) are also compared to the estimates satisfying the following extended Kalman-Bucy filtering equations for the quadratic state (3.117) over the linear observations (3.120), obtained using the direct copy of the state dynamics (3.119) in the estimate equation and assigning the filter gain as the solution of the Riccati equation: m˙ 2 (t) = 0.09m22(t) + P2(t)[y(t) − m2(t − 0.8)],

(3.123)

with the initial condition m2 (0) = E(x(0) | y(0)) = m0 , P˙2 (t) = 0.18P2(t) − P22(t), with the initial condition P2 (0) = E((x(0) − m(0))2 | y(0)) = P0 .

(3.124)

3.7 Filtering Problem for Nonlinear State over Delayed Observations

129

Numerical simulation results are obtained solving the systems of filtering equations (3.119)–(3.120), (3.121)–(3.122), and (3.123)–(3.124). The obtained values of the estimates m(t), m1 (t), and m2 (t) satisfying the equations (3.119),(3.121), and (3.123), respectively, are compared to the real values of the state variable x(t) in (3.117). For each of the three filters (3.119)–(3.120), (3.121)–(3.122), and (3.123)–(3.124) and the reference system (3.117) involved in simulation, the following initial values are assigned: x0 = 1.1, m0 = 0.1, P0 = 1. Gaussian disturbance ψ in (3.118) is realized using the built-in MatLab white noise function. The following graphs are obtained: graph of the reference state variable x(t) for the system (3.117); graph of the optimal filter estimate m(t) satisfying the equations (3.119)–(3.120); graph of the estimate m1 (t) satisfying the equations (3.121)–(3.122); graph of the estimate m2 (t) satisfying the equations (3.123)–(3.124). The graphs of all those variables are shown on the entire simulation interval from T = 0 to T = 4 (Fig. 3.15). Note that the gain matrices P(t), and P1 (t), do not converge to zero as time tends to infinity, since the polynomial dynamics of third order is stronger than the quadratic Riccati terms in the right-hand sides of the equations (3.120) and (3.122). The following values of the reference state variable x(t) and the estimates m(t), m1 (t), m2 (t) are obtained at the reference time points T = 1, 2, 3, 4: for T = 1, x(1) = 1.221, m(1) = 0.377, m1 (1) = 0.272, m2 (1) = 0.205; for T = 2, x(2) = 1.372, m(2) = 1.1, m1 (2) = 0.744, m2 (2) = 0.611; for T = 3, x(3) = 1.566, m(3) = 1.533,

State

2 1.8 1.5

Optimal estimate

1

0

0.5

0.8

1

1.5

2 time

2.5

3

3.5

4

0

0.5

0.8

1

1.5

2 time

2.5

3

3.5

4

0

0.5

1

1.5

2 time

2.5

3

3.5

4

0

0.5

1

1.5

2 time

2.5

3

3.5

4

2 1.8 1.5 1 0.5 0

Estimate 1

1.5 1 0.5 0

Estimate 2

1.5 1 0.5 0

Fig. 3.15. Graph of the reference state variable x(t) satisfying the equation (3.117) (State); graph of the optimal filter estimate m(t) satisfying the equations (3.119),(3.120) (Optimal estimate); graph of the estimate m1 (t) satisfying the equations (3.121),(3.122) (Estimate 1); graph of the estimate m2 (t) satisfying the equations (3.123),(3.124) (Estimate 2) on the entire simulation interval [0, 4]

130

3 Optimal Filtering Problems for Time-Delay Systems

m1 (3) = 1.136, m2 (3) = 0.901; for T = 4, x(4) = 1.821, m(4) = 1.820, m1 (4) = 1.466, m2 (4) = 1.119. Thus, it can be concluded that the obtained optimal filter (3.119),(3.120) for a quadratic state over linear observations with delay yield definitely better estimates than the conventional filter for a quadratic state over linear observations without delay (3.121),(3.122) or the conventional extended Kalman-Bucy filter (3.123),(3.124). Discussion The simulation results show that the values of the estimate calculated by using the obtained optimal filter for a quadratic state over linear observations with delay are noticeably closer to the real values of the reference variable than the values of the estimates given by the best filter available for a quadratic state over linear observations without delay or the conventional extended Kalman-Bucy filter. Moreover, it can be seen that the estimate produced by the optimal filter converges to the real values of the reference variables as time tends to the asymptotic time point, although the reference quadratic system state (3.117) itself is unstable and even goes to infinity for a finite time. This significant improvement in the estimate behavior is obtained due to the more careful selection of the filter gain matrix in the equations (3.119),(3.120), as it should be in the optimal filter. Although this conclusion follows from the developed theory, the numerical simulation serves as a convincing illustration. Design of the optimal filter for uncertain stochastic systems with an unknown or even time-varying delay in the observation equation is viewed as a feasible direction of the future research.

4 Optimal Control Problems for Time-Delay Systems

4.1 Optimal Control Problem for Linear Systems with Multiple Input Delays 4.1.1

Problem Statement

Consider a linear system with multiple time delays in control input p

x(t) ˙ = a0 (t) + a(t)x(t) + ∑ Bi (t)u(t − hi ),

(4.1)

i=0

with the initial condition x(t0 ) = x0 . Here, x(t) ∈ Rn is the system state, u(t) ∈ Rm is the control variable, hi > 0, i=1, . . . , p, are positive time delays, h0 =0, h = max{h1, . . . , h p } is the maximum delay shift, and a0 (t), a(t), and Bi (t), i = 0, . . . , p, are piecewise continuous matrix functions of appropriate dimensions. Existence of the unique solution of the equation (4.1) is thus assured by the Carath´eodory theorem (see, for example, [29]). The control function u(t) regulates the system state by fusing the values of u(t) at various delayed time moments t − hi , i = 1, . . . , p, as well at the current time t, which assumes that the current system state depends not only on the current value of u(t) but also on its values after certain time lags hi , i = 1, . . . , p. This situation is frequently encountered in, for example, network control systems [36, 97]. The quadratic cost function to be minimized is defined as follows: 1 1 J = [x(T )]T ψ [x(T )] + 2 2

 T

(uT (s)R(s)u(s) + xT (s)L(s)x(s))ds,

(4.2)

t0

where R is positive and ψ , L are nonnegative definite symmetric matrices, and T > t0 is a certain time moment. The optimal control problem is to find the control u∗ (t), t ∈ [t0 , T ], that minimizes the criterion J along with the trajectory x∗ (t), t ∈ [t0 , T ], generated upon substituting u∗ (t) into the state equation (4.1). The solution to this problem is given in the next subsection. The optimality of the solution is then proved in Subsection 4.1.4. M. Basin: New Trends in Optimal Filtering, LNCIS 380, pp. 131–173, 2008. c Springer-Verlag Berlin Heidelberg 2008 springerlink.com 

132

4 Optimal Control Problems for Time-Delay Systems

4.1.2

Optimal Control Problem Solution

The solution to the optimal control problem for the linear system with input delay (4.1) and the quadratic criterion (4.2) is given as follows. The optimal control law for t ≥ t0 + h is given by p

p

p

i=0

i=0

k=0

∑ Bi (t)u∗(t − hi) = ∑ Bi (t)(R(t))−1[ ∑ BTk (t) exp (−

 t t−hk

aT (s)ds)]Q(t)x(t − hi), (4.3)

where the matrix function Q(t) satisfies the matrix Riccati equation ˙ = −aT (t)Q(t) − Q(t)a(t) + L(t)− Q(t) p

Q(t)[ ∑ exp(− i=0

 t

p

t−hi

a(s)ds)Bi (t)]R−1 (t)[ ∑ BTi (t) exp (− i=0

(4.4)

 t

aT (s)ds)]Q(t),

t−hi

with the terminal condition Q(T ) = −ψ . Substituting the optimal control (4.3) into the state equation (4.1), the optimally controlled state equation is obtained x(t) ˙ = a0 (t) + a(t)x(t)+ p

p

i=0

k=0

∑ Bi (t)(R(t))−1 [ ∑ BTk (t) exp (−

 t t−hk

aT (s)ds)]Q(t)x(t − hi ),

(4.5) x(t0 ) = x0 .

Since the delayed control terms u(t − hi ) cannot be properly defined in the entire time interval t ∈ [t0 ,t0 + h), h = max{h1, . . . , h p } (the system coefficient values are not given for t < t0 ), the optimal control law (4.3) is applied starting from t = t0 + h. The optimal LQ regulator for linear systems without delays should be applied in the interval [t0 ,t0 + h), in accordance with the dynamic programming principle [14] claiming that a part of the optimal control function in any subinterval must be the optimal control itself in this subinterval. For the system with many input delays, h1 ≤ . . . ≤ h p , the optimal control must be designed, according to the dynamic programming principle, as the optimal control for the system without input delays for t ∈ [t0 ,t0 + h1 ), with one input delay for t ∈ [t0 + h1 ,t0 + h2 ), with two input delays for t ∈ [t0 + h2 ,t0 + h3 ), etc. Note that this optimal control applicability restriction is consistent with the results of [2], where the same restriction should be imposed on the optimal control law obtained in Examples 1,2 in [2], in the case of time-varying system coefficients. 4.1.3

Example

This section presents an example of designing the optimal regulator for a system (4.1) with a criterion (4.2) using the scheme (4.3)–(4.5) and comparing it to the regulator, where the matrix Q(t) is selected as in the optimal linear regulator for a system without delays, and to some other feasible feedback regulators linear in state. Consider a scalar linear system x(t) ˙ = x(t) + u(t − 0.1) + u(t),

(4.6)

4.1 Optimal Control Problem for Linear Systems with Multiple Input Delays

133

with the initial condition x(0) = 1. The optimal control problem is to find the control u(t), t ∈ [0, T ], that minimizes the criterion J=

1 2

 T

(u2 (t) + x2 (t))dt,

(4.7)

0

where T = 0.25. In other words, the optimal control problem is to minimize the state x using the minimum energy of control u. Let us first construct the regulator where the optimal control law and the matrix Q(t) are calculated in the same manner as for the optimal linear regulator for a linear system without delays in control input, that is u∗ (t) = (R(t))−1 BT (t)Q(t)x(t) (see [49] for reference). Since B(t) = 1 in (4.6) and R(t) = 1 in (4.7), the optimal control is actually equal to u(t) = Q(t)x(t), (4.8) where Q(t) satisfies the Riccati equation ˙ = −aT (t)Q(t) − Q(t)a(t) + L(t) − Q(t)B(t)R−1(t)BT (t)Q(t), Q(t) with the terminal condition Q(T ) = −ψ . Since a(t) = 1, B(t) = 1 in (4.6), and L(t) = 1 and ψ = 0 in (4.7), the last equation turns to ˙ = 1 − 2Q(t) − (Q(t))2, Q(t)

Q(0.25) = 0.

(4.9)

Upon substituting the optimal control (4.8) into (4.6), the controlled system takes the form x(t) ˙ = x(t) + Q(t − 0.1)x(t − 0.1) + Q(t)x(t), (4.10) where the undefined values of x(t − 0.1) are set to zero for t ∈ [0, 0.1). The results of applying the regulator (4.8),(4.9) to the system (4.6) yield the following values of the state (4.10) and the criterion (4.7) at the final moment T = 0.25: x(0.25) = 1.2234 and J(0.25) = 0.1563. Let us now apply the optimal regulator (4.3)–(4.5) for linear systems with multiple time delays in control input to the system (4.6). Since a(t) = 1, B(t) = 1, B1 (t) = 1, and t h = 0.1 in (4.6) and ψ = 0, R(t) = 1, and L(t) = 1 in (4.7), hence, exp (− t−h aT (s)ds) = exp (−0.1) and the optimal control law (4.3) takes the form u∗ (t) + u∗(t − 0.1) = (1 + exp(−0.1))Q(t)[x(t) + x(t − 0.1)],

(4.11)

where Q(t) satisfies the Riccati equation ˙ = 1 − 2Q(t) − ((1 + exp(−0.1))Q(t))2 , Q(t)

Q(0.25) = 0.

(4.12)

Upon substituting the optimal control (4.11) into (4.6), the optimally controlled system takes the form x(t) ˙ = x(t) + (1 + exp(−0.1))Q(t)[x(t − 0.1) + x(t)].

(4.13)

Since the delayed control term u(t − 0.1) cannot be properly defined for t ∈ [0, 0.1) (the system coefficient values are not given for t < t0 ), the control law (4.8) is used in the

134

4 Optimal Control Problems for Time-Delay Systems

interval [0, 0.1), in accordance with the dynamic programming principle [14] claiming that a part of the optimal control function in any subinterval must be the optimal control itself in this subinterval. The optimal control (4.11) is thus applied starting from t = 0.1 in the interval [0.1, 0.25]. Thus, the obtained optimal control law is the feedback linear control u(t) = kQ(t)x(t) with k = 1 and Q(t) satisfying (4.9), for t ∈ [0, 0.1), and k = 1 + exp (−0.1) and Q(t) satisfying (4.12), for t ∈ [0.1, 0.25]. Note that if an original controlled system contains many input delays, h1 ≤ . . . ≤ h p , the optimal control must be designed, according to the dynamic programming principle, as the optimal control for the system without input delays for t ∈ [t0 ,t0 + h1 ), with one input delay for t ∈ [t0 + h1 ,t0 + h2 ), with two input delays for t ∈ [t0 + h2 ,t0 + h3 ), etc. The results of applying the optimal regulator (4.11),(4.12) to the system (4.6) yield the following values of the state (4.13) and the criterion (4.7) at the final moment T = 0.25: x(0.25) = 1.1973 and J(0.25) = 0.155605. To better understand behavior of the controlled system (4.6), the optimal regulator (4.11),(4.12) is also compared to feedback regulators in the form u(t) = kQ(t)x(t), where Q(t) satisfies (4.9) and k is set to 1, for t ∈ [0, 0.1), and Q(t) satisfies the Riccati equation ˙ = 1 − 2Q(t) − (kQ(t))2, Q(0.25) = 0, Q(t) (4.14) and k is set to a constant in accordance to the time-invariance of the system (4.6), for t ∈ [0.1, 0.25]. The values k = 1, k = 2, and k = 3 are examined. (Note that the optimal regulator (4.11),(4.12) corresponds to k = 1 + exp(−0.1) < 2.) The value k = 2 yields the final values x(0.25) = 1.195 and J(0.25) = 0.15562, which is a bit worse than those for the optimal regulator (4.11),(4.12). The value k = 1 yields the final values x(0.25) = 1.2239 and J(0.25) = 0.1563; k = 3 yields the final values x(0.25) = 1.1685 and J(0.25) = 0.1563. Thus, it can be observed that the optimal regulator (4.11),(4.12), corresponding to k = 1 + exp(−0.1) for t ∈ [0.1, 0.25], minimizes the value of the criterion (4.7) among all feedback regulators u(t) = kQ(t)x(t), where k = 1 for t ∈ [0, 0.1) and Q(t) satisfies the equation (4.14) for t ∈ [0.1, 0.25] and the equation (4.9) for t ∈ [0, 0.1). (The final state value x(0.25) naturally decreases as k increases.) In the next section, the regulator (4.11)-(4.12) is proved to satisfy the necessary optimality condition, maximum principle [67, 44], over all admissible piecewise-continuous control functions. 4.1.4

Proof of Optimal Control Problem Solution

Necessity Define the Hamiltonian function [67, 44] for the optimal control problem (4.1),(4.2) as 1 H(x, u, q,t) = (uT R(t)u + xT L(t)x) + qT [a0 (t) + a(t)x + u1(u)], 2 p

where u1 (u) = ∑ Bi (t)u(t − hi ). Applying the maximum principle condition i=0

∂ H/∂ u = 0 to this specific Hamiltonian function (4.15) yields ∂ H/∂ u = 0 ⇒ R(t)u(t) + (∂ u1(t)/∂ u)T q(t) = 0.

(4.15)

4.1 Optimal Control Problem for Linear Systems with Multiple Input Delays

135

Upon denoting (∂ u(t − hi )/∂ u(t)) = Mi (t) and observing that p

(∂ u1 (u)/∂ u) = ∑ Bi (t)Mi (t), the optimal control law is obtained as i=0

p

u∗ (t) = −R−1 (t)[ ∑ MiT (t)BTi (t)]q(t). i=0

Taking linearity and causality of the problem into account, let us seek q(t) as a linear function of x(t)1 q(t) = −Q(t)x(t), (4.16) where Q(t) is a square symmetric matrix of dimension n. This yields the complete form of the optimal control p

u∗ (t) = R−1 (t)[ ∑ MiT (t)BTi (t)]Q(t)x(t).

(4.17)

i=0

Note that the transversality condition [67, 44] for q(T ) implies that q(T ) = ∂ J/∂ x(T ) = ψ x(T ) and, therefore, Q(T ) = −ψ . Using the co-state equation dq(t)/dt = −∂ H/∂ x, which gives −dq(t)/dt = L(t)x(t) + a(t)q(t),

(4.18)

and substituting (4.16) into (4.18), we obtain ˙ Q(t)x(t) + Q(t)d(x(t))/dt = L(t)x(t) − a(t)Q(t)x(t).

(4.19)

Substituting the expression for x(t) ˙ from the state equation (4.1) into (4.19) yields p

˙ Q(t)x(t) + Q(t)a0 (t) + Q(t)a(t)x(t) + Q(t)[ ∑ Bi (t)u(t − hi )] = i=0

= L(t)x(t) − a(t)Q(t)x(t).

(4.20)

In view of linearity of the problem, differentiating the last expression in x does not imply loss of generality. Upon taking into account that (∂ u(t − hi )/∂ x(t)) = p

(∂ u(t − hi )/∂ u(t))(∂ u(t)/∂ x(t)) = Mi (t)R−1 (t)[ ∑ MiT (t)BTi (t)]Q(t) and differentiati=0

ing the equation (4.20) in x, it is transformed into the Riccati equation ˙ = L(t) − Q(t)a(t) − a(t)Q(t)− Q(t) 1

(4.21)

In some papers (see, for example, [2]), the optimal control u∗ (t) is obtained as an integral of the previous values of the state x(t) over the interval [t − h,t]. However, since the backward solution of the linear equation (4.1) exists and is unique (see [38]), any previous value x(τ ), τ ∈ [t − h,t] can be uniquely represented as a function of the current value x(t) (as well as of any delayed value x(t − r), r > 0). Thus, the optimal control u∗ (t) can be chosen as a linear function of x(t). Using the current value x(t), the transversality condition [67, 44] induced by the cost function (4.2) is readily satisfied (see the end of this paragraph). Moreover, the least retarded causal feedback control is thus obtained for the plant (4.1).

136

4 Optimal Control Problems for Time-Delay Systems p

p

i=0

i=0

Q(t)[ ∑ Bi (t)Mi (t)]R−1 (t)[ ∑ MiT (t)BTi (t)]Q(t),

Q(T ) = −ψ .

Let us find the values of matrices Mi (t) for this problem. Substituting the optimal control law (4.17) into the equation (4.1) gives p

x(t) ˙ = a0 (t) + a(t)x(t) + ∑ Bi (t)(R(t − hi))−1 ×

(4.22)

i=0

p

[ ∑ MkT (t − hi )BTk (t − hi)]Q(t − hi )x(t − hi ), k=0

By virtue of the Cauchy formula (see [49, 19]), the following relation is valid for the solution of (4.22) x(t) = Φ (t, r)x(r) +  t r

p

p

i=0

k=0

 t r

Φ (t, τ )a0 (τ )d τ +

(4.23)

Φ (t, τ ) ∑ Bi (τ )(R(τ − hi ))−1 [ ∑ MkT (τ − hi )BTk (τ − hi )]Q(τ − hi )x(τ − hi )d τ ,

where t, r ≥ t0 and Φ (t, τ ) is the matrix of fundamental solutions of the homogeneous equation (4.1), that is the solution of the matrix equation d(Φ (t, τ ))/dt = a(t)Φ (t, τ ),

Φ (t,t) = I, 

t where I is the identity matrix. In other words, Φ (t − hi ,t) = exp (− t−h a(s)ds). i Since the integral terms in the right-hand side of (4.23) do not explicitly depend on u(t), ∂ x(t)/∂ u(t) = Φ (t, r)∂ x(r)/∂ u(t). Inverting the last equality implies that ∂ u(t)/∂ x(t) = (∂ u(t)/∂ x(r))Φ (r,t). Hence, the equality Tu(t) = K1 Φ (t, r)K2 x(r) holds, where T ∈ Rn×m and K1 , K2 ∈ Rn×n can be selected the same for any t, r ≥ t0 . Note that matrices T , K1 , and K2 serve as dimension-matching connectors between vectors x ∈ Rn and u ∈ Rm . Writing the last equality for x(t + hi ), hi > 0, yields Tu(t + hi ) = K1 Φ (t + hi , r)K2 x(r). Thus, (∂ (Tu(t))/∂ (Tu(t + hi )) = Φ (t, r)(Φ (t + hi , r))−1 = Φ (t,t + hi ), which leads to (∂ (Tu(t))/∂ u(t + hi ) = Φ (t,t + hi )T . Setting now T = Bi (t) and using t − hi instead of t yields Bi (t)(∂ u(t − hi )/∂ u(t)) = t Bi (t)Mi (t) = Φ (t − hi ,t)Bi (t) = exp (− t−h a(s)ds)Bi (t) for t ≥ t0 + hi . Upon substituti t ing Bi (t)Mi (t) = exp (− t−hi a(s)ds)Bi (t) into (4.21), the equation (4.21) for Q(t) takes the form (4.4). Let us finally show that the equality (4.17) can be transformed into the form (4.3) and the equation (4.22) into the form (4.5). Indeed, taking into account ∂ (Tu(t))/∂ (Tu(t + hi )) = ∂ x(t)/∂ x(t + hi ) = Φ (t,t + hi ) (the last equality directly follows from (4.23)) implies that the equality ∂ u(t)/∂ x(t) = ∂ u(t + hi )/∂ x(t + hi ) holds p

for t ≥ t0 . Thus, the expressions R−1 (t − hi )[ ∑ MkT (t − hi )BTk (t − hi )]Q(t − hi ) in the k=0

p

formulas (4.17) for u(t − hi ) can be replaced by R−1 (t)[ ∑ MiT (t)BTi (t)]Q(t) for any i=0

i = 1, . . . , p and t ≥ t0 + hi , thus yielding the control law (4.3). Substituting the formula

4.1 Optimal Control Problem for Linear Systems with Multiple Input Delays

137

(4.3) into (4.1), the equation (4.5) is obtained. The necessity part of the optimal control problem solution is proved. Remark. For linear systems with multiple time delays in state and control, the uniqueness of the optimal control and the sufficiency of the necessary optimality condition, i.e., the optimality of the solution obtained from the maximum principle, was proved in the works of Kharatashvili in 1960s (see Section 27 in [67] and [44]). Thus, the solution found in the preceding subsection gives us the solution of the optimal control problem stated in Subsection 4.1.1. The sufficiency of the maximum principle also follows from the satisfaction of the Hamilton-Jacobi-Bellman (HJB) equation, which is proved in the next subsection. Satisfaction of Hamilton-Jacobi-Bellman Equation Let us additionally show that the obtained solution also solves the HJB equation, thus assuring the time-consistency and robustness properties, as well as sufficiency of the necessary optimality condition. Indeed, for the considered problem (4.1),(4.2), the HJB equation takes the form

∂ V (x,t)/∂ t + min H1 (x, u, ∂ V (x,t)/∂ x,t) = 0, u

1 V (x, T ) = [x(T )]T ψ [x(T )], (4.24) 2

where 1 V (x,t) = min( [x(T )]T ψ [x(T )]+ u 2 1 2

 T

(4.25)

(uT (s)R(s)u(s) + xT (s)L(s)x(s))ds)

t

is the Bellman function [14], and H1 (x, u, q,t) = qT [a0 (t) + a(t)x + u1(u)]. Substituting (4.25) into (4.24) yields 1 min(− (uT (t)R(t)u(t) + xT (t)L(t)x(t))+ u 2 (∂ V (x,t)/∂ x)T [a0 (t) + a(t)x + u1(u)])) = 0,

(4.26)

1 V (x, T ) = [x(T )]T ψ [x(T )]. 2

Seeking the Bellman function in the form V (x,t) = − 12 xT Qx reduces the problem of determining the control minimizing the left-hand side of (4.26) to the problem of finding the control maximizing (4.15) with q(t) in the form (4.16). The solution is then given by the optimal control (4.17) and the equation (4.21) for Q(t). The terminal condition for (4.21), Q(T ) = −ψ , also satisfies the terminal condition for the HJB equation (4.24). The equality of the minimized left-hand side of (4.26) to zero is assured by the fact that, for time-varying systems, the sum  of the maximized value of the Hamiltonian (4.15) and ∗T ) = ∂ ( q∗T dx∗ ) is also equal to zero (see [67, 44, 32]): ˙ the term 12 (x∗T Q(t)x ∂t 1 H(x∗ , u∗ , q∗ ,t) = (u∗T R(t)u∗ + x∗T L(t)x∗ + d(x∗T Q(t)x∗ )/dt) = 0, 2

(4.27)

138

4 Optimal Control Problems for Time-Delay Systems

where u∗ is the maximizing value of control and x∗ , q∗ are the corresponding values of state and co-state. Integrating (4.27) from t to T yields   T 1 1 T T T [u (s)R(s)u(s) + x (s)L(s)x(s)]ds + x (T )Q(T )x(T ) = − max u 2 t 2 1 = − x∗T (t)Q(t)x∗ (t) = V (x∗ ,t), 2 which demonstrates that the selected form of the Bellman function, V (x,t) = − 12 xT Qx, is equal to the required expression (4.25) over the arguments maximizing (4.15). Hence, the control law u∗ (t) (4.17) and the corresponding state trajectory x∗ (t) satisfy the HJB equation (4.24) with the Bellman function V (x,t) = − 12 xT Qx, where Q(t) satisfies the equation (4.21) with the terminal condition Q(T ) = −ψ . Thus, the solution obtained from the maximum principle in the necessity part also has the time-consistency property and is robust with respect to parameter variations. 4.1.5

Duality between Filtering and Control Problems for Time-Delay Systems

This section establishes duality between the solutions of the optimal filtering problem for linear systems over observations with multiple time delays and the optimal control problem for linear systems with multiple time delays in control input and a quadratic criterion. For this purpose, the optimal filtering equations for a linear state equation over linear observations with multiple time delays (obtained in Section 3.1) are briefly reviewed in the next subsection. Then, both results are compared and discussed. Optimal Filter for Linear System States over Observations with Multiple Delays Let the unobservable random process x(t) be described by an ordinary differential equation for the dynamic system state dx(t) = (a0 (t) + a(t)x(t))dt + b(t)dW1(t),

x(t0 ) = x0 ,

(4.28)

and a delay-differential equation be given for the observation process: p

dy(t) = (A0 (t) + ∑ Ai (t)x(t − hi ))dt + F(t)dW2 (t),

(4.29)

i=0

where x(t) ∈ Rn is the state vector, y(t) ∈ Rm is the observation process, hi > 0, i = 1, . . . , p, are positive time delays, h0 = 0, the initial condition x0 ∈ Rn is a Gaussian vector such that x0 , W1 (t), W2 (t) are independent. The observation process y(t) depends on delayed states x(t − hi ), i = 1, . . . , p, as well as non-delayed state x(t), which assumes that collection of information on the system state for the observation purposes is made not only at the current time but also after certain time lags hi > 0, i = 1, . . . , p. The vector-valued function a0 (s) describes the effect of system inputs (controls and disturbances). It is assumed that at least one of Ai (t), i = 0, . . . , p, is a nonzero matrix and F(t)F T (t) is a positive definite matrix. All coefficients in (4.28),(4.29) are deterministic piecewise-continuous matrix functions of appropriate dimensions.

4.1 Optimal Control Problem for Linear Systems with Multiple Input Delays

139

The estimation problem is to find the estimate of the system state x(t) based on the observation process Y (t) = {y(s), 0 ≤ s ≤ t}, which minimizes the Euclidean 2-norm J = E[(z(t) − zˆ(t))T (z(t) − zˆ(t)) | FtY ] at every time moment t. Here, E[ξ (t) | FtY ] means the conditional expectation of a stochastic process ξ (t) = (z(t) − zˆ(t))T (z(t) − zˆ(t)) with respect to the σ - algebra FtY generated by the observation process Y (t) in the interval [t0 ,t]. As known, this optimal estimate is given by the conditional expectation x(t) ˆ = m(t) = E(x(t) | FtY ) of the system state x(t) with respect to the observations Y (t). As usual, the matrix function P(t) = E[(x(t) − m(t))(x(t) − m(t))T | FtY ] is the estimation error variance. The solution to the stated problem is given by the following system of filtering equations, which is closed with respect to the introduced variables, m(t) and P(t): p

dm(t) = (a0 (t) + a(t)m(t))dt + P(t)[ ∑ exp (− i=0

 t t−hi

aT (s)ds)ATi (t)]×

(4.30)

p  −1 F(t)F T (t) (dy(t) − (A0 (t) + ∑ Ai (t)m(t − hi))dt). i=0

T

dP(t) = (P(t)a (t) + a(t)P(t) + b(t)bT (t)− p

P(t)[ ∑ exp (− i=0

 t t−hi

(4.31)

 −1 aT (s)ds)ATi (t)] F(t)F T (t) ×

p

[ ∑ Ai (t) exp (− i=0

 t

a(s)ds)]P(t))dt. t−hi

The system of filtering equations (4.30) and (4.31) should be complemented with the initial conditions m(t0 ) = E[x(t0 ) | FtY0 ] and P(t0 ) = E[(x(t0 ) − m(t0 )(x(t0 ) − m(t0 )T | FtY0 ]. This system is similar to the conventional Kalman-Bucy filter, except the adjustments for delays in the estimate and variance equations, calculated due to the Cauchy formula for the linear state equation. In the case of a constant matrix the optimal filter takes the  t aTin the state equation, a ds) = exp(−aT h)) especially simple form (exp(− t−h p

dm(t) = (a0 (t) + am(t))dt + P(t)[ ∑ exp (−aT hi )ATi (t)]×

(4.32)

i=0

p  −1 F(t)F T (t) (dy(t) − (A0 (t) + ∑ Ai (t)m(t − hi))dt), i=0

T

dP(t) = (P(t)a + aP(t) + b(t)bT (t)−

(4.33)

140

4 Optimal Control Problems for Time-Delay Systems p  −1 p P(t)[ ∑ exp (−aT hi )ATi (t)] F(t)F T (t) [ ∑ Ai (t) exp (−ahi )]P(t))dt. i=0

i=0

Thus, the equation (4.30) (or (4.32)) for the optimal estimate m(t) and the equation (4.31) (or (4.33)) for its covariance matrix P(t) form a closed system of filtering equations in the case of a linear state equation and linear observations with multiple time delays. Discussion It can be readily seen that there exists the duality between the solution (4.3)–(4.5) of the optimal control problem and the solution (4.30),(4.31) of the optimal filtering one. Indeed, the optimal filter gain matrix in (4.30) p

K f = P(t)[ ∑ exp(− i=0

 t t−hi

aT (s)ds)ATi (t)](F(t)F T (t))−1 ,

is the dual transpose to the optimal regulator gain matrix in (4.5) p

Kc = (R(t)) [ ∑ −1

i=0

BTi (t) exp (−

 t

aT (s)ds)]Q(t),

t−hi

and the optimal filter gain matrix equation (4.31) is precisely dual to the optimal regulator gain matrix equation (4.4). The pairs of dual functions are (a, −aT ), (B, AT ), (Bi , ATi ), (R, FF T ), (L, bbT ). The duality also changes the initial conditions in the filtering equations to the terminal conditions in the regulator ones and inverts the integration direction in the exponent integrals. This is quite consistent with the dual function pairs indicated for linear systems in ([19], Section 5.4, Theorem 5.10) and with the time inversion established between the optimal control and filtering problems for linear systems in ([49], Subsection 4.4.2, Theorem 4.8). Thus, it can be concluded that the duality principle is valid for linear systems with multiple time delays in input and observations. It should also be noted that using the duality between control and filtering problems for linear systems with multiple time delays allows one to know a ”candidate” to the solution before applying rigorous optimality tools, such as the maximum principle and dynamic programming. This knowledge much facilitates, of course, the following substantiation of the ”candidate” result. Let us finally comment that the duality principle is based on the fact the an optimal filtering problem with additive Gaussian noises and the minimum error variance criterion can always be represented as the optimal regulator problem with a quadratic criterion in the inverse time, even for nonlinear systems, not only for linear ones. This gives one quite a transparent hypothesis that the duality principle should also be valid for nonlinear systems, as well as for linear ones. To prove it, one should start working with systems with certain nonlinear dynamics but linear input and observation functions and then gradually extending validity of the duality principle for a broader class of nonlinear systems.

4.2 Optimal Control Problem for Linear Systems with Equal State and Input Delays

141

4.2 Optimal Control Problem for Linear Systems with Equal State and Input Delays 4.2.1

Problem Statement

Consider a linear system with time delay in the state x(t) ˙ = a0 (t) + a(t)x(t − h) + B(t)u(t − h),

(4.34)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], where x(·) ∈ C([t0 − h,t0 ]; Rn ) is the system state, u(t) ∈ Rm is the control input variable, coefficients a0 (t), a(t), and B(t) are continuous functions of time, and φ (s) is a continuous function given in the interval [t0 − h,t0 ]. Note that the state of a delayed system (4.34) is infinite-dimensional (see [56], Section 2.3). Under the made assumptions, existence of the unique solution of the unforced equation (4.34) with zero input is assured by the results given in [56] (see Theorem 3.2.1 on p. 62 therein and the preceding comments). The quadratic cost function to be minimized is defined as follows: 1 1 J = [x(T1 )]T ψ [x(T1 )] + 2 2

 T1

uT (s)R(s)u(s)ds +

t0

1 2

 T1

xT (s)L(s)x(s)ds,

(4.35)

t0

where R(s) and L(s) are positive and nonnegative definite symmetric continuous matrix functions, ψ is a nonnegative definite symmetric matrix, and T1 > t0 + h is the terminal time (control horizon). The optimal control problem is to find the control u∗ (t), t ∈ [t0 , T1 ], that minimizes the criterion J along with the trajectory x∗ (t), t ∈ [t0 , T1 ], generated upon substituting u∗ (t) into the state equation (4.34). 4.2.2

Optimal Control Problem Solution

The solution to the optimal control problem for the linear system with state delay (4.34) and the quadratic criterion (4.35) is given as follows. The optimal control law is given by (4.36) u∗ (t) = (R(t))−1 BT (t)Q(t)x(t), where the matrix function Q(t) satisfies the matrix equation ˙ = L(t) − Q(t)M1 (t)a(t) − aT (t)M1T (t)Q(t) − Q(t)B(t)R−1(t)BT (t)Q(t), (4.37) Q(t) with the terminal condition Q(T1 ) = −ψ . The auxiliary matrix M1 (t) is defined as M1 (t) = (∂ x(t − h)/∂ x(t)), whose value is equal to zero, M1 (t) = 0, if t ∈ [t0 ,t0 + h), and is equal to I, M1 (t) = I, where I is the identity matrix of dimension n, if t ≥ t0 + h. Upon substituting the optimal control (4.36) into the state equation (4.34), the optimally controlled state equation is obtained x(t) ˙ = a0 (t) + a(t)x(t − h) + B(t)R−1(t − h)BT (t − h)Q(t − h)x(t − h),

(4.38)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ]. Note that the system (4.34) is not controlled in the interval [t0 ,t0 + h).

142

4 Optimal Control Problems for Time-Delay Systems

It should also be noted that the obtained optimal regulator makes an advance with respect to general optimality results for time delay systems (such as given in [2, 56, 46, 47]), since (a) the optimal control law is given explicitly and not as a solution of a system of integro-differential or PDE equations, and (b) the quasi-Riccati equation for the gain matrix does not contain any time advanced arguments and does not depend on the state variables and, therefore, leads to a conventional two points boundary-valued problem generated in the optimal control problems with quadratic criterion and finite horizon (see, for example, [49]). Thus, the obtained optimal regulator is realizable using a delay-differential equation for the state and an ordinary differential one for the gain matrix. Remark. Existence of the unique solution to the forward-time Cauchy problem for the linear state delay equation (4.38) readily follows from the results given in [56] (see Theorem 3.2.1 on p. 62 therein and the preceding comments), in view of continuity of the initial condition and coefficients of the equation (4.34) and continuity of the weight matrix functions in the criterion (4.35). Thus, the obtained optimal regulator (4.36)–(4.38) can be equally applied to unstable as well as stable plants (4.34), since the forward-time Cauchy problem for linear state delay systems (4.38) always has a unique solution in the whole interval [t0 − h, ∞), which does not diverge to infinity for a finite time. 4.2.3

Example

This section presents an example of designing the optimal regulator for a system (4.34) with a criterion (4.35), using the scheme (4.36)–(4.38), and comparing it to the regulator where the matrix Q(t) is selected as in the optimal linear regulator for a system without delays. Let us start with a scalar linear system x(t) ˙ = 10x(t − 0.25) + u(t − 0.25),

(4.39)

with the initial conditions x(s) = 1 for s ∈ [−0.25, 0]. The control problem is to find the control u(t), t ∈ [0, T1 ], T1 = 0.5, that minimizes the criterion  T1

1 J= [ 2

u2 (t)dt +

0

 T1

x2 (t)dt].

(4.40)

0

In other words, the control problem is to minimize the overall energy of the state x using the minimal overall energy of control u. Let us first construct the regulator where the control law and the matrix Q(t) are calculated in the same manner as for the optimal linear regulator for a linear system without delays, that is u(t) = R−1 (t)BT (t)Q(t)x(t) (see [49] for reference). Since B(t) = 1 in (4.39) and R(t) = 1 in (4.40), the optimal control is actually equal to u(t) = Q(t)x(t), where Q(t) satisfies the Riccati equation ˙ = −aT (t)Q(t) − Q(t)a(t) + L(t) − Q(t)B(t)R−1(t)BT (t)Q(t), Q(t)

(4.41)

4.2 Optimal Control Problem for Linear Systems with Equal State and Input Delays

143

with the terminal condition Q(T1 ) = −ψ . Since a(t) = 10 in (4.39) and L(t) = 1 and ψ = 0 in (4.40), the last equation turns to ˙ = 1 − 20Q(t) − Q2(t), Q(t)

Q(0.5) = 0,

(4.42)

whose solution can be found analytically and is equal to √ √ √ √ Q(t) = [1 − exp( 404(0.5 − t))][10 + 101 − exp( 404(0.5 − t))(10 − 101)]−1 . Upon substituting the control (4.41) into (4.39), the controlled system takes the form x(t) ˙ = 10x(t − 0.25) + Q(t − 0.25)x(t − 0.25).

(4.43)

The results of applying the regulator (4.41)–(4.43) to the system (4.39) are shown in Fig. 4.1, which presents the graphs of the criterion (4.40) J(t) and the control (4.41) u(t) in the interval [0, T1 ]. The value of criterion (4.40) at the final moment T1 = 0.5 is J(0.5) = 140.32. Let us now apply the optimal regulator (4.36)–(4.38) for linear states with time delay to the system (4.39). The control law (4.36) takes the same form as (4.41) u∗ (t) = Q∗ (t)x(t),

(4.44)

where Q∗ (t) satisfies the equation Q˙ ∗ (t) = 1 − 20Q∗(t)M1 (t) − Q∗2 (t),

Q∗ (0.5) = 0,

(4.45)

where M1 (t) = 0 for t ∈ [0, 0.25) and M1 (t) = 1 for t ∈ [0.25, 0.5]. Since the solution Q∗ (t) of the equation (4.45) is not smooth, it has been numerically solved with the approximating terminal condition Q∗ (0.5) = 0.04, in order to avoid chattering. Upon substituting the control (4.44) into (4.39), the optimally controlled system takes the same form as (4.43) x(t) ˙ = 10x(t − 0.25) + Q∗(t − 0.25)x(t − 0.25).

(4.46)

The results of applying the regulator (4.44)–(4.46) to the system (4.39) are shown in Fig. 4.2, which presents the graphs of the criterion (4.40) J(t) and the control (4.44) u∗ (t) in the interval [0, T1 ]. The value of the criterion (4.40) at the final moment T1 = 0.5 is J(0.5) = 6.218. There is a significant (about twenty times) improvement in the values of the cost function in comparison to the preceding case, due to the optimality of the regulator (4.36)–(4.38) for a linear system with equal delays in state and input. Finally, let us compare the optimal regulator (4.36)-(4.38) to the best linear regulator based on a rational approximation of the time-delay system (4.39). The input-state transfer function of (4.39), G(s) = (s exp (sh) − 10)−1, h = 0.25, is approximated by a rational function up to the first order of h: G−1 (s) = s + s2 h − 10 + O(h2 ). Note that since the third derivative of the solution of the equation (4.46) is virtually a constant equal to zero, the omission of more than quadratic powers of s should not introduce a considerable error into the dynamics. This approximation results in the time-domain realization (h = 0.25) 0.25(d 2(x(t))/dt 2 ) + d(x(t))/dt = 10x(t) + u(t),

144

4 Optimal Control Problems for Time-Delay Systems

which is represented in the standard form of two first-order differential equations x(t) ˙ = z(t),

z˙(t) = 40x(t) − 4z(t) + 4u(t),

(4.47)

with the initial conditions x(0) = 1, z(0) = 0.25. The control law is calculated as the optimal control for the linear system without delays (4.47): u1 (t) = BT1 Q1 (t)[x(t) z(t)]T ,

(4.48)

and the two-dimensional matrix Q1 (t) satisfies the Riccati equation Q˙ 1 (t) = −aT1 Q1 (t) − Q1(t)a1 + L1 − Q1 (t)B1 BT1 Q1 (t),

(4.49)

with the terminal condition Q1 (T1 ) = 0, where B1 = [0 4]T , a1 = [0 1 | 40 − 4] and L1 = diag[1 0]. The control (4.48) is then substituted into the original time-delay system (4.39). The results of applying the regulator (4.47)–(4.49) to the system (4.39) are shown in Fig. 4.3, which presents the graphs of the criterion (4.40) J(t) and the control (4.48) u1 (t) in the interval [0, T1 ]. The value of the criterion (4.40) at the final moment T1 = 0.5 is J(0.5) = 10.296. Thus, the simulation results show that application of the regulator (4.47)–(4.49), based on a rational approximation, leads to an increase in the problem dimension, producing additional computational difficulties, and yields still unsatisfactory values of the cost function in comparison to the optimal regulator (4.36)–(4.38).

criterion

150

100

50

0

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

10

control

0 −10 −20 −30 −40

Fig. 4.1. Best linear regulator available for linear systems without delays. Graphs of the criterion (4.40) J(t) and the control (4.41) u(t) in the interval [0, 0.5].

4.2 Optimal Control Problem for Linear Systems with Equal State and Input Delays

145

7 6

criterion

5 4 3 2 1 0

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

1

control

0 −1 −2 −3 −4

Fig. 4.2. Optimal regulator obtained for linear systems with equal delays in state and input. Graphs of the criterion (4.40) J(t) and the optimal control (4.44) u∗ (t) in the interval [0, 0.5]. 12 10

criterion

8 6 4 2 0

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.3 time

0.35

0.4

0.45

0.5

time

0.5 0

control

−0.5 −1 −1.5 −2 −2.5

0

0.05

0.1

0.15

0.2

0.25

Fig. 4.3. Regulator based on a rational approximation of the original time-delay system. Graphs of the criterion (4.40) J(t) and the control (4.48) u1 (t) in the interval [0, 0.5].

146

4.2.4

4 Optimal Control Problems for Time-Delay Systems

Proof of Optimal Control Problem Solution

Define the Hamiltonian function [67, 44] for the optimal control problem (4.34), (4.35) as 1 H(x, u, q,t) = (uT R(t)u + xT L(t)x) + qT [a0 (t) + a(t)x1 + B(t)u1], 2

(4.50)

where x1 (t) = x(t − h) and u1 (t) = u(t − h). Applying the maximum principle condition ∂ H/∂ u = 0 to this specific Hamiltonian function (4.50) yields

∂ H/∂ u = 0 ⇒ R(t)u(t) + (∂ u1(t)/∂ u)T BT (t)q(t) = 0. Upon denoting (∂ u1 (t)/∂ u) = M(t), the optimal control law is obtained as u∗ (t) = −R−1 (t)M T (t)BT (t)q(t). Taking linearity and causality of the problem into account, the co-state q(t) is sought as a linear function in x(t) to readily satisfy the transversality condition [67, 44] induced by the cost function (4.35): q(t) = −Q(t)x(t), (4.51) where Q(t) is a square symmetric matrix of dimension n. This yields the complete form of the optimal control u∗ (t) = R−1 (t)M T (t)BT (t)Q(t)x(t),

(4.52)

and the transversality condition for q(T1 ) implies that q(T1 ) = ∂ J/∂ x(T1 ) = ψ x(T1 ), therefore, Q(T1 ) = −ψ . Using the co-state equation dq(t)/dt = −∂ H/∂ x and denoting (∂ x1 (t)/∂ x) = M1 (t) yields (4.53) −dq(t)/dt = L(t)x(t) + aT (t)M1T (t)q(t), and substituting (4.51) into (4.53), we obtain ˙ Q(t)x(t) + Q(t)d(x(t))/dt = L(t)x(t) − aT (t)M1T (t)Q(t)x(t).

(4.54)

Substituting the expression for x(t) ˙ from the state equation (4.34) into (4.54) yields ˙ Q(t)x(t) + Q(t)a0 (t) + Q(t)a(t)x(t − h) + Q(t)B(t)u(t − h) = = L(t)x(t) − aT (t)M1T (t)Q(t)x(t).

(4.55)

In view of linearity of the problem, differentiating the last expression in x does not imply loss of generality. Upon substituting the optimal control law (4.52) into (4.55), taking into account that (∂ x(t − h)/∂ x(t)) = M1 (t) and (∂ u(t − h)/∂ u) = M(t), and differentiating the equation (4.55) in x, it is transformed into the quasi-Riccati equation ˙ = L(t) − Q(t)M1 (t)a(t) − aT (t)M1T (t)Q(t)− Q(t) Q(t)B(t)M(t)R−1 (t)M T (t)BT (t)Q(t), with the terminal condition Q(T1 ) = −ψ .

(4.56)

4.2 Optimal Control Problem for Linear Systems with Equal State and Input Delays

147

Let us obtain the value of M1 (t). By definition, M1 (t) = (∂ x(t − h)/∂ x(t)). Substituting the optimal control law (4.52) into the equation (4.34) gives x(t) ˙ = a0 (t) + a(t)x(t − h) + B(t)R−1(t − h)BT (t − h)Q(t − h)x(t − h),

(4.57)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ]. Integrating (4.57) yields x(t0 + h) = x(t0 ) +  t0 +h t0

 t0 +h t0

(a0 (s) + a(s)x(s − h))ds+

B(s)R−1 (s − h)BT (s − h)Q(s − h)x(s − h)ds.

(4.58)

Analysis of the formula (4.58) shows that x(t) does not depend on x(t − h), if t ∈ [t0 ,t0 + h). Therefore, M1 (t) = 0 for t ∈ [t0 ,t0 + h). On the other hand, if t ≥ t0 , the following formula is valid for the solution x(t) of the equation (4.58)  t+h

x(t + h) = x(t) + t

 t+h t

(a0 (s) + a(s)x(s − h))ds+

B(s)R−1 (s − h)BT (s − h)Q(s − h)x(s − h)ds.

(4.59)

Analysis of the formula (4.59) shows that x(t + h) depends on x(t) only via the nonintegral term. Therefore, (∂ x(t + h)/∂ x(t)) = I, if t ≥ t0 , and, consequently, M1 (t) = (∂ x(t − h)/∂ x(t)) = I, where I is the identity matrix of dimension n, if t ≥ t0 + h. Let us now find the value of matrix M(t) for this problem. Note that integrating (4.57) also yields the formula x(t + h) = x(τ ) +  t+h τ

 t+h τ

(a0 (s) + a(s)x(s − h))ds+

B(s)R−1 (s − h)BT (s − h)Q(s − h)x(s − h)ds.

(4.60)

where t ≥ t0 and τ is taken in the interval [t,t + h]. Since the integral terms do not depend on u(t), ∂ x(t + h)/∂ u(t) = ∂ x(τ )/∂ u(t). Inverting the last equality implies that ∂ u(t)/∂ x(t + h) = ∂ u(t)/∂ x(τ ) for any τ ∈ [t,t + h]. Hence, u(t) = K1 (t)x(τ ), where K1 (t) is the same for any τ ∈ [t,t + h], therefore, K1 (t) = K1 = const in the interval [t,t + h]. Analogously, since the integral terms in (4.60) do not depend on u(t + h) as well, u(t +h) = K2 x(τ ), where K2 (t) = K2 = const. Thus, ∂ u(t)/∂ u(t +h) = K1 (K2 )−1 = K = const for any t ≥ t0 . Since ∂ u(t)/∂ u(t) = I, K = I and M(t) = (∂ u(t − h)/∂ u(t)) = I for t ≥ t0 + h. In addition, analysis of the formula (4.58) implies that ∂ x(t0 + r)/∂ u(t) = ∂ x(t0 + r)/∂ u(t + h) for any t ∈ [t0 − h,t0 ] and any r ∈ [0, h], yielding again M(t) = I for t ∈ [t0 ,t0 + h]. Thus, the equation (4.56) for Q(t) finally takes the form (4.37). The optimal control problem solution is proved.

148

4 Optimal Control Problems for Time-Delay Systems

4.3 Optimal Control Problem for Linear Systems with Multiple State Delays 4.3.1

Problem Statement

Consider a linear system with multiple time delays in the state p

x(t) ˙ = ∑ ai (t)x(t − hi) + B(t)u(t),

(4.61)

i=0

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ], h0 = 0, h = max (h1 , . . . , h p ), where x(·) ∈ C([t0 − h,t0 ]; Rn ) is the system state, u(t) ∈ Rm is the control variable, and φ (s) is a continuous function given in the interval [t0 − h,t0]. The coefficients ai (t) and B(t) are considered continuous functions of time. Note that the state of a delayed system (4.61) is infinite-dimensional (see [56], Section 2.3). Existence of the unique forward solution of the equation (4.61) is thus assured by the Carath´eodory theorem (see, for example, [29]), and existence of the unique backward solution follows from analyticity of the right-hand side functional with respect to the system state (see [38, 72]) The quadratic cost function to be minimized is defined as follows: 1 J = [x(T )]T ψ [x(T )]+ 2 1 2

 T

(uT (s)R(s)u(s)ds + xT (s)L(s)x(s))ds,

(4.62)

t0

where R is positive and ψ , L are nonnegative definite continuous symmetric matrix functions, and T > t0 is a certain time moment. The optimal control problem is to find the control u∗ (t), t ∈ [t0 , T ], that minimizes the criterion J along with the trajectory x∗ (t), t ∈ [t0 , T ], generated upon substituting u∗ (t) into the state equation (4.61). 4.3.2

Optimal Control Problem Solution

The solution to the optimal control problem for the linear system with multiple state delays (4.61) and the quadratic criterion (4.62) is given as follows. The optimal control law takes the form u∗ (t) = (R(t))−1 BT (t)Q(t)x(t), (4.63) where the matrix function Q(t) satisfies the matrix Riccati equation with time-varying and state-dependent coefficients p

p

i=0

i=0

˙ = L(t) − Q(t)( ∑ ai (t)Ni (t)) − ( ∑ NiT (t)aTi (t))Q(t)− Q(t) Q(t)B(t)R−1 (t)BT (t)Q(t), with the terminal condition Q(T ) = −ψ .

(4.64)

4.3 Optimal Control Problem for Linear Systems with Multiple State Delays

149

Each of the auxiliary matrices Ni (t), i = 0, . . . , p, is defined as a solution of the algebraic equation Ni (t)x(t) = x(t − hi ), which can always be satisfied, if x(t) is not the vector of zeros. On the other hand, if the equality x(t) = 0 is satisfied only in a set of isolated time points t = τk , k = 1, 2, . . ., where the value of the optimal control (4.63) is also equal to zero, then the values of Q(τk ) can be assigned as the limits of the values of Q(t) from the right, thus assuring the continuity of Q(t) as the solution of (4.64). If the equality x(t) = 0 is satisfied in a continuous time interval [t1 ,t2 ], t2 ≤ T , where the value of the optimal control (4.63) is again equal to zero, then the matrix Q(t) in this interval can be found as the solution of (4.64) continued from the interval [t2 , T ]. Moreover, since the state velocity dx(t)/dt is also equal to zero in [t1 ,t2 ], the values of x(t − hi ), i = 1, . . . , p should be equal to zero, x(t − hi ) = 0, in this interval. Thus, the matrices Ni (t), i = 1, . . . , p, can be assigned arbitrarily for t ∈ [t1 ,t2 ], for example, as the limits of the values of Ni (t) from the interval [t2 , T ], thus assuring the smoothness of Q(t) as the solution of (4.64). Details of the proof are given in Subsection 4.3.4. Note that the matrices Ni (t) are actually the state transition matrices from the current moment t to the moment t − hi for the original time-delay system (4.61). The matrix N0 (t) corresponding to the non-delayed term a0 (t)x(t) is equal to the identity matrix, N0 (t) = I, everywhere. Upon substituting the optimal control (4.63) into the state equation (4.61), the optimally controlled state equation is obtained p

x(t) ˙ = ∑ ai (t)x(t − hi ) + B(t)R−1(t)BT (t)Q(t)x(t),

(4.65)

i=0

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ]. It should be noted that the obtained optimal regulator makes an advance with respect to general optimality results for time-delay systems (such as given in [2, 56, 46, 47]), since (a) the optimal control law is given explicitly and not as a solution of a system of integro-differential or partial differential equations, and (b) the Riccati equation for the gain matrix does not contain any time-advanced arguments and, therefore, leads to a two points boundary-valued problem generated in the optimal control problems with quadratic criterion and finite horizon (see, for example, [49]). Thus, the obtained optimal regulator is realizable using two delay-differential equations. Remark 1. At the first glance, the matrices Ni (t) seem to depend also on the system states x(t) and x(t − hi ). However, this is not so. As follows from Theorem 3.2.6 in [56], the formula x(t) = Φ (t, τ )x(τ ), which is conventional for linear systems without delays (see [49]), also takes place for the states x(t), x(τ ) and the fundamental matrix Φ (t, τ ) of the linear time-delay system (4.61), where the control u(t) is given by (4.63). Note that Φ (t, τ ) does not depend on x(t) or x(τ ); in particular, Φ (t,t − hi ) does not depend on x(t) or x(t − hi ). Therefore, any matrix Ni (t) forming a linear combination between x(t) and x(t − hi ) is independent of the system states. Remark 2. It can be observed that the matrix functions Ni (t) may be unbounded at neighborhoods of the isolated time points t = τk , k = 1, 2, . . ., where x(τ ) = 0. This raises the question if the optimal LQR theory is applicable to the transformed system

150

4 Optimal Control Problems for Time-Delay Systems p

x(t) ˙ = ∑ ai (t)Ni (t)x(t) + B(t)u(t),

(4.66)

i=0

where the coefficients depending on Ni (t) may also be unbounded. The affirmative answer to this question follows from the general assumptions of the optimal control system theory (see [67], chapter 2], where only piecewise continuity of the right-hand side is required, if the system equation solution exists. Since the solution of (4.66) coincides, by construction, with the solution of the original system equation (4.61), whose existence is substantiated in the paragraph after (4.61), the optimal LQR theory is still applicable to the system (4.66). Remark 3. In some papers (see, for example, [2]), the optimal control u∗ (t) is obtained as an integral of the previous values of x(t) over the interval [t − h,t]. However, since the backward solution of the equation (4.61) exists and is unique, as indicated in Subsection 4.3.1, any previous value x(s), s ∈ [t − h,t] can be uniquely represented as a function of the current value x(t) (as well as of any delayed value x(t − r), r > 0). Thus, the optimal control u∗ (t) can be obtained as a function of x(t) in the form (4.63). The current value x(t) is selected to form the closed-loop control (4.63), first, because the transversality condition (see [67]) induced by the cost function (4.62) can readily be satisfied and, second, due to practical applicability of the current-time control in real technical problems. 4.3.3

Example

This section presents an example of designing the optimal regulator for a system (4.61) with a criterion (4.62), using the scheme (4.63)–(4.65), and comparing it to the regulator where the matrix Q is selected as in the optimal linear regulator for a system without delays. In addition, the optimal regulator (4.63)–(4.65) is compared to two linear regulators based on rational approximations of the original time-delay system. Let us start with a scalar linear system x(t) ˙ = x(t) + 10x(t − 0.25) + u(t),

(4.67)

with the initial conditions x(s) = 1 for s ∈ [−0.25, 0]. The control problem is to find the control u(t), t ∈ [0, T ], T = 0.5, that minimizes the criterion  T

1 J= [ 2

u2 (t)dt +

0

 T

x2 (t)dt].

(4.68)

0

In other words, the control problem is to minimize the overall energy of the state x using the minimal overall energy of control u. Let us first construct the regulator where the control law and the matrix Q(t) are calculated in the same manner as for the optimal linear regulator for a linear system without delays, that is u(t) = R−1 (t)BT (t)Q(t)x(t) (see [49] for reference). Since B(t) = 1 in (4.67) and R(t) = 1 in (4.68), the optimal control is actually equal to u(t) = Q(t)x(t), where Q(t) satisfies the Riccati equation

(4.69)

4.3 Optimal Control Problem for Linear Systems with Multiple State Delays

151

˙ = −(a0 (t) + a1(t))T Q(t) − Q(t)(a0(t) + a1(t)) + L(t)− Q(t) Q(t)B(t)R−1 (t)BT (t)Q(t), with the terminal condition Q(T ) = −ψ . Since a0 (t) = 1, a1 (t) = 10, a0 (t)+ a1(t) = 11, B(t) = 1 in (4.67), and L(t) = 1, ψ = 0 in (4.68), the last equation turns to ˙ = 1 − 22Q(t) − Q2(t), Q(t)

Q(0.5) = 0.

(4.70)

Upon substituting the control (4.69) into (4.67), the controlled system takes the form x(t) ˙ = x(t) + 10x(t − 0.25) + Q(t)x(t).

(4.71)

The results of applying the regulator (4.69)–(4.71) to the system (4.67) are shown in Fig. 4.4, which presents the graphs of the criterion (4.68) J(t) and the control (4.69) u(t) in the interval [0, T ]. The value of criterion (4.68) at the final moment T = 0.5 is J(0.5) = 19.16. Let us now apply the optimal regulator (4.63)–(4.65) for linear systems with multiple state delays to the system (4.67). The control law (4.63) takes the same form as (4.69) u∗ (t) = Q∗ (t)x(t),

(4.72)

where Q∗ (t) satisfies the equation Q˙ ∗ (t) = 1 − 2Q∗(t) − 20Q∗(t)N1 (t) − Q∗ 2 (t),

Q∗ (0.5) = 0,

(4.73)

where N0 (t) = 1 and N1 (t) = x(t − 0.25)/x(t) for t ∈ [0, 0.5]. Note that the obtained equation (4.73) does not contain any advanced arguments and, therefore, can be solved using simple numerical methods, such as ”shooting.” This method consists in varying initial conditions of (4.73), taking into account monotonicity of the solution of (4.73) with respect to initial conditions, until the terminal condition is satisfied. In this example, the equation (4.73) has been solved with the approximating terminal condition Q∗ (0.5) = 0.05, in order to reduce the computation time. Upon substituting the control (4.72) into (4.67), the optimally controlled system takes the same form as (4.71) x(t) ˙ = x(t) + 10x(t − 0.25) + Q∗(t)x(t).

(4.74)

The results of applying the regulator (4.72)–(4.74) to the system (4.67) are shown in Fig. 4.5, which presents the graphs of the criterion (4.68) J(t) and the control (4.72) u∗ (t) in the interval [0, T ]. The value of the criterion (4.68) at the final moment T = 0.5 is J(0.5) = 6.56. There is a definitive improvement (about three times) in the values of the cost function in comparison to the preceding case, due to the optimality of the regulator (4.63)–(4.65) for linear systems with multiple state delays. Let us also compare the optimal regulator (4.63)–(4.65) to the best linear regulators based on the linear and quadratic rational approximations of the time-delay system (4.67). First, the input-state transfer function of (4.67), G(s) = (s− 1 − 10 exp(−sh))−1 ,

152

4 Optimal Control Problems for Time-Delay Systems

h = 0.25, is approximated by a rational function up to the first order of h: G−1 (s) = s(1 + 10h) − 11 + O(h2), which results in the time-domain realization (h = 0.25) x(t) ˙ = (22/7)x(t) + (2/7)u(t),

(4.75)

with the initial condition x(0) = 1. The control law is calculated as the optimal control for the linear system without delays (4.75): u1 (t) = (2/7)Q1 (t)x(t),

(4.76)

and Q1 (t) satisfies the Riccati equation Q˙ 1 (t) = 1 − (44/7)Q1(t) − (2/7)2Q21 (t),

(4.77)

with the terminal condition Q1 (0.5) = 0. The control (4.76) is then substituted into the original time-delay system (4.77). The results of applying the regulator (4.75)–(4.77) to the system (4.77) are shown in Fig. 4.6, which presents the graphs of the criterion (4.68) J(t) and the control (4.76) u1 (t) in the interval [0, T ]. The value of the criterion (4.68) at the final moment T = 0.5 is J(0.5) = 7.39. Thus, the simulation results show that application of the regulator (4.75)–(4.77), based on a rational approximation of the first order, yields still unsatisfactory values of the cost function in comparison to the optimal regulator (4.63)–(4.65). Next, the transfer function G(s) is approximated by a rational function up to the second order of h: G−1 (s) = s(1 + 10h) − 11 − 10(h2/2)s2 + O(h3). This approximation results in the time-domain realization (h = 0.25) (10/32)(d 2(x(t))/dt 2 ) = (7/2)d(x(t))/dt − 11x(t) − u(t), which is represented in the standard form of two first-order differential equations x(t) ˙ = z(t),

z˙(t) = 11.2z(t) − 35.2x(t) − 3.2u(t),

(4.78)

with the initial conditions x(0) = 1, z(0) = 0. The control law is calculated as the optimal control for the linear system without delays (4.78): u2 (t) = BT2 Q2 (t)[x(t)z(t)]T ,

(4.79)

and the two-dimensional matrix Q2 (t) satisfies the Riccati equation Q˙ 2 (t) = L2 − aT2 Q2 (t) − Q2 (t)a2 − Q2(t)B2 BT2 Q2 (t),

(4.80)

with the terminal condition Q2 (T ) = 0, where B2 = [0 − 3.2]T , a2 = [0 1 | −35.2 11.2] and L2 = diag[1 0]. The control (4.79) is then substituted into the original time-delay system (4.67). The results of applying the regulator (4.78)–(4.80) to the system (4.67) are shown in Fig. 4.5, which presents the graphs of the criterion (4.68) J(t) and the control (4.79) u2 (t) in the interval [0, T ]. The value of the criterion (4.68) at the final moment T = 0.5 is J(0.5) = 52.25, which is much worse than in any preceding case. Thus, the second-order rational approximation occurs to be practically useless, since

4.3 Optimal Control Problem for Linear Systems with Multiple State Delays

153

20

criterion

15

10

5

0

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

5 0

control

−5 −10 −15 −20 −25

Fig. 4.4. Best linear regulator available for linear systems without delays. Graphs of the criterion (4.68) J(t) and the control (4.69) u(t) in the interval [0, 0.5].

it leads to an increase in the problem dimension, producing additional computational difficulties, and yields outrageous values of the cost function. The last phenomenon is explained by the fact that the second-order polynomial transfer function G(s) = (s(1 + 10h) − 11 − 10(h2/2)s2 )−1 has two complex unstable poles with a positive real part, which poorly approximate the real unstable pole (s = 3.83). This creates additional large unstable dynamics in the approximating system, which is not present in the original system (4.67), thus producing too large distortions in the control design. Finally note that the rational transfer function corresponding to the Pade approximation exp (−sh) = (1 − sh/2)/(1 + sh/2) is not considered here, since it yields an even larger unstable pole s = 21.16, does not approximate the real unstable pole at all (the second produced pole is negative, i.e., stable), and also leads to an increase in the problem dimension. 4.3.4

Proof of Optimal Control Problem Solution

Upon introducing the auxiliary functions Ni (t), 0 = 1, . . . , p, as defined in Subsection 4.3.2, the state equation (4.61) can be written in the form p

x(t) ˙ = ∑ ai (t)Ni (t)x(t) + B(t)u(t),

(4.81)

i=0

with the initial condition x(0) = φ (0). Since the matrices Ni (t) are known timedependent functions at the current time moment t, the original optimal control problem

154

4 Optimal Control Problems for Time-Delay Systems

7 6

criterion

5 4 3 2 1 0

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

1 0 −1

control

−2 −3 −4 −5 −6 −7 −8

Fig. 4.5. Optimal regulator obtained for linear systems with multiple state delays. Graphs of the criterion (4.68) J(t) and the optimal control (4.72) u∗ (t) in the interval [0, 0.5].

Fig. 4.6. Regulator based on a first-order approximation of the original time-delay system. Graphs of the criterion (4.68) J(t) and the control (4.76) u1 (t) in the interval [0, 0.5].

4.3 Optimal Control Problem for Linear Systems with Multiple State Delays

155

Fig. 4.7. Regulator based on a second-order approximation of the original time-delay system. Graphs of the criterion (4.68) J(t) and the control (4.79) u2 (t) in the interval [0, 0.5].

for a time-delay system is now reduced to the conventional optimal LQR problem for the state (4.81) and the same quadratic cost function (4.62), where the state dynamics p

matrix A is equal to A = ∑ ai (t)Ni (t). The well-known solution to this problem (see, i=0

for example, [49]) directly yields the optimal control law (4.63) and the gain matrix equation in the form (4.64). It should however be demonstrated that the optimal control can be consistently defined at the time moments where x(t) = 0. Indeed, if the set {τk : x(τk ) = 0, k = 1, 2, . . .} is a set of isolated time points, then the optimal control u∗ (τk ) is equal to zero at any time τk , in view of linearity and continuity of the control law (4.63). Thus, the value of the gain matrix Q(τk ) does not actually participate in formation of the optimal control and can be assigned as the limit of the values of Q(t) from the right, Q(τk ) = lim Q(t), t→τk ,t>τk

thus assuring the continuity of Q(t) as the solution of (4.64). Assume now that the set {t : x(t) = 0} includes a continuous time interval, [t1 ,t2 ], t2 ≤ T . This implies that the optimal control (4.63) is equal to zero in this interval, u∗ (t) = 0, t ∈ [t1 ,t2 ]. Hence, the values of the matrix Q(t) for t ∈ [t1 ,t2 ] do not participate in formation of the optimal control in the interval [t1 ,t2 ] and can be assigned arbitrarily. On the other hand, the optimal control problem reduced to the interval [t0 ,t1 ] does not induce any additional transversality condition at the point t1 , since the fixed zero state value at t = t1 , x(t1 ) = 0, is assumed (see [67]). Therefore, any function Q(t) satisfying (4.64) in [t0 ,t2 ] would be appropriate for forming the optimal control law (4.63). Thus, the gain matrix Q(t) can be found in the interval [t0 ,t2 ] as the solution of (4.64) continued from the interval [t2 , T ]. Furthermore, the state velocity dx(t)/dt is also equal to zero in the interval

156

4 Optimal Control Problems for Time-Delay Systems

[t1 ,t2 ], dx(t)/dt = 0, t ∈ [t1 ,t2 ], if x(t) = 0 for t ∈ [t1 ,t2 ]. Analysis of the state equation (4.61) shows that the equality dx(t)/dt = 0 in a continuous time interval is reached if x(t − hi ) = 0 for any i = 1, . . . , p in this interval. Thus, the matrices Ni (t), i = 1, . . . , p, can also be assigned arbitrarily for t ∈ [t1 ,t2 ], for example, as the limits of the values of Ni (t) from the interval [t2 , T ], thus assuring the smoothness of Q(t) as the solution of (4.64). Finally, consider the case of the zero state value at the initial point, x(t0 ) = 0, and verify that no special situation is generated. Indeed, since all the coefficients in (4.61),(4.62) and the initial function φ (s) are continuous, the optimal control law is also continuous with respect to initial conditions. Upon approximating the given initial function by initial functions with nonzero values at t = t0 converging to zero in the limit, the optimal control laws (4.63) are obtained for each pre-limiting approximation, which yield the limiting zero control values, u(t0 ) = 0, in view of the limiting zero state value, x(t0 ) = φ (t0 ) = 0, and the continuity with respect to initial conditions. Thus, the matrix initial value Q(t0 ) is not important for forming the control value u(t0 ) and can be left such as obtained upon solving the equation (4.64). The optimal control problem solution is proved.

4.4 Optimal Control Problem for Linear Systems with Multiple State and Input Delays 4.4.1

Problem Statement

Consider a linear system with multiple time delays in the state and control input p

q

i=0

j=0

x(t) ˙ = ∑ ai (t)x(t − hi ) + ∑ B j (t)u(t − τ j ),

(4.82)

with the initial conditions x(s) = φ (s) ∈ Rn , s ∈ [t0 − h,t0 ], h0 = 0, h = max (h1 , . . . , h p ), where x(·) ∈ C([t0 − h,t0 ]; Rn ) is the system state, and u(s) = φ1 (s) ∈ Rm , s ∈ [t0 − τ ,t0 ], τ0 = 0, τ = max (τ1 , . . . , τ p ), where u(·) ∈ C([t0 − τ ,t0 ]; Rm ) is the control variable, and φ (s) and φ1 (s) are continuous functions given in the intervals [t0 − h,t0] and [t0 − τ ,t0 ], respectively. The coefficients ai (t) and B j (t) are considered continuous functions of time. Note that the state of a delayed system (4.82) is infinite-dimensional (see [56], Section 2.3). Existence of the unique forward solution of the equation (4.82) is thus assured by the Carath´eodory theorem (see, for example, [29]), and existence of the unique backward solution follows from analyticity of the right-hand side functional with respect to the system state (see [38, 72]) The quadratic cost function to be minimized is defined as follows:



1 J = [x(T )]T ψ [x(T )]+ 2

(4.83)

1 T T (u (s)R(s)u(s)ds + xT (s)L(s)x(s))ds, 2 t0 where R is positive and ψ , L are nonnegative definite continuous symmetric matrix functions, and T > t0 is a certain time moment.

4.4 Optimal Control Problem for Linear Systems with Multiple State and Input Delays

157

The optimal control problem is to find the control u∗ (t), t ∈ [t0 , T ], that minimizes the criterion J along with the trajectory x∗ (t), t ∈ [t0 , T ], generated upon substituting u∗ (t) into the state equation (4.82). 4.4.2

Optimal Control Problem Solution

The solution to the optimal control problem for the linear system with multiple state delays (4.82) and the quadratic criterion (4.83) is given as follows. The optimal control law takes the form q

u∗ (t) = R−1 (t)( ∑ M Tj (t)BTj (t))Q(t)x(t),

(4.84)

j=0

where the matrix function Q(t) satisfies the matrix Riccati equation with time-varying and state-dependent coefficients p

p

˙ = L(t) − Q(t)( ∑ ai (t)Ni (t)) − ( ∑ NiT (t)aTi (t))Q(t)− Q(t) i=0

i=0

q

q

j=0

j=0

Q(t)( ∑ B j (t)M j (t))R−1 (t)( ∑ M Tj (t)BTj (t))Q(t),

(4.85)

with the terminal condition Q(T ) = −ψ . Each of the auxiliary matrices Ni (t), i = 0, . . . , p, is defined as a solution of the algebraic equation Ni (t)x(t) = x(t − hi ), which can always be satisfied, if x(t) is not the vector of zeros. On the other hand, if the equality x(t) = 0 is satisfied only in a set of isolated time points t = τk , k = 1, 2, . . ., where the value of the optimal control (4.84) is also equal to zero, then the values of Q(τk ) can be assigned as the limits of the values of Q(t) from the right, thus assuring the continuity of Q(t) as the solution of (4.85). If the equality x(t) = 0 is satisfied in a continuous time interval [t1 ,t2 ], t2 ≤ T , where the value of the optimal control (4.84) is again equal to zero, then the matrix Q(t) in this interval can be found as the solution of (4.85) continued from the interval [t2 , T ]. Moreover, since the state velocity dx(t)/dt is also equal to zero in [t1 ,t2 ], the values of x(t − hi ), i = 1, . . . , p should be equal to zero, x(t − hi ) = 0, in this interval. Thus, the matrices Ni (t), i = 1, . . . , p, can be assigned arbitrarily for t ∈ [t1 ,t2 ], for example, as the limits of the values of Ni (t) from the interval [t2 , T ], thus assuring the smoothness of Q(t) as the solution of (4.85). Details of the proof are given in Subsection 4.4.4. Each of the auxiliary matrices M j (t), j = 0, . . . , q, is defined as a solution of the algebraic equation M j (t)u∗ (t) = u∗ (t − τ j ), which can always be satisfied, if u∗ (t) is not the vector of zeros. The equality u∗ (t) = 0 could be satisfied in view of the following q

reasons: 1) x(t) = 0; 2) Q(t) = 0; or 3) ∑ M Tj (t)BTj (t) = 0. The first case, x(t) = 0, j=0

has already been discussed and the methods for extending the definitions of the corresponding matrices have been pointed out. The second case, Q(t) = 0, allows one not to define the matrices M j (t), since the value of Q(t) is already known. Finally, the third q

case, ∑ M Tj (t)BTj (t) = 0, implies that the quadratic term in the Riccati equation (4.85) j=0

158

4 Optimal Control Problems for Time-Delay Systems

is equal to zero (while the other terms are unchanged) and, therefore, also allows one not to define the matrices M j (t). Note that the matrices Ni (t) and M j (t) are actually the state and control transition matrices from the current moment t to the moments t − hi and t − τ j for the original timedelay system (4.82). The matrices N0 (t) and M0 (t) corresponding to the non-delayed terms a0 (t)x(t) and B0 (t)u∗ (t) are equal to the identity matrix, N0 (t) = M0 (t) = I, everywhere. Upon substituting the optimal control (4.84) into the state equation (4.82), the optimally controlled state equation is obtained p

q

i=0

j=0

x(t) ˙ = ∑ ai (t)x(t − hi ) + B(t)R−1(t)( ∑ M Tj (t)BTj (t))Q(t)x(t),

(4.86)

with the initial conditions x(s) = φ (s), s ∈ [t0 − h,t0], and u(s) = φ1 (s), s ∈ [t0 − τ ,t0 ] It should be noted that the obtained optimal regulator makes an advance with respect to general optimality results for time-delay systems (such as given in [2, 56, 46, 47]), since (a) the optimal control law is given explicitly and not as a solution of a system of integro-differential or partial differential equations, and (b) the Riccati equation for the gain matrix does not contain any time-advanced arguments and, therefore, leads to a two points boundary-valued problem generated in the optimal control problems with quadratic criterion and finite horizon (see, for example, [49]). Thus, the obtained optimal regulator is realizable using two delay-differential equations. Remark 1. At the first glance, the matrices Ni (t) seem to depend also on the system states x(t) and x(t − hi ). However, this is not so. As follows from Theorem 3.2.6 in [56], the formula x(t) = Φ (t, τ )x(τ ), which is conventional for linear systems without delays (see [49]), also takes place for the states x(t), x(τ ) and the fundamental matrix Φ (t, τ ) of the linear time-delay system (4.82), where the control u(t) is given by (4.84). Note that Φ (t, τ ) does not depend on x(t) or x(τ ); in particular, Φ (t,t − hi ) does not depend on x(t) or x(t − hi ). Therefore, any matrix Ni (t) forming a linear combination between x(t) and x(t − hi ) is independent of the system states. Remark 2. It can be observed that the matrix functions Ni (t) may be unbounded at neighborhoods of the isolated time points t = τk , k = 1, 2, . . ., where x(τ ) = 0. This raises the question if the optimal LQR theory is applicable to the transformed system p

q

i=0

j=0

x(t) ˙ = ∑ ai (t)Ni (t)x(t) + ∑ B j (t)u(t − τ j ),

(4.87)

where the coefficients depending on Ni (t) may also be unbounded. The affirmative answer to this question follows from the general assumptions of the optimal control system theory (see [67], chapter 2], where only piecewise continuity of the right-hand side is required, if the system equation solution exists. Since the solution of (4.87) coincides, by construction, with the solution of the original system equation (4.82), whose existence is substantiated in the paragraph after (4.82), the optimal LQR theory is still applicable to the system (4.87). Remark 3. In some papers (see, for example, [2]), the optimal control u∗ (t) is obtained as an integral of the previous values of x(t) over the interval [t − h,t]. However, since the

4.4 Optimal Control Problem for Linear Systems with Multiple State and Input Delays

159

backward solution of the equation (4.87) exists and is unique, as indicated in Subsection 4.4.1, any previous value x(s), s ∈ [t − h,t] can be uniquely represented as a function of the current value x(t) (as well as of any delayed value x(t − r), r > 0). Thus, the optimal control u∗ (t) can be obtained as a function of x(t) in the form (4.84). The current value x(t) is selected to form the closed-loop control (4.84), first, because the transversality condition (see [67]) induced by the cost function (4.83) can readily be satisfied and, second, due to practical applicability of the current-time causal control in real technical problems. 4.4.3

Example

This section presents an example of designing the optimal regulator for a system (4.82) with a criterion (4.83), using the scheme (4.84)–(4.86), and comparing it to the regulator where the matrix Q is selected as in the optimal linear regulator for a system without delays. In addition, the optimal regulator (4.84)–(4.86) is compared to two linear regulators based on rational approximations of the original time-delay system. Let us start with a scalar linear system x(t) ˙ = x(t) + 10x(t − 0.25) + u(t − 0.5),

(4.88)

with the initial conditions x(s) = 0.1 for s ∈ [−0.25, 0] and u(s) = 10 for s ∈ [−0.5, 0]. The control problem is to find the control u(t), t ∈ [0, T ], T = 1, that minimizes the criterion   T 1 T 2 u (t)dt + x2 (t)dt]. (4.89) J= [ 2 0 0 In other words, the control problem is to minimize the overall energy of the state x using the minimal overall energy of control u. Let us first construct the regulator where the control law and the matrix Q(t) are calculated in the same manner as for the optimal linear regulator for a linear system without delays, that is u(t) = R−1 (t)BT (t)Q(t)x(t) (see [49] for reference). Since B(t) = 1 in (4.88) and R(t) = 1 in (4.89), the optimal control is actually equal to u(t) = Q(t)x(t),

(4.90)

where Q(t) satisfies the Riccati equation ˙ = −(a0 (t) + a1(t))T Q(t) − Q(t)(a0(t) + a1(t))+ Q(t) L(t) − Q(t)B(t)R−1(t)BT (t)Q(t), with the terminal condition Q(T ) = −ψ . Since a0 (t) = 1, a1 (t) = 10, a0 (t)+ a1(t) = 11, B(t) = 1 in (4.88), and L(t) = 1, ψ = 0 in (4.89), the last equation turns to ˙ = 1 − 22Q(t) − Q2(t), Q(t)

Q(1) = 0.

(4.91)

Upon substituting the control (4.90) into (4.88), the controlled system takes the form x(t) ˙ = x(t) + 10x(t − 0.25) + Q(t)x(t).

(4.92)

160

4 Optimal Control Problems for Time-Delay Systems

The results of applying the regulator (4.90)–(4.92) to the system (4.88) are shown in Fig. 4.8, which presents the graphs of the criterion (4.89) J(t) and the control (4.90) u(t) in the interval [0, T ]. The value of criterion (4.89) at the final moment T = 1 is J(1) = 9113.63. Let us now apply the optimal regulator (4.84)–(4.86) for linear systems with multiple state delays to the system (4.88). The control law (4.84) takes the same form as (4.90) u∗ (t) = Q∗ (t)x(t),

(4.93)

where Q∗ (t) satisfies the equation Q˙ ∗ (t) = 1 − 2Q∗(t) − 20Q∗(t)N1 (t) − M12 (t)Q∗ 2 (t),

Q∗ (1) = 0,

(4.94)

(t) = u∗ (t − 0.5)/u∗(t)

for t ∈ [0, 1]. where N0 (t) = 1, N1 (t) = x(t − 0.25)/x(t), and M1 Note that the obtained equation (4.94) does not contain any advanced arguments and, therefore, can be solved using simple numerical methods, such as ”shooting.” This method consists in varying initial conditions of (4.94), taking into account monotonicity of the solution of (4.94) with respect to initial conditions, until the terminal condition is satisfied. Upon substituting the control (4.93) into (4.88), the optimally controlled system takes the form (4.95) x(t) ˙ = x(t) + 10x(t − 0.25) + M1(t)Q∗ (t)x(t). The results of applying the regulator (4.93)–(4.95) to the system (4.88) are shown in Fig. 4.9, which presents the graphs of the criterion (4.89) J(t) and the control (4.93) u∗ (t) in the interval [0, T ]. The value of the criterion (4.89) at the final moment T = 1 is J(1) = 465.54. There is a definitive improvement (about twenty times) in the values of the cost function in comparison to the preceding case, due to the optimality of the regulator (4.84)–(4.86) for linear systems with multiple state and input delays. Note that, due to the specialty of the MatLab layout, the discontinuous control u∗ (t) is represented by a continuous curve in Fig. 4.9. In reality, there is a jump in values of u∗ (t − 0.5) at t = 0.5, given that u(s) = 10 for s ∈ [−0.5, 0] (the value from the left) and u(0+) = Q∗ (0)x(0) (the value from the right). The real graph of u∗ (t − 0.5) is then continuous between t = 0.5 and 1, and the correct value of u∗ (t −0.5) from the left at t = 1 is -8.673. The discontinuous control jump at t = 0.5 also takes place for u∗ (t − 0.5), u∗1 (t − 0.5), and u∗2 (t − 0.5) in Figs. 4.8, 4.10, and 4.11, where the control value from the left is equal to 10 and the control values from the right are shown correctly. Let us also compare the optimal regulator (4.84)–(4.86) to the best linear regulators based on the linear and quadratic rational approximations of the time-delay system (4.88). First, the input-state transfer function of (4.88), G(s) = (exp (−sτ ))(s − 1 − 10 exp(−sh))−1 , h = 0.25, τ = 0.5 is approximated by a rational function up to the first order of h and τ : G−1 (s) = s(1 + 10h) − 11 − 11sτ + o(h + τ ), which results in the time-domain realization (h = 0.25, τ = 0.5) x(t) ˙ = −(11/2)x(t) − (1/2)u1(t),

(4.96)

with the initial condition x(0) = 1. The control law is calculated as the optimal control for the linear system without delays (4.96): u1 (t) = −(1/2)Q1(t)x(t),

(4.97)

4.4 Optimal Control Problem for Linear Systems with Multiple State and Input Delays

161

and Q1 (t) satisfies the Riccati equation Q˙ 1 (t) = 1 + 11Q1(t) − (1/4)Q21(t),

(4.98)

with the terminal condition Q1 (1) = 0. The control (4.97) is then substituted into the original time-delay system (4.88). The results of applying the regulator (4.96)-(4.98) to the system (4.86) are shown in Fig. 4.10, which presents the graphs of the criterion (4.89) J(t) and the control (4.97) u1 (t) in the interval [0, T ]. The value of the criterion (4.89) at the final moment T = 1 is J(1) = 544.44. Thus, the simulation results show that application of the regulator (4.96)-(4.98), based on a rational approximation of the first order, yields still unsatisfactory values of the cost function in comparison to the optimal regulator (4.84)-(4.86). Next, the transfer function G(s) is approximated by a rational function up to the second order of h and τ : G−1 (s) = s(1 + 10h) − 11 − 10(h2/2)s2 − 11sτ − (11/2)(τ 2 /2)s2 + o(h2 + τ 2 ). This approximation results in the time-domain realization, which is represented in the standard form of two first-order differential equations (h = 0.25, τ = 0.5) x(t) ˙ = z(t), (4.99) z˙(t) = −(32/27)z(t) − (176/27)x(t) − (16/27)u2(t), with the initial conditions x(0) = 1, z(0) = 0. The control law is calculated as the optimal control for the linear system without delays (4.99): u2 (t) = BT2 Q2 (t)[x(t)z(t)]T ,

(4.100)

and the two-dimensional matrix Q2 (t) satisfies the Riccati equation Q˙ 2 (t) = L2 − aT2 Q2 (t) − Q2 (t)a2 − Q2(t)B2 BT2 Q2 (t),

(4.101)

with the terminal condition Q2 (T ) = 0, where B2 = [0 − 16/27]T , a2 = [0 1 | −176/27 − 32/27] and L2 = diag[1 0]. The control (4.100) is then substituted into the original time-delay system (4.88). The results of applying the regulator (4.99)-(4.101) to the system (4.88) are shown in Fig. 4.11, which presents the graphs of the criterion (4.89) J(t) and the control (4.100) u2 (t) in the interval [0, T ]. The value of the criterion (4.89) at the final moment T = 1 is J(1) = 539.68, which is only a bit better than in the preceding case. Thus, the second-order rational approximation occurs to be practically useless, since it leads to an increase in the problem dimension, producing additional computational difficulties, and insignificantly improves values of the cost function in comparison to the first-order one. Finally note that the rational transfer function corresponding to the Pade approximation exp (−sh) = (1 − sh/2)/(1 + sh/2) is not considered here, since it yields a too large unstable pole s = 21.16, which does not approximate the real unstable pole, s = 3.83, of the original transfer function of (4.88) at all (the other two produced poles are negative, i.e., stable), and also leads to an increase in the problem dimension up to the third order system. It could also be readily observed that the unstable pole of the system without delays assumed in designing the regulator (4.90),(4.91) is equal to s = 11, which is even less than that of the Pade approximation.

162

4 Optimal Control Problems for Time-Delay Systems

10000 9000

criterion

8000

6000

4000

2000

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.6

0.7

0.8

0.9

1

time

50

10 0

control

-50

-100

-150

-200

-250

0

0.1

0.2

0.3

0.4

0.5 time

Fig. 4.8. Best linear regulator available for linear systems without delays. Graphs of the criterion (4.88) J(t) and the control (4.89) u(t) in the interval [0, 1].

Fig. 4.9. Optimal regulator obtained for linear systems with multiple state delays. Graphs of the criterion (4.88) J(t) and the optimal control (4.89) u∗ (t) in the interval [0, 1].

4.4 Optimal Control Problem for Linear Systems with Multiple State and Input Delays

163

600 500

criterion

400

200

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.6

0.7

0.8

0.9

1

0.6

0.7

0.8

0.9

1

time 10 8

control

6 4 2 0 -2

0

0.1

0.2

0.3

0.4

0.5

0 time

control in [0.5,1]

-0.002 -0.004 -0.006 -0.008 -0.01

0

0.1

0.2

0.3

0.4

0.5 time

Fig. 4.10. Regulator based on a first-order approximation of the original time-delay system. Graphs of the criterion (4.88) J(t) and the control (4.96) u1 (t) in the interval [0, 1] and augmented graph of the control (4.96) u1 (t) in the interval [0.5, 1].

4.4.4

Proof of Optimal Control Problem Solution

Upon introducing the auxiliary functions Ni (t), 0 = 1, . . . , p, and M j (t), 0 = 1, . . . , q, as defined in Subsection 4.4.2, the state equation (1) can be written in the form p

q

i=0

j=0

x(t) ˙ = ∑ ai (t)Ni (t)x(t) + ∑ B j (t)M j (t)u(t),

(4.102)

with the initial condition x(0) = φ (0). Since the matrices Ni (t) and M j (t) are known time-dependent functions at the current time moment t, the original optimal control problem for a time-delay system is now reduced to the conventional optimal LQR problem for the state (4.102) and the same quadratic cost function (4.83), where the state p

dynamics matrix A is equal to A(t) = ∑ ai (t)Ni (t) and the control matrix B is equal i=0

164

4 Optimal Control Problems for Time-Delay Systems

600 500

criterion

400

200

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.6

0.7

0.8

0.9

1

0.6

0.7

0.8

0.9

1

time 10 8

control

6 4 2 0 -2

0

0.1

0.2

0.3

0.4

0.5

0 time

control in [0.5,1]

-0.02 -0.04 -0.06 -0.08 -0.1 -0.12

0

0.1

0.2

0.3

0.4

0.5

Fig. 4.11. Regulator based on a second-order approximation of the original time-delay system. Graphs of the criterion (4.88) J(t) and the control (4.99) u2 (t) in the interval [0, 0.5] and augmented graph of the control (4.99) u2 (t) in the interval [0.5, 1]. q

to B(t) = ∑ B j (t)M j (t). The well-known solution to this problem (see, for example, j=0

[49]) directly yields the optimal control law (4.84) and the gain matrix equation in the form (4.85). It should however be demonstrated that the optimal control can be consistently defined at the time moments where x(t) = 0 or u∗ (t) = 0. Consider first the case x(t) = 0. Indeed, if the set {τk : x(τk ) = 0, k = 1, 2, . . .} is a set of isolated time points, then the optimal control u∗ (τk ) is equal to zero at any time τk , in view of linearity and continuity of the control law (4.84). Thus, the value of the gain matrix Q(τk ) does not actually participate in formation of the optimal control and can be assigned as the limit of the values of Q(t) from the right, Q(τk ) = lim Q(t), thus assuring the continuity of t→τk ,t>τk

Q(t) as the solution of (4.85). Assume now that the set {t : x(t) = 0} includes a continuous time interval, [t1 ,t2 ], t2 ≤ T . This implies that the optimal control (4.84) is equal

4.4 Optimal Control Problem for Linear Systems with Multiple State and Input Delays

165

to zero in this interval, u∗ (t) = 0, t ∈ [t1 ,t2 ]. Hence, the values of the matrix Q(t) for t ∈ [t1 ,t2 ] do not participate in formation of the optimal control in the interval [t1 ,t2 ] and can be assigned arbitrarily. On the other hand, the optimal control problem reduced to the interval [t0 ,t1 ] does not induce any additional transversality condition at the point t1 , since the fixed zero state value at t = t1 , x(t1 ) = 0, is assumed (see [67]). Therefore, any function Q(t) satisfying (4.85) in [t0 ,t2 ] would be appropriate for forming the optimal control law (4.84). Thus, the gain matrix Q(t) can be found in the interval [t0 ,t2 ] as the solution of (4.85) continued from the interval [t2 , T ]. Furthermore, the state velocity dx(t)/dt is also equal to zero in the interval [t1 ,t2 ], dx(t)/dt = 0, t ∈ [t1 ,t2 ], if x(t) = 0 for t ∈ [t1 ,t2 ]. Analysis of the state equation (4.82) shows that the equality dx(t)/dt = 0 in a continuous time interval is reached if x(t − hi ) = x(t − τ j ) = 0 for any i = 1, . . . , p, j = 1, . . . , q, in this interval. Thus, the matrices Ni (t), i = 1, . . . , p, and M j (t), j = 1, . . . , q, can also be assigned arbitrarily for t ∈ [t1 ,t2 ], for example, as the limits of their values from the interval [t2 , T ], thus assuring the smoothness of Q(t) as the solution of (4.85). Consider now the case u∗ (t) = 0. Indeed, the equality u∗ (t) = 0 could be satisfied in q

view of the following reasons: 1) x(t) = 0; 2) Q(t) = 0; or 3) ∑ M Tj (t)BTj (t) = 0. j=0

The first case, x(t) = 0, has already been discussed and the methods for extending the definitions of the corresponding matrices have been pointed out. The second case, Q(t) = 0, allows one not to define the matrices M j (t), since the value of Q(t) is already q

known. Finally, the third case, ∑ M Tj (t)BTj (t) = 0, implies that the quadratic term in j=0

the Riccati equation (4.85) is equal to zero (while the other terms are unchanged) and, therefore, also allows one not to define the matrices M j (t). Finally, consider the case of the zero state value at the initial point, x(t0 ) = 0, and verify that no special situation is generated. Indeed, since all the coefficients in (4.82),(4.83) and the initial function φ (s) are continuous, the optimal control law is also continuous with respect to initial conditions. Upon approximating the given initial function by initial functions with nonzero values at t = t0 converging to zero in the limit, the optimal control laws (4.84) are obtained for each pre-limiting approximation, which yield the limiting zero control values, u(t0 ) = 0, in view of the limiting zero state value, x(t0 ) = φ (t0 ) = 0, and the continuity with respect to initial conditions. Thus, the matrix initial value Q(t0 ) is not important for forming the control value u(t0 ) and can be left such as obtained upon solving the equation (4.85). The optimal control problem solution is proved. Discussion The optimal control problem is considered for al linear system with multiple state and input delays and a quadratic criterion. The original problem is reduced to a conventional LQR problem for systems without delays, using the state and control transition matrices approach. The obtained solution is based on the optimal LQR for non-delayed systems, whereas the features distinguishing between those cases are explicitly pointed out. Performance of the obtained optimal regulator is verified against the linear-quadratic regulators for the non-delayed system and two rational approximations of the original

166

4 Optimal Control Problems for Time-Delay Systems

time-delay system, revealing a definitive advantage of the optimal one. The structure of the developed transition matrices approach makes it applicable to the optimal regulator and controller design for linear time-delay stochastic systems with some unmeasured states.

4.5 Optimal Controller Problem for Linear Systems with Input and Observation Delays 4.5.1

Problem Statement

Let (Ω , F, P) be a complete probability space with an increasing right-continuous family of σ -algebras Ft ,t ≥ 0, and let (W1 (t), Ft ,t ≥ t0 ) and (W2 (t), Ft ,t ≥ t0 ) be independent Wiener processes. The partially observed Ft -measurable random process (x(t), y(t)) is described by an ordinary differential equation for the dynamic system state dx(t) = (a0 (t) + a(t)x(t))dt + B(t)u(t − τ )dt + b(t)dW1 (t),

(4.103)

with the initial condition x(s) = φ (s), s ∈ [t0 − τ ,t0 ], and a delay-differential equation is given for the observation process: dy(t) = (A0 (t) + A(t)x(t − h))dt + F(t)dW2 (t). Here, x(t) ∈ Rn

is the state vector, u(t) ∈ Rm

(4.104)

is the control input, y(t) ∈ R p

is the observation process, φ (s) is a mean square piecewise-continuous Gaussian stochastic process (see [41], Section 3.6, or [70], Subsection 2.7.5, for definition) given in the interval [t0 − h,t0 ], such that φ (s), W1 (t), and W2 (t) are independent. The observation process y(t) depends on the delayed state x(t − h), where h is an observation delay, which assumes that collection of information on the system state for the observation purposes is possible only after a certain time h. It is assumed that A(t) is a nonzero matrix and F(t)F T (t) is a positive definite matrix. All coefficients in (4.103),(4.104) are deterministic functions of appropriate dimensions. The control function u(t) regulates the system state employing the values of u(t) at the previous time moment t − τ , where τ is a control delay, which assumes that there is a certain time lag τ between the moment of elaborating the control signal by the actuator and the moment of reaching the system input by it. This situation is frequently encountered in, for example, network control systems [36, 97]. In addition, the quadratic cost function J to be minimized is defined as follows



1 J = E[ xT (T )Φ x(T )+ 2 

(4.105)

1 T T 1 T T u (s)R(s)u(s)ds + x (s)L(s)x(s)ds], 2 t0 2 t0 where K is positive definite and Φ , L are nonnegative definite symmetric matrices, T > t0 is a certain time moment, the symbol E[ f (x)] means the expectation (mean) of a function f of a random variable x, and aT denotes transpose to a vector (matrix) a. The optimal controller problem is to find the control u∗ (t), t ∈ [t0 , T ], that minimizes the criterion J along with the unobserved trajectory x∗ (t), t ∈ [t0 , T ], generated upon substituting u∗ (t) into the state equation (4.103).

4.5 Optimal Controller Problem for Linear Systems with Input and Observation Delays

4.5.2

167

Separation Principle for Time-Delay Systems

As well as for a linear stochastic system without delays, the separation principle remains valid for a linear stochastic system with control and observation delays. Indeed, let us replace the unobservable system state x(t) by its optimal estimate m(t) given by the equation (see Section 3.1 for statement and derivation) dm(t) = (a0 (t) + a(t)m(t))dt + B(t)u(t − τ )dt + P(t)×  t

exp (− t−h

(4.106)

aT (s)ds)AT (t)(F(t)F T (t))−1 (dy(t) − (A0(t) + A(t)m(t − h))dt).

with the initial condition m(s) = E(φ (s)), s ∈ [t0 − τ ,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 . Here, m(t) is the best estimate for the unobservable process x(t) at time t based on the observation process Y (t) = {y(s),t0 ≤ s ≤ t}, that is the conditional expectation m(t) = E(x(t) | FtY ). The best estimate m(t) minimizes the criterion H = E[(x(t) − x(t)) ˆ T (x(t) − x(t)) ˆ | FtY ],

(4.107)

with respect to selection of the estimate m as a function of observations y(t), at every time moment t. ([41], Theorem 5.3 or [70], Subsection 5.10.2). The complementary equation for the error variance matrix P(t) takes the form (see Section 3.1 for derivation) ˙ = P(t)aT (t) + a(t)P(t) + b(t)bT (t)− P(t)  t

P(t) exp (−

 −1 aT (s)ds)AT (t) F(t)F T (t) A(t) exp(−

t−h

(4.108)

 t

a(s)ds)P(t), t−h

with the initial condition P(t0 ) = E((x(t0 ) − m(t0 ))(x(t0 ) − m(t0 ))T | y(t0 )). It is readily verified (see [49], Section 5 for similar derivations) that the optimal control problem for the system state (4.103) and cost function (4.105) is equivalent to the optimal control problem for the estimate (4.106) and the cost function J represented as  1 T T 1 u (s)R(s)u(s)ds+ (4.109) J = E{ mT (T )Φ m(T ) + 2 2 t0 1 2

 T t0

mT (s)L(s)m(s)ds +

1 2

 T t0

tr[P(s)L(s)]ds + tr[P(T )Φ ]},

where tr[A] denotes trace of a matrix A. Since the latter part of J is independent of control u(t) or state x(t), the reduced effective cost function M to be minimized takes the form 1 M = E{ mT (T )Φ m(T )+ (4.110) 2   1 T T 1 T T u (s)R(s)u(s)ds + m (s)L(s)m(s)ds}. 2 t0 2 t0 Thus, the solution for the optimal control problem specified by (4.103),(4.105) can be found solving the optimal control problem given by (4.106),(4.110). However, the

168

4 Optimal Control Problems for Time-Delay Systems

minimal value of the criterion J should be determined using (4.109). This conclusion presents the separation principle in linear systems with control and observation delays. 4.5.3

Optimal Control Problem Solution

Based on the solution of the optimal control problem obtained Section 4.1 in the case of an observable system state with time delay in control input, the following results are valid for the optimal control problem (4.106),(4.110), where the system state (the estimate m(t)) is completely available and, therefore, observable. The optimal control law is given by  t

u∗ (t − h) = (R(t))−1 BT (t) exp (

t−τ

−aT (s)ds)Q(t)m(t − h),

(4.111)

where the matrix function is the solution of the following equation dual to the variance equation ˙ = −aT (t)Q(t) − Q(t)a(t) + L(t)− Q(t)  t

Q(t) exp (

t−τ

 t

−a(s)ds)B(t)R−1 (t)BT (t) exp (

t−τ

−aT (s)ds)Q(t),

(4.112)

with the terminal condition Q(T ) = −ψ . In the process of derivation of the equation (4.112), it has been taken into account that the last term in the equation (4.106), t aT (s)ds) AT (t)(F(t)F T (t))−1 (dy(t) − (A0 (t) + A(t)m(t − h))dt), is a P(t) exp (− t−h Gaussian white noise. Upon substituting the optimal control (4.111) into the equation (4.106) for the reconstructed system state m(t), the following optimally controlled state estimate equation is obtained (4.113) dm(t) = (a0 (t) + a(t)m(t) + B(t)(R(t)−1BT (t)×  t

exp (

t−τ

−aT (s)ds)Q(t)m(t − τ ))dt + P(t) exp(−

 t

t−h

−aT (s)ds)AT (t)×

(F(t)F T (t))−1 (dy(t) − (A0 (t) + A(t)m(t − h))dt), with the initial condition m(s) = E(φ (s)), s ∈ [t0 − τ ,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 . Thus, the optimally controlled state estimate equation (4.113), the gain matrix constituent equation (4.112), the optimal control law (4.111), and the variance equation (4.107) give the complete solution to the optimal controller problem for unobserved states of linear systems with control and observation delays. 4.5.4

Example

This section presents an example of designing the controller for a linear system with input and observation delays using the scheme (4.111)–(4.113) and comparing it to the

4.5 Optimal Controller Problem for Linear Systems with Input and Observation Delays

169

controller, where the matrices P and Q are selected as in the optimal linear controller for a system without delays. Let us start with a scalar linear system x(t) ˙ = x(t) + u(t − 0.1),

x(s) = φ (s), s ∈ [−0.1, 0],

(4.114)

where φ (s) = N(0, 1) for s ∈ [−0.1, 0], and N(0, 1) is a Gaussian random variable with zero mean and unit variance. The observation process is given by y(t) = x(t − 0.2) + ψ (t),

(4.115)

where ψ (t) is a white Gaussian noise, which is the weak mean square derivative of a standard Wiener process (see ([41], Subsection 3.8, or [70], Subsection 4.6.5)). The equations (4.114) and (4.115) present the conventional form for the equations (4.103) and (4.104), which is actually used in practice [6]. The control problem is to find the control u(t), t ∈ [0, T ], that minimizes the criterion 1 1 J = E{ [x(T ) − x∗ ]2 + 2 2

 T

u2 (t)dt},

(4.116)

0

where T = 0.45, and x∗ = 25 is a large value of x(t) a priori unreachable for time T . In other words, the control problem is to maximize the unobserved state x(t) using the minimum energy of control u. Let us first construct the controller where the control law and the matrices P(t) and Q(t) are calculated in the same manner as for the optimal linear controller for a linear system without delays in control input, that is u∗ (t) = (R(t))−1 BT (t)Q(t)m(t) (see [67] for reference). Since B(t) = 1 in (4.114) and R(t) = 1 in (4.116), the control law is actually equal to u(t) = Q(t)m(t), (4.117) where m(t) satisfies the equation m(t) ˙ = a0 (t) + a(t)m(t) + B(t)u(t − τ ) + P(t)AT (t)× (F(t)F T (t))−1 (y(t) − (A0 (t) + A(t)m(t − h))). with the initial condition m(s) = E(φ (s)), s ∈ [t0 − τ ,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 ; Q(t) satisfies the Riccati equation ˙ = −aT (t)Q(t) − Q(t)a(t) + L(t) − Q(t)B(t)R−1(t)BT (t)Q(t)), Q(t) with the terminal condition Q(T ) = ψ ; and P(t) satisfies the Riccati equation ˙ = P(t)aT (t) + a(t)P(t) + b(t)bT (t) − P(t)AT (t)(F(t)F T (t))−1 A(t)P(t), P(t) with the initial condition P(t0 ) = E((x(t0 )− m(t0 ))(x(t0 )− m(t0 ))T | y(t0 )). Since t0 = 0, a0 (t) = 0, a(t) = 1, B(t) = 1, b(t) = 0, τ = 0.1 in (12), A0 (t) = 0, A(t) = 1, F(t) = 1, h = 0.2 in (4.115), and L = 0 and ψ = 1 in (4.116), the last equations turn to m(t) ˙ = m(t) + u(t − 0.1) + P(t)(y(t) − m(t − 0.2)), m(s) = 0, s < 0, m(0) = m0 ,

(4.118)

170

4 Optimal Control Problems for Time-Delay Systems

˙ = −2Q(t) − (Q(t))2, Q(t) ˙ = 2P(t) − (P(t)) , P(t) 2

Q(0.45) = 1,

(4.119)

P(0) = P0 .

(4.120)

Upon substituting the control (4.117) into (4.118), the controlled estimate equation takes the form m(t) ˙ = m(t) + Q(t − 0.1)m(t − 0.1) + P(t)(y(t) − m(t − 0.2)), m(s) = 0, s < 0, m(0) = m0 .

(4.121)

For numerical simulation of the system (4.114),(4.115), the initial value x(0) = 0.05 is assigned for realization of the Gaussian variable x(0) = φ (0) in (4.114), the values m0 = 1.9 and P0 = 10 are assigned as the initial conditions of the estimate m(t) and the filter gain P(t), respectively, and the disturbance ψ (t) in (4.115) is realized using the built-in MatLab white noise function. The results of applying the controller (4.117)–(4.121) to the system (4.114),(4.115) are shown in Fig. 4.12, which presents the graphs of the state (4.114) x(t) controlled by (4.117), the controlled estimate (4.121) m(t), the criterion (4.116) J(t), the control (4.117) u(t), the filter gain (4.120) P(t), and the control gain (4.119) Q(t), in the interval [0, T ]. The values of the state (4.114), the estimate (4.121), and the criterion (4.116) at the final moment T = 0.45 are x(0.45) = 4.0064, m(0.45) = 4.7841, and J(0.45) = 236.0656. Let us now apply the optimal controller (4.108)–(4.113) for linear systems with control and observation delays to the system (4.114), (4.115). Since t t T aT (s)ds) = exp (−0.2) and exp ( t− a (s)ds) = exp (0.1), the control law exp (− t−h τ (4.111) takes the form u∗ (t − 0.1) = (exp (−0.1))Q(t)m(t − 0.1),

(4.122)

where m(t) ˙ = m(t) + u∗(t − 0.1) + (exp(−0.2))P(t)(y(t) − m(t − 0.2)), m(s) = 0, s < 0, m(0) = m0 , ˙ = −2Q(t) − (exp(−0.1)Q(t))2 , Q(t) ˙ = 2P(t) − (exp(−0.2)P(t))2 , P(t)

(4.123)

Q(0.45) = 1,

(4.124)

P(0) = P0 .

(4.125)

Upon substituting the control (4.122) into (4.123), the controlled estimate equation takes the form m(t) ˙ = m(t) + (exp(−0.1))Q(t)m(t − 0.1)+ (4.126) (exp (−0.2))P(t)(y(t) − m(t − 0.2)), m(s) = 0, s < 0, m(0) = m0 .

4.5 Optimal Controller Problem for Linear Systems with Input and Observation Delays

171

5

5

4

4

estimate

state

The results of applying the controller (4.108)–(4.113) to the system (4.114), (4.115) are shown in Fig. 4.13, which presents the graphs of the state (4.114) x(t) controlled by (4.122), the controlled estimate (4.126) m(t), the criterion (4.116) J(t), and the control (4.122) u∗ (t), the variance (4.125) P(t), and the control gain (4.124) Q(t), in the interval [0, T ]. The values of the state (4.114), the controlled estimate (4.126), and the criterion (4.116) at the final moment T = 0.45 are x(0.45) = 6.87, m(0.45) = 6.876, and J(0.45) = 215.43. There is a definite improvement in the values of the controlled state and the controlled estimate to be maximized and the criterion to be minimized, in comparison to the preceding case, due to the optimality of the controller (4.108)–(4.113) for linear systems with input and observation delays.

3 2

2

1

1

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

320

20

300

15

275

control

criterion

0

250

6 4 2 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

10

0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time 10 control gain

8

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

5

225 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time 10

filter gain

3

5

1 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

Fig. 4.12. Graphs of the state (4.114) x(t) governed by the best linear controller (4.117) available for systems without delay, the controlled estimate (4.121) m(t), the criterion (4.116) J(t), the control (4.117) u(t), the filter gain (4.120) P(t), and the control gain (4.119) Q(t), in the interval [0, 0.45]

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

50

300

40

275

30

250 225 200

20

0

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

control gain

30

8 6 4

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

10

10 variance

7 6 5 4 3 2 1 0

325

control

criterion

7 6 5 4 3 2 1 0

estimate

4 Optimal Control Problems for Time-Delay Systems

state

172

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

20 10 0

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

Fig. 4.13. Graphs of the state (4.114) x(t) controlled by the optimal linear regulator (4.122) designed for systems with input and observation delays, the controlled estimate (4.126) m(t), the criterion (4.116) J(t), the control (4.122) u∗ (t), the variance (4.125) P(t), and the control gain (4.124) Q(t), in the interval [0, 0.45]

Thus, the designed controller for linear systems with input and observation delays regulates the system variables better than the best linear controller available for systems without delays from both points of view, thus illustrating the theoretical conclusion. Discussion The optimal controller for linear system states with input delay, linear observations with delay confused with white Gaussian noises, and a quadratic criterion has been designed. The optimality of the obtained controller has been proved using the previous results in the optimal filtering over observations with delay and the optimal regulation with delay in control input. Simulation results show that application of the obtained controller has yielded 30% better values of the criterion (with respect to the difference between the initial and final criterion values) and one and half greater values of the controlled

4.5 Optimal Controller Problem for Linear Systems with Input and Observation Delays

173

variable in comparison to the best linear controller available for systems without delays. Moreover, the final value the controlled estimate provided by the designed controller practically coincides with the final value of the controlled state, whereas the controlled estimate given by the best controller available for linear systems without delays is still far from the real state value. Although these conclusions follow from the developed theory, the numerical simulation serves as a convincing illustration.

5 Sliding Mode Applications to Optimal Filtering and Control

5.1 Optimal Robust Sliding Mode Controller for Linear Systems with Input and Observation Delays 5.1.1

Problem Statement

In practical applications, a control system operates under uncertainty conditions that may be generated by parameter variations or external disturbances. Consider a real trajectory of the disturbed control system x(t) ˙ = f (x(t)) + B(t)u + g1(x(t),t) + g2(x(t − τ ),t).

(5.1)

Here, x(t) ∈ Rn is the system state, u(t) ∈ Rm is the control input, f (x(t)) is a known function determining the proper state dynamics, the rank of matrix B(t) is complete and equal to m for any t > t0 , and the pseudoinverse matrix of B is uniformly bounded: B+ (t) ≤ b+ , b+ = const > 0, B+ (t) := [BT (t)B(t)]−1 BT (t), and B+ (t)B(t) = I, where I is the m-dimensional identity matrix. Uncertain inputs g1 and g2 represent smooth disturbances corresponding to perturbations and nonlinearities in the system. For g1 , g2 , the standard matching conditions are assumed to be held: g1 , g2 ∈ spanB, or, in other words, there exist smooth functions γ1 , γ2 such that g1 (x(t),t) = B(t)γ1 (x(t),t),

(5.2)

g2 (x(t − τ ),t) = B(t)γ2 (x(t − τ ),t), ||γ1 (x(t),t)|| ≤ q1 ||x(t)|| + p1, q1 , p1 > 0, ||γ2 (x(t − τ ),t)|| ≤ q2 ||x(t − τ )|| + p2, q2 , p2 > 0. The last two conditions provide reasonable restrictions on the growth of the uncertainties. Let us also consider the nominal control system x˙0 (t) = f (x0 (t)) + B(t)u0(x0 (t − τ ),t), M. Basin: New Trends in Optimal Filtering, LNCIS 380, pp. 175–197, 2008. c Springer-Verlag Berlin Heidelberg 2008 springerlink.com 

(5.3)

176

5 Sliding Mode Applications to Optimal Filtering and Control

where a certain delay-dependent control law u0 (x(t − τ ),t) is realized. The problem is to reproduce the nominal state motion determined by (5.3) in the disturbed control system (5.1). The following initial conditions are assumed for the system (5.3) x(s) = φ (s),

(5.4)

where φ (s) is a piecewise continuous function given in the interval [t0 − τ ,t0 ]. Thus, the control problem now consists in robustification of control design in the nominal system (5.3) with respect to uncertainties g1 , g2 : to find such a control law u = u0 (x(t − τ ),t) + u1 (t) that the disturbed trajectories (5.1) with initial conditions (5.4) coincide with the nominal trajectories (5.3) with the same initial conditions (5.4). 5.1.2

Design Principles for State Disturbance Compensator

Let us design the control law for (5.1) in the form u(t) = u0 (x(t − τ ),t) + u1(t),

(5.5)

where u0 (x(t − τ ),t) is the nominal feedback control designed for (5.3), and u1 (t) ∈ Rm is the relay control generating the integral sliding mode (see [87, 88] for definition and details) in some auxiliary space to reject uncertainties g1 , g2 . Substitution of the control law (5.5) into the system (5.1) yields x(t) ˙ = f (x(t)) + B(t)u0 (x(t − τ ),t) + B(t)u1(t)+

(5.6)

g1 (x(t),t) + g2(x(t − τ ),t). Define the auxiliary function s(t) = z(t) + s0 (x(t),t),

(5.7)

where s0 (x(t),t) = B+ (t)x(t), and z(t) is an auxiliary variable defined below. Then, s(t) ˙ = z˙(t) + G(t)[ f (x(t)) + B(t)u0(x(t − τ ),t)+

(5.8)

B(γ1 (x(t),t)) + γ2 (x(t − τ ),t)) + B(t)u1(t)] + (∂ s0 (x(t),t)/∂ t), G(t) = ∂ s0 (x(t),t)/∂ x = B+ (t) and ∂ s0 (x(t),t)/∂ t = (d(B+ (t))/dt)x(t). Note that in the framework of this integral sliding mode approach, the optimal control u0 (x(t)) is not required to be differentiable and the sliding mode manifold matrix GB = B+ B = I is always invertible. The philosophy of integral sliding mode control is the following: in order to achieve x(t) = x0 (t) at all t ∈ [t0 , ∞), the sliding mode should be organized on the surface s(t), since the following disturbance compensation should have been obtained in the sliding mode motion B+ (t)B(t)u1eq (t) = −B+ (t)B(t)γ1 (x(t),t) − B+(t)B(t)γ2 (x(t − h),t),

5.1 Optimal Robust Sliding Mode Controller for Linear Systems

that is

177

u1eq (t) = −γ1 (x(t),t) − γ2 (x(t − h),t).

Note that the equivalent control u1eq (t) can be unambiguously determined from the last equality and the initial condition for x(t). Define the auxiliary variable z(t) as the solution to the differential equation z˙(t) = −B+ (t)[ f (x(t)) + B(t)u0 (x(t − τ ),t)] + d((B+(t))/dt)x(t), with the initial condition z(0) = −s0 (0) = −B+ (0)ϕ (0). Then, the sliding manifold equation takes the form s(t) ˙ = B+ (t)[B(t)(γ1 (x(t),t)) + γ2 (x(t − τ ),t)) + B(t)u1(t)] = = γ1 (x(t),t) + γ2 (x(t − τ ),t) + u1(t) = 0. Finally, to realize sliding mode, the relay control is designed u1 (t) = −M(x(t), x(t − τ ),t)sign[s(t)],

(5.9)

M = q(||x(t)|| + ||x(t − τ )||) + p, q > q1 , q2 , p > p1 + p2 , where sign[s(t)] = [sign(s1 (t)), . . . , sign(sm (t))] for s(t) ∈ Rm , The convergence to and along the sliding mode manifold s(t) = 0 is assured by the Lyapunov function V (t) = sT (t)s(t)/2 for the system (5.2) with the control input u1 (t) of (5.9): V˙ (t) = sT (t)[γ1 (x(t),t) + γ2 (x(t − h),t) + u1(t)] ≤ −|s(t)|([q(||x(t)|| + ||x(t − τ )||) + p]+ [γ1 (x(t),t) + γ2 (x(t − τ ),t)]) < 0, m

where |s(t)| = ∑ |si (t)|. i=1

5.1.3

Design Principles for Observation Disturbance Compensator

Let the observation process (4.104) be corrupted with unknown disturbances dy(t) = (A0 (t) + A(t)x(t − h))dt + F(t)dW2 (t)+

(5.10)

(k1 (x(t),t) + k2 (x(t − h),t))dt, where k1 (x(t),t) and k2 (x(t − h),t) are non-Gaussian and, possibly, deterministic noises not bearing any useful information and depending on the current and delayed states. Such disturbances obviously deteriorate the quality of estimation and should be eliminated. For this purpose, assume that the disturbances satisfy the following conditions (note that no matching conditions are assumed) ||k1 (x(t),t)|| ≤ q3 ||x(t)|| + p3, q3 , p3 > 0,

178

5 Sliding Mode Applications to Optimal Filtering and Control

||k2 (x(t − h),t)|| ≤ q4 ||x(t − h)|| + p4, q4 , p4 > 0, providing reasonable restrictions on their growth. The observation process (5.10) consists of the useful and parasitic parts, y(t) = yu (t) + y p (t), where dyu (t) = (A0 (t) + A(t)x(t − h))dt + B(t)dW2 (t) and dy p (t) = (g1 (x(t),t) + g2 (x(t − h),t))dt. If only the useful signal yu (t) is present, the optimal filter based on the observations yu (t) yields the desirable estimate mu (t) for the unobserved state x(t). At this point, the problem is to suggest a tuning adjustment y1 (t) that, being added to the actual observations y(t) = yu (t)+ y p (t), compensates for observation disturbances k1 , k2 . The following sliding mode technique solves this problem: define the sliding manifold σ (t) as σ (t) = z(t) + σ0 (t), where σ0 (t) = m(t) and z(t) is an auxiliary variable to be assigned. The condition of motion along the sliding manifold, d σ (t)/dt = 0, yields: d σ (t) = dz(t) + (E( f (x(t)) | FtY0 ) + B(t)u(t − τ ))dt+ C(t)[dy(t) − (A0(t) + A(t)m(t − h))dt + y1 (t)dt] = 0,

(5.11) 

t where y(t) is the disturbed observation process (5.10), C(t) = P(t) exp(− t−h aT (s)ds) AT (t) (F(t)F T (t))−1 is the filter gain matrix, and y1 (t) is the tuning adjustment to observations, whose values on the sliding manifold are denoted by y1eq (t). The value of the tuning adjustment on the sliding manifold must be equal to

y1eq (t) = −(k1 (x(t),t) + k2 (x(t − h),t)), thus compensating for unknown disturbances. In doing so, in view of d σ (t)/dt = dz(t)/dt + d σ0(t)/dt, the auxiliary variable z(t) is assigned by the equation dz(t) = (−E( f (x(t)) | FtY0 ) − B(t)u(t − τ ))dt−

(5.12)

C(t)[dyu (t) − (A0 (t) + A(t)m(t − h))dt], with the initial condition z(0) = −σ0 (0) = −m(0), where yu (t) is an average of some a priori known realizations of the useful signal. Thus, the estimate m(t) based on the disturbed observations with compensator y(t) + y1 (t) and the desired estimate mu (t) coincide in the mean square as random variables on the sliding manifold σ (t) = 0, and the sliding manifold equation takes the form

σ˙ (t) = k1 (x(t),t) + k2(x(t − h),t) + y1eq(t) = 0, thus assuring compensation of the observation disturbances. Finally, to realize sliding mode, the relay compensator control is designed y1 (t) = −M1 (x(t), x(t − h),t))sign[σ (t)],

(5.13)

˜ + ||x(t − h)||) + p, ˜ q˜ > q3 , q4 , p˜ > p3 + p4 . The mean square where M1 = q(||x(t)|| convergence to and along the sliding mode manifold is proved using the same Lyapunov function as in the preceding subsection. The next section presents the robustification of the designed optimal controller (4.108)–(4.113). This robust regulator is designed assigning the sliding mode manifolds according to (5.7),(5.8) and (5.11),(5.12) and subsequently moving to and along this manifold using relay control (5.9),(5.13).

5.1 Optimal Robust Sliding Mode Controller for Linear Systems

5.1.4

179

Robust Sliding Mode Controller Design for Linear System with Input and Observation Delays

Consider the disturbed state equation (4.103), whose behavior is affected by uncertainties g1 , g2 , presenting perturbations and nonlinearities in the system dx(t) = a(t)x(t)dt + B(t)u(t − τ )dt+

(5.14)

b(t)dW1 (t) + g1(x(t),t) + g2(x(t − h),t), with the initial condition x(s) = φ (s), s ∈ [t0 − τ ,t0 ], and the observation equation (4.104), which is now affected by uncertainties k1 , k2 , presenting perturbations and nonlinearities in the observations (4.104) dy(t) = (A0 (t) + A(t)x(t − h))dt + F(t)dW2 (t)+

(5.15)

(k1 (x(t),t) + k2 (x(t − h),t))dt. It is also assumed that the uncertainties satisfy the standard matching and growth conditions (5.2) given in Subsection 5.1.1, and the quadratic cost function is the same as (4.105) in Section 4.5.1. The optimally controlled estimate equation (4.113) for the state (5.14) over the observations (5.15) takes the form dm(t) = (a(t)m(t) + B(t)(R(t)−1BT (t)×  t

exp(−

t−τ

(5.16)

aT (s)ds)Q(t)m(t − τ ) + Bg1(x(t),t)+

Bg2 (x(t − h),t))dt + P(t) exp(−

 t

aT (s)ds)AT (t)×

t−h

(F(t)F T (t))−1 (dy(t) − (A0 (t) + A(t)m(t − h))dt), with the initial condition m(s) = E(φ (s)), s ∈ [t0 − τ ,t0 ) and m(t0 ) = E(φ (t0 ) | FtY0 ), s = t0 . The problem is to robustify the obtained optimal controller (4.108)–(4.113), using the methods specified by (5.7)–(5.9) and (5.11)–(5.13). First, define the new control (5.5) as follows: u(t) = u0 (x(t − τ ),t) + u1 (t), where the optimal control u0 (x(t − τ ),t) coincides with (4.111) and the robustifying component u1 (t) is obtained according to (5.9) u1 (t) = −M(x(t), x(t − τ ),t)sign[s(t)], M = q(||x(t)|| + ||x(t − τ )||) + p, q > q1 , q2 , p > p1 + p2 . Consequently, the sliding mode manifold function s(t) is defined as s(t) = z(t) + s0 (x(t),t), where s0 (m(t),t) = B+ (t)m(t), and the auxiliary variable z(t) satisfies the delay differential equation dz(t) = −B+ (t)[a(t)m(t)dt+

180

5 Sliding Mode Applications to Optimal Filtering and Control

B(t)(R(t)−1 BT (t) exp (−

 t t−τ

aT (s)ds)Q(t)m(t − τ )dt+

C(t)(dy(t) − (A0 (t) + A(t)m(t − h))dt)],



t with the initial condition z(0) = −B+ (0)ϕ (0)), where C(t) = P(t) exp(− t−h aT (s)ds) AT (t) (F(t)F T (t))−1 . Note that the compensator u1 (t) introduced at this step can also compensate for observation disturbances k1 , k2 , if the filter gain matrix C(t) belongs to the matrix B(t) span, i.e., C(t) = B(t)M(t). This fact readily follows from the sliding mode equation (5.8) and the structure of the disturbed controlled estimate equation (5.16). Thus, the introduced control u1 (t) can compensate for state disturbances g1 , g2 and observation disturbances k1 , k2 , if the matching condition C(t) = B(t)M(t) holds. However, the compensator (5.13) should still be applied to compensate for observation disturbances in the unmatched case. For this purpose, define the new observation process y(t) + y1 (t), where y(t) are actual observations, and the robustifying component y1 (t) is obtained according to (5.13)

y1 (t) = −M1 (x(t), x(t − h),t))sign[σ (t)], ˜ + ||x(t − h)||) + p, ˜ q˜ > q3 , q4 , p˜ > p3 + p4 . Consequently, the where M1 = q(||x(t)|| sliding mode manifold function σ (t) is defined as

σ (t) = z(t) + σ0(t), where σ0 (t) = m(t), and the auxiliary variable z(t) satisfies the delay differential equation dz(t) = (−a(t)m(t)− B(t)(R(t)−1 BT (t) exp (−

 t t−τ

aT (s)ds)Q(t)m(t − τ ))dt−

C(t)[dyu (t) − (A0 (t) + A(t)m(t − h))dt], with the initial condition z(0) = −σ0 (0) = −m(0). The undisturbed observations yu (t) could be determined from the nominal system corresponding to (5.14),(5.15), where all disturbances and white noises are absent and the initial condition for (5.14) coincides with m0 . Note that the compensator y1 (t) introduced at this step can also compensate for state disturbances g1 , g2 , if the state disturbances satisfy the matching conditions with the filter gain matrix C(t), i.e., g1 (x(t),t) = C(t)β1 (x(t),t), g2 (x(t − h),t) = C(t)β2 (x(t − h),t). This fact readily follows from the sliding manifold equation (5.11) and the structure of the disturbed controlled estimate equation (5.16). A case of joint compensation of state and observation disturbances using the only observation disturbance compensator (5.13) is presented in the following example.

5.1 Optimal Robust Sliding Mode Controller for Linear Systems

5.1.5

181

Example

This section presents an example of designing the controller for a linear system with input and observation delays using the scheme (4.108)–(4.113), disturbing the obtained controller by noises in state and observation equations, and designing a robust sliding mode observation disturbance compensator for those noises using the scheme (5.11)– (5.13). Let us consider a scalar linear system x(t) ˙ = x(t) + u(t − 0.1),

x(s) = φ (s), s ∈ [−0.1, 0],

(5.17)

where φ (s) = N(0, 1) for s ∈ [−0.1, 0], and N(0, 1) is a Gaussian random variable with zero mean and unit variance. The observation process is given by y(t) = x(t − 0.2) + ψ (t),

(5.18)

where ψ (t) is a white Gaussian noise, which is the weak mean square derivative of a standard Wiener process. The control problem is to find the control u(t), t ∈ [0, T ], that minimizes the criterion 1 1 J = E{ [x(T ) − x∗ ]2 + 2 2

 T

u2 (t)dt},

(5.19)

0

where T = 0.45, and x∗ = 25 is a large value of x(t), which would a priori be unreachable for the optimally controlled system at the time T . In other words, the control problem is to maximize the unobserved state x(t) using the minimum energy of control u. Let us apply the optimal controller (4.108)–(4.113) for linear systems with cont aT (s)ds) = trol and observation delays to the system (5.17),(5.18). Since exp (− t−h t T exp (−0.2) and exp (− t− τ a (s)ds) = exp (−0.1), the control law (4.111) takes the form (5.20) u∗ (t − 0.1) = (exp (−0.1))Q(t)m(t − 0.1), where m(t) ˙ = m(t) + u(t − 0.1) + (exp(−0.2))P(t)(y(t) − m(t − 0.2)),

(5.21)

m(s) = 0, s < 0, m(0) = m0 , ˙ = −2Q(t) − (exp(−0.1)Q(t))2 , Q(t) ˙ = 2P(t) − (exp(−0.2)P(t))2 , P(t)

Q(0.45) = 1,

(5.22)

P(0) = P0 .

(5.23)

Upon substituting the control (5.20) into (5.21), the optimally controlled estimate equation takes the form m(t) ˙ = m(t) + (exp(−0.1))Q(t)m(t − 0.1)+ (exp (−0.2))P(t)(y(t) − m(t − 0.2)), m(s) = 0, s < 0, m(0) = m0 .

(5.24)

182

5 Sliding Mode Applications to Optimal Filtering and Control

For numerical simulation of the system (5.17),(5.18), the initial value x(0) = 0.05 is assigned for realization of the Gaussian variable x(0) = φ (0) in (5.17), the values m0 = 1.9 and P0 = 10 are assigned as the initial conditions of the estimate m(t) and the filter gain P(t), respectively, and the disturbance ψ (t) in (5.18) is realized using the built-in MatLab white noise function. The results of applying the controller (4.108)–(4.113) to the system (5.17),(5.18) are shown in Fig. 4.13 from Section 4.5, which presents the graphs of the state (5.17) x(t) controlled by (5.20), the controlled estimate (5.24) m(t), the criterion (5.19) J(t), the control (5.20) u∗ (t), the variance (5.23) P(t), and the control gain (5.22) Q(t), in the interval [0, T ]. The values of the state (5.17), the controlled estimate (5.24), and the criterion (5.19) at the final moment T = 0.45 are x(0.45) = 6.87, m(0.45) = 6.876, and J(0.45) = 215.43. The next task is to introduce state and observation disturbances into the system (5.17),(5.18). These disturbances are realized as a constant: g(t) = k(t) = 100. The matching conditions are valid, because state x(t), control u(t), and observations y(t) have the same dimension: dim(x) = dim(u) = dim(y) = 1. The restrictions on the disturbance growth hold with q1 = q2 = p2 = q3 = q4 = p4 = 0 and p1 = p3 = 100, since ||g(t)|| = ||k(t)|| = 100. The disturbed controller equation (5.24) takes the form m(t) ˙ = m(t) + (exp(−0.1))Q(t)m(t − 0.1) + 100+

(5.25)

(exp (−0.2))P(t)(y(t) − m(t − 0.2) + 100), m(s) = 0, s < 0, m(0) = m0 . The system behavior significantly deteriorates upon introducing the disturbances. Figure 5.1 presents the graphs of the state (5.17) x(t) controlled by (5.20), the controlled estimate (5.25) m(t), the criterion (5.19) J(t), and the control (5.20) u(t), in the interval [0, T ]. The value of the state (5.17), the controlled estimate (5.25) and the criterion (5.19) at the final moment T = 0.45 are x(0.45) = 255.7, m(0.45) = 511, and J(0.45) = 96660. The deterioration of the criterion value in comparison to that obtained for the optimal controlled estimate (5.24) is more than 300 times. Let us finally design the robust integral sliding mode observation compensator for the introduced disturbances. The new controlled state equation should be m(t) ˙ = m(t) + (exp(−0.1))Q(t)m(t − 0.1) + 100+

(5.26)

(exp (−0.2))P(t)(y(t) − m(t − 0.2) + 100 + y1(t)), m(s) = 0, s < 0, m(0) = m0 , where the compensator y1 (t) is obtained according to (5.13) y1 (t) = −M(x(t), x(t − h),t)sign[σ (t)],

(5.27)

and M = 230.4 > p1 (exp (0.2))( max (P−1 (t))) + p3 . The sliding mode manifold σ (t) t≤0.45

is defined by where σ0 (t) = m(t).

σ (t) = z(t) + σ0(t),

5.1 Optimal Robust Sliding Mode Controller for Linear Systems

183

The auxiliary variable z(t) satisfies the delay differential equation z˙(t) = −m(t) − (exp(−0.1))Q(t)m(t − 0.1)−

(5.28)

(exp (−0.2))P(t)(yu (t) − m(t − 0.2)), with the initial condition z(0) = −m(0) = −1.9, where the undisturbed observations yu (t) are determined from the undisturbed system (5.17),(5.18) x˙u (t) = x(t) + (exp(−0.1))Q(t)xu (t − 0.1), yu (t) = xu (t − 0.2), with the initial condition xu (s) = 0, s < 0, xu (0) = m0 . Upon introducing the compensator (5.27) into the controller equation (5.26), the controlled estimate behavior is very much improved. Figure 5.2 presents the graphs of of the state (5.17) x(t) controlled by (5.20), the controlled estimate (5.26) m(t), the criterion (5.19) J(t), and the control (5.20) u(t), after applying the compensator (5.27), in the interval [0, T ]. The value of the state (5.17), the controlled estimate (5.26) and the criterion (5.19) at the final moment T = 0.45 are x(0.45) = 8.087, m(0.45) = 8.066, and J(0.45) = 215.31. Thus, the values of the criterion and state after applying the compensator (5.27) are even better than those for the controller (5.24), although approximation of the true state by the estimate m(t) is a bit worse. This phenomenon is produced by difference in random realizations of the observation white noise in (5.18) in both cases. Nevertheless, the obtained values of the criterion, state, and controlled variable are quite admissible for representation of the optimally controlled variables (5.20)–(5.24) in the undisturbed problem, taking into account that the maximum over absolute differences between both realizations of the MatLab white noise functions reaches values between 6 and 8 in each interval of the length 0.05. Remark. This example shows that the observation compensator (5.27) summarizes the actions of the state and observation compensators (5.9) and (5.13), if they were designed separately. Indeed, for both those compensators, the sliding mode manifolds coincide: s(t) = σ (t) = m(t) + z(t), where z(t) is defined by (5.28), and the compensators themselves are equal to u1 (t) = −M1 (x(t), x(t − h),t)sign[s(t)], M1 = 100.1 > p1 , and y1 (t) = −M2 (x(t), x(t − h),t)sign[σ (t)], M2 = 100.1 > p3 . Taking into account that the observation compensator y1 (t) enters the equation (5.26) through the filter gain matrix (exp (−0.2))P(t), the proposed value M = 230.4 > p1 (exp (0.2))( max (P−1 (t))) + p3 t≤0.45

can be obtained for the compensator (5.27). Discussion The robust integral sliding mode compensator has been designed for the optimal controller for linear stochastic systems with input and observation delays. The proposed technique can be considered as a universal method for robustifying the optimal filtering and control algorithms in linear stochastic time-delay systems, providing simultaneous suppression of non-Gaussian disturbances in state and observation equations.

5 Sliding Mode Applications to Optimal Filtering and Control

300

600

250

500

200

400

estimate

state

184

150

300

100

200

50

100

0

0

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

4

10

x 10

800

600

6

control

criterion

8

4

200

2 0

400

0

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

9

9

8

8

7

7

6

6

estimate

state

Fig. 5.1. Controlled system in the presence of disturbance. Graphs of the disturbed state (5.17) x(t), the disturbed controlled estimate (5.25) m(t), the disturbed criterion (5.19) J(t), and the disturbed control (5.20) u(t) in the interval [0, 0.45].

5 4

5 4

3

3

2

2

1

1

0

0

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

325

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

70 60

300

control

criterion

50 275 250

40 30 20

225 200

10 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

0

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 time

Fig. 5.2. Controlled system after applying robust integral sliding mode compensator. Graphs of the compensated state (5.17) x(t), the compensated controlled estimate (5.26) m(t), the compensated criterion (5.19) J(t), and the compensated control (5.20) u(t) in the interval [0, 0.45].

5.2 Optimal and Robust Control for Linear State Delay Systems

185

Application of the designed robustification algorithm to robust controller design in the illustrative example has yielded promising results, even improving the criterion value in comparison to the nominal optimal controller. It has also been shown that the compensator, suppressing simultaneously state and observation disturbances, actually summarizes the actions of the state and observation compensators, if they were designed separately.

5.2 Optimal and Robust Control for Linear State Delay Systems 5.2.1

Optimal Control Problem

Consider a linear system with time delay in the state x(t) ˙ = a0 (t) + a(t)x(t − h) + B(t)u(t),

(5.29)

with the initial condition x(s) = ϕ (s), s ∈ [t0 − h,t0 ], where x(t) ∈ Rn is the system state, u(t) ∈ Rm is the control variable, and ϕ (s) is a piecewise continuous function given in the interval [t0 − h,t0 ]. Existence of the unique solution of the equation (1) is thus assured by the Carath´eodory theorem (see, for example, [29]). The quadratic cost function to be minimized is defined as follows: 1 J = [x(T )]T ψ [x(T )]+ 2 1 2

 T t0

uT (s)R(s)u(s)ds +

1 2

 T

xT (s)L(s)x(s)ds,

(5.30)

t0

where R is a positive definite symmetric matrix, ψ and L are nonnegative definite symmetric matrices, and T > t0 is a certain time moment. The optimal control problem is to find the control u∗ (t), t ∈ [t0 , T ], that minimizes the criterion J along with the trajectory x∗ (t), t ∈ [t0 , T ], generated upon substituting u∗ (t) into the state equation (5.29). The solution to the stated optimal control problem is given in the next section and then proved using the maximum principle [67, 44] in Appendix. 5.2.2

Optimal Control Problem Solution

The solution to the optimal control problem for the linear system with state delay (5.29) and the quadratic criterion (5.30) is given as follows. The optimal control law is given by (5.31) u∗ (t) = (R(t))−1 BT (t)Q(t)x(t), where the matrix function Q(t) satisfies the matrix equation ˙ = L(t) − Q(t)M1 (t)a(t) − aT (t)M1T (t)Q(t) − Q(t)B(t)R−1(t)BT (t)Q(t), (5.32) Q(t) with the terminal condition Q(T ) = −ψ . The auxiliary matrix M1 (t) is defined as M1 (t) = (∂ x(t − h)/∂ x(t)), whose value is equal to zero, M1 (t) = 0,

186

5 Sliding Mode Applications to Optimal Filtering and Control

if t ∈ [t0,t0 + h), and is determined as M1 (t) = Φ −1 (t,t − h) = Φ (t − h,t) t B(s)R−1 (s)BT (s)Q(s)ds), if t ≥ t0 + h, where Φ (t, τ ) satisfies the matrix = exp (− t−h equation d Φ (t, τ ) = B(t)R−1 (t)BT (t)Q(t)Φ (t, τ ), dt with the initial condition Φ (t,t) = I, and I is the identity matrix. Upon substituting the optimal control (5.31) into the state equation (5.29), the optimally controlled state equation is obtained x(t) ˙ = a0 (t) + a(t)x(t − h) + B(t)R−1(t)BT (t)Q(t)x(t),

(5.33)

with the initial condition x(s) = ϕ (s), s ∈ [t0 − h,t0]. The results obtained in this section by virtue of the duality principle are proved in Subsection 5.2.5 using the general equations of the Pontryagin maximum principle [67, 44]. It should also be noted that the obtained optimal regulator makes an advance with respect to general optimality results for time delay systems (such as given in [66, 26, 2, 20, 85]), since (a) the optimal control law is given explicitly and not as a solution of a system of integro-differential or PDE equations, and (b) the quasi-Riccati equation (5.32) for the gain matrix does not contain any time advanced arguments and does not depend on the state variables and, therefore, leads to a conventional two points boundary-valued problem generated in the optimal control problems with quadratic criterion and finite horizon (see, for example, [49]). Thus, the obtained optimal regulator is realizable using two delay-differential equations. Taking into account that the state space of a delayed system (5.29) is infinite-dimensional [56], this seems to be a significant advantage. 5.2.3

Robust Control Problem

Consider a nominal control system with state delay, which for generality is assumed to be nonlinear with respect to the state x, x(t) ˙ = f (x(t − h)) + B(t)u(t),

(5.34)

where u(t) ∈ Rm is the control input, the rank of matrix B(t) is complete and equal to m for any t > 0, and the pseudoinverse matrix of B is uniformly bounded: B+ (t) ≤ b+ , b+ = const > 0, B+ (t) := [BT (t)B(t)]−1 BT (t), and B+ (t)B(t) = I, where I is the m-dimensional identity matrix. Suppose that there exists a state feedback control law u0 (x(t),t), such that the dynamics of the nominal closed loop system takes the form x˙0 (t) = f (x0 (t − h)) + B(t)u0(x0 (t),t),

(5.35)

and has certain desired properties. However, in practical applications, system (5.34) operates under uncertainty conditions that may be generated by parameter variations

5.2 Optimal and Robust Control for Linear State Delay Systems

187

and external disturbances. Let us consider the real trajectory of the disturbed closed loop control system x(t) ˙ = f (x(t − h)) + B(t)u(t) + g1(x(t),t) + g2(x(t − h),t),

(5.36)

where g1 , g2 are smooth uncertainties presenting perturbations and nonlinearities in the system (5.34). For g1 , g2 , the standard matching and conditions are assumed to be held: g1 , g2 ∈ spanB, or, in other words, there exist smooth functions γ1 , γ2 such that g1 (x(t),t) = B(t)γ1 (x(t),t), g2 (x(t − h),t) = B(t)γ2 (x(t − h),t), ||γ1 (x(t),t)|| ≤ q1 ||x(t)|| + p1 , q1 , p1 > 0, ||γ2 (x(t − h),t)|| ≤ q2 ||x(t − h)|| + p2, q2 , p2 > 0. The last two conditions provide reasonable restrictions on the growth of the uncertainties. The following initial conditions are assumed for system (5.34) x(θ ) = ϕ (θ ),

(5.37)

where ϕ (θ ) is a piecewise continuous function given in the interval [t0 − h,t0 ]. Thus, the control problem now consists in robustification of the control design in the system (5.35) with respect to uncertainties g1 , g2 : to find such a control law that the trajectories of system (5.36) with initial conditions (5.37) coincide with the trajectories (5.35) x0 (t) with the same initial conditions (5.37). The integral sliding mode technique [88, 17], enabling one to follow the sliding mode manifold from the initial time moment, is first developed for the general nonlinear state-delay system and then specified for the original linear state-delay system (5.29) in the next two subsections. 5.2.4

Design Principles

Let us redesign the control law for system (5.35) in the form u(t) = u0 (x(t),t) + u1 (t),

(5.38)

where u0 (x(t),t) is the ideal feedback control designed for (5.34), and u1 (t) ∈ Rm is the relay control generating the integral sliding mode in some auxiliary space to reject uncertainties g1 , g2 . Substitution of the control law (5.38) into the system (5.34) yields x(t) ˙ = f (x(t − h)) + B(t)u0(x(t),t) + B(t)u1(t) + g1(x(t),t) + g2(x(t − h),t). (5.39) Define the auxiliary function s(t) = z(t) + s0 (x(t),t),

(5.40)

where s0 (x(t),t) = B+ (t)x(t), and z(t) is an auxiliary variable defined below. Then, s(t) ˙ = z˙(t) + G(t)[ f (x(t)) + B(t)u0(x(t),t)+

188

5 Sliding Mode Applications to Optimal Filtering and Control

B(γ1 (x(t),t)) + γ2 (x(t − h),t)) + B(t)u1(t)] + (∂ s0 (x(t),t)/∂ t),

(5.41)

G(t) = ∂ s0 (x(t),t)/∂ x = B+ (t) and ∂ s0 (x(t),t)/∂ t = d(B+ (t))/dt)x(t). Note that in the framework of this modified (with respect to [88, 10]) integral sliding mode approach, the optimal control u0 (x(t)) is not required to be differentiable and the sliding mode manifold matrix GB = B+ B = I is always invertible. The philosophy of integral sliding mode control is the following: in order to achieve x(t) = x0 (t) at all t ∈ [t0 , ∞), the sliding mode should be organized on the surface s(t) = 0, since the following disturbance compensation should have been obtained in the sliding mode motion B+ (t)B(t)u1eq (t) = −B+ (t)B(t)γ1 (x(t),t) − B+ (t)B(t)γ2 (x(t − h),t), that is u1eq (t) = −γ1 (x(t),t) − γ2 (x(t − h),t). Note that the equivalent control u1eq (t) can be unambiguously determined from the last equality and the initial condition for x(t). Define the auxiliary variable z(t) as the solution to the differential equation z˙(t) = −B+ (t)[ f (x(t − h)) + B(t)u0(x(t),t)] + d(B+(t))/dt)x(t), with the initial conditions z(t0 ) = −s0 (t0 ) = −B+ (t0 )ϕ (t0 ). Then, the sliding manifold equation takes the form s(t) ˙ = B+ (t)[B(t)(γ1 (x(t),t)) + γ2 (x(t − h),t)) + B(t)u1(t)] = = γ1 (x(t),t) + γ2 (x(t − h),t) + u1(t) = 0. Finally, to realize sliding mode, the relay control is designed u1 (t) = −M(x(t), x(t − h),t)sign[s(t)],

(5.42)

M = q(||x(t)|| + ||x(t − h)||) + p, q > q1 , q2 , p > p1 + p2 . The convergence to and along the sliding mode manifold s(t) = 0 is assured by the Lyapunov function V (t) = sT (t)s(t)/2 for the system (5.39) with the control input u1 (t) of (5.42): V˙ (t) = sT (t)[γ1 (x(t),t) + γ2 (x(t − h),t) + u1(t)] ≤ −|s(t)|([q(||x(t)|| + ||x(t − h)||) + p] + [γ1(x(t),t) + γ2 (x(t − h),t)]) < 0, m

where |s(t)| = ∑ |si (t)|. i=1

The next subsection presents the robustification of the designed optimal control (5.31). This robust regulator is designed assigning the sliding mode manifold according to (5.40),(5.41) and subsequently moving to and along this manifold using relay control (5.42).

5.2 Optimal and Robust Control for Linear State Delay Systems

5.2.5

189

Robust Sliding Mode Control Design for Linear State Delay Systems

Returning to the original particular linear case, consider the disturbed linear state delay system (5.29), whose behavior is affected by uncertainties g1 , g2 presenting perturbations and nonlinearities in the system x(t) ˙ = a0 (t) + a(t)x(t − h) + B(t)u(t) + g1(x(t),t) + g2(x(t − h),t).

(5.43)

It is also assumed that the uncertainties satisfy the standard matching and growth conditions g1 (x(t),t) = B(t)γ1 (x(t),t), g2 (x(t − h),t) = B(t)γ2 (x(t − h),t), ||γ1 (x(t),t)|| ≤ q1 ||x(t)|| + p1 , q1 , p1 > 0, ||γ2 (x(t − h),t)|| ≤ q2 ||x(t − h)||, q2 , p2 > 0. The quadratic cost function (5.30) is the same as in Section 5.2.1. The problem is to robustify the obtained optimal control (5.31), using the method specified by (5.40),(5.41). Define this new control in the form (5.38): u(t) = u0 (x(t),t) + u1 (t), where the optimal control u0 (x(t),t) coincides with (5.31) and the robustifying component u1 (t) is obtained according to (5.42) u1 (t) = −M(x(t), x(t − h),t)sign[s(t)], M = q(||x(t)|| + ||x(t − h)||) + p, q > q1 , q2 , p > p1 + p2 . Consequently, the sliding mode manifold function s(t) is defined as s(t) = z(t) + s0 (x(t),t), (5.44) where s0 (x(t),t) = B+ (t)x(t),

(5.45)

and the auxiliary variable z(t) satisfies the delay differential equation z˙(t) = −B+ (t)[a0 (t) + a(t)x(t − h) + B(t)u0(x(t),t)],

(5.46)

with the initial conditions z(t0 ) = −B+ (t0 )ϕ (t0 ). 5.2.6

Example

This section presents an example of designing the optimal regulator for a system (5.29) with a criterion (5.30), using the scheme (5.31)–(5.33), and comparing it to the regulator where the matrix Q is selected as in the optimal linear regulator for a system without delays, disturbing the obtained regulator by a noise, and designing a robust sliding mode compensator for that disturbance, using the scheme (5.44)–(5.46). Consider a scalar linear system x(t) ˙ = 10x(t − 0.25) + u(t),

(5.47)

190

5 Sliding Mode Applications to Optimal Filtering and Control

with the initial conditions x(s) = 1 for s ∈ [−0.1, 0]. The control problem is to find the control u(t), t ∈ [0, T ], T = 0.5, that minimizes the criterion  T

1 J= [ 2

u2 (t)dt +

0

 T

x2 (t)dt].

(5.48)

0

In other words, the control problem is to minimize the overall energy of the state x using the minimal overall energy of control u. Since the initial criterion value is zero, the criterion is quadratic, and the state initial condition is positive, the criterion value would necessarily increase: in the part of the state, if the control is small, or in the part of control, if the control is large. Thus, it is required to find such a balanced value of the control input that the total system energy, i.e., state energy plus control one, would increase in the interval [0, T ] as minimally as possible. Note that it is not assumed to maintain the state at a given point, such as x = 0; both, the state and absolute control values, are permitted to increase while keeping the total system energy at the minimal possible level. Let us first construct the regulator where the control law and the matrix Q(t) are calculated in the same manner as for the optimal linear regulator for a linear system without delays, that is u(t) = R−1 (t)BT (t)Q(t)x(t) (see [49] for reference). Since B(t) = 1 in (5.47) and R(t) = 1 in (5.48), the optimal control is actually equal to u(t) = Q(t)x(t),

(5.49)

where Q(t) satisfies the Riccati equation ˙ = −aT (t)Q(t) − Q(t)a(t) + L(t) − Q(t)B(t)R−1(t)BT (t)Q(t), Q(t) with the terminal condition Q(T ) = −ψ . Since a(t) = 10, B(t) = 1 in (5.47), and L(t) = 1 and ψ = 0 in (5.48), the last equation turns to ˙ = 1 − 20Q(t) − Q2(t), Q(t)

Q(0.5) = 0.

(5.50)

Upon substituting the control (5.49) into (5.47), the controlled system takes the form x(t) ˙ = 10x(t − 0.25) + Q(t)x(t).

(5.51)

The results of applying the regulator (5.49)–(5.51) to the system (5.47) are shown in Fig. 5.3, which presents the graphs of the criterion (5.48) J(t) and the control (5.49) u(t) in the interval [0, T ]. The value of criterion (5.48) at the final moment T = 0.5 is J(0.5) = 15.94. Let us now apply the optimal regulator (5.31)–(5.33) for linear states with time delay to the system (5.47). The control law (5.31) takes the same form as (5.49) u∗ (t) = Q∗ (t)x(t),

(5.52)

where Q∗ (t) satisfies the equation Q˙ ∗ (t) = 1 − 20Q∗(t)M1 (t) − Q∗2 (t),

Q∗ (0.5) = 0,

(5.53)

5.2 Optimal and Robust Control for Linear State Delay Systems

191



t where M1 (t) = 0 for t ∈ [0, 0.25) and M1 (t) = exp (− t−0.25 Q∗ (s)ds) for t ∈ [0.25, 0.5]. ∗ Since the solution Q (t) of the equation (5.53) is not smooth, it has been numerically solved with the approximating terminal condition Q∗ (0.5) = 0.04, in order to avoid chattering. Upon substituting the control (5.52) into (5.47), the optimally controlled system takes the same form as (5.51)

x(t) ˙ = 10x(t − 0.25) + Q∗(t)x(t).

(5.54)

The results of applying the regulator (5.52)–(5.54) to the system (5.47) are shown in Fig. 5.4, which presents the graphs of the criterion (5.48) J(t) and the control (5.52) u∗ (t) in the interval [0, T ]. The value of the criterion (5.48) at the final moment T = 0.5 is J(0.5) = 4.63. There is a definitive improvement (three and half times) in the values of the criterion to be minimized in comparison to the preceding case, due to the optimality of the regulator (5.52)–(5.54) for linear states with time delay. Let us also compare the optimal regulator (5.52)–(5.54) to the best linear regulators based on linear approximation of the original time-delay system (5.47). The input-state transfer function of (5.47), G(s) = (s − 10 exp(−sh))−1 , h = 0.25, is approximated by a rational function up to the first order of h: G−1 (s) = s(1 + 10h) − 10 + O(h2) (h = 0.25) x(t) ˙ = (20/7)x(t) + (2/7)u(t),

(5.55)

with the initial condition x(0) = 1. The control law is calculated as the optimal control for the linear system without delays (5.55): u1 (t) = (2/7)Q1 (t)x(t),

(5.56)

and Q1 (t) satisfies the Riccati equation Q˙ 1 (t) = 1 − (40/7)Q1(t) − (2/7)2Q21 (t),

(5.57)

with the terminal condition Q1 (0.5) = 0. The control (5.56) is then substituted into the original time-delay system (5.47). The results of applying the regulator (5.55)-(5.57) to the system (5.47) are shown in Fig. 5.5, which presents the graphs of the criterion (5.48) J(t) and the control (5.56) u1 (t) in the interval [0, T ]. The value of the criterion (5.48) at the final moment T = 0.5 is J(0.5) = 9.77. Thus, the simulation results show that application of the regulator (5.55)-(5.57), based on the first-order approximation, yields still unsatisfactory values of the cost function in comparison to the optimal regulator (5.52)-(5.54). The next task is to introduce a disturbance into the controlled system (5.54). This deterministic disturbance is realized as a constant: g(t) = 100. The matching conditions are valid, because state x(t) and control u(t) have the same dimension: dim(x) = dim(u) = 1. The restrictions on the disturbance growth hold with q1 = q2 = p2 = 0 and p1 = 100, since ||g(t)|| = 100. The disturbed system equation (5.54) takes the form x(t) ˙ = 100 + 10x(t − 0.25) + Q∗(t)x(t).

(5.58)

The system state behavior significantly deteriorates upon introducing the disturbance. Figure 5.6 presents the graphs of the criterion (5.48) J(t) and the control (5.52)

192

5 Sliding Mode Applications to Optimal Filtering and Control

u(t) in the interval [0, T ]. The value of the the criterion (5.48) at the final moment T = 0.5 is J(0.5) = 398.68. The deterioration of the criterion value in comparison to that obtained using the optimal regulator (5.52) is more than 80 times. Let us finally design the robust integral sliding mode control compensating for the introduced disturbance. The new controlled state equation should be x(t) ˙ = 100 + 10x(t − 0.25) + Q∗(t)x(t) + u1 (t),

(5.59)

where the compensator u1 (t) is obtained according to (5.52) u1 (t) = −M(x(t), x(t − h),t)sign[s(t)],

(5.60)

and M = 100.4 > p1 = 100. The sliding mode manifold s(t) is defined by (5.40) s(t) = z(t) + s0 (x(t),t), where s0 (x(t),t) = B+ (t)x(t) = x(t), and the auxiliary variable z(t) satisfies the delay differential equation z˙(t) = −B+ (t)[10x(t − 0.25) + u0(t)] = −[10x(t − 0.25) + Q∗(t)x(t)], with the initial conditions z(0) = −x(0) = −1. Upon introducing the compensator (5.60) into the state equation (5.59), the system state behavior is much improved. Figure 5.7 presents the graphs of the criterion (5.48) J(t) and the control (5.52) u(t), after applying the compensator (5.60), in the interval [0, T ]. The value of the criterion (5.48) at the final moment T = 0.5 is J(0.5) = 4.64. Thus, the criterion value after applying the compensator (5.60) is only slightly different from the criterion value given by the optimal regulator (5.52)–(5.54) for linear state delay systems. 5.2.7

Proof of Optimal Control Problem Solution

Define the Hamiltonian function [67, 44] for the optimal control problem (5.29),(5.30) as 1 (5.61) H(x, u, q,t) = (uT R(t)u + xT L(t)x) + qT [a0 (t) + a(t)x1 + B(t)u], 2 where x1 (x) = x(t − h). Applying the maximum principle condition ∂ H/∂ u = 0 to this specific Hamiltonian function (5.61) yields

∂ H/∂ u = 0 ⇒ R(t)u(t) + BT (t)q(t) = 0, and the optimal control law is obtained as u∗ (t) = −R−1 (t)BT (t)q(t). Taking linearity and causality of the problem into account, let us seek q(t) as a linear function in x(t) q(t) = −Q(t)x(t), (5.62)

5.2 Optimal and Robust Control for Linear State Delay Systems

193

18 16

criterion

14 12 10 8 6 4 2 0

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

5

control

0 −5 −10 −15 −20

Fig. 5.3. Best linear regulator available for linear systems without state delay. Graphs of the criterion (5.48) J(t) and the control (5.49) u(t) in the interval [0, 0.5].

where Q(t) is a square symmetric matrix of dimension n. This yields the complete form of the optimal control u∗ (t) = R−1 (t)BT (t)Q(t)x(t). (5.63) Note that the transversality condition [67, 44] for q(T ) implies that q(T ) = ∂ J/∂ x(T ) = ψ x(T ) and, therefore, Q(T ) = −ψ . Using the co-state equation dq(t)/dt = −∂ H/∂ x and denoting (∂ x1 (t)/∂ x) = M1 (t) yields (5.64) −dq(t)/dt = L(t)x(t) + aT (t)M1T (t)q(t), and substituting (5.62) into (5.64), we obtain ˙ Q(t)x(t) + Q(t)d(x(t))/dt = L(t)x(t) − aT (t)M1T (t)Q(t)x(t).

(5.65)

Substituting the expression for x(t) ˙ from the state equation (5.29) into (5.65) yields ˙ Q(t)x(t) + Q(t)a(t)x(t − h) + Q(t)B(t)u(t) = L(t)x(t) − aT (t)M1T (t)Q(t)x(t). (5.66) In view of linearity of the problem, differentiating the last expression in x does not imply loss of generality. Upon substituting the optimal control law (5.63) into (5.66), taking into account that (∂ x(t − h)/∂ x(t)) = M1 (t), and differentiating the equation (5.66) in x, it is transformed into the quasi-Riccati equation ˙ = L(t) − Q(t)M1 (t)a(t) − aT (t)M1T (t)Q(t) − Q(t)B(t)R−1(t)BT (t)Q(t). (5.67) Q(t) with the terminal condition Q(T ) = −ψ .

194

5 Sliding Mode Applications to Optimal Filtering and Control

5

criterion

4 3 2 1 0

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0.5 0

control

−0.5 −1 −1.5 −2 −2.5 −3

Fig. 5.4. Optimal regulator obtained for linear systems with state delay. Graphs of the criterion (5.48) J(t), and the optimal control (5.52) u∗ (t) in the interval [0, 0.5].

Let us now obtain the value of M1 (t). By definition, M1 (t) = (∂ x(t − h)/∂ x(t)). Substituting the optimal control law (5.63) into the equation (5.29) gives x(t) ˙ = a0 (t) + a(t)x(t − h) + B(t)R−1(t)BT (t)Q(t)x(t),

(5.68)

with the initial condition x(s) = φ (s), s ∈ [t0 − h,t0 ]. Integrating (5.68) yields x(t0 + h) = x(t0 ) +

 t0 +h t0

(a0 (s) + a(s)x(s − h))ds +

 t0 +h

B(s)R−1 (s)BT (s)Q(s)x(s)ds.

t0

(5.69) Analysis of the formula (5.69) shows that x(t) does not depend on x(t − h), if t ∈ [t0 ,t0 + h). Therefore, M1 (t) = 0 for t ∈ [t0 ,t0 + h). On the other hand, if t ≥ t0 + h, the following Cauchy formula is valid for the solution x(t) of the equation (5.68) x(t) = Φ (t,t − h)x(t − h) +

t t−h

Φ (t, s)(a0 (s) + a(s)x(s − h))ds,

(5.70)

where Φ (t, τ ) satisfies the matrix equation d Φ (t, τ ) = B(t)R−1 (t)BT (t)Q(t)Φ (t, τ ), dt with the initial condition Φ (t,t) = I, and I is the identity matrix. The expression (5.70) t B(s)R−1 (s)BT (s)Q(s)ds) implies that M1 (t) = Φ −1 (t,t − h) = Φ (t − h,t) = exp (− t−h for t ≥ t0 + h. The optimal control problem solution is proved.

5.2 Optimal and Robust Control for Linear State Delay Systems

195

10 9 8 7

criterion

6 5 4 3 2 1 0

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.3

0.35

0.4

0.45

0.5

time

0.2 0.1 0 −0.1

control

−0.2 −0.3 −0.4 −0.5 −0.6 −0.7 −0.8

0

0.05

0.1

0.15

0.2

0.25 time

Fig. 5.5. Best linear regulator based on the first-order approximation of the transfer function of the original time-delay system. Graphs of the criterion (5.48) J(t), and the control (5.56) u1 (t) in the interval [0, 0.5].

Discussion The optimal regulator for linear system with state delay and a quadratic cost function has been designed in a closed form. It is represented as a real-time feedback control whose gain matrix satisfies a quasi-Riccati equation without time advanced arguments, which provides a significant advantage with respect to previously obtained results in the area of optimal control for time delay systems. A robustifying control for the obtained optimal regulator has then been designed based on the integral sliding mode technique. The integral sliding mode compensator is realized as a relay control in a such way that the sliding mode motion starts from the initial moment, thus eliminating the matched uncertainties from the beginning of system functioning. This constitutes the crucial advantage of the integral sliding modes in comparison to the conventional ones. Performance of the optimal regulator for linear systems with state delay has been verified in the illustrative example against the best linear regulators available for the system

196

5 Sliding Mode Applications to Optimal Filtering and Control

400

criterion

300

200

100

0

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

5 0

control

−5 −10 −15 −20 −25

Fig. 5.6. Controlled system in the presence of disturbance. Graphs of the criterion (5.48) J(t) and the control (5.52) u(t) in the interval [0, 0.5].

5

criterion

4 3 2 1 0

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0

0.05

0.1

0.15

0.2

0.25 time

0.3

0.35

0.4

0.45

0.5

0.5 0

control

−0.5 −1 −1.5 −2 −2.5 −3

Fig. 5.7. Controlled system after applying robust integral sliding mode compensator. Graphs of the criterion (5.48) J(t) and the control (5.52) u(t) in the interval [0, 0.5].

5.2 Optimal and Robust Control for Linear State Delay Systems

197

without delay and the first-order approximation of the original state-delay system. The simulation results show a definitive improvement in the values of the criterion in favor of the designed regulator. Subsequent introduction of disturbances significantly affects system behavior in the example: the criterion value to be minimized increases more than 80 times. However, upon applying the robust integral sliding mode compensator, the system behavior is much improved: the criterion value after applying the compensator is insignificantly different from the criterion value given by the optimal regulator. Thus, it can be concluded that the designed optimal regulator and robust integral sliding mode compensator provide together the optimal control technique for linear state delay systems, which is also resistible to influence of external disturbances.

References

1. Albrekht, E.G.: On the optimal stabilization of nonlinear systems. J. Appl. Math. Mech. 25, 1254–1266 (1962) 2. Alekal, Y., Brunovsky, P., Chyung, D.H., Lee, E.B.: The quadratic problem for systems with time delays. IEEE Trans. Automat. Contr. 16, 673–687 (1971) 3. Alessandri, A.: Design of sliding mode observers and filters for nonlinear dynamic systems. In: Proc. 39th IEEE Conference on Decision and Control, pp. 2593–2598 (2000) 4. Alexander, H.L.: State estimation for distributed systems with sensing delay. SPIE Data Structures and Target Classification, 1470 (1991) 5. Azemi, A., Yaz, E.: Sliding mode adaptive observer approach to chaotic synchronization. ASME Transactions J. Dynamic Systems, Measurements and Control 122, 758–765 (2000) ˚ om, K.J.: Introduction to Stochastic Control Theory. Academic Press, New York (1970) 6. Astr¨ 7. Bar–Shalom, Y.: Optimal simultaneous state estimation and parameter identification in linear discrete-time systems. IEEE Transactions on Automatic Control 17, 308–319 (1972) 8. Basin, M.V., Alcorta-Garcia, M.A.: Optimal control for third degree polynomial systems and its automotive application. In: Proc. 41st IEEE Conference on Decision and Control, pp. 1745–1750 (2002) 9. Basin, M.V., Fridman, L.M., Skliar, M.: Optimal and robust sliding mode filter for systems with continuous and delayed measurements. In: Proc. 41st Conference on Decision and Control, pp. 2594–2599 (2002) 10. Basin, M.V., Fridman, L.M., Acosta, P., Rodriguez-Gonzalez, J.: Optimal and robust sliding mode control for linear systems with multiple time delays in control input. Asian J. Control 5, 557–567 (2003) 11. Basin, M.V., Rodriguez-Gonzalez, J.G., Martinez-Zuniga, R.: Optimal control for linear systems with time delay in control input. Journal of The Franklin Institute 341, 267–278 (2004) 12. Basin, M.V., Fridman, L.M., Rodriguez-Gonzalez, J., Acosta, P.: Integral sliding mode design for robust filtering and control of linear stochastic time-delay systems. Int. J. Robust Nonlinear Control 15, 407–421 (2005) 13. Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory. SIAM, Philadelphia (1998) 14. Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957) 15. Benes, V.E.: Exact finite-dimensional filters for certain diffusions with nonlinear drift. Stochastics 5, 65–92 (1981) 16. Boukas, E.K., Liu, Z.K.: Deterministic and Stochastic Time–Delayed Systems. Birkhauser, Boston (2002)

200

References

17. Cao, W.J., Xu, J.X.: Nonlinear integral type sliding surfaces for both matched and unmatched uncertain systems. In: Proc. American Control Conference, pp. 4369–4374 (2001) 18. Charalambous, C.D., Logothetis, A.: Maximum likelihood parameter estimation from incomplete data via the sensitivity equations: The continuous-time case. IEEE Transactions on Automatic Control 45, 928–934 (2000) 19. Chen, C.T.: Linear System Theory and Design. Holt, Rinehart and Winston, New York (1984) 20. Delfour, M.C.: The linear quadratic control problem with delays in space and control variables: a state space approach. SIAM J. Contr. Optim. 24, 835–883 (1986) 21. Dion, J.M.: Linear Time Delay Systems. Pergamon, London (1999) 22. Drakunov, S.: On adaptive quasioptimal filter with discontinuous parameters. Automation and Remote Control 44, 1167–1175 (1983) 23. Duncan, T.E., Mandl, P., Pasik–Duncan, B.: A note on sampling and parameter estimation in linear stochastic systems. IEEE Transactions on Automatic Control 44, 2120–2125 (1999) 24. Dugard, J.L., Verriest, E.I.: Stability and Control of Time-Delay Systems. Springer, New York (1998) 25. Edwards, C., Spurgeon, S.: On the development of discontinuous observers. Int. J. Control 59, 1211–1229 (1994) 26. Eller, D.H., Aggarwal, J.K., Banks, H.T.: Optimal control of linear time-delay systems. IEEE Trans. Aut. Contr. 14, 678–687 (1969) 27. Elliott, L.J., Krishnamurthy, V.: New finite–dimensional filters for estimation of continuous–time linear gaussian systems. SIAM Journal on Control and Optimization 35, 1908–1923 (1997) 28. Elliott, L.J., Krishnamurthy, V.: New finite–dimensional filters for parameter estimation of discrete–time linear gaussian models. IEEE Transactions on Automatic Control 44, 938– 951 (1999) 29. Filippov, A.F.: Differential Equations with Discontinuous Right-Hand Sides. Kluwer, Dordrecht (1989) 30. Fleming, W.H., Rishel, R.W.: Deterministic and Stochastic Optimal Control. Springer, New York (1975) 31. Fridman, L., Levant, A.: Higher Order Sliding Modes. In: Barbot, J.P., Perruquetti, W. (eds.) Sliding Mode Control in Engineering, pp. 53–102. Marcel Dekker, New York (2002) 32. Gantmacher, F.R.: Lectures in Analytical Mechanics, Mir, Moscow (1975) 33. Gao, H., Lam, J., Xie, L., Wang, C.: New approach to mixed H2 /H∞ –filltering for polytopic discrete–time systems. IEEE Transactions on Signal Processing 53, 3183–3192 (2005) 34. Gelb, A.: Applied Optimal Estimation. MIT Press, Cambridge (1974) 35. Gu, K., Niculescu, S.I.: Survey on recent results in the stability and control of time–delay systems. ASME Transactions J. Dyn. Syst. Measur. Contr. 125, 158–165 (2003) 36. Gupta, P., Kumar, P.: A system and traffic dependent adaptive routing algorithm for ad hoc networks. In: Proc. 36th Conference on Decision and Control, pp. 2375–2380 (1997) 37. Haime, A., Hamalainen, R.: On the nonlinear regulator problem. J. Opt. Theory and Appl. 16, 3–4 (1975) 38. Hale, J.K., Verduyn–Lunel, S.M.: Introduction to Functional Differential Equations. Springer, New York (1993) 39. Hazewinkel, M., Marcus, S.I., Sussmann, H.J.: Nonexistence of exact finite–dimensional filters for conditional statistics of the cubic sensor problem. Systems and Control Letters 5, 331–340 (1983) 40. Hsiao, F.H., Pan, S.T.: Robust Kalman filter synthesis for uncertain multiple time–delay stochastic systems. ASME Transactions J. of Dynamic Systems, Measurement, and Control 118, 803–807 (1996)

References

201

41. Jazwinski, A.H.: Stochastic Processes and Filtering Theory. Academic Press, New York (1970) 42. Kalman, R.E., Bucy, R.S.: New results in linear filtering and prediction. Theory ASME Trans., Part D 83, 95–108 (1961) 43. Kaszkurewicz, E., Bhaya, A.: Discrete–time state estimation with two counters and measurement delay. In: Proc. 35th IEEE Conference on Decision and Control, pp. 1472–1476 (1996) 44. Kharatashvili, G.L.: A maximum principle in external problems with delays. In: Balakrishnan, A.V., Neustadt, L.W. (eds.) Mathematical Theory of Control. Academic Press, New York (1967) 45. Koivo, H.N.: Least-squares estimator for hereditary systems with time–varying delay. IEEE Transactions on Systems, Man, and Cybernetics 4, 275–283 (1974) 46. Kolmanovskii, V.B., Shaikhet, L.E.: Control of Systems with Aftereffect. American Mathematical Society, Providence (1996) 47. Kolmanovskii, V.B., Myshkis, A.D.: Introduction to the Theory and Applications of Functional Differential Equations. Kluwer, New York (1999) 48. Kushner, H.J.: On differential equations satisfied by conditional probability densities of Markov processes. SIAM J. Control 12, 106–119 (1964) 49. Kwakernaak, H., Sivan, R.: Linear Optimal Control Systems. Wiley-Interscience, New York (1972) 50. Kwakernaak, H.: Optimal filtering in systems with time delay. IEEE Trans. Automat. Contr. 19, 169–173 (1974) 51. Lee, E.B., Marcus, L.: Foundations of the Optimal Control Theory. Wiley, New York (1967) 52. Larsen, T.D., Andersen, N.A., Ravn, O., Poulsen, N.K.: Incorporation of the time–delayed measurements in a discrete–time Kalman filter. In: Proc. 37th IEEE Conference on Decision and Control, pp. 3972–3977 (1998) 53. Lukes, D.L.: Optimal regulation of nonlinear dynamic systems. SIAM J. Control Opt. 7, 75–100 (1969) 54. Mahmoud, M.S.: Robust Control and Filtering for Time-Delay Systems. Marcel Dekker, New York (2000) 55. Mahmoud, M., Shi, P.: Robust Kalman filtering for continuous time–lag systems with Markovian jump parameters. IEEE Transactions on Circuits and Systems 50, 98–105 (2003) 56. Malek-Zavarei, M., Jamshidi, M.: Time-Delay Systems: Analysis, Optimization and Applications. North-Holland, Amsterdam (1987) 57. Manitius, A.: Optimum control of linear time–lag processes with quadratic performance indexes. In: Proc. 4th IFAC Congress (1969) 58. Mirkin, L., Raskin, N.: Every stabilizing dead–time controller has an observer–predictor– based structure. Automatica 39, 1747–1754 (2003) 59. Misawa, E.A.: Nonlinear state estimation using sliding observers. PhD Thesis. MIT Press, Cambridge (1989) 60. Murray, R., Sastry, S.: Nonholonomic motion planning: steering using sinusoids. IEEE Trans. Automat. Control 38, 700–716 (1993) 61. Mitter, S.K.: Filtering and stochastic control: a historical perspective. IEEE Control Systems Magazine 16, 67–76 (1996) 62. Niculescu, S.I.: Delay Effects on Stability: A Robust Control Approach. Springer, Heidelberg (2001) 63. Niu, Y., Ho, D.W.C., Lam, J.: Robust integral sliding mode control for uncertain stochastic systems with time-varying delay. Automatica 41, 873–880 (2005) 64. Ogunnaike, B.A.: On–line modelling and predictive control of an industrial terpolymerization reactor. Int. J. Control 59, 711–729 (1994)

202

References

65. Oguztoreli, M.N.: A time optimal control problem for systems described by differential difference equations. SIAM J. Control 1, 290–310 (1963) 66. Oguztoreli, M.N.: Time–Lag Control Systems. Academic Press, New York (1966) 67. Pontryagin, L.S., Boltyanskii, V.G., Gamkrelidze, R.V., Mishchenko, E.F.: The Mathematical Theory of Optimal Processes. Interscience, New York (1962) 68. Poznyak, A.S.: Stochastic output noise effects in sliding mode state estimation. International Journal of Control 76, 986–999 (2003) 69. Pugachev, V.S.: Probability Theory and Mathematical Statistics for Engineers. Pergamon, London (1984) 70. Pugachev, V.S., Sinitsyn, I.N.: Stochastic Systems: Theory and Applications. World Scientific, Singapore (2001) 71. Rao, C.R.: Linear Statistical Inference and its Applications. Wiley-Interscience, New York (1973) 72. Richard, J.P.: Time–delay systems: an overview of some recent advances and open problems. Automatica 39, 1667–1694 (2003) 73. Sain, M.K.: Applications of tensors to modeling and control. Control Systems Technical Report 38, Department of Electrical Engineering. Notre Dame University (1985) 74. Sheng, J., Chen, T., Shah, S.L.: Optimal filtering for multirate systems. IEEE Transactions on Circuits and Systems 52, 228–232 (2005) 75. Sheng, J.: Optimal filtering for multirate systems based on lifted models. In: Proc. American Control Conference, pp. 3459–3461 (2005) 76. Shi, P.: Filtering on sampled–data systems with parametric uncertainty. IEEE Transactions on Automatic Control 43, 1022–1027 (1998) 77. Shi, P., Mahmoud, M., Nguang, S., Ismail, A.: Robust filtering for jumping systems with mode–dependent delays. Signal Processing 86, 140–152 (2006) 78. Shi, P., Boukas, E., Agarwal, R.K.: Control of markovian jump discrete–time systems with norm-bounded uncertainty and unknown delay. IEEE Transactions on Automatic Control 44, 2139–2144 (1999) 79. Shi, P., Xia, Y., Liu, G.P., Rees, D.: On designing of sliding mode control for stochastic jump systems. IEEE Transactions on Automatic Control 51, 97–103 (2006) 80. Shtessel, Y.B., Zinober, A.S.I., Shkolnikov, I.: Sliding mode control for nonlinear systems with output delay via method of stable system center. ASME Trans. J. Dyn. Syst. Measur. Contr. 125, 253–257 (2003) 81. Slotine, J.J., Hedrick, J.K.: On sliding observers for nonlinear systems ASME Transactions. J. Dynamic Systems, Measurement, and Control 109, 245–252 (1987) 82. Smith, O.J.M.: Feedback Control Systems. McGraw Hill, New York (1958) 83. Souza de, C.E., Palhares, R.M., Peres, P.L.D.: Robust H∞ –filtering design for uncertain linear systems with multiple time–varying state delays. IEEE Trans. Signal Processing 49, 569–576 (2001) 84. Theodor, Y., Shaked, U.: Robust discrete–time minimum variance filtering. IEEE Trans. Signal Processing 44, 181–189 (1996) 85. Uchida, K., Shimemura, E., Kubo, T., Abe, N.: The linear–quadratic optimal control approach to feedback control design for systems with delay. Automatica 24, 773–780 (1988) 86. Utkin, V.I.: Sliding Modes in Control and Optimization. Springer, New York (1992) 87. Utkin, V.I., Shi, L.: Integral sliding mode in systems operating under uncertainty conditions. In: Proc. 35th Conference on Decision and Control, pp. 4591–4596 (1996) 88. Utkin, V.I., Guldner, J., Shi, J.: Sliding Mode Control. In: Electromechanical Systems. Taylor and Francis, London (1999) 89. Walcott, B.L., Corless, M.J., Zak, S.K.: Comparative study of nonlinear state observation techniques. Int. J. Control 45, 2109–2132 (1987)

References

203

90. Willemstein, A.P.: Optimal regulation of nonlinear dynamical systems in a finite interval. SIAM J. Control Opt. 15, 1050–1069 (1977) 91. Wonham, W.M.: Some applications of stochastic differential equations to nonlinear filtering. SIAM J. Control 2, 347–369 (1965) 92. Xu, S., Chen, T.: Robust H–infinity control for uncertain stochastic systems with state delay. IEEE Transactions on Automatic Control 47, 2089–2094 (2002) 93. Xia, Y., Jia, Y.: Robust sliding mode control for uncertain stochastic time-delay systems. IEEE Trans. Automatic Control 48, 1086–1092 (2003) 94. Xu, S., van Dooren, P.V.: Robust H∞ –filtering for a class of nonlinear systems with state delay and parameter uncertainty. Int. J. Control 75, 766–774 (2002) 95. Yau, S.S.T.: Finite–dimensional filters with nonlinear drift I: a class of filters including both Kalman-Bucy and Benes filters. J. Math. Systems, Estimation, and Control 4, 181– 203 (1994) 96. Yoshida, T., Loparo, K.: Quadratic regulatory theory for analytic nonlinear systems with additive controls. Automatica 25, 531–544 (1989) 97. Zhang, L., Zhao, Z., Shu, Y., Wang, L., Yang, O.W.W.: Load balancing of multipath source routing in ad hoc networks. In: Proc. IEEE International Conference on Communications, pp. 3197–3201 (2002) 98. Zhang, H., Duan, G., Xie, L.: Linear quadratic regulation for linear time–varying systems with multiple input delays. Automatica 42, 1465–1476 (2006) 99. Zhang, H., Lu, X., Cheng, D.: Optimal estimation for continuous–time systems with delayed measurements. IEEE Trans. Automat. Contr. 51, 823–827 (2006) 100. Zheng, W.X.: On unbiased parameter estimation of linear systems using noisy measurements. Cybernetics and Systems 34, 59–70 (2003) 101. Zubov, V.I.: Lectures in Control Theory (in Russian), Nauka, Moscow (1975)

Index

advanced arguments 142 automotive system 8, 62 backward solution bilinear state 5

identification 47 infinite-dimensional space 84 initial condition 1 initial state 84 innovations 4 input delay 131 integral sliding mode 176 Ito differential 2

148

conditional expectation 2 continuous-time systems 1 cost function 53, 66, 131 criterion 53, 66, 131 cubic sensor problem 38

Kalman-Bucy filter V, 15 Kushner equation V linear quadratic Gaussian (LQG) linear quadratic regulator (LQR) linear: input 53 observations 1 state 5 time-delay system 75

delayed input 131 delayed measurements 75 differential equation 1 duality 138 duality principle 53, 140 dynamic programming 138 energy 69, 133 equal state and input delays estimation error 2 Euclidean 2 expectation 2 extended Kalman filter 40 gain matrix 24 Gaussian: random variable noise 8

1

finite-dimensional filter 21 forward-time solution 142 Hamiltonian

55

141

maximum principle 134 minimizing control 53, 66, 131 multiple: observation delays 75 state delays 84, 148 state and observation delays 94 state and input delays 156 multiplicative noise 25, 33 network control system 131 non-negative definite 68, 131 observation process 1 optimal: control 53, 131 controller 65, 166

168 132

206

Index

joint state filtering and parameter identification 47 filter 2 filtering 1 partially measured linear part polynomial 1 controller 65 drift 2 filter 2 regulator 56, 63 state 1 positive definite 67, 131

17, 33

separation principle 67, 167 sliding mode 175 smoothing 76 state delay 84 stochastic system 1 terpolymerization 10, 56 third degree polynomial state time delay 75, 131 transition matrix 119, 146 uncertainty 175 unknown stochastic parameters variance

quadratic function Riccati equation

2

67, 131 V, 16

Wiener process 1 white Gaussian noise

30

6

47

Lecture Notes in Control and Information Sciences Edited by M. Thoma, M. Morari Further volumes of this series can be found on our homepage: springer.com Vol. 380: Basin M. New Trends in Optimal Filtering and Control for Polynomial and Time-Delay Systems 206 p. 2008 [978-3-540-70802-5] Vol. 379: Mellodge P.; Kachroo P.; Model Abstraction in Dynamical Systems: Application to Mobile Robot Control 116 p. 2008 [978-3-540-70792-9] Vol. 378: Femat R.; Solis-Perales G.; Robust Synchronization of Chaotic Systems Via Feedback 199 p. 2008 [978-3-540-69306-2] Vol. 377: Patan K. Artificial Neural Networks for the Modelling and Fault Diagnosis of Technical Processes 206 p. 2008 [978-3-540-79871-2] Vol. 376: Hasegawa Y. Approximate and Noisy Realization of Discrete-Time Dynamical Systems 245 p. 2008 [978-3-540-79433-2] Vol. 375: Bartolini G.; Fridman L.; Pisano A.; Usai E. (Eds.) Modern Sliding Mode Control Theory 465 p. 2008 [978-3-540-79015-0] Vol. 374: Huang B.; Kadali R. Dynamic Modeling, Predictive Control and Performance Monitoring 240 p. 2008 [978-1-84800-232-6] Vol. 373: Wang Q.-G.; Ye Z.; Cai W.-J.; Hang C.-C. PID Control for Multivariable Processes 264 p. 2008 [978-3-540-78481-4] Vol. 372: Zhou J.; Wen C. Adaptive Backstepping Control of Uncertain Systems 241 p. 2008 [978-3-540-77806-6] Vol. 371: Blondel V.D.; Boyd S.P.; Kimura H. (Eds.) Recent Advances in Learning and Control 279 p. 2008 [978-1-84800-154-1] Vol. 370: Lee S.; Suh I.H.; Kim M.S. (Eds.) Recent Progress in Robotics: Viable Robotic Service to Human 410 p. 2008 [978-3-540-76728-2]

Vol. 369: Hirsch M.J.; Pardalos P.M.; Murphey R.; Grundel D. Advances in Cooperative Control and Optimization 423 p. 2007 [978-3-540-74354-5] Vol. 368: Chee F.; Fernando T. Closed-Loop Control of Blood Glucose 157 p. 2007 [978-3-540-74030-8] Vol. 367: Turner M.C.; Bates D.G. (Eds.) Mathematical Methods for Robust and Nonlinear Control 444 p. 2007 [978-1-84800-024-7] Vol. 366: Bullo F.; Fujimoto K. (Eds.) Lagrangian and Hamiltonian Methods for Nonlinear Control 2006 398 p. 2007 [978-3-540-73889-3] Vol. 365: Bates D.; Hagström M. (Eds.) Nonlinear Analysis and Synthesis Techniques for Aircraft Control 360 p. 2007 [978-3-540-73718-6] Vol. 364: Chiuso A.; Ferrante A.; Pinzoni S. (Eds.) Modeling, Estimation and Control 356 p. 2007 [978-3-540-73569-4] Vol. 363: Besançon G. (Ed.) Nonlinear Observers and Applications 224 p. 2007 [978-3-540-73502-1] Vol. 362: Tarn T.-J.; Chen S.-B.; Zhou C. (Eds.) Robotic Welding, Intelligence and Automation 562 p. 2007 [978-3-540-73373-7] Vol. 361: Méndez-Acosta H.O.; Femat R.; González-Álvarez V. (Eds.): Selected Topics in Dynamics and Control of Chemical and Biological Processes 320 p. 2007 [978-3-540-73187-0] Vol. 360: Kozlowski K. (Ed.) Robot Motion and Control 2007 452 p. 2007 [978-1-84628-973-6] Vol. 359: Christophersen F.J. Optimal Control of Constrained Piecewise Affine Systems 190 p. 2007 [978-3-540-72700-2] Vol. 358: Findeisen R.; Allgöwer F.; Biegler L.T. (Eds.): Assessment and Future Directions of Nonlinear Model Predictive Control 642 p. 2007 [978-3-540-72698-2]

Vol. 357: Queinnec I.; Tarbouriech S.; Garcia G.; Niculescu S.-I. (Eds.): Biology and Control Theory: Current Challenges 589 p. 2007 [978-3-540-71987-8] Vol. 356: Karatkevich A.: Dynamic Analysis of Petri Net-Based Discrete Systems 166 p. 2007 [978-3-540-71464-4] Vol. 355: Zhang H.; Xie L.: Control and Estimation of Systems with Input/Output Delays 213 p. 2007 [978-3-540-71118-6] Vol. 354: Witczak M.: Modelling and Estimation Strategies for Fault Diagnosis of Non-Linear Systems 215 p. 2007 [978-3-540-71114-8] Vol. 353: Bonivento C.; Isidori A.; Marconi L.; Rossi C. (Eds.) Advances in Control Theory and Applications 305 p. 2007 [978-3-540-70700-4] Vol. 352: Chiasson, J.; Loiseau, J.J. (Eds.) Applications of Time Delay Systems 358 p. 2007 [978-3-540-49555-0] Vol. 351: Lin, C.; Wang, Q.-G.; Lee, T.H., He, Y. LMI Approach to Analysis and Control of Takagi-Sugeno Fuzzy Systems with Time Delay 204 p. 2007 [978-3-540-49552-9] Vol. 350: Bandyopadhyay, B.; Manjunath, T.C.; Umapathy, M. Modeling, Control and Implementation of Smart Structures 250 p. 2007 [978-3-540-48393-9] Vol. 349: Rogers, E.T.A.; Galkowski, K.; Owens, D.H. Control Systems Theory and Applications for Linear Repetitive Processes 482 p. 2007 [978-3-540-42663-9] Vol. 347: Assawinchaichote, W.; Nguang, K.S.; Shi P. Fuzzy Control and Filter Design for Uncertain Fuzzy Systems 188 p. 2006 [978-3-540-37011-6] Vol. 346: Tarbouriech, S.; Garcia, G.; Glattfelder, A.H. (Eds.) Advanced Strategies in Control Systems with Input and Output Constraints 480 p. 2006 [978-3-540-37009-3]

Vol. 341: Commault, C.; Marchand, N. (Eds.) Positive Systems 448 p. 2006 [978-3-540-34771-2] Vol. 340: Diehl, M.; Mombaur, K. (Eds.) Fast Motions in Biomechanics and Robotics 500 p. 2006 [978-3-540-36118-3] Vol. 339: Alamir, M. Stabilization of Nonlinear Systems Using Receding-horizon Control Schemes 325 p. 2006 [978-1-84628-470-0] Vol. 338: Tokarzewski, J. Finite Zeros in Discrete Time Control Systems 325 p. 2006 [978-3-540-33464-4] Vol. 337: Blom, H.; Lygeros, J. (Eds.) Stochastic Hybrid Systems 395 p. 2006 [978-3-540-33466-8] Vol. 336: Pettersen, K.Y.; Gravdahl, J.T.; Nijmeijer, H. (Eds.) Group Coordination and Cooperative Control 310 p. 2006 [978-3-540-33468-2] Vol. 335: Kozłowski, K. (Ed.) Robot Motion and Control 424 p. 2006 [978-1-84628-404-5] Vol. 334: Edwards, C.; Fossas Colet, E.; Fridman, L. (Eds.) Advances in Variable Structure and Sliding Mode Control 504 p. 2006 [978-3-540-32800-1] Vol. 333: Banavar, R.N.; Sankaranarayanan, V. Switched Finite Time Control of a Class of Underactuated Systems 99 p. 2006 [978-3-540-32799-8] Vol. 332: Xu, S.; Lam, J. Robust Control and Filtering of Singular Systems 234 p. 2006 [978-3-540-32797-4] Vol. 331: Antsaklis, P.J.; Tabuada, P. (Eds.) Networked Embedded Sensing and Control 367 p. 2006 [978-3-540-32794-3] Vol. 330: Koumoutsakos, P.; Mezic, I. (Eds.) Control of Fluid Flow 200 p. 2006 [978-3-540-25140-8] Vol. 329: Francis, B.A.; Smith, M.C.; Willems, J.C. (Eds.) Control of Uncertain Systems: Modelling, Approximation, and Design 429 p. 2006 [978-3-540-31754-8]

Vol. 345: Huang, D.-S.; Li, K.; Irwin, G.W. (Eds.) Intelligent Computing in Signal Processing and Pattern Recognition 1179 p. 2006 [978-3-540-37257-8]

Vol. 328: Loría, A.; Lamnabhi-Lagarrigue, F.; Panteley, E. (Eds.) Advanced Topics in Control Systems Theory 305 p. 2006 [978-1-84628-313-0] Vol. 327: Fournier, J.-D.; Grimm, J.; Leblond, J.; Partington, J.R. (Eds.) Harmonic Analysis and Rational Approximation 301 p. 2006 [978-3-540-30922-2]

Vol. 344: Huang, D.-S.; Li, K.; Irwin, G.W. (Eds.) Intelligent Control and Automation 1121 p. 2006 [978-3-540-37255-4]

Vol. 326: Wang, H.-S.; Yung, C.-F.; Chang, F.-R. H∞ Control for Nonlinear Descriptor Systems 164 p. 2006 [978-1-84628-289-8]