Control of uncertain systems--modelling, approximation, and design: a workshop on the occasion of Keith Glover's 60th birthday

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari 329 B. A. Francis M. C. Smith J. C. W...

Author: Bruce A. Francis | Malcolm C. Smith | Jan C. Willems

254 downloads 1019 Views 10MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari

329

B. A. Francis M. C. Smith J. C. Willems (Eds.)

Control of Uncertain Systems: Modelling, Approximation, and Design A Workshop on the Occasion of Keith Glover's 60th Birthday With 109 Figures

Series Advisory Board

F. Allgöwer · P. Fleming · P. Kokotovic · A.B. Kurzhanski · H. Kwakernaak · A. Rantzer · J.N. Tsitsiklis

Editors Prof. Bruce A. Francis

Prof. Malcolm C. Smith

University of Toronto Department of Electrical and Computer Engineering King’s College Road 10 M5S 3G4 Toronto Canada

Cambridge University Department of Engineering Trumpington Street CB2 1PZ Cambridge United Kingdom

Prof. Jan C. Willems Katholieke Universiteit Leuven Departement Elektrotechniek (ESAT) SISTA/COSIC Division Kasteelpark Arenberg 10 3001 Leuven-Heverlee Belgium ISSN 0170-8643 ISBN-10 3-540-31754-6 Springer Berlin Heidelberg New York ISBN-13 978-3-540-31754-8 Springer Berlin Heidelberg New York Library of Congress Control Number: 2006920069 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in other ways, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under German Copyright Law. Springer is a part of Springer Science+Business Media springer.com © Springer-Verlag Berlin Heidelberg 2006 Printed in Germany The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Data conversion by editors. Final processing by PTP-Berlin Protago-TEX-Production GmbH, Germany Cover-Design: design & production GmbH, Heidelberg Printed on acid-free paper 89/3141/Yu - 5 4 3 2 1 0

This Festschrift contains a collection of articles by friends, co-authors, colleagues, and former Ph.D. students of Keith Glover. They are a tribute to his seminal work, which has been a source of inspiration to all. The editors are grateful to all authors for their efforts in writing their articles and for meeting the time schedule. They also thank the referees for reviewing the draft manuscripts, and the publisher Springer Verlag for accepting this book as a volume in the Lecture Notes in Control and Information Sciences and for making this book available in time for the workshop Control of Uncertain Systems: Modelling, Approximation, and Design. This workshop to celebrate Keith Glover’s sixtieth birthday takes place on April 21 and 22, 2006, at St John’s and Gonville and Caius Colleges of the University of Cambridge. The generous financial support from the Engineering and Physical Sciences Research Council (EPSRC, UK) and from the MathWorks is gratefully acknowledged. Last but not least, the editors would like to thank Malcolm Smith from the University of Cambridge, the General Chair of the workshop, for organizing the event surrounding Keith Glover’s sixtieth birthday, and for taking care of all the practical details.

November 2005

Bruce Francis and Jan Willems

To Keith Glover on the occasion of his sixtieth birthday

CV — Keith Glover Keith Glover was born on April 23, 1946, in Bromley, Kent, UK. He studied electrical engineering at Imperial College in London and graduated in June, 1967. He worked as an engineer in the communications division for the Marconi Company from 1967 until 1969. From 1969 until 1973 he was a Kennedy Scholar at the Massachusetts Institute of Technology, where he received his Ph.D. degree in 1973 with a dissertation entitled Structural Aspects of System Identification. After completing his Ph.D., he was an assistant professor with the Department of Electrical Engineering of the University of Southern California. In 1976, he joined the Department of Engineering of the University of Cambridge, where he has been since, with a one year leave of absence in 1983–84 as a Visiting Fellow at the Australian National University in Canberra. At Cambridge, Keith Glover rose through the ranks: University Lecturer from 1976 to 1987, Reader in Control Engineering in 1987, and Professor of Engineering since 1989. He has been Head of the Information Engineering Division from 1993 until 2002, and since 2002, he is Head of the Department of Engineering. Keith is a fellow of Sidney Sussex College. His scientific work spans a wide variety of topics, with as main themes: system identification, model reduction and approximation, robust controller synthesis, and control of aircrafts and engines. He is the author and co-author of two very highly cited articles, both among the most influential papers in the field of systems and control in the 20-th century. His 1984 article in the International Journal on Control on Hankel norm and balanced model reduction marked the beginning of a systematic study on model simplification of dynamical systems. The four-author paper (by J. C. Doyle, K. Glover, P. P. Khargonekar, and B. A. Francis) in the 1989 IEEE Transactions on Automatic Control revealed the role of the Riccati equation in H∞ -control. These and related results were incorporated in a successful software package, the µ-Analysis and Synthesis Toolbox (for use with Matlab), distributed by The Mathworks Inc. and written by G. J. Balas, J. C. Doyle, K. Glover, A. Packard, and R. Smith. This work has been honored by many awards and several best paper prizes, among these the IEEE W. R. G. Baker Award for the most outstanding paper reporting original work in the Transactions, Journals and Magazines of the Societies or in the Proceedings of the IEEE in 1991, for the four-author paper mentioned earlier. Keith Glover was elected a Fellow of the Royal Society (1993), of the IEEE (1993), of the Institute of Measurement and Control (1999), and of the Royal Academy of Engineering (2000). In 1998 he was awarded the Sir Harold Hartley Medal by the Institute of Measurement and Control. He was the recipient of the 2001 IEEE Control Systems Award. Keith Glover married Liz (née Jean Elizabeth Priestley) on New Year’s Day 1970. They live in Cambridge, UK, and have two children, Sarah and William.

VIII

Curriculum Vitae

Research students supervised 1. M. de Glas, A Theory of Fuzzy Systems, 1981. 2. M. J. Saxon, Digital Control Strategies for an Industrial Batch Process, 1982. 3. G. O. Corrêa, Identifiable Structures in Linear System Estimation, 1983. 4. P. L. Mills, The Design of Systolic Arrays for Finite Numerical Functions, 1984. 5. G. A. Bundell, Robust Decentralised Adaptive Control in the Manipulator Control Problem, 1985. 6. J. Lam, Model Reduction of Delay Systems, 1988. 7. D. C. McFarlane, Robust Controller Design using Normalized Co-prime Factor Plant Descriptions, 1988. 8. D. Mustafa, Minimum Entropy H∞ Control, 1989. 9. P. A. Iglesias, Robust and Adaptive Control for Discrete-Time Systems, 1991. 10. R. A. Hyde, The Application of Robust Control to VSTOL Aircraft, 1991. 11. J. A. Sefton, A Geometrical Approach to Feedback Stability, 1991. 12. F. S. T. van Diggelen, Hadamard Weighting in Robust Control, 1992. 13. G. Vinnicombe, Measuring Robustness of Feedback Systems, 1992. 14. G. E. Dullerud, Control of Uncertain Sampled-Data Systems, 1994. 15. R. A. Davis, Model Validation for Robust Control, 1995. 16. P. J. Goddard, Performance-preserving Controller Approximation, 1995. 17. S. Lall, Robust Control Synthesis in the Time Domain, 1995. 18. Y. C. Chu, Control of Systems with Repeated Scalar Nonlinearities, 1996. 19. M. Brandstetter, Robust Air-Fuel Ratio Control for Combustion Engines, 1996. 20. M. W. Cantoni, Linear Periodic Systems: Robustness Analysis and Sampleddata Control, 1998. 21. E. Moynihan, Robust Control of Diesel Internal Combustion Engines, 1998. 22. G. Papageorgiou, Robust Control System Design: H∞ Loop Shaping and Aerospace Applications, 1998. 23. R. G. Ford, Robust Automotive Idle Speed Control in a Novel Framework, 2000. 24. S. J. Cornelius, Modelling and Control of Automotive Catalysts, 2001. 25. K. M. Halsey, Nested Feedback Systems: Analysis and Design within an H∞ -Loopshaping Framework, 2002. 26. A. U. Genc, Linear Parameter-Varying Modelling and Robust Control of Variable Cam Timing Engines, 2002. 27. M. Jung, Mean-Value Modelling and Robust Control of the Airpath of a Turbocharged Diesel Engine, 2003. 28. C. Papageorgiou, Robustness Analysis of Nonlinear Dynamic Inversion Control Laws for Flight Control Application, 2003.

Curriculum Vitae

IX

Publications Books 1. K. Zhou, J. C. Doyle and K. Glover, Robust and Optimal Control, Prentice Hall, 1996. 2. D. C. McFarlane and K. Glover, Robust Controller Design using Normalised Coprime Factor Plant Descriptions, Springer Verlag, Lecture Notes in Control and Information Sciences, volume 138, 1989. 3. D. Mustafa and K. Glover, Minimum Entropy H∞ Control, Springer Verlag, Lecture Notes in Control and Information Sciences, volume 146, 1990.

Articles in scientific journals and edited books 1. P. R. Kleindorfer and K. Glover, Linear convex stochastic optimal control with applications in production planning, IEEE Transactions on Automatic Control, volume AC-18, pages 56–59, 1973. 2. K. Glover and J. C. Willems, On the stability of numerical integration routines for ordinary differential equations, Journal of the Institute of Mathematics and its Applications, volume 11, pages 171–180, 1973. 3. K. Glover and J. C. Willems, Parametrization of linear dynamical systems: Canonical forms and identifiability, IEEE Transactions on Automatic Control, volume AC-19, pages 640–645, 1974. 4. K. Glover and L. M. Silverman, Characterization of structural controllability, IEEE Transactions on Automatic Control, volume AC-21, pages 534– 537, 1976. 5. M. S. Grewal and K. Glover, Identifiability of linear and nonlinear dynamical systems, IEEE Transactions on Automatic Control, volume AC-21, pages 833–837, 1976. 6. E. Emre, L. M. Silverman and K. Glover, Generalized dynamic covers for linear systems with applications to deterministic identification and realization problems, IEEE Transactions on Automatic Control, volume AC-22, pages 26–35, 1977. 7. B. A. Francis and K. Glover, Bounded peaking in the optimal linear regulator with cheap control, IEEE Transactions on Automatic Control, volume AC-23, pages 608–617, 1978. 8. K. Glover, New techniques for approximate realization, Proceedings of the IEE, volume 126, pages 596–604, 1979. 9. L. Ljung and K. Glover, Frequency domain versus time domain methods in system identification - A brief discussion, Automatica, volume 17, pages 71–88, 1981. 10. K. Glover, All optimal Hankel-norm approximations of linear multivariable systems and their L∞ -error bounds, International Journal of Control, volume 6, pages 1115–1193, 1984. 11. G. Correa and K. Glover, Pseudo-canonical forms, identifiable parametrizations and simple parameter estimation for linear multivariable systems: Input-output models, Automatica, volume 20, pages 429–442, 1984.

X

Curriculum Vitae

12. G. Correa and K. Glover, Pseudo-canonical forms, identifiable parametrizations and simple parameter estimation for linear multivariable systems: Parameter estimation, Automatica, volume 20, pages 443–452, 1984. 13. R. F. Curtain and K. Glover, Controller design for distributed systems based on Hankel-norm approximation, IEEE Transactions on Automatic Control, volume AC-31, pages 173–176, 1986. 14. G. O. Correa and K. Glover, On the choice of parametrization for identification, IEEE Transactions on Automatic Control, volume AC-31, pages 8– 15, 1986. 15. K. Glover, Robust stabilization of linear multivariable systems: Relations to approximation, International Journal of Control, volume 43, pages 741– 766, 1986. 16. R. F. Curtain and K. Glover, Robust stabilization of infinite dimensional systems by finite dimensional controllers, Systems & Control Letters, volume 7, pages 41–47, 1986. 17. J. B. Moore, L. Xia and K. Glover, On improving control-loop robustness of model-matching controllers, Systems & Control Letters, volume 7, pages 83–88, 1986. 18. Y. S. Hung and K. Glover, Optimal Hankel-norm approximation of stable systems with first-order stable weighting functions, Systems & Control Letters, volume 7, pages 165–172, 1986. 19. K. Glover and J. R. Partington, Bounds on achievable accuracy in model reduction, in Modelling Robustness and Sensitivity Reduction in Control Systems, edited by R.F. Curtain, NATO ASI Series F, pages 95–118, 1987. 20. K. Glover, Identification: Frequency domain methods, in Systems and Control Encyclopedia, edited by M.G. Singh, Pergamon Press, pages 2264– 2270, 1987. 21. J. R. Partington, K. Glover, H. J. Zwart and R. F. Curtain, L∞ approximation and nuclearity of delay systems, Systems & Control Letters, volume 10, pages 59–65, 1988. 22. H. J. Zwart, R. F. Curtain, J. R. Partington and K. Glover, Partial fraction expansions for delay systems, Systems & Control Letters, volume 10, pages 235–243, 1988. 23. K. Glover, R. F. Curtain and J. R. Partington, Realisation and approximation of linear infinite dimensional systems with error bounds, SIAM Journal of Control, volume 26, pages 863–898, 1988. 24. K. Glover and J. C. Doyle, State-space formulae for all stabilizing controllers that satisfy an H∞ -norm bound and relations to risk sensitivity, Systems & Control Letters, volume 11, pages 167–172, 1988. 25. J. C. Doyle, K. Glover, P. P. Khargonekar and B. A. Francis, State-space solutions to standard H2 and H∞ control problems, IEEE Transactions on Automatic Control, volume AC-34, pages 831–847, 1989. 26. K. Glover and D. C. McFarlane, Robust stabilization of normalized coprime factor plant descriptions with H∞ -bounded uncertainty, IEEE Transactions on Automatic Control, volume AC-34, pages 821–830, 1989.

Curriculum Vitae

XI

27. K. Glover and D. Mustafa, Derivation of the maximum entropy H∞ controller and a state-space formula for its entropy, International Journal of Control, volume 50, pages 899–916, 1989. 28. D. J. Clements and K. Glover, Spectral factorization via Hermitian pencils, Linear Algebra and its Applications, volume 122/123/124, pages 797– 846, 1989. 29. K. Glover and J. C. Doyle, A state space approach to H∞ optimal control, in Three Decades of Mathematical Systems Theory: A Collection of Surveys at the Occasion of the 50th Birthday of Jan C. Willems, edited by H. Nijmeijer and J. M. Schumacher, Springer-Verlag Lecture Notes in Control and Information Sciences volume 135, pages 179–218, 1989. 30. K. Glover, A tutorial on Hankel-norm approximation, in From Data to Model, by J. C. Willems, Springer-Verlag, pages 26–48, 1989. 31. D. J. N. Limebeer, G. Halikias and K. Glover, State-space algorithm for the computation of super-optimal matrix interpolating functions, International Journal of Control, volume 50, pages 2431–2466, 1989. 32. J. B. Moore, K. Glover and A. Telford, All stabilizing controllers as frequency shaped state estimate feedback, IEEE Transactions on Automatic Control, volume AC-35, pages 203–208, 1990. 33. D. C. McFarlane, K. Glover and M. Vidyasagar, Reduced order controller design using coprime factor model reduction, IEEE Transactions on Automatic Control, volume AC-35, pages 369–373, 1990. 34. P. A. Iglesias, D. Mustafa and K. Glover, Discrete time H∞ controllers satisfying a minimum entropy criterion, Systems & Control Letters, volume 14, pages 275–286, 1990. 35. J. A. Sefton and K. Glover, Pole/zero cancellations in general H∞ problem with reference to a two block design, Systems & Control Letters, volume 14, pages 295–306, 1990. 36. J. R. Partington and K. Glover, Robust stabilization of delay systems by approximation of coprime factors, Systems & Control Letters, volume 14, pages 325–332, 1990. 37. K. Glover, J. Lam and J. R. Partington, Rational approximation of a class of infinite-dimensional systems I: Singular values of Hankel operators, Mathematics of Control, Systems and Signals, volume 3, pages 325–344, 1990. 38. M. Green, K. Glover, D. J. N. Limebeer and J. C. Doyle, A J-spectral factorization approach to H∞ control, SIAM Journal on Control and Optimimization, volume 28, pages 1350–1371, 1990. 39. K. Glover, J. Lam and J. R. Partington, Rational approximation of a class of infinite-dimensional systems II: Optimal convergence rates of L∞ approximants, Mathematics of Control, Systems and Signals, volume 4, pages 233–246, 1991. 40. K. Glover, J. Lam and J. R. Partington, Rational approximation of a class of infinite-dimensional systems: The L2 case, in Progress in Approximation

XII

41. 42.

43. 44. 45.

46. 47. 48. 49. 50. 51. 52. 53. 54.

Curriculum Vitae

Theory, edited by P. Nevai and A. Pinkus, Academic Press, pages 405–440, 1991. D. Mustafa, K. Glover and D. J. N. Limebeer, Solutions to the H∞ general distance problem which minimize an entropy integral, Automatica, volume 27, pages 193–199, 1991. K. Glover, D. J. N. Limebeer, J. C. Doyle, E. M. Kasenally, and M. G. Safonov, A characterization of all solutions to the four block general distance problem, SIAM Journal on Control and Optimization, volume 29, pages 283–324, 1991. D. Mustafa and K. Glover, Model reduction by H∞ -balanced truncation, IEEE Transactions on Automatic Control, volume 36, pages 668–682, 1991. P. A. Iglesias and K. Glover, State-space approach to discrete-time H∞ control, Int. J. Control, volume 54, pages 1031–1073, 1991. K. Glover, D. J. N. Limebeer and Y. S. Hung, A structured approximation problem with applications to frequency weighted model reduction, IEEE Transactions on Automatic Control, volume 37, pages 447–465, April 1992. D. C. McFarlane and K. Glover, A loop shaping design procedure using H∞ synthesis, IEEE Transactions on Automatic Control, volume 37, pages 759–769, 1992. G. D. Halikias, D. J. N. Limebeer and K. Glover, A state-space algorithm for the superoptimal Hankel-norm approximation problem, SIAM Journal on Control and Optimization, volume 31, pages 960–982, 1993. R. A. Hyde and K. Glover, The application of scheduled H∞ controllers to a VSTOL aircraft, IEEE Transactions on Automatic Control, volume 38, pages 1021–1039,1993. G. E. Dullerud and K. Glover, Robust stabilization of sampled-data systems to structured LTI perturbations, IEEE Transactions on Automatic Control, volume 38, pages 1497–1508, 1993. F. S. T. van Diggelen and K. Glover, State space solutions to Hadamard weighted H∞ and H2 control problems, International Journal of Control, volume 59, pages 357–394, 1994. F. S. T. van Diggelen and K. Glover, A Hadamard weighted loop shaping design procedure for robust decoupling, Automatica, volume 30, pages 831–845, 1994. K. Zhou, K. Glover, B. Bodenheimer and J. C. Doyle, Mixed H2 and H∞ performance objectives I: Robust performance analysis, IEEE Transactions on Automatic Control, volume 39, pages 1564–1574, 1994. J. C. Doyle, K. Zhou, K. Glover and B. Bodenheimer, Mixed H2 and H∞ performance objectives II: Optimal control, IEEE Transactions on Automatic Control, volume 39, pages 1575–1587, 1994. G. E. Dullerud and K. Glover, Analysis of structured LTI uncertainty in sampled-data systems, Automatica, volume 31, pages 99–113, 1995.

Curriculum Vitae

XIII

55. R. A. Hyde, K. Glover and G. T. Shanks, VSTOL first flight of an Hinfinity control law, Computing and Control Engineering Journal, volume 6, pages 11–16, 1995. 56. G. E. Dullerud and K. Glover, Robust performance of periodic systems, IEEE Transactions on Automatic Control, volume 41, pages 1146– 1159,1996. 57. C. L. Beck, J. C. Doyle and K. Glover, Model-reduction of multidimensional and uncertain systems IEEE Transactions on Automatic Control, volume 41, pages 1466–1477, 1996. 58. M. W. Cantoni and K. Glover, H∞ sampled-data synthesis and related numerical issues, Automatica, volume 33, pages 2233–2241, 1997. 59. P. J. Goddard and K. Glover, Controller approximation: Approaches for preserving H∞ performance, IEEE Transactions on Automatic Control, volume 43, pages 858–871, 1998. 60. Y-C. Chu and K. Glover, Bounds of the induced norm and model reduction errors for systems with repeated scalar nonlinearities, IEEE Transactions on Automatic Control, volume 44, pages 471–483, 1999. 61. Y-C. Chu and K. Glover, Stabilization and performance synthesis for systems with repeated scalar nonlinearities, IEEE Transactions on Automatic Control, volume 44, pages 484–496, 1999. 62. M. W. Cantoni and K. Glover, Gap-metric robustness analysis of linear periodically time-varying feedback systems, SIAM Journal on Control and Optimization, volume 38, pages 803–822, 2000. 63. M. W. Cantoni and K. Glover, Existence of right and left representations of the graph for linear periodically time-varying systems, SIAM Journal on Control and Optimization, volume 38, pages 786–802, 2000. 64. K. Glover and M. G. Safonov, George Zames — 7 January 1934–10 August 1997, International Journal on Robust and Nonlinear Control, volume 10, pages 849–850, 2000. 65. S. J. Cornelius, N. Collings and K. Glover, The role of oxygen storage in NO conversion in automotive catalysts, Topics in Catalysis, volume 16, pages 57–62, 2001. 66. R. G. Ford and K. Glover, Spark ignition engine idle speed control using a novel framework and enabling control of the tradeoff between fuel consumption and load rejection capability, Vehicle System Dynamics, volume 36 , pages 225–251, 2001. 67. G. Papageorgiou and K. Glover, Two-degree-of-freedom control of an actively controlled wind-tunnel model, Journal of Guidance, Control and Dynamics, volume 25, pages 510–516, 2002. 68. M. Jung, R. G. Ford, K. Glover, N. Collings, and M. Watts, Parameterization and transient validation of a variable geometry turbocharger for mean-value modeling at low and medium speed-load points, Journal on Engines, pages 2480–2493, 2002.

XIV

Curriculum Vitae

69. Y-C. Chu, K. Glover and A. P. Dowling, Control of combustion instabilities via H∞ -loop shaping, µ-analysis and integral quadratic constraints, Automatica volume 39, pages 219–231, 2003. 70. M. Jung, K. Glover and U. Christen, Comparison of uncertainty parameterisations for H-infinity robust control of turbocharged diesel engines, Control Engineering Practice, volume 13, pages 15–25, 2005. 71. C. Papageorgiou and K. Glover, Robustness analysis of nonlinear flight controllers, Journal of Guidance, Control and Dynamics, volume 28, pages 639–648, 2005. 72. K. M. Halsey and K. Glover, Analysis and synthesis of nested feedback systems, IEEE Transactions on Automatic Control, volume 50, pages 984– 996, 2005. 73. K. M. Halsey and K. Glover, Analysis and synthesis on a generalized stability region, IEEE Transactions on Automatic Control, volume 50, pages 997–1009, 2005.

Selected conference papers 1. K. Glover and D. J. N. Limebeer, Robust Multivariable Control System Design Using Optimal Reduced Order Plant Model, Proceedings of the American Control Conference, San Francisco, pages 644–649, 1983. 2. K. Glover, Multiplicative Approximation of Linear Multivariable Systems with L∞ Error Bounds, Proceedings of the American Control Conference, Seattle, pages 1705–1709, 1986. 3. K. Glover, Minimum Entropy and Risk Sensitive Control : the Continuous Time Case, Proceedings of the 28-th IEEE Conference on Decision and Control, Tampa, pages 388–391, Dec. 1989. 4. K. Glover, State space formulae for the multiplicative approximation of multivariable transfer functions, Proceedings of the International Symposium on Networks and Systems, MTNS-91, Kobe, Japan, 1991. Published as Recent Advances in Mathematical Theory of Systems, Control, Networks and Signal Processing I, Mita Press, Tokyo, pages 467–472, 1992. 5. R. A. Davis and K. Glover, An Application of recent model validation techniques to flight test data,Proceedings of the Third European Control Conference ECC95, Rome, 1995. 6. G. Papageorgiou and K. Glover, A Systematic Procedure for Designing Non-Diagonal Weights to Facilitate H∞ Loop Shaping, Proceedings of the 36-th IEEE Conference on Decision and Control, San Diego, pages 2127– 2132, 1997. 7. K. Glover, G. Vinnicombe and G. Papageorgiou, Guaranteed multi-loop stability margins and the gap metric, Proceedings of the 39-th IEEE Conference on Decision and Control, Sydney, pages 4084-4085, 2000.

Contents

LMI-Based Gain Scheduled Controller Synthesis for a Class of Linear Parameter Varying Systems Brian D. O. Anderson, Alexander Lanzon, Jan Bendtsen . . . . . . . . . . . . .

1

Control of High-Speed Underwater Vehicles Gary J. Balas, J´ ozsef Bokor, B´ alint Vanek, Roger E.A. Arndt . . . . . . . . 25 Using Feedback to Improve System Identification Roger W. Brockett . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 On the Gap Metric for Finite-Dimensional Linear Time-Varying Systems in Continuous Time Michael Cantoni . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Robustly Stabilizing Controllers with Internal Loop Ruth F. Curtain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Model Reduction of Strongly Stable Nonstationary LPV Systems Mazen Farhood and Geir E. Dullerud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Getting Mobile Autonomous Robots to Rendezvous Zhiyun Lin, Bruce Francis, Manfredi Maggiore . . . . . . . . . . . . . . . . . . . . . . . 119 Experiment Design for Robust Control: Why Do More Work than Is Needed? M. Gevers, X. Bombois, G. Scorletti, P. Van den Hof, R. Hildebrand . . . 139 Past, Present and Future of Automotive Control Lino Guzzella and Christopher Onder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

XVI

Contents

Review of Multivariable Control Applied to the VAAC Harrier and Remaining Open Questions Rick A. Hyde . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Regulating Cell Shape During Cytokinesis Janet C. Effler, Pablo A. Iglesias, Douglas N. Robinson . . . . . . . . . . . . . . . 203 Intrinsic Uncertainty in Gene Regulation Network Hidenori Kimura . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems Randy Cogill, Michael Rotkowitz, Benjamin Van Roy, Sanjay Lall . . . . . . 243 An H∞ Loop-Shaping Approach to Steering Control for High-Performance Motorcycles Simos Evangelou, David J. N. Limebeer, Robin S. Sharp, Malcolm C. Smith . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Frequency Domain Versus Time Domain Methods in System Identification –Revisited Lennart Ljung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Optimal, Worst Case Filter Design via Convex Optimization Kunpeng Sun, Andy Packard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Distance Measures, Robust Stability Conditions and Robust Performance Guarantees for Uncertain Feedback Systems George Papageorgiou, Alexander Lanzon . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 Stochastic Modelling Over a Finite Alphabet and Algorithms for Finding Genes from Genomes M. Vidyasagar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Feedback Networks Glenn Vinnicombe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 Thoughts on System Identification Jan C. Willems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 Sample Reuse Techniques for Probabilistic Robust Control Xinjia Chen, Jorge L. Aravena and Kemin Zhou . . . . . . . . . . . . . . . . . . . . . 417

LMI-Based Gain Scheduled Controller Synthesis for a Class of Linear Parameter Varying Systems Brian D.O. Anderson1 , Alexander Lanzon1 , and Jan Bendtsen2 1

2

Research School of Information Sciences and Engineering, The Australian National University Canberra ACT 0200, Australia National ICT Australia Ltd., Locked Bag 8001, Canberra ACT 2601, Australia [email protected], [email protected] Department of Control Engineering, Aalborg University Fredrik Bajers Vej 7C, 9220 Aalborg East, Denmark [email protected]

This paper presents a novel method for constructing controllers for a class of single-input multiple-output (SIMO) linear parameter varying (LPV) systems. This class of systems encompasses many physical systems, in particular systems where individual components vary with time, and is therefore of significant practical relevance to control designers. The control design presented in this paper has the properties that the system matrix of the closed loop is multi-affine in the various scalar parameters, and that the resulting controller ensures a certain degree of stability for the closed loop even when the parameters are varying, with the degree of stability related directly to a bound on the average rate of allowable parameter variations. Thus, if knowledge of the parameter variations is available, the conservativeness of the design can be kept at a minimum. The construction of the controller is formulated as a standard linear time-invariant (LTI) design combined with a set of linear matrix inequalities, which can be solved efficiently with software tools. The design procedure is illustrated by a numerical example.

1 Introduction Many nonlinear systems can be formulated within the framework of linear parameter varying (LPV) systems and, as a consequence, LPV systems have received much attention in the control research community within the last two decades.

B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 1–23, 2006. © Springer-Verlag Berlin Heidelberg 2006

2

B.D.O. Anderson, A. Lanzon, and J. Bendtsen MΛ (s)

NΛ (s) Λ

u

M(s)−1

N(s)

y

Fig. 1. Perturbed plant represented using coprime factorizations.

In general, LPV systems are systems that can be described by linear transfer functions or state space realizations, but where certain parameters in the description are non-constant. Obviously, such a description is quite general; it captures, for instance, many nonlinear systems that can be modeled using linearized dynamics, but where the parameters vary as the operating point changes. Many examples of such systems can be found in the literature; [1] describes a method for formulating LPV models of propagation phenomena and shows two examples involving temperature control for a plate cooled by convection and mass flow rate regulation in a pneumatic network. [2] contains an example of gain scheduled H∞ control design for the pitch axis of a (simplified) LPV model of a missile. [3] shows how to write the Van der Pol equation in LPV form; and so forth. Due to the close relation to LTI plants, it is natural to want to use gain scheduling techniques to control LPV plants. Care must be exercised, however, when designing gain scheduled controllers that do not explicitly take into account the rate of parameter variations for LPV plants, since a gain scheduled closed-loop system can become unstable unless the scheduling happens “sufficiently slowly” [4]. It is thus desirable to take knowledge of the rate of parameter variations into account in the design, since otherwise one may be forced to introduce conservativeness by allowing arbitrarily fast variations. [5] provides a general survey of gain scheduling techniques and also lists references to various applications. 1.1 Systems of Interest In this paper, we are interested in LPV plants where the parameter dependence can be written using a matrix fractional description, and where the parameter dependence terms are affine or in fact multi-affine in the parameters. Specifically, we look at SIMO plants that permit a description of the form P (λ, s) = (N (s) + NΛ (s)Λ)(M (s) + MΛ (s)Λ)−1

(1)

where N (s), NΛ (s), M (s) and MΛ (s) are appropriately chosen stable transfer function matrices and N (s), M (s) are coprime factors over RH∞ of P (0, s), and Λ is a vector multi-linear in some scalar λi , thus

LMI-Based Gain Scheduling

3

Λ = [λ1 , λ2 , . . . , λr , λ1 λ2 , . . . , λr−1 λr , . . . , λ1 λ2 . . . λr ]T ¯ i ≤ λi ≤ λ ¯i , i = 1, . . . , r}. The with λ = [λ1 , . . . , λr ]T ∈ λbox = {−λ term multi-affine allows products of different λi , but no powers. The vector λ describes the variation from a specific ’central’ or ’nominal’ value of the LPV system. This configuration is illustrated in Figure 1, from which it is clear that with λ = 0, the system is described by the ’nominal’ plant P (0, s) = N (s)M (s)−1 . Reference [6] pointed out that many physical linear systems with variations in individual physical parameters could be modelled in this way; essentially, the only restriction is that the varying components must not allow crosscouplings of energy storage. Examples of systems with multi-affine dependence on parameters thus include electric circuits where individual components such as resistors, capacitors and inductors vary, corresponding mechanical systems (masses, friction coefficients, inertias etc.), thermo-dynamical systems (fluid storage, valves etc.), and so forth3 . Such systems often arise as a selection of linearizations of a nonlinear plant in various operating points, but may also appear from applications of system identification or from first-principles modeling (see e.g. [7]). Note also that parameter variations in physical systems may not be symmetrical about a central value of zero in the way implied by the definition of λbox above; however, it is always possible, through a simple variable substitution, to rewrite the system such that the parameter variation becomes symmetrical. We are going to embed the problem of designing controllers to stabilise the plant set of (1) within the problem of designing controllers to stabilise a larger set of plants. Denote the 2r corners of λbox by λcor . Thus λ ∈ λcor if and ¯ i for each i. Call the set of such λ values λcori , i ∈ {1, . . . , 2r }. only if λi = ±λ Each corner value of λ gives rise to a plant P (λcori , s). It is not hard to see, and indeed argued in [8], that for a generic value of Λ generated by a generic λ ∈ λbox , there exists a set of nonnegative weights δi summing to 1 such that 2r

δi [N (s) + NΛ (s)Λcori ]

N (s) + NΛ (s)Λ =

(2)

i=1

Indeed, we review this below, where it is also pointed out that the same set of δi yield 2r

δi [M (s) + MΛ (s)Λcori ]

M (s) + MΛ (s)Λ =

(3)

i=1 3

Note that sometimes the entry corresponding to an individual λi may correspond to the inverse of a physical component value rather than the component value itself; this is of course not relevant in terms of stability etc., since e.g., the behavior of an electric circuit may be equally well described by a resistor’s conductance as by its resistance.

4

B.D.O. Anderson, A. Lanzon, and J. Bendtsen

[Here, Λcori denotes the value of Λ corresponding to λcori ]. So we cover all plants of the set (1) by considering all plants of the form

P¯ (δ, s) = {

2r

2r

δi [N (s) + NΛ (s)Λcori ]}{ i=1

δi [M (s) + MΛ (s)Λcori ]}−1

i=1

δi ≥ 0

δi = 1 (4)

Call the set (1) Sλ and call the set just defined Sδ . There holds Sλ ⊆ Sδ . We will actually explain how to design controllers for the set Sδ . The construction of the δi from a particular λ can be made independent of NΛ (s), MΛ (s) etc, as indicated in the following argument, referred to in [9], which is a minor development of [8]. Let g(λ1 , λ2 , . . . , λr ) be any multi-affine function of the λi , with −1 ≤ λi ≤ 1.Then successive use of the multi-affine property yields: g(λ1 , λ2 , . . . , λr ) = (1/2)(1+λ1 )g(1, λ2 , . . . , λr )+(1/2)(1−λ1 )g(−1, λ2 , . . . , λr ) (5) = (1/2)(1 + λ1 )[(1/2(1 + λ2 )g(1, 1, λ3 , . . . , λr ) + (1/2)(1 − λ2 )g(1, −1, λ3 , . . . , λr )] + (1/2)(1 − λ1 )[(1/2(1 + λ2 )g(−1, 1, λ3 , . . . , λr ) + (1/2)(1 − λ2 )g(−1, −1, λ3, . . . , λr )]

(1/2r )[1 + (−1)i1 λ1 ][1 + (−1)i2 λ2 ]

= ij ∈1,2,j=1,...,r

. . . [1 + (−1)ir λr ]g((−1)i1 , (−1)i2 , . . . , (−1)ir ) (6) The δi are evidently the quantities (1/2r )[1 + (−1)i1 λ1 ][1 + (−1)i2 λ2 ] . . . [1 + (−1)ir λr ]. We shall describe this decomposition as the standard convex representation. Now, in order to treat the controller synthesis problem and furthermore be able to handle time-varying parameter values without losing stability guarantees, we will appeal to two different (though somewhat related) approaches for dealing with LPV systems. The first approach relates directly to affine parameter dependencies, but does not consider time variation. This idea was developed in [9], which considered strict positive realness of families of transfer function matrices and [10], which provided a convex parameterization of all fixed controllers that simultaneously stabilize a parameter-dependent system for any permissible fixed values of the parameters. The second idea was


5

developed in [11] and relates the rate of parameter variation for time varying systems to degree of stability. We will briefly review the latter ideas in the following section. 1.2 Robust Stability Results for LPV Systems Reference [10] addressed the synthesis of robust controllers for linear timeinvariant systems. The uncertain real parameters were assumed to appear linearly in the closed loop characteristic polynomial, and the parameters were assumed to lie in a bounded set, i.e., a description that matched the setup in (1). The paper proceeded to give a convex parameterization of all controllers that stabilize P (λ, s) for fixed frozen values of λ. The stability proof, which was directly inspired by [9], relied on showing the equivalence between the stability of a certain transfer function involving the parameters in an affine way and the existence of a frequency-dependent (stable, positive-real) multiplier π such that the inverse of the transfer function multiplied by π has positive real part for all frequencies. If a π can be found, it is then possible to compute a controller that is independent of λ and that stabilizes P for any value of λ. However, no stability guarantees were given if the parameters are allowed to vary. In the method presented here, the controller is allowed to depend on the time varying parameters; in fact, some of the parameters in the controller will be tracking the parameters in the plant directly. This offers the prospect of better performance, or ability to deal with a larger uncertainty set. A significant contribution of this paper is to give a theoretical justification for being able to tolerate time varying parameters in a setup somewhat similar to the one given in [10]. To this end, we will appeal to the second approach to dealing with systems that are affine in the parameters mentioned in the previous subsection. Recall that an autonomous linear time varying (LTV) system χ(t) ˙ = A(λ(t))χ(t)

(7)

where χ ∈ Rn is the state vector and λ ∈ λbox ⊂ Rr is a parameter vector describing the variation in the system parameters at time t, is said to be exponentially asymptotically stable with degree of stability γ > 0 if there exist real scalars c, α > 0 such that for all χ(t0 ) and t ≥ t0 , we have χ(t) eγ(t−t0 ) ≤ c χ(t0 ) e−α(t−t0 ) Reference [11] dealt with systems of this form for which the existence can be demonstrated of a quadratic Lyapunov function χT P (λ)χ with P (λ) = P T (λ) positive definite, multi-affine in λ, and such that A(λ)T P (λ) + P (λ)A(λ) < −2σP (λ)

(8)

6


[Existence of such P (λ) is demonstrated in [11] for the case where A(λ) = A + h(λ)g T with h(λ) ∈ Rn affine in the elements of λ; we will however not appeal to this part of reference [11]]. Equation (8) ensures that for any frozen value of λ, the eigenvalues of A(λ) lie to the left of Re[s] = −σ. Propositions 5.1 and 5.2 of reference [11] proceed to quantify the stability of the system (7) in terms of the permissible rate of change of parameter variation. Theorem 1. For λ ∈ λbox , let A(λ) be a set of n × n matrices such that there exists a positive definite symmetric P (λ), multi-affine in λ, satisfying (8) for some σ > 0. Then the time-varying system (7) is exponentially asymptotically stable with degree of stability γ ∈ [0, σ) if there exist T > 0 and 1i , 2i > 0, i = 1, . . . , r such that for all i = 1, . . . , r and all t > 0 we have ¯i + λi (t) ∈ [−λ and sup t≥0

t+T t

¯ −

1i , λi

2i ]

(9)

¯

r

|

)+λi d ln λλ¯ ii (τ −λi (τ )

i=1

dτ

|dτ < 4(σ − γ)

(10)

Proof. See [11]. This theorem implies that if it is possible to synthesize controllers for an LPV plant such that the closed loop is σ-Hurwitz for all (frozen) values of λ, and an associated quadratic Lyapunov function can be found with multi-affine dependence on λ, then we will have stability guarantees for the time-varying closed-loop system provided known bounds on the average rate of variation of λ are observed. It is important to note here that (10) is a bound on the average rate of parameter variation, and that abrupt parameter variations such as steps are allowed, as long as they occur sufficiently infrequently. 1.3 Contribution of This Paper The contribution of this paper is thus to provide a method for constructing controllers for such systems with the property that the A-matrix of the closed loop has the crucial property set out in Theorem 1. The construction of these controllers is formulated by means of a set of linear matrix inequalities, which can be solved efficiently with readily available software tools. If these matrix inequalities are feasible, the resulting controller ensures a certain degree of stability for the closed loop with any allowed constant parameter vector, and we can then quantify the rate at which the parameter variations are allowed to occur if certain extra conditions are satisfied, and build this knowledge into the controller. This provides a potential for less conservative design than other LFT-based methods in the sense that the resulting gain scheduled controller does not have to take unreasonably fast parameter variations into account.


7

1.4 Notation Note that throughout the paper, 0 and I indicate zero and identity elep×m ments of appropriate vector spaces (of compatible dimension). RH∞ denotes the space of all proper, real rational stable transfer matrices mapping m-dimensional input signals to p-dimensional output signals. A system P (s) p×m is σ-Hurwitz if P (s + σ) ∈ RH∞ . We will also use the notion of a Laguerre basis (see e.g., [12]). If a transfer function matrix P (s) with m inputs and p outputs can be written as P (s) = P˘ Bτk (s)

(11)

where P˘ = P˘0 P˘1 · · · P˘k Bτk (jω) = Im

2 τ −s 2 τ +s

Im

∈ Rp×(k+1)m , 2 τ −s 2 τ +s

2

Im · · ·

2 τ 2 τ

−s +s

k

T

Im

,

p×m for some τ ∈ R+ and k ∈ Z+ , then P (s) obviously belongs to RH∞ . Finally, Co{·} denotes the convex hull of (·).

1.5 Outline of Paper The outline of the rest of this paper is as follows. In the following section we provide an overview of the systems we are considering and discuss how to handle the parameter dependencies in a matrix fractional description of the plant and controller. Then, in Section 3, we formulate a finite set of linear matrix inequalities, which, if feasible, provides a controller that not only stabilizes the plant under consideration for all possible parameter values, but also preserves the appropriate parameter dependency for the closed loop characteristic polynomial allowing application of Theorem 1. In Section 4 we provide an explicit implementation of our proposed design procedure. Finally, we present a numerical example illustrating the method in Section 5 and make some closing remarks in Section 6.

2 LPV Framework As stated in the introduction, we will employ a matrix fractional description of the plant with parameter variations and demonstrate the existence of a set of synthesis LMIs which, if feasible, will provide a controller that stabilizes the LPV system for all permissible values of λ, and which allows λ to vary with time. As will be shown, the controller constructed in this way depends on λ and the closed-loop characteristic polynomial of the controller-plant interconnection will have multi-affine dependency on λ.

8


2.1 Plant and Controller Fractional Descriptions Let P be a SIMO plant of the form (1), i.e., mapping a single input signal to p ≥ 1 output signals; we assume the parameter dependence allows us to find 1×1 p×l p×1 , M (s) ∈ RH∞ , NΛ (s) ∈ RH∞ stable transfer functions N (s) ∈ RH∞ 1×l and MΛ (s) ∈ RH∞ such that P (λ, s) = (N (s) + NΛ (s)Λ)(M (s) + MΛ (s)Λ)−1 where

(12) T

Λ = [λ1 , λ2 , . . . , λr , λ1 λ2 , . . . , λr−1 λr , . . . , λ1 λ2 . . . λr ] ¯ i ≤ λi ≤ λ ¯ i , i = 1, . . . , r}. One can check with λ = [λ1 , . . . , λr ]T ∈ λbox = {−λ r that l = 2 − 1. This configuration is illustrated in Figure 1, from which it is clear that with λ = 0 we have the nominal LTI plant P (0, s) = N (s)M (s)−1 . Specifically, N (s) and M (s) are chosen to be right coprime factorizations of P (0, s), while NΛ (s) and MΛ (s) are chosen such that N (s) + NΛ (s)Λ and M (s) + MΛ (s)Λ represent a right coprime factorization of P (λ, s) over RH∞ for almost all λ ∈ λbox . There is a nontrivial assumption buried here. Note though that a continuity argument shows that the coprimeness existing at λ = 0 persists for sufficiently small values of λ. So the assumption is harmless in a suitably small region. Also, in case p > 1, one could argue generically that the zeros of the scalar denominator are never likely to be simultaneously zeros of all entries of the vector numerator in (12), even if as λ varies, these zeros may move backwards and forwards along the real axis in the complex plane. We shall further assume that NΛ and MΛ are strictly proper; if the form of P (λ, s) arises as described in Section 1.1 or reference [6], this will occur. Using this matrix fractional description, we define the following transfer function G(λ, s) =

N (s) + NΛ (s)Λ M (s) + MΛ (s)Λ

(p+1)×1 ∈ RH∞ .

(13)

We now postulate a controller of basically the same structure as the plant, i.e., ˜ (s) + ΛT U ˜Λ (s)) C(λ, s) = (V˜ (s) + ΛT V˜Λ (s))−1 (U (14) 1×p 1×1 l×p l×1 ˜ (s) ∈ RH , V˜ (s) ∈ RH , U ˜Λ (s) ∈ RH ˜ where U ∞ ∞ ∞ and VΛ (s) ∈ RH∞ ˜ (s), V˜ (s) are left coprime factors over RH∞ of the are chosen such that U ˜ (s) + ΛT U ˜Λ (s) and V˜ (s) + ΛT V˜Λ (s) represent nominal controller C(0, s) and U a left coprime factorization of C(λ, s) over RH∞ for almost all λ ∈ λbox . In ˜ (s) should be designed (using any standard method) fact, C(0, s) = V˜ (s)−1 U to stabilize the nominal plant P (0, s) = N (s)M (s)−1 and indeed achieve other specified performance objectives. C(0, s) might for instance be designed to provide certain stability margins etc. Analogously with G, we introduce the transfer function ˜ ˜Λ (s) V˜ (s) + ΛT V˜Λ (s) ˜ (s) − ΛT U K(λ, s) = −U corresponding to the controller C(λ, s).

1×(p+1) ∈ RH∞

(15)


+$

*

9

,

( "!& )#

+%

'"!& )# Fig. 2. Closed loop of plant and controller.

2.2 Internal Stability and Multi-affine Parameter Dependence The controller is placed in closed loop with the plant as depicted in Figure 2. The internal stability of the closed loop can then be determined by examining the collection of transfer functions P (λ, s) (I − C(λ, s)P (λ, s))−1 −C(λ, s) I I ˜ −1 K ˜ = G(KG)

H(P, C) =

(16)

Inserting the plant and controller factorizations in this expression yields H(P, C) = G

˜ − ΛT U ˜Λ V˜ + ΛT V˜Λ −U

N + NΛ Λ M + MΛ Λ

−1

˜ K

˜ N ) + ΛT (V˜Λ M − U ˜Λ N ) + (V˜ MΛ − U ˜ NΛ )Λ = G (V˜ M − U ˜Λ NΛ )Λ + ΛT (V˜Λ MΛ − U

−1

˜ K

(17)

where the dependencies on s have been suppressed in the notation for the sake of brevity. Note that the inverted term in the above expression is a scalar function of the entries of Λ and thus of λ; this term (without the inverse) plays the role of the characteristic polynomial of the closed loop transfer matrix of the ˜ are stable controlled system including the λ-dependence. Since both G and K transfer functions, the zeros of this “characteristic polynomial” determine the stability of the closed loop, as any unstable poles must occur here. In order to yield the desired multi-affine dependence on λ of the closed-loop characteristic polynomial, this scalar denominator must be affine in Λ (cf. [6]), which is ˜Λ (s)NΛ (s) to be skew-symmetric. Since equivalent to enforcing V˜Λ (s)MΛ (s)−U NΛ (s) and MΛ (s) are given from the plant description, it is clear that we must ˜Λ (s) and V˜Λ (s) to achieve this. choose U Towards this end, let F =

NΛ (p+1)×l ∈ RH∞ MΛ

(18)

10


and suppose the normal rank of F is r. When r < p + 1, F has a nontrivial left kernel of dimension q = p + 1 − r. We pick a basis for this kernel F ⊥ ∈ q×(p+1) RH∞ that has full row normal rank and satisfies F ⊥ (s)F (s) = 0 ∀s ∈ C ⊥ (such an F always exists when r < p+1 and this is easily seen via a McMillan decomposition of F ). We shall further require that F ⊥ is strictly proper. This is a mild restriction, but appears necessary for the results to follow. The case r < p + 1 is guaranteed to arise when l < p + 1, a situation which corresponds to having at least as many outputs as scalar parameters. The case may arise even if l >= p+ 1, but it is not generic. If p = 1, i.e. we are working with SISO systems; then r = 1 when there is one scalar parameter only. When r = p + 1, there is of course no nontrivial left kernel for F ; for consistency of notation q×(p+1) . Then a simple choice however, we let q = 1 and set F ⊥ = 0 ∈ RH∞ ⊥ T ˜ ˜ (19) −UΛ (s) VΛ (s) = α(s)F (s) + F (s)β(s) (p+1)×(p+1)

l×q with an arbitrary α ∈ RH∞ and an arbitrary β ∈ RH∞ that T satisfies β = −β (i.e. is skew-symmetric) does the job, since V˜Λ (s)MΛ (s) − ˜Λ (s)NΛ (s) is clearly skew-symmetric and hence U ˜Λ (s)NΛ (s))Λ = 0 ∀s ∈ C. ΛT (V˜Λ (s)MΛ (s) − U

Moreover, some lines of linear algebra shows that (19) includes all possible choices which ensure the quadratic form is zero. Notice for future reference ˜Λ (s) and V˜Λ (s) are necessarily strictly proper. that U For notational convenience, we define ˜ (s)NΛ (s) Φ(s) = V˜ (s)MΛ (s) − U (20) Inserting this expression in equation (17), we arrive at ˜ (s)N (s)] + Φ(s)Λ H(P, C) = G(λ, s) [V˜ (s)M (s) − U + ΛT [α(s)F ⊥ (s) + F T (s)β(s)]G(0, s)

−1

˜ K(λ, s).

Note that there is no loss of generality in forcing the nominal controller co˜ to satisfy the Bezout identity V˜ (s)M (s) − U ˜ (s)N (s) = prime factors V˜ and U ˜ 1, as can be seen from the following argument. Consider V (s)M (s)−U˜(s)N (s) = Z(s) = 1, where Z(s), Z(s)−1 ∈ RH∞ since the nominal controller is assumed to stabilize the nominal plant. Then it will be possible to extract a common ˜ −1 , which cancels the common factor Z(s) factor Z(s)−1 on the right of (KG) ˜ that can be extracted on the left of K(λ, s).

3 Controller Synthesis We are now ready to state our main results. We first present a controller synthesis result, and then show that the corresponding controller-plant interconnection has an appropriate multi-affine parameter dependence that allows us to invoke Theorem 1 in order to deal with time variations.


11

3.1 Parameter-Dependent Controller Construction The synthesis result presented in the following theorem relates the existence of a parameter-dependent controller to the feasibility of a set of linear matrix inequalities. Theorem 2. Consider the plant P (λ, s) given in (12) where T

Λ = [λ1 , λ2 , . . . , λr , λ1 λ2 , . . . , λr−1 λr , . . . , λ1 λ2 . . . λr ] , l = 2r − 1, ¯i ≤ and the parameter vector λ belongs to the polytope λbox = {λ ∈ Rr : −λ ¯i , i = 1, . . . , r}, and assume that the fractional description in (12) λi ≤ λ is coprime for all λ ∈ λbox , with NΛ (s) and MΛ (s) strictly proper. Assume furthermore that a nominal LTI controller C(0, s) that stabilizes P (0, s) has ˜ (s) be a left coprime factorisation over been found and let C(0, s) = V˜ −1 (s)U ˜ , V˜ satisfy the Bezout identity V˜ (s)M (s) − U ˜ (s)N (s) = 1. RH∞ such that U ˜Λ (s) Then there exists a controller C(λ, s) of the form (14), where U ˜ and VΛ (s) are chosen in accordance with (18), (19) and the associated remarks, that stabilizes P (λ, s) for all (frozen) λ ∈ Λ if there exist integers ˘ ∈ R1×(N1 +1) , Γ˘ ∈ N1 , N2 , N3 ∈ Z+ , constants τ1 , τ2 , τ3 ∈ R+ and matrices Π l×q(N2 +1) ˘ T (p+1)×(p+1) ˘ R , Ξi = −Ξi ∈ R for i ∈ {0, 1, . . . , N3 } that satisfy ˘ N1 (jω) + ΛT [Γ˘ B N2 (jω)F ⊥ (jω) Re [1 + Φ(jω)Λ]ΠB τ2 τ1 ˘N3 BτN3 (jω)]G(0, jω) > 0 ˘0 Ξ˘1 . . . Ξ + F T (jω) Ξ 3

(21)

for all ω ∈ R ∪ {∞} and all Λ = Λcori , where Λcori denotes the value assumed by Λ when λ ∈ λcori , i.e. λ assumes a value at a corner of λbox , and where G, F, F ⊥ , Φ are all defined in Section 2. Proof. We shall first establish that (21) is a necessary and sufficient condition for the stability of a closed loop formed from one of the set of plants Sδ , and an associated controller. The set Sλ of plants (12) is a subset of Sδ , and thus establishing the claim for Sδ will yield the result for Sλ . To establish the claim in relation to Sδ , we shall link a series of equivalent statements. Consider the plant set introduced in Section 1 defined by

P¯ (δ, s) = {

2r

2r

δi [N (s) + NΛ (s)Λcori ]}{ i=1

δi [M (s) + MΛ (s)Λcori ]}−1

i=1

δi ≥ 0 Consider also a corresponding set of controllers:

δi = 1

(22)

12


¯ s) = { C(δ,

2r

δi [V˜ (s) +

ΛTcori V˜Λ (s)]}−1 {

2r

˜ (s) + ΛT V˜Λ (s)]} δ i [U cori

i=1

i=1

δi ≥ 0

δi = 1

(23)

˜Λ (s) V˜Λ (s)] is chosen as described in the Theorem hypothesis. where [−U We assert now the equivalence of the following statements. ¯ s) of the form (23), where U ˜Λ (s) and V˜Λ (s) 1. there exists a controller C(δ, are chosen in accordance with (18), (19) and the associated remarks, that stabilizes P¯ (δ, s) in (22) for all (frozen) δi with δi ≥ 0 and δi = 1. (p+1)×(p+1) l×q 2. ∃α ∈ RH∞ and β ∈ RH∞ that satisfies β = −β T such that ¯ = G(λ, s) 1 + Φ(s) H(P¯ , C)

2r

δi Λcori

+

i=1 2r

δi ΛTcori

[α(s)F ⊥ (s) + F T (s)β(s)]G(0, s)

−1

˜ K(λ, s) ∈ RH∞

i=1

∀δi ≥ 0,

δi = 1,

˜ F, F ⊥ , Φ are all defined in Section 2. This is merely a rewhere q, G, K, statement of (a) above and the algebra for massaging H(P, C) into the form given here was discussed in Section 2. (p+1)×(p+1) l×q 3. ∃α ∈ RH∞ and β ∈ RH∞ that satisfies β = −β T such that 2r

2r

δi Λcori +

1+Φ(s) i=1

δi ΛTcori [α(s)F ⊥ (s)+F T (s)β(s)]G(0, s)

−1

i=1

∈ RH∞

∀δi ≥ 0,

δi = 1.

˜ Sufficiency is obvious since G(λ, s), K(λ, s) ∈ RH∞ . Necessity follows via the following argument: Since coprimeness of the fractional descriptions4 ˜ of P (λ, s) and C(λ, s) means that G(λ, s) and K(λ, s) have full column ¯ and full row rank respectively at all s ∈ C+ , then there exist left and right ˜ stable inverses for G(λ, s) and K(λ, s) respectively. (p+1)×(p+1) 1×1 l×q 4. ∃π ∈ RH∞ , α ∈ RH∞ and β ∈ RH∞ that satisfies β = −β T such that 4

Notice that if the fractional description of the plant P (λ, s) (resp. the controller ˆ ∈ Λ, then G(λ, s) (resp. K(λ, ˜ C(λ, s)) is not coprime for some λ s)) will drop ¯ + and for λ, ˆ and the interconnection of plant P (λ, s) in rank for some s0 ∈ C and controller C(λ, s) will not be internally stable. Thus the presumption that the controller C(λ, s) stabilises P (λ, s) automatically requires coprimeness of the fractional description for all λ of interest.


13

2r

Re

δi Λcori

1 + Φ(jω) i=1 2r

+

δi ΛTcori

[α(jω)F ⊥ (jω) + F T (jω)β(jω)]G(0, jω) π(jω)

>0

i=1

∀ω ∈ R ∪ {∞}, λ ∈ Λ. This equivalence is via [9]. (p+1)×(p+1) 1×1 l×q 5. ∃π ∈ RH∞ , α ∈ RH∞ and β ∈ RH∞ that satisfies β = −β T such that Re

1 + Φ(jω)Λcori + ΛTcori [α(jω)F ⊥ (jω) + F T (jω)β(jω)]G(0, jω) π(jω)

>0

∀ω ∈ R ∪ {∞}, i = 1, . . . , 2r , Necessity is obvious since if condition (d) holds for all δi ≥ 0, δi = 1, then it is also satisfied for the corner points. On the other hand, sufficiency follows by taking a positive linear combination of the inequalities of condition (e). (p+1)×(p+1) 1×1 l×q 6. ∃π ∈ RH∞ , γ ∈ RH∞ and ξ ∈ RH∞ that satisfies ξ = −ξ T such that Re [1 + Φ(jω)Λcori ]π(jω) + ΛTcori [γ(jω)F ⊥ (jω) + F T (jω)ξ(jω)]G(0, jω)

>0

∀ω ∈ R ∪ {∞}, i = 1, . . . , 2r . Necessity is easy via a relabling of γ = απ and ξ = βπ. Sufficiency follows via the following argument: Summing up the 2r inequalities above yields 2r Re{π(jω)} > 0

∀ω ∈ R ∪ {∞}

since all the other terms disappear due to the symmetry about zero of the polytope considered. Consequently, π is stable and strictly positive 1×1 real, which implies it must be a unit in RH∞ . Hence, given any γ ∈ (p+1)×(p+1) l×q RH∞ and any ξ ∈ RH∞ that satisfies ξ = −ξ T , it is possible (p+1)×(p+1) l×q to construct α = γ/π ∈ RH∞ and β = ξ/π ∈ RH∞ that T satisfies β = −β . 7. there exist integers N1 , N2 , N3 ∈ Z+ , constants τ1 , τ2 , τ3 ∈ R+ and ma˘ ∈ R1×(N1 +1) , Γ˘ ∈ Rl×q(N2 +1) , Ξ ˘i = −Ξ ˘ T ∈ R(p+1)×(p+1) for trices Π i i ∈ {0, 1, . . . , N3 } that satisfy

14


˘ τN1 (jω) + ΛT [Γ˘ BτN2 (jω)F ⊥ (jω) Re [1 + Φ(jω)Λ]ΠB 2 1 ˘0 Ξ ˘1 . . . Ξ˘N3 BτN3 (jω)]G(0, jω) > 0 + F T (jω) Ξ 3 for all ω ∈ R ∪ {∞} and all Λ = Λcori . Here, BτNi i (s) (i ∈ {1, 2, 3}) denotes Laguerre basis matrices as defined in (11). The index m in (11) takes the values 1, q and p + 1 respectively. 1×1 ˘ τN1 (s) ∈ RH∞ Sufficiency is trivially easy via a labelling π = ΠB ,γ = 1 (p+1)×(p+1) N2 N3 l×q ˘ ˘ ˘ ˘ Γ Bτ2 (s) ∈ RH∞ and ξ = Ξ0 Ξ1 . . . ΞN3 Bτ3 (s) ∈ RH∞ ˘i is skewand noting that the constructed ξ also satisfies ξ = −ξ T since Ξ symmetric for all i ∈ {0, 1, . . . , N3 }. Necessity on the other hand follows via the following argument: Since Laguerre parametrisations provide a uniform approximation of objects in RH∞ and furthermore since the inequalities in (f) are strict inequalities, it follows that there always exists sufficiently large Ni and sufficiently small τi (i ∈ {1, 2, 3}) to ensure that the truncation error does not alter the sign of the inequalities. At this point, we have a necessary and sufficient condition for the controllers (23) to stabilize the corresponding plant in the set Sδ , see (22). Now recall from Section 1 that for any Λ generated by a generic λ ∈ λbox , there exists a set of nonnegative weights δi summing to 1 such that 2r

δi [N (s) + NΛ (s)Λcori ]

N (s) + NΛ (s)Λ =

(24)

i=1

This is the standard convex representation and there are three similar equations for M (s) + MΛ (s)Λ and the two fractional components of the controller. Recall also that the injective mapping from Λ to the δi is independent of NΛ (s) etc. It follows that the set of plants P (λ, s) and controllers C(λ, s), λ ∈ λbox , are respectively subsets of the set of plants P¯ (δ, s) and ¯ s). Hence condition (g) is a sufficient condition guaranteeing controllers C(δ, the C(λ, s) stabilize the corresponding P (λ, s). 3.2 Incorporating a Degree of Stability in the Frozen Parameter Design As pointed out in the Introduction, Theorem 1 relates a certain degree of stability of a collection of frozen parameter systems to the degree of stability of a time-variable parameter system given a bound on the average rate of change of the parameters. In order to allow a reasonable amount of timevariation in the parameter λ, we have to ensure that the closed loop with frozen parameter values has a degree of stability of σ and that this property is expressible with a particular Lyapunov function. We shall deal with the Lyapunov function in the next subsection, and the degree of stability issue here. The results of Subsection 3.1 only ensure the closed-loop is stable with


15

no extra degree of stability, i.e., that the closed-loop poles are only guaranteed to be in the left half of the complex plane for all λ ∈ λbox . Fortunately, this difficulty is straightforward to address. We will first shift the given plant by an amount σ > 0 to the right in the complex plane, work with this σ-shifted version of the plant, design a controller for this shifted plant such that the plant/controller interconnection is stable for all permissible values of λ, and then shift the controller back to obtain the necessary degree of stability σ for the closed-loop interconnection. In more detail, assume the original given plant that we wish to control is Pˆ (λ, s) ∈ Rp×1 and that the shifted plant, denoted by P (λ, s), is defined by P (λ, s) := Pˆ (λ, s − σ) ˆ (s − σ) + N ˆΛ (s − σ)Λ)(M ˆ (s − σ) + M ˆ Λ (s − σ)Λ)−1 (25) = (N p×1 ˆ p×l ˆ 1×1 ˆ (s − σ) ∈ RH∞ where N , NΛ (s − σ) ∈ RH∞ , M (s − σ) ∈ RH∞ and 1×l ˆ MΛ (s − σ) ∈ RH∞ and σ > 0 is a real constant. Of course, (25) must be a coprime factorization for all λ of interest, which is a stronger condition than ˆ (s) + N ˆΛ (s)Λ)(M ˆ (s) + M ˆ Λ (s)Λ)−1 being a coprime realization of requiring (N ˆ P (λ, s). Design a controller C(λ, s) for the shifted plant P (λ, s) according to the results presented in the previous two subsections. This controller C(λ, s) ˆ s) that will should then be shifted back to get the correct controller C(λ, actually be implemented in closed loop with the original given plant Pˆ (λ, s). ˆ s) is given by This controller C(λ,

ˆ s) := C(λ, s + σ). C(λ, Note that this σ-shift does not ruin the multi-affine parameter dependency appearing in the fractional descriptions of the plant and controller, nor in the closed-loop quantity required to have no zeros in the closed right half plane. 3.3 Securing a Quadratic Lyapunov Function with Multi-affine Parameter Dependence Theorem 1 is the key result from which one can conclude retention of stability even in the face of parameter variations. In this subsection we explain how the Lyapunov equation (8) can be satisfied with a multi-affine P (λ). The argument is akin to that in [11]. With notation as arising in the proof of Theorem 2, define for i = 1, 2, . . . , 2r strictly positive real functions Zi (s) by Zi (jω) = 1 + Φ(jω)Λcori + ΛTcori α(jω)F ⊥ (jω) + F T (jω)β(jω) G(0, jω) π(jω)

(26)

where, without loss of generality, we assume π(s) has been normalised to satisfy π(∞) = 1. Suppose that each Zi (jω) has a minimal state-variable

16


realization 1 + cTi (sI − A)−1 b. [Recall that Φ(s) and α(s)F ⊥ (s) + F T (s)β(s) are strictly proper; this ensures Z(∞) = 1]. We are implicitly ruling out the possibility of any Λcori being non-generic in the sense of giving rise to a pole-zero cancellation, or nonminimality for some i of the state-variable realization. As recalled in [11] by the Kalman-Yakubovic Lemma, there exist positive definite symmetric Pi and Qi with −Pi A − AT Pi − Qi Pi b − ci (Pi b − ci )T 2

>0

(27)

Now we know that for any λ ∈ λbox , the transfer function Z(λ, jω) = [1 + Φ(jω)Λ + ΛT [α(jω)F ⊥ (jω) + F T (jω)β(jω)]G(0, jω)]π(jω) (28) is expressible as a convex combination of the Zi (jω), and accordingly is also strictly positive real. Moreover, the standard convex representation in Section 1 implies the existence of a unique multi-affine P (λ), Q(λ) and c(λ) (expressible with the same weights as a convex linear combination of the respective corner values as for Z(λ, jω)) for which Z(λ, jω) = 1 + cT (λ)(jωI − A)−1 b

(29)

and −P (λ)A − AT P (λ) − Q(λ) P (λ)b − c(λ) >0 (P (λ)b − c(λ))T 2

(30)

Of course, P (λ), Q(λ) and c(λ) assume the values Pi , Qi and ci at the corners of λbox . Now when the shifting procedure of subsection 3.3 is used, it actually results that Zi (jω − σ) and Z(λ, jω − σ) are strictly positive real, which means that −P (λ)A − AT P (λ) − Q(λ) − 2σP (λ) P (λ)b − c(λ) >0 (P (λ)b − c(λ))T 2

(31)

Performing a congruence transformation I c(λ) 0 I on inequality (31) yields inequality (32) below −P (λ)[A − bcT (λ)] − [A − bcT (λ)]T P (λ) − Q(λ) − 2σP (λ) P (λ)b + c(λ) >0 (P (λ)b + c(λ))T 2 (32) which is associated with X −1 (s) = 1 − cT (sI − A + bcT )−1 b being strictly positive real, due to X(s) = 1 + cT (sI − A)−1 b being strictly positive real.


17

From inequality (32), it is immediate that [A − bcT (λ)]T P (λ) + P (λ)[A − bcT (λ)] < −2σP (λ)

(33)

Now the frozen closed-loop characteristic polynomial as derived in the proof of Theorem 2 is the numerator of 1 + Φ(s)Λ + ΛT [α(s)F ⊥ (s) + F T (s)β(s)]G(0, s)

(34)

which is a factor of the numerator of [1 + Φ(s)Λ + ΛT [α(s)F ⊥ (s) + F T (s)β(s)]G(0, s)]π(s) = 1 + cT (λ)(sI − A)−1 b (35) Thus the closed-loop characteristic polynomial is a factor of the characteristic polynomial of A − bcT (λ). Equation (33) above, Theorem 1 and the multi-affine character of P (λ) then assure stability for time-varying λ, as summarized in the following theorem. Theorem 3. Consider a plant Pˆ (λ, s) and the associated σ-shifted plant P (λ, s) as given in (25). Assume that a controller of the form (14) that satisfies the suppositions of Theorem 2 has been synthesised for the σ-shifted plant ˆ s) := C(λ, s+σ) as the controller that is actually P (λ, s). Further, define C(λ, implemented with the original plant Pˆ (λ, s). Then the closed-loop interconnecˆ s) is exponentially asymptotically stable with degree tion of Pˆ (λ, s) and C(λ, of stability γ ∈ [0, σ] for all time-varying λ(t) ∈ λbox that satisfy conditions (9) and (10) of Theorem 1.

4 Design Procedure We will now provide an explicit procedure that can be followed in order to construct the LPV controller proposed above. We will assume that a description of the plant Pˆ (λ, s) with multi-affine parameter dependence of the form (1) is available. If the plant is described in state space, it can be rewritten in transfer matrix form by following the procedure in [6]. 1. Rescale, if necessary, the polytope λbox so that it is symmetric around λ = 0 and obtain P (λ, s) by shifting Pˆ (λ, s) by σ > 0 as detailed in (25). 2. Design a nominal controller C(0, s) for the nominal shifted plant P (0, s) ˜ (s) and V˜ (s) satisfying a Bezout identity and calculate coprime factors U with the nominal plant coprime factors N (s) and M (s). 3. Define G(λ, s) as in (13), F (s) as in (18) and construct F ⊥ (s) as described immediately after (18) ensuring F ⊥ (s) is strictly proper. Define Φ(s) as in (20) and BτNi i (s) for i ∈ {1, 2, 3} as in (11), with m = 1, q, (p + 1) respectively. 4. Set up the 2r linear matrix inequalities (21) corresponding to every vertex in λbox and specify an appropriate ω-grid in order to obtain a finitedimensional problem (a set of LMIs). Attempt to solve (as is standard,

18


˘ ∈ R1×(N1 +1) , Γ˘ ∈ Rl×q(N2 +1) , Ξ ˘i = see [13]) these LMIs for matrices Π (p+1)×(p+1) T ˘ ∈ R for i ∈ {0, 1, . . . , N }: start with small integers −Ξ 3 i N1 , N2 , N3 and large τ1 , τ2 , τ3 > 0, yielding a crude Laguerre parametrisation, and if infeasible, gradually increase N1 , N2 , N3 and decrease τ1 , τ2 , τ3 > 0 until the LMIs can be solved. ˘ ∈ R1×(N1 +1) , Γ˘ ∈ Rl×q(N2 +1) , Ξ ˘i = −Ξ ˘ T ∈ R(p+1)×(p+1) 5. Once matrices Π i ˘ τN1 (s) ∈ for i ∈ {0, 1, . . . , N3 } have been found, construct π = ΠB 1 1×1 N2 l×q ˘ ˘ ˘ ˘ RH∞ , γ = Γ Bτ2 (s) ∈ RH∞ and ξ = Ξ0 Ξ1 . . . ΞN3 BτN33 (s) ∈ (p+1)×(p+1)

RH∞ . (p+1)×(p+1) l×q 6. Define α = γ/π ∈ RH∞ and β = ξ/π ∈ RH∞ , and model reduce if necessary (one should check that the reduced quantities still satisfy the LMIs). 7. Compute ˜Λ (s) V˜Λ (s) = α(s)F ⊥ (s) + F T (s)β(s) −U as described in (19) and construct C(λ, s) as given in (14). ˆ s) = C(λ, s+σ). 8. Shift the controller by σ to obtain the final controller C(λ, ˆ s) and Pˆ (λ, s) will then have degree of The closed loop involving C(λ, stability σ > 0 for all (frozen) λ ∈ λbox , and will also be guaranteed to be exponentially asymptotically stable with degree of stability γ ∈ [0, σ) for all time-varying λ(t) ∈ λbox that satisfy conditions (9) and (10) in Theorem 1. ¯ i , without really For a given (fixed) σ > 0, we can attempt to maximize λ worrying about the possible isolated points where we might lose coprimeness, as the existence of a solution to the LMIs is still sufficient for H(P, C) to ¯i at a time (or all together, be stable. Maximization over each separate λ ¯ ¯ ¯ assuming λ1 = λ2 = · · · = λl ) is a generalized eigenvalue (i.e., quasi-convex) problem, which can be easily solved using commercially available software. ¯i and optimize Consequently, one strategy is to first set the same value for all λ this value until the largest hyper-cube of feasible parameters is found. Then, ¯ i is fixed for some i and the optimization is repeated for a hyper-cube of λ lower dimension, and so forth. By this strategy, eventually, a large polytope symmetric around λ = 0 for which the LMIs are satisfied will be found, allowing for correspondingly large parameter variations. ¯ a bisection algorithm may be To find a good trade-off between σ and λ, run for different values of σ, where the quasi-convex optimization strategy outlined above can be carried out for each chosen value of σ. One has to employ a bisection algorithm (or similar technique) to optimise σ because σ enters the expressions for Φ(s), F (s), F ⊥ (s), G(0, s), . . . in a highly nonlinear fashion.


19

Perturbed system, nominal control

4

1

2

y (−) , y (−.−)

2 0 −2 −4 −6 −8

0

5

10

15

20

25

30

0

5

10

15 Time

20

25

30

2

u (−)

0 −2 −4 −6 −8

ˆ s); Fig. 3. Simulation of Pˆ (λ, s) interconnected with the nominal controller C(0, the parameters are fixed to λ = [−1, −0.98]T . The system is disturbed by a step in w2 (see Figure 2) to w2 = −1 at time t = 0 followed by another step back to w2 = 0 at time t = 5.

5 Numerical Example Here, we will illustrate the procedure outlined in Section 3 with a numerical example. Consider the unshifted LPV plant Pˆ (λ, s) given by Pˆ (λ, s) =

n1 (s)+λ1 ν11 (s) m(s)+λ1 λ2 µ(s) n2 (s)+λ2 ν22 (s) m(s)+λ1 λ2 µ(s)

where the nominal part is given as Pˆ (0, s) =

n1 (s) m(s) n2 (s) m(s)

=

1 s4 +2.32s3 +7.48s2 +9.16s+5 s s4 +2.32s3 +7.48s2 +9.16s+5

and the system variations given by ν1 (s) = 0.13s + 0.0494, ν2 (s) = 0.156 and µ(s) = 0.195s + 0.2041. The parameters λ = [λ1 , λ2 ]T are allowed to vary within the square λbox = [−1 ; 1]2 . The plant is shifted by σ = 0.38 and an observer-based state space compensator C(0, s) is designed to place the closed-loop poles in s = −1, s = −2, ..., s = −8. Figure 3 shows a closed-loop simulation with λ = [−1, −0.98]T ; as can be seen, the nominal controller is able to stabilize the system for this fixed value of the parameters. The nominal controller also stabilises the system for the four corners of λbox and indeed throughout λbox .

20

B.D.O. Anderson, A. Lanzon, and J. Bendtsen LPV system, nominal control

5

−10

1

2

y (−) , y (−.−)

0 −5

−15 −20

0

5

10

15

20

25

30

0

5

10

15 Time

20

25

30

5

u (−)

0 −5 −10 −15 −20

ˆ s) and Fig. 4. Simulation of Pˆ (λ, s) interconnected with the nominal controller C(0, time varying parameters. The same disturbance as in Figure 3 is used. The system becomes unstable due to the time variation.

However, when λ is allowed to vary as   sin 5t λ = λ(t) =  cos 5t  sin 5t cos 5t the nominal controller is not able to stabilize the system, even in case of a vanishing disturbance. This is shown in Figure 4. Notice that the frequency of variation is substantial in comparison with the degree of stability of the frozen-parameter nominal closed-loop system. The plant and controller are now factorized as   1 d(s)

N (s) s−0.38   = , d(s) M (s) 4 3 2 s +0.8s +5.702s +4.261s+2.493  0.13s d(s)

NΛ (s)  = 0 MΛ (s) 0

d(s)

0

0.156 d(s)

0

0 0

0.195s+0.13 d(s)

  

where d(s) = s4 + 10s3 + 35s2 + 50s + 24, and 3 4 2 5 4 ˜ (s) = −6046s − 7.679 × 10 s − 2.248 × 10 s + 6.125 × 10 U 4 3 2 s + 26s + 251s + 1066s + 1680 3 −4280s − 2.867 × 104 s2 + 3.394 × 104 s + 1.888 × 105 s4 + 26s3 + 251s2 + 1066s + 1680


21

LPV system, LPV control

2

0 −1

1

2

y (−) , y (−.−)

1

−2 −3

0

5

10

15

20

25

30

0

5

10

15 Time

20

25

30

2

u (−)

1 0 −1 −2 −3

ˆ Fig. 5. Simulation of Pˆ (λ, s) interconnected with the LPV controller C(λ, s). The same disturbance and parameter variation as in Figure 4 is used; the closed-loop system remains stable.

s4 + 35.2s3 + 512.1s2 + 3921s + 1.196 × 104 V˜ (s) = , s4 + 26s3 + 251s2 + 1066s + 1680 respectively. NΛ (s) In our case, F (s) = is diagonal and obviously has full normal MΛ (s) ⊥ rank. Hence, the annihilator F (s) for F (s) must be zero, and α(s) is not needed. We define a grid over the jω axis (in this case, equidistant in the interval from ω = 10−2 rad/s to ω = 105 rad/s, along with infinity) and set up the synthesis LMIs (21) using Matlab’s LMILAB toolbox. In this example, the LMIs were found to be feasible for N1 = 1,N3 = 0 and τ1 = 0.05. Note that N2 does not enter the picture because α(s) is absent, and τ3 is not needed because N3 = 0. We were able to compute the parameter-dependent factors in the controller as   50.72s2 +2029s 0 ρ(s)  −60.86s−2435 ˜Λ =  0 U   ρ(s)  V˜Λ =

30.15s2 +1226s+804 181.1s2 +7367s+4830 ρ(s) ρ(s)



20.1s2 +804s ρ(s)   144.9s+5797   ρ(s)

0

with ρ(s) = s5 + 50.27s4 + 437.7s3 + 1459s2 + 2037s + 966.4. This controller is then shifted back (by σ = −0.38) and implemented in closed loop with Pˆ (λ, s). Figure 5 shows a simulation of the closed loop with the LPV controller, in which it is seen that the controller is now able to stabilize the system.

22


6 Discussion In this paper we presented a novel procedure for design of controllers for LPV systems whose transfer functions display multi-affine parameter dependence. This class of systems encompasses many physical systems, and is therefore of significant practical relevance to control designers. The controller structure was chosen to reflect the structure of the plant, and the terms associated with parameter dependencies in the controller were chosen to remove terms that are not multi-affine in the parameters. The idea was to obtain a multiaffine dependence for the parameters of the closed loop A-matrix, and thus allow the designer to obtain guarantees on the extent of time variation in the parameters that can be allowed. The problem of finding the controller parameter dependencies was cast as a finite-dimensional LMI problem. We outlined the procedure in a constructive manner for a particular choice of controllers and illustrated the feasibility of the method with a numerical example. With r parameters, we actually have 2r different frequency domain conditions which have to be fulfilled. As it stands, the method is limited to apply to single-input (but possibly multi-output) plants. This is due to the fact that some of the manipulations of ˜ −1 rely on this quantity being scalar. This is a weakness of the current (KG) design and should be alleviated in future research if possible. Finally, it should be noted that all the results presented deal with stability, not performance, i.e., we are not able to quantify loss of performance compared to the nominal designs, either due to time variations or due to the constrained way of handling the parameter dependence in the controller. This would be interesting to deal with in the future.

Acknowledgements This work has been supported in part by ARC Discovery-Projects Grant DP0342683 and National ICT Australia Ltd. National ICT Australia Ltd. is funded through the Australian Government’s Backing Australia’s Ability initiative, in part through the Australian Research Council. Large parts of this research were performed during a month’s visit by the third author at the Australian National University and National ICT Australia Ltd.

References 1. M. Mattei, “An LPV approach to the robust control of a class of quasi-linear processes,” Journal of Process Control, vol. 14, pp. 651–660, 2004. 2. P. Apkarian, J. M. Biannic, and P. Gahinet, “Self-scheduled H∞ control of missile via linear matrix inequalities,” AIAA Journal on Guidance, Control and Dynamics, vol. 18, pp. 532–538, 1995.


23

3. F. Bruzelius, S. Petterson, and C. Breitholtz, “Linear parameter-varying descriptions of nonlinear systems,” in Proc. of the 2004 American Control Conference, 2004. 4. J. S. Shamma and M. Athans, “Analysis of gain scheduled control for nonlinear plants,” IEEE Transactions on Automatic Control, vol. 35, pp. 898–907, 1990. 5. W. Rugh and J. S. Shamma, “Research on gain scheduling,” Automatica, vol. 36, pp. 1401–1425, 2000. 6. S. Dasgupta and B. D. O. Anderson, “Physically based parameterizations for designing adaptive algorithms,” Automatica, vol. 23, pp. 469–477, 1987. 7. D. J. Leith, A. Tsourdos, B. A. White, and W. E. Leithead, “Application of velocity-based gain-scheduling to lateral autopilot design for an agile missile,” Control Engineering Practice, vol. 9, pp. 1079–1093, 2001. 8. L. Zadeh and Desoer, Linear Systems Theory. Krieger Publishing Company, 1979. 9. B. D. O. Anderson, S. Dasgupta, P. Khargonekar, F. J. Kraus, and M. Mansour, “Robust strict positive realness: Characterization and construction,” IEEE Transactions on Circuits and Systems, vol. 37, pp. 869–876, 1990. 10. A. Rantzer and A. Megretski, “A convex parameterization of robustly stabilizing controllers,” IEEE Transactions on Automatic Control, vol. 39, pp. 1802–1808, 1994. 11. S. Dasgupta, B. D. O. Anderson, G. Chockalingam, and M. Fu, “Lyapunov functions for uncertain systems with applications to the stability of time varying systems,” IEEE Transactions on Circuits and Systems, vol. 41, pp. 93–106, 1994. 12. A. Lanzon, “Pointwise in frequency performance weight optimization in musynthesis,” International Journal of Robust and Nonlinear Control, vol. 15, pp. 171–199, 2005. 13. P. Gahinet, A. Nemirovski, A. J. Laub, and M. Chilali, LMI control toolbox, The MathWorks, Inc., 1995, for use with Matlab.

Control of High-Speed Underwater Vehicles Gary J. Balas1 , József Bokor2 , Bálint Vanek1 , and Roger E.A. Arndt3 1 2 3

Aerospace Engineering and Mechanics, 110 Union St. SE, University of Minnesota, Minneapolis, MN 55455 [email protected], [email protected] MTA SZTAKI, Hungarian Academy of Science, Budapest XI Kende u. 13-17, H-1518 Budapest, POB 63, Hungary [email protected] Civil Engineering and St. Anthony Falls Laboratory, 2 Third Ave. SE, University of Minnesota, Minneapolis, MN 55414 [email protected]

I (GB) was fortunate that early in my career I had the opportunity to work with Keith Glover. Keith had just begun his obsession with golf. A simple game where a ball stands still and the player advances it from different parts of the landscape into a hole. During a golf outing, Keith vigorously struck a ball off the fairway. It drove through a water hazard and miraculously ended up a few feet off the green. I was amazed by the image I saw as the ball moved through the water. Due to its velocity, a cavitation bubble was generated behind it. I had no inkling that my golf outing with Keith would lead many years later to an interest in the control of supercavitating vehicles.

1 Introduction An object traveling at very high speeds generates cavitation bubbles on the corner of sharp contours due to pressure gradients that result in flow separation. This is often the case if a propeller spins fast enough, the surrounding liquid is vaporized due to the decrease in pressure generating cavitation bubbles [1]. As pressure surrounding the liquid decreases, the bubbles collapse. B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 25–44, 2006. © Springer-Verlag Berlin Heidelberg 2006

26

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt

Cavitation bubbles contain significant energy that can lead to pitting on the blades and reduced efficiency. Minimization or reduction of cavitation is the focus of significant engineering effort. Alternatively there is a significant benefit in some circumstances to increased cavitation or supercavitation. Supercavitation can be used by vehicles to dramatically reduce drag, enabling vehicles able to maintain high high speed flight under water. Supercavitation results in a single, stable bubble that can envelope an entire vehicle. Hence only the front tip of the vehicle, upon which is often mounted a cavitator, is in contact with the fluid. The cavitator is designed to create a low pressure cavity around the vehicle when traveling through the fluid at high speeds. The cavity may be ventilated with gas to increase its size and reduce the operating speed needed to generate a cavity. Drag on a supercavitating vehicle is dramatically reduced due to the vehicle flying in a gas rather than a fluid. The Russian military has exploited the benefits of supercavitation with the development of the Shkval torpedo which is claimed to reach speeds of 100 m/s [2]. This is roughly a factor of 5 faster than the fastest traditional underwater vehicles. A picture of Shkval vehicle is shown in Figure 1.

Fig. 1. Shkval vehicle

Steering and stabilization within the cavity area are significant technical challenges facing high speed supercavitating vehicles (HSSVs) [3]. The stability of the vehicle is a function of the cavitator size, shape and the size and immersion of the control surfaces. The cavity generated by the cavitator surrounds the vehicle though they may interact at the vehicle trailing edge and at the points where the control surfaces impinge on the cavity wall. The low buoyancy acting on the body requires the vehicle weight to be supported by the cavitator and control surfaces, if present. Body interaction with the cavity walls which leads to high restoring forces known as planing. Planing can be used as a support force to improved maneuverability and eliminate the fin actuators, there by reducing cost and complexity. Planing though can lead to limit cycle oscillations and increases drag if not controlled. Hence, there are many configurations and tradeoffs associated with control surface selection [5].

Control of High-Speed Underwater Vehicles

27

Even if the body is stable inside the cavity, the vehicle may not be stable when in contact with the cavity. Nonlinear interaction of the control surfaces and the body with the cavity wall is very important when calculating the fin and planing forces acting on the vehicle. Note that the distance between the body and the cavity wall is on the order of 4-10 cm. The cavity wall exerts a large restoring force over the short period of time. The nature of this instability forces the vehicle back into the cavity, often resulting in limit cycle behavior [3]. Cavity-vehicle interaction also exhibits strong memory effects and cavity shape is a function of the history of the vehicle motion. Suppression of limit cycles oscillations and disturbance attenuation require high bandwidth actuators and high sample rate real-time control algorithms. This paper investigates the control challenges associated with supercavitating vehicles using a low order, longitudinal axis vehicle model. The first part of the paper presents the simplified pitch-plane model of a HSSV vehicle. This model draws heavily on the benchmark HSSV model proposed by Dzielski and Kurdila [3]. That model is extended to include delay dependent interaction of the vehicle and cavity and a more refined planing model. The second part of the paper describes the control approach applied to the supercavitating vehicle model. The plant model is described as a switched, bimodal system whose switching surface, the cavity boundary, is delay dependent [9]. Controllability of delay dependent, bimodal systems is discussed and a systematic control design strategy based on dynamic inversion is proposed. A controller is synthesized to track pitch angle, angular rate, vertical position and vertical speed for the HSSV vehicle model using the proposed approach. Simulations of the closed-loop vehicle are performed and analyzed in the third section of the paper. Challenges facing the control designer are highlighted with respect to actuator and sensor requirements, modeling issues, robustness and performance. The paper concludes with a summary of the results and directions for future study.

2 Modeling of Supercavitating Vehicles Several mathematical descriptions of high speed supercavitation vehicles (HSSVs) are available in the literature [3, 6, 8]. A brief synopsis of these models follows. •

•

A high fidelity, nonlinear six degrees-of-freedom (DOF) mathematical model was proposed in [8]. Lookup tables are used to calculate the forces generated by control surfaces as a function of immersion depth. The centerline of the cavity bubble is a function of the time history of the path of the vehicle nose affected by buoyancy. A two DOF longitudinal axis model for control design purposes was proposed in [3]. The simplified model is linear within the cavity and nonlinear when the vehicle is in contact with the cavity (planing). The planing con-

28

•


dition is defined as a function of vertical velocity and no time history dependency of the cavity is included. A one DOF longitudinal axis model was proposed to study the switching behavior of the supercavitating vehicle in [6]. The model contains only longitudinal translational motion and its interaction with the cavity. The shape of the cavity boundary is described as a function of the time delayed position of the vehicle nose.

This paper extends the two DOF longitudinal axis model developed in [3] to include the time dependent behavior of the cavity and redefines the planing condition to be a function of the vehicle vertical position, pitch angle and time history of the cavity. The only characteristic of the longitudinal axis, nonlinear six DOF model not included in the extended model is the nonlinear control surface force and moments dependence on the surface immersion depth and angle-of-attack. 2.1 Two Degree of Freedom Model with Delay A natural way to simplify the complex behavior of the 6-DOF model is to restrict the motion of the vehicle to the longitudinal plane. Consider the two degree of freedom vehicle shown in Figure 2.

Fig. 2. Schematic diagram of two degree of freedom model

The geometry of the vehicle is intended to approximate the geometry of the actual vehicle. The body consist of a conical and a cylindrical section, where the cylindrical section is twice as long as the front conical section. The reference coordinate system is placed into the cavitator’s center, the x axis is the horizontal axis (perpendicular to z) in direction of the cavitator. θ is the angular rotation of the x-axis. The z axis is oriented towards the center of Earth. The body length is denoted by L and its radius is R. The body has uniform density ρb , from which the mass and inertia are calculated. The reference point is at the cavitator, hence the inertia matrix is computed at


29

that point. If the full vehicle body is in the cavity, hydrodynamic forces will only act on the cavitator and fins. The cavitator drag coefficient is modeled as Cx = Cx0 (1 + σ) where σ is the cavitation number and Cx0 = 0.82 [4]. The resulting lift on the cavitator is approximately: Fcav =

1 πρRn2 V 2 Cx αc = Cl αc 2

(1)

where Rn is the cavitator radius, ρ water density, V vehicle velocity, δc cavitator deflection and αc = Vw + δc is the cavitator angle of attack. The force acting on the fins is modeled as Ff in = −nCl αf

(2)

where n denotes the relative effectiveness of the fins to the cavitator, δf the + δf is the fins angle of attack. All fin deflection, q pitch rate and αf = w+qL V angles are assumed to be small, hence small angle approximations apply to calculate the forces. The dynamic equations of motion, after dividing through by (πρmR2 L), are       1−n −n 7 17L w ˙ w 36    = CV  mL m     9 (3) 133 2 17L 11 2 −n −n q˙ q 36 60 R + 405 L m mL  +V 

0 0

 

7 9  w  17L q 36



+ CV

 

−n 1 2  mL mL  δf  −n 0 δc m

+ Fgrav + Fplane (t, τ )

2

where C = 21 Cx RRn . The time delay between the fluid impinging on the cavity and the aft end of the vehicle is δ(t) = VL . The time dependence of the states and controls is suppressed in the paper unless it is needed for clarity. The force and moments due to planing, Fplane [7], and gravity, Fgrav , are     7 1 R 1 + h )2 ( )αp , Fgrav =  9  g, (4) Fplane = −V 2  mL  (1 − 1 17L h +R 1 + 2h m

36

h is the normalized immersion depth corresponding to the distance the body penetrates the water and αp is the immersion angle corresponding to the angle the body surface makes with the cavity. R is the normalized median distance Rc −R between the transom and the cavity. The parameter values used in the R numerical calculations are given in Table 1. In the original two degree-of-freedom model proposed in [3], the interaction of the vehicle and cavity was not a function of the cavitator position. Hence it was not delay dependent. Vertical velocity, w(t), was used to predict the occurrence of planing assuming zero pitch angle and steady state

30

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt Table 1. System parameters for simulation model [3] Parameter

Description

Value and Units

g m n Rn R L V σ Cx0

Gravitational acceleration Density ratio, ρρm Fin effectiveness Cavitator radius Vehicle radius Length Velocity Cavitation number Lift coefficient

9.81 sm2 2 0.5 0.0191m 0.0508m 1.80m 75 m s 0.03 0.82

horizontal speeds. For short delay times, this is a valid assumption. In the extended model, the planing condition is a function of the cavity boundary and a delayed function of the cavitator position. This requires an extension of the vehicle state space description by two additional states: dz = w − V θ. z(t) vertical position of vehicle at the nose, dt dθ • θ(t) vehicle pitch angle, = q. dt The centerline of the cavity is determined by the path of the nose through the fluid, see Figure 2. It is assumed that the diameter of the cavity at any given location depends only on the length between its centerline position and the actual position of the nose. The resulting equations of motion are     z˙ z  θ˙  θ    = A   + B δe + Fgrav + Fplane (t, τ ) (5) w˙  w  δc q q˙

•

The matrices A and B can be derived from equations (3, 2.1) and the definitions of the added states z and θ. The vehicle motion as stated above is a nonlinear, switched delayed-state dependent system. Contact with the gas filled cavity bubble is responsible for the nonlinear effects. Three situations are possible during the longitudinal motion: (i) the entire vehicle is inside the cavity no forces are generated by planing, (ii) the transom (aft body) is immersed in the bottom of the cavity (+ direction), or (iii) the planing occurs as the transom hits the top of the cavity. In both situations the resulting planing force acts in the opposite direction of the immersion, providing a large enough force to bounce the body back into the cavity, leading to oscillatory motion like a fast boat bouncing on the top of water. The planing conditions and immersion angle are given by the following equations:


31

 1  R [z(t) + θL + R − z(t − τ ) − Rc ] if z(t − τ ) + Rc < z(t) + θL + R h = 0 (inside cavity) 1 [z(t − τ ) − R − z(t) − θL + R] if z(t − τ ) − Rc > z(t) + θL + R c R (6)  

θ−

w(t − τ ) + R˙ c V

bottom contact

(7) inside cavity ˙c w(t − τ ) − R θ − top contact V It can be seen by the geometric relationships how the two different planing conditions are stated. The position of the vehicle transom is a function of the vehicle nose position, rotation and vehicle radius at the transom. The cavity’s boundary is located at Rc distance from centerline position of the cavity τ time before (z(t − τ )), the immersion is the difference between the two. The immersion angle is a function of three different variables: The nominal cavity shape is due to a nonconstant cavity radius through the length of the body (R˙ c ), the shape of the cavity distorted by the past vertical velocity of the vehicle (w(t − τ )), and the vehicle rotation during boundary penetration, (θ). All these expressions use small angle approximations around straight level flight, assuming also that the horizontal speed of the vehicle (V ) is constant and the surrounding environment including the water temperature, pressure do not change. To simplify the nonlinear equations of the planing forces, two numerical constants are defined [7]: αp =

κ1 =

0

L 1.92 ( − 3)−1 − 1, Rn σ

κ2 = (1 − (1 −

4.5σ 40 1 )κ 17 ) 2 1+σ 1

(8)

Using constants κ1 and κ2 , the relationship for the cavity radius at the transom is: 1+σ 1 Rc = Rn (0.82 (9) ) 2 κ2 σ The expression for the contraction rate of the cavity (R˙ c ) is: 23

4.5σ 1 + σ 1 (1 − 1+σ κ117 ) 20 )2 V R˙ c = − (0.82 17 σ κ2 ( 1.92 σ − 3)

(10)

The equations for the cavity shape are valid if the following condition is satisfied: 1.92 − 3). (11) L > Rn ( σ The state space equations describing the system in equation (3) represent a bimodal, switched system. There are some specific characteristics of this model that we will use later. These are (i) in the first mode the system dynamics is

32


linear (inside cavity) and in the second mode it is nonlinear (planing) input affine, i.e. the control inputs effect the dynamics linearly in both modes, (ii) the switching condition does not depend on the control inputs, (iii) the switching hyperplane depends on the delayed state variable z¯(t − τ ). To develop a control strategy for this system, the following considerations will be applied. Properties (i) and (ii) allow for feedback linearization in both modes. This will be performed in a coordinate system that reflect the most the geometric structure of the problem. It will be shown that this design step results in linear dynamics in both modes ensuring continuous dynamics on the switching hyper surface. Since the latter depends on delayed state variables, controllability has to be analyzed and a controller has to be designed that ensures stability and tracking performance. This is investigated in the following section.

3 Theoretical Background of the Controller Design Our approach relies on the assumption that the delay in the equations of motion can be eliminated by applying a suitable feedback. Then the controllability analysis and the controller design can be performed for bimodal LTI systems. Since the concept of relative degree plays a central role in this approach, its definition for nonlinear, time delay and LTI systems will be given first. 3.1 Feedback Linearization Given a nonlinear input affine system: m

gi (x)ui ,

x˙ = f (x) +

x ∈ X,

u∈U

(12)

i=1

yj = hj (x),

yj ∈ Y,

j = 1, . . . , p,

(13)

has a vector relative degree r = [r1 , . . . , rp ], ri ≥ 0, ∀i, if at a point x0 (i) Lgj Lkf hi (x) = 0, . . . j = 1, . . . , m, k < ri−1 , (ii) The matrix  Lg1 Lrf1 −1 h1 (x), . . . , Lg1 Lrf1 −1 h1 (x)   .. =  ., rp −1 rp −1 Lgm Lf hp (x), . . . , Lgm Lf hp (x) 

AIA

has rank p at x0 .

(14)


33

For linear time invariant (LTI) systems given by (A, B, C), we have that Lgj Lfri −1 hi (x) = ci Ari −1 bj and if p = m then the vector relative degree is defined if rankALT I = n where n is the state dimension. The concept of relative degree can be extended to time delay systems, too. Usually this is defined for a discrete time equivalent of the continuous time systems by introducing the discrete time shift operator Δ as Δxt = xt−τ with τ denoting the given time delay. The time delay system is given now by (A(Δ), B(Δ), C(Δ)), i.e. the matrices depend on the delay operator. This implies that the coefficients are elements of the polynomial ring R[Δ]. The relative degree is defined similarly to the LTI case as follows. Definition 1 Given the single input - single output linear time delay system (A(Δ), b(Δ), c(Δ)). It has relative degree r > 0 if cAk b = 0, k = 0, . . . , r − 1 and cAr b = 0. It has pure relative degree r if in addition cAr b is an invertible element of R[Δ]. This definition has an obvious extension to the multivariable case. It requires the matrix AT D is invertible over Rp×p [Δ]. To perform the analysis and design a controller, new state variables for equation 3 are chosen as:     z(t) x1 (t) x2 (t) −V θ(t) + w(t)     (15)  x3 (t) =  θ(t) q(t) x4 (t) The matrix used for this coordinate transformation is:  T    c1 1 0 00 cT1 A 0 −V 1 0    Tc =   cT2  = 0 1 0 0 cT2 A 0 0 01

(16)

The state space equations in the new coordinate system are: x ¯˙ = where

Ac x¯(t) + Bc u(t) + F¯g Ac x¯(t) + F¯p (t, x, δ) + Bc u(t) + F¯g

if c¯T (δ)¯ x(t) ≤ 0, T if c¯ (δ)¯ x(t) ≥ 0,

(17)



   0 1 0 0 0 −α110 −α111 −α120 −α121   T   Bc = c1 AB  Ac =   0   0  0 0 1 −α210 −α211 −α220 −α221 , cT2 AB

(18)

and F¯grav = Tc Fgrav + C1 where C1 is a constant associated with the shift in the origin of the coordinate system. Similarly F¯plane = Tc Fplane .

34


In our situation the inputs enter linearly in the state equations in both modes. In addition, it is assumed that all states can be measured. This allows us to select two outputs defined as y1 = x1 and y2 = x3 , such that there exists pure vector relative degree in both modes, and in addition, these are identical, i.e. by defining: 1000 Cc = (19) 0010 The relative degree for the modes are: r12 = 2,

r21 = 2,

r12 + r21 = n = 4

r12

r22

r12

= 2,

= 2,

+

r22

=n=4

Mode 1

(20)

Mode 2

(21)

The consequence of this property is that one can apply state feedback in both modes such that this will eliminate time delay in Mode 1 and nonlinearity (exact feedback linearization) in Mode 2. This feedback is given by: M1−1 (y˙ 13 (t) + Fα x(t) + Fg + vI (t)) M1−1 (y˙ 13 (t) + Fα x(t) + Fg − Fp (x, δ) + vII (t))

if cT (δ)x(t) ≤ 0, if cT (δ)x(t) ≥ 0, (22) where M1 = (Cc Ac Bc )−1 , y13 = [y1 , y3 ]T , and the feedback gain Fα is defined by the controllability invariants αijk , i = 1, 2, j = 1, 2, k = 0, 1 of the linear part A, B of the system. uf lc =

Applying this control with identical external signals vI = vII = v, the feedback linearized closed loop has the following form in both modes: x˙ F L = AF L xF L + BF L v, rwhere AF L

 0 0 = 0 0

10 00 00 00

 0 0 , 1 0

BF L

  00 1 0  = 0 0 , 01

(23)

(24)

and the switching condition is given by the sign of ys = c(δ)xc . 3.2 Controllability Analysis of the Bimodal System The controllability of the linearized, bimodal dynamics needs to be analyzed and a tracking controller designed. Results on controllability of single input single LTI systems with single switching surface and relative degrees r = r1 = r2 = 1 has been published by [12].


35

In their approach the problem is reduced to analyzing the dynamics of the system on the switching surface. This is given by the zero dynamics derived with respect to the “switching output” ys . It was shown that the zero dynamics have to be controllable when using positive ys in one mode (negative ys in the second mode, respectively). It was assumed that the system is both left and right invertible and the dynamics is continuous on the switching surface, i.e. A1 x + b1 u = A2 x + b2 u. Since the relative degree r = 1, under the above assumptions, the zero dynamics can be written as: η(t) ˙ = Hη +

g1 ys (t) g2 ys (t)

if ys (t) ≤ 0, if ys (t) ≥ 0,

(25)

with η ∈ Rn−1 . It can be proved that the problem (25) is equivalent to the following (sign constrained) switching problem with systems (H, g1 ) and (H, −g2 ) with a nonnegative input u, see [19]. Then the same results can be obtained using the following reasoning: in the unconstrained input case to compute the reachability set the following Lie-algebra of the vector spaces Hη + g1 u and Hη − g2 u, i.e., the Lie-algebra generated by Hη + [g1 − g2 ][u u]T , needs to be defined. Denote this set by R(H, [g1 , −g2 ]). Thus a necessary condition of controllability is that R(H, [g1 , −g2 ]) = Rn , i.e., the pair (H, [g1 , −g2 ]) has to be controllable. This is a Kalman - like rank condition. Since one can use only sign constrained inputs, this imposes an additional condition on H. A sufficient condition is that if H has an even number of eigenvalues with zero real parts, then the zero dynamics is controllable with nonnegative inputs. More results on controllability with nonnegative inputs can be found in [11, 13]. This result is extended for our application as follows. Consider the MISO system with B ∈ Rn×m and ys = Cx. Also consider the case, when there is a direction p ∈ ImB such that the system is left and right invertible cor¯ one has the responding to the direction p. Using the notation B = [ p B], system: ¯u x˙ = Ax + pup + B ¯,

ys = Cx.

(26)

Let us denote by V ∗ the largest (A, p) - invariant subspace in C = kerC and by W∗ the smallest (C, A) invariant subspace over Imp. It follows that system has the following decomposition induced by a choice of basis in V ∗ and W∗ : ξ˙ = A11 ξ + γv 1 ¯21 u ¯ + v) up = (−A12 η − B γ ¯22 u¯ + Gys , η˙ = A22 η + B

(27) (28) (29)

36


Since r = 1, ξ = ys , equation (29) describes the dynamics of the system on C. Rewrite the equation of this zero dynamics as η˙ = P η + Q¯ u + Rys .

(30)

assuming that Q is monic. Proposition 1 If the pair (P, Q) is controllable, then η is controllable “without” using ys , e.g. by applying u¯ = Q# (−Rys + w). If the pair (P, Q) is not controllable, then the conditions of controllability with unconstrained u ¯ but nonnegative ys is the following. 1. The pair (P, [Q R]) has to be controllable. 2. Consider the decomposition induced by the reachability subspace R(P, Q), η˙ 1 = P11 η1 + P12 η2 + Q1 u ¯ + R1 ys

(31)

η˙ 2 =

(32)

P22 η2 + R2 ys ,

where R2 = 0. Then P22 has no real eigenvalues. Remark 1 The first condition is a Kalman-rank condition. The second one can be given in some alternative forms using e.g. results from [11, 13]. For the high speed supercavitating vehicle model, this result has to be applied to a time delay system. The following approach is taken. Since only one delay time is present in the switching condition, it is possible to discretize the system with extended state space by including the delayed state variable. Since feedback linearization has been already applied, it is possible to use a backward difference scheme defined for LTI systems that preserves the geometry needed to analyze the zero dynamics. The resulting discrete time state equations are: x(t + 1) = Ad x(t) + Bd v(t), where

 1 0  Ad =  0 0 1

T 1 0 0 0

0 0 0 0 1T 0 1 0 0

 0 0  0 , 0 0



0

0

ys = Cd x(t)

(33)



β21 T β22 T    0  Bd =   0 , β41 T β42 T  0 0

Cd = [1, 0, v, 0, −1]

(34)

where T denoted the sample time. The next step is to find the relative degrees by selecting one of the inputs, say v1 first. They are identically r = 2 for both modes since the feedback linearization and state transform resulted in the same linear canonic form in both modes.


37

To obtain the zero dynamics one has to construct a state transform matrix Tcd from the row vectors spanning the orthogonal complement of V ∗ and imBd1 where imBd1 is the first column of Bd . It can be shown that V ∗⊥ = span{cs , cs Ad } and that the remaining three rows of Tcd is selected from ImB1⊥ resulting in the transform:   cs   cs A     1 0 0 0 0 (35) Tcd =   0 0 1 0 0 0 β41 T 0 −β21 T 0 Using this state transform [ξ T (t), η T (t)]T = Tcd x(t) and that V ∗ is (Ad , Bd1 ) invariant, the following decomposition is obtained: ξ(t + 1) =

0 a12 0 0 ξ(t) + v (t) + v (t) b21 1 e22 2 0 a22

ys = 1 0 ξ(t)

switching condition

η(t + 1) = P η(t) + Rξ(t) + Qv2 (t), where

  p11 p12 p13 P =  0 p22 p23  , 0 0 p33

  0 r12 R = 0 r22  , 0 0

(36) (37) (38)

 0 Q =  0 . q31 

(39)

The zero dynamics are described by the last equation. (The same approach can be repeated when selecting the second column of Bd .) Using Proposition 1, it can be seen that due to their special structure, the (P, Q), pair is controllable. This implies that the dynamic inversion controller with switching and pole placement for tracking error stability can be applied to control the bimodal system.

4 Multivariable Pole Placement With feedback linearization the system behaves the same regardless of the interior switching state, hence a single outer loop controller can guarantee stability and appropriate tracking properties for the complex system. A variety of linear design approaches can be used for that purpose [15–18], but the focus of this paper is on the feedback linearization controller. Hence a simple poleplacement controller is synthesized as the outer-loop tracking controller. The inner loop dynamics after feedback linearization is given by x˙ F L = AF L xF L + BF L v,

y = Cs xF L ,

(40)

38


where using the structure of Cs one can see that the first and the third state variables are measured. The system (Cc , AF L , BF L )has relative degrees for r1 = 2, r2 = 2 and the dynamic inversion controller designed for tracking can be written as: ref (41) v = M1−1 (x˙ ref c24 − Fppl (xF L − xc )), where xref is the reference state vector transformed to the canonic state coorc dinates, xref c24 is a 2-dimensional vector containing the 2nd and the 4th elements of xref and the gain matrix designed for pole allocation has the structure: c Fppl = as

−α ¯ 110 −α ¯ 111 0 0 0 0 −α ¯ 220 −α ¯ 221

(42)

Defining the tracking error as etr = xF L −xref c , its dynamics can be written e˙ tr = Acl etr , where Acl = AF L − BF L Fppl , and   0 1 0 0  −α ¯ 110 −α ¯ 111 0 0  . Acl =   0 0 0 1  0 0 −α ¯ 220 −α ¯ 221

(43)

This shows that the coefficients of the tracking error characteristic equation (thus the poles, too) are arbitrarily assigned by the choice of the feedback gain Fppl . The closed-loop and the tracking error system are stable for a given set of α ¯ coefficients. The structure of the complete control-loop system is shown in Figures 3 and 4 where the inner-loop is responsible to feedback linearize the system and the outer-loop is responsible for reference tracking. As stated before the plant is switched and nonlinear, hence different controllers are used in the three different modes, which are selected by a state dependent switching based on the planing model and measurements. As noise enters the cavity wall, which is the switching surface also these selections are not always perfect, hence the outer-loop must be robust enough to handle that “noise.” After feedback linearization the reference tracking part needs only to be designed for the linear model. It is possible to track position, velocity, angle and angle rate commands with different weights if they are selected consistently. The pole placement controller is synthesized in the canonic coordinates. The special structure of the feedback linearized system results in independent control position and angle tracking.


v

✲

✲ Bc ✻

✲

❄

vehicle

x0

Fgrav

✲

✲

❄

✲

▼ ✻ Ac

✛

39

✲ ys

Cs

x ¯(t − τ ) ✻ ✲ δ

x ¯(t)

✛

Fplane (Bimodal) ✛ Feedback Linearizing Controller

✛ ✛ ✛

(switching) Fig. 3. Inner loop: Switching feedback linearizing controller Feedback linearized system xref ✲ −1 Tc

✲ ✻

✲ Fppl vp✲ BF L ✲

xF L

x˙ F ✲ L

✲

✻ AF L ✛

Fig. 4. Outer loop: Inversion controller for tracking

5 Control of a Supercavitating Vehicle Model Simulations are performed in Matlab/Simulink environment and all parameter dependencies are analyzed with respect to the reference setup. The maneuver is an obstacle avoidance maneuver: the horizontal speed is constant 75m/s while the vehicle moves up 17.5m and returns to continue its straight path as seen in Figure 5. It is assumed that pressure and cavitation number remain constant during the 4 second maneuver. In addition to the original problem, an additional noise component is added to the cavity wall leading to uncertainty in the cavity wall position uncertain - the controller switching surface. The cavity wall is not in Rc distance from the cavity centerline, it ranges between Rc ± 0.1(Rc − R) 1 . Recall the baseline control problem assumes a with a filter Gn = 600s+1

40


z (m)

20

Basic AMP 15 AMP 20

10 0 −10 0

1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

w (m/s)

10 5 0 −5 −10 0

Fig. 5. Simulated trajectory of high speed supercavtating vehicle

unity actuator model. Figure 7 shows the effect of including a 30 Hz, first order actuator model. The performance specifications are to track θ reference commands and reduce limit cycle oscillations. The reference tracking properties received the higher priority as compared with the damping the oscillatory behavior. The following controller gains were selected: −α ¯ 110 = −40000; −α ¯ 111 = −400; −α ¯ 220 = −90000; −α ¯ 221 = −600

(44)

With which the resulting eigenvalues are −300; −300; −200; −200. The resulting contribution from the tracking part of the controller with these high gains is negligible comparing with the inversion based part’s contribution to compensate against the effect of planing. Actuator bandwidth has a significant impact on closed-loop performance. The time response of the closed-loop system with the addition of a 30 Hz first order actuator is shown in Figure 7. Increased oscillations of the vehicle in the cavity are clearly evident after cavity strikes during the maneuver. The baseline response in Figure 6 has significantly less cavity wall strikes. All other simulation results presented have unity as the actuator model, i.e. no actuator. The value of the delay time also plays an important role in the controller performance. This is due to the feedback controller directly inverting the delay. To see this, a 5 ms reduction in the delay time is introduced into the simulation. The vehicle tracks the reference signal well though oscillations due to cavity wall strikes last longer and the maximum planing depth increases when the delay is not known precisely, see Figure 8. The cavity wall is often not a clear glass boundary between the vapor and the fluid. Rather there may be a turbulent boundary associated with that interface. The effect of a non-perfect cavity boundary is modeled as the nominal boundary with the addition of a “noisy” fluid. The boundary is modeled as Rc ± 0.1(Rc − R). In the simulation the maximum planing depth remains the same but the actuator deflection is more radical, see Figure 9.

Control of High-Speed Underwater Vehicles 4

No. act (rad)

0.1

fin

0

δ

q (rad/sec)

0.2

−0.1 −0.2 0

1

2

3

4

5

6

h

4 2 0 0

0 −2 1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0.2 δcav (rad)

prime

(m/m)

6

No act.

2

−4 0

7

41

1

2

3 4 time (s)

5

6

0.1 0 −0.1 −0.2 0

7

Fig. 6. Trajectory tracking: No actuator 4

Basic (rad)

0.2

fin

0

δ

q (rad/sec)

0.4

−0.2 −0.4 0

1

2

3

4

5

6

h

1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0.4 δcav (rad)

prime

(m/m)

6 4 2 0 0

0 −2 −4 0

7

Basic

2

1

2

3 4 time (s)

5

6

0.2 0 −0.2 −0.4 0

7

Fig. 7. Trajectory tracking: First order 30 Hz actuator model 4

25ms delay (rad)

0.2

δ

fin

0 −0.2

h

prime

(m/m)

−0.4 0

1

2

3

4

5

6

2 0 −2 −4 0

7

4

0.4

3

0.2

δcav (rad)

q (rad/sec)

0.4

2 1 0 0

1

2

3 4 time (s)

5

6

7

25ms delay 1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0 −0.2 −0.4 0

Fig. 8. Trajectory tracking: Uncertain time delay

42

G.J. Balas, J.Bokor, B.Vanek, and R.E.A. Arndt 4 (rad)

q (rad/sec)

0.5

δ

fin

0

−0.5 0

Slow noise 1

2

3

4

5

6

Slow noise 1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0.5

6

δcav (rad)

(m/m) prime

h

0 −2 −4 0

7

8

4 2 0 0

2

1

2

3 4 time (s)

5

6

0 −0.5 −1 0

7

Fig. 9. Trajectory tracking: Cavity noise

4

0.2

2

(rad)

0.4

δ

fin

0 −0.2 −0.4 0

1

2

3

4

5

h

prime

(m/m)

8

6

4 2 0 0

1

2

3 4 time (s)

5

6

1

2

3

4

5

6

7

1

2

3 4 time (s)

5

6

7

0.4

Basic AMP 15 AMP 20

6

0 −2 −4 0

7

δcav (rad)

q (rad/sec)

Increasing the noise magnitude results in larger actuator deflection spikes and has longer settling times. A larger magnitude noise of size Rc ± 0.5(Rc − R) cavity radius and a 1 slow noise case when the noise filter is Gn = 60s+1 , Figure 10. As expected the maximum planing depth increases as the maneuver become more radical but it is interesting to note that the control signals are not following the same trend, they fin deflection remain basically the same, while the cavitator deflection has small contribution from the increased planing, possibly because the planing angle is different in the different cases.

7

0.2 0 −0.2 −0.4 0

Fig. 10. Trajectory tracking: 15, 18, 20m amplitude maneuvers

6 Summary and Future Research This paper describes the control challenges associated with supercavitating vehicles. The feedback control problem features a delayed switched system


43

with significant nonlinearities affecting the design. There are a number of open questions to be resolved before such a vehicle can be successfully controlled. These include: • • •

Desired maneuvering requirements and performance metrics Actuation methods, sensing requirements and accuracy needed to meet performance requirements. Robustness of feedback controller to modeling errors and poor physical models

The performance and robustness requirements facing the design of a feedback controller will significantly shape the design of future supercavitating vehicles.

Acknowledgments This work was funded by the Office of Naval Research, award number N000140110229, Dr. Kam Ng Program Manager. The first author would like to thank Professor Keith Glover for many stimulating conversations on the design of feedback control algorithms and model reduction for engineering systems. I am a better control engineer because of the lessons he taught me.

References 1. Sturgeon, V (2001), “Racing through water: Supercavitation,” Caltech Undergraduate Research Journal vol. 1, no. 2, 16–21 2. Ashley S (2001) “Warp drive underwater,” Scientific American, 42–51. 3. Dzielski J, Kurdila, A (2003), “A Benchmark control problem for supercavitating vehicles and an initial investigation of solutions,” J. Vibration and Control 9:791–804 4. Logvinovich GV (1972), “Hydrodynamics of free-boundary flows,” translated from the Russian (NASA-TT-F-658), US Department of Commerce, Washington D.C. 5. Arndt REA, Balas GJ, Wosnik M (2005), “Control of cavitating flows: A perspective,” accepted for publication, Japan Society of Mechanical Engineers International Journal, Japan 6. Kirschner IN, Rosenthal BJ, Uhlman JS (2003), “Simplified dynamical system analysis of supercavitating high-speed bodies,” Fifth International Symposium on Cavitation, Osaka Japan 7. Paryshev EV, (2002), “The Plane problem of immersion of an expanding cylinder through a cylindrical free surface of variable radius,” International Summer Scientific School on High-Speed Hydrodynamics, Cheboksary, Russia 8. Kirschner IN, Kring DC, Stokes AW, Uhlman JS (2002), “Control strategies for supercavitating vehicles,” J Vibration and Control 8:219–242 9. Shao Y, Mesbahi M, Balas GJ (2003), “Planing, switching and supercavitating flight control,” AIAA Guidance, Navigation and Control Conference, AIAA2003-5724

44


10. Balas G, Bokor J, Szabo Z (2003), “Invariant Subspaces for LPV Systems and Their Applications,” IEEE Trans Automat Contr, 48:2065–2068 11. Brammer RF (1972), “Controllability in linear autonomous systems with positive controllers,” SIAM J. Control, 10:329–353 12. C ¸ amlibel MK, Heemels WPMH, Schumacher JM (2004), On the controllability of bimodal piecewise linear systems. In: Alur R, Pappas GJ (eds.) Hybrid Systems: Computationand Control, LNCS 2993, Springer, Berlin, 250–264 13. Saperstone SH, Yorke JA (1971), “Controllability of linear oscillatory systems using positive controls,” SIAM J. Control, 9:253–262 14. Balas G, Bokor J, Szab´ o (2004), “Tracking of continuous LPV systems using dynamic inversion,” Conference on Decision and Control 3:2929–2933 15. Doyle JC, Glover K, Khargonekar P, Francis B (1989), “State-space solutions to standard H2 and H∞ control problems,” IEEE Trans Auto Control 34:831–847 16. McFarlane DC, Glover K (1989), Robust Controller Design using Normalised Coprime Factor Plant Descriptions, Springer-Verlag Lecture Notes in Control and Information Sciences, V138 17. McFarlane DC, Glover K (1992), “A Loop Shaping Design Procedure using H∞ Synthesis,” IEEE Trans Auto Control 37:759–769 18. Balas GJ, Packard AK, Doyle JC, Glover K, Smith R (1993) µ-Analysis and Synthesis Toolbox, MUSYN Inc. and The MathWorks, Natick MA 19. G. Balas , Z. Szab´ o and J. Bokor, Controllability of bimodal LTI systems. Submitted to the 2006 American Control Conference, Minneapolis, MN 2005.

Using Feedback to Improve System Identification Roger W. Brockett Division of Engineering and Applied Sciences Harvard University

1 Introduction In his Doctoral thesis and subsequent publications (e.g., [1]) Keith Glover explored the parameterization of multivariable time invariant linear systems investigating identifiability and suitable standard forms for use in system identification. Over the years the idea of identifiability has come to play an important role in the literature. In this paper we describe some new results centering around a quantitative measure of identifiability defined in terms of a suitable Gramian. The motivation comes from a desire to provide a quantitative evaluation of the so-called “two dimensional method” widely used in nuclear magnetic resonance (NMR) spectroscopy and we provide this in terms of a Cramer-Rao bound. In the final section we expand the scope of these ideas, providing a more general system theoretic development which discusses a new role for feedback in system identification. An important part of the methodology to be discussed involves merging the results of a number of independent experiments in order to reduce the identification error. The use of a sequence of independent experiments is not a new concept in system identification in that process control engineers have often used sine wave testing to determine the frequency response of a linear system. This requires a sequence of inputs of the form ui (t) = sin ωi t, in each case waiting for the system to come to steady state, and then recording the amplitude and phase of the output signal. In the implementation of two dimensional NMR there is again a sequence of independent experiments but now what is recorded is a set of transient responses. The elements of the set are distinguished by the difference between the time at which the transient is initiated and the time at which an impulsive feedback action is applied. These responses are stitched together to form a function of two variables which is then used as the basis for identification. Further explanation is given below.


46

R.W. Brockett

2 Moore-Penrose and Cramer-Rao Let † denote conjugate transpose. It is a familiar fact that if A is an n by m complex matrix of rank m then the choice of x that minimizes (Ax−b)† (Ax−b) is x = (A† A)−1 A† b. Thus (A† A)−1 A† is an explicit expression for the MoorePenrose inverse applicable when columns of A are linearly independent. Of course if it happens that b is in the range space of A then this choice of x leads to zero error and minimizes ||x|| relative to any other choice that results in zero error. Suppose f is a random variable taking on values in a complex inner product space F and that f has a finite mean and variance. If A is a linear operator defined on a second inner product space, A : X → F it often happens that it is desirable to find an estimator such that the expected value of Ax(f ) agrees with the expected value of f and the variance of x(f ) is minimal. If † −1 1 f has a Gaussian density, ρ(f ) = (1/N )e− 2 (f −m) Σ (f −m) then, again, the Moore-Penrose inverse can play a role. More specifically, if it happens that Ef lies in the range space of A and if A† A is invertible, then x(f ) = (A† A)−1 A† f is the minimum variance unbiased estimate. It results in a random variable x with variance (A† A)−1 A† ΣA(A† A)−1 , where Σ is the variance of f . If the equations are scaled so that Σ = κ2 I then the variance of x is just κ2 (A† A)−1 . This result holds even when A : Cn → H with H being an infinite dimensional Hilbert space, provided that the operator A† A is well defined and invertible. For example, if we want an unbiased estimate of b and can observe y(k) = ak b + κn(k) for k = 0, 1, ..., with n(k) being zero mean, unity variance Gaussians, then the minimum variance, unbiased estimate of b is ∞

ˆb =

a2k

−1 ∞

k=0

ak y(k)

k=0

provided that |a| < 1 so that the sums are defined. We prefer to formulate our results in continuous time so as to avoid certain technicalities. The usual interpretation of what is meant by a continuoustime measurement with additive white noise involves a limiting process which begins with discrete-time measurements with an additive Gaussian random variable, independent from sample to sample. By repeatedly doubling the sampling frequency while doubling the variance of the noise one can obtain a meaningful limit. The precise description of this limit, however, depends on an initial scaling. Adopting the usual interpretation of continuous-time measurements, as in y(t) ˙ = eλt b + κν(t) ˙ the previous analysis carries over to continuous time as ˆb =

∞ t=0

e2λt dt

−1

∞ t=0

eat y(t)dt

provided λ < 0. Using the above formulae with a = eλ and λ < 0, we see that in the two cases we have error variances

Using Feedback to Improve System Identification

σcontinuous =

47

κ2 −κ2 ; σdiscrete = 2λ 1 − e2λ

which agree in the limit as λ goes to zero, but not otherwise. With some limitations, this idea can be applied to nonlinear problems in which the goal is to find x(f ) such that Eφ(x) = Ef . For example, if φ is differentiable and some means for finding an approximate solution is available then rewrite φ(x) as φ(x) = φ(x0 + δx) = f and make a Taylor series approximation ∂φ δx ≈ f φ(x0 ) + ∂x Letting A denote the Jacobian we are then led to δx = (A† A)−1 A† (f − φ(x0 )) ; A =

∂φ ∂x

x0

This can be used iteratively as in x(k + 1) = x(k) − h(k) A† A)−1 A† (f − φ(x(k)) with h(k) either constant or becoming smaller with increasing k as it would in stochastic approximation. Again, this applies in the case where the range space is an infinite dimensional Hilbert space provided that the operator A† A is well defined and invertible. For example, if λ0 is real and if we observe y(t) ˙ = eλt b + ν(t) ˙ but now regard both b and λ as numbers to be estimated, we may linearize about (b0 , λ0 ) and work with [eλ0 t , teλ0 t b0 ]

δb = y(t) ˙ − eλ0 t b0 δλ

In this case the Moore-Penrose inverse involves the operator A† A =

∞ 0

eλ0 t teλ0 t b0

[eλ0 t , teλ0 t b0 ]

dt =

−1/2λ0 b0 /4λ20 b0 /4λ20 −b20 /8λ30

If the probability density of f is known and if it has finite variance then the Cramer-Rao inequality provides a lower bound on the variance of an unbiased estimate. For example, if, as we assumed above, f is Gaussian with density ρ(f ) =

1 −12 (f −φ(x))† (f −φ(x)) e 2κ N

and compute the Fisher information matrix M =E

∂ ln ρ(f ) ∂x

∂ ln ρ(f ) ∂x

T

48

R.W. Brockett

we have M −1 as a lower bound on the variance associated with any unbiased estimate. In the above setting M coincides with κ−2 A† A evaluated at the true value of (λ, b). The idea behind the Cramer-Rao bound is that there is a “gain factor” that describes how much a change in the parameter to be estimated changes the probability distribution of the quantities being observed. This circle of ideas provides a way to make precise the intuitive idea that the quality of any estimate is limited both by the variance of the noise and by the magnitude of the change in the probability distribution caused by a change in the quantity being estimated.

3 Moore-Penrose and Smoothing We can also consider the linear problem discussed above as an infinite time smoothing problem. It is a textbook fact that for the Gauss-Markov estimation problem (all parameters are real here) x˙ = Ax + B w˙ ; y˙ = Cx + ν˙ with A, C observable, the variance of the error satisfies Σ˙ = AΣ + ΣAT + BB T − ΣC T CΣ In the special situation in which B is zero, a steady state solution would need to satisfy 0 = AΣ + ΣAT − ΣC T CΣ Of course Σ = 0 is one possible solution and it is easy to see that only when A has unstable eigenvalues will there be solutions with Σ both nonzero and nonnegative definite. On the other hand, when considering smoothing problems the matter is different. If all the eigenvalues of A have negative real parts then in view of the fact that the solution of the smoothing problem runs backward in time, the positive definite solution of 0 = −AΣ − ΣAT − ΣC T CΣ given by −1

(Σ∞ )

=

∞ 0

T

eA t C T CeAt dt

is of interest. More specifically, suppose we want to estimate x(0), given y on [0, ∞) but no prior information on x(0). Rearranging the B = 0 version of the variance equation, we see that A − Σ∞ C T C = −Σ∞ AT (Σ∞ )−1 Thus, on the basis of the observation of y on [t, ∞), the best estimate of x(t) is given by the solution of


49

d x ˆ = ΣAT Σ −1 x ˆ + ΣC T y dt By the variation of constants formula, x ˆ(t) =

∞ t

Σe−A

T

or, equivalently, x ˆ(t) = ΣeAt

(t−σ)

∞ t

Σ −1 ΣC T y(σ)dσ

e−A

T

σ

C T y(σ)dσ

Setting t = 0 we see that the solution of this infinite time smoothing problem coincides with the Moore-Penrose solution of CeAt x(0) = y(t) ; t ∈ [0, ∞), i.e., x ˆ(0) = Σ

∞

0

eA

T

σ

C T y(σ)dσ

4 Stable Systems; Partial Fraction Coordinates Our main new results are given in sections 5-7; this section is largely devoted to additional background material needed for comparison. The basic problem, fitting a function of time with a sum of exponentials, has roots going back to work of Prony in 1795. (See [2] for references and more recent developments.) Our treatment of these ideas, based on continuous observation and taking full advantage of state-space notation, seems efficient and, in any case, is essential for our later work. Suppose we are given y(tk ) + κn(tk ) with the n(tk ) being zero mean Gaussian random variables of unit variance and wish to find (possibly complex) vectors b = [b1 , b2 , ...bn ]T and λ = [λ1 , λ2 , ..., λn ]T such that n

bi eλi tk ; k = 1, 2, 3, ...

y(tk ) = i=1

As suggested by the Cramer-Rao bound, a key role is played by the linear term in the Taylor series expansion n i=1

(bi + δi )e(λi +γi )tk =

n i=1

bi eλi tk +

n i=1

δi eλi tk +

n

bi γi tk eλi tk + ...

i=1

It is convenient to adopt a vector-matrix notation. Let Λ and b be defined as     b1 λ1 0 ... 0  b2   0 λ2 ... 0     Λ=  ... ... ... ...  ; b =  ...  0 0 ... λn bn

50

R.W. Brockett

and let η denote a row vector whose entries are all ones. Then, in a notation in which the derivative is expressed as a row vector, we have ∂ηeΛt b ∂ηeΛt b ∂b ∂Λ

= ηeΛt tbT eΛt

Observe that if the real parts of λi and λj are negative then for all nonnegative integers k ∞

(−1)k+1 k! ¯ tk eλi t eλj t dt = ¯ (λi + λj )k+1

0

so that

∞ 0

¯

tk eΛt eΛt dt = Wk+1 (Λ)

with Wk being the Hermitean matrix    Wk (Λ) = (−1)k (k − 1)!  

1 1 ¯ 1 +λ1 )k (λ ¯ 1 +λ2 )k (λ 1 1 ¯ 2 +λ1 )k (λ ¯ 2 +λ2 )k (λ

...

...

1 1 ¯ n +λ1 )k (λ ¯ n +λ2 )k (λ

... ... ... ...

1 ¯ 1 +λn )k (λ 1 ¯ 2 +λn )k (λ

...

1 ¯ n +λn )k (λ

    

Given two matrices of the same size we can form a product using element by element multiplication; we write Q ∗ H to denote the matrix whose ij th entry is qij hij . This is variously called the Schur product or the Hadamard product. Note that A ∗ B = B ∗ A, that (A + B) ∗ C = A ∗ C + B ∗ C and that A ∗ (B ∗ C) = (A ∗ B) ∗ C but that mixtures of ordinary matrix product and Schur product, such as (AB) ∗ C, can not be simplified except in special cases. Our uses of the Schur product are mostly related to the identity ∞ 0

¯

tk eΛt QeΛt dt = Wk+1 (Λ) ∗ Q

valid for diagonal matrices Λ whose diagonal entries have negative real parts. As is well known, if Q and H are Hermitian and nonnegative definite their Schur product is also nonnegative definite. However, it is possible for the Schur product to be positive definite even if neither factors are even positive semidefinite. Lemma 1: Let η be the row vector [1, 1, ..., 1], let Λ be a diagonal matrix whose diagonal entries have negative real parts, and let b be a column vector with complex entries. Considered as a mapping from (b, λ) to L2 [0, ∞), the Jacobian of ψ(b, λ)(t) = ηeΛt b is

J=

∂ψ ∂ψ ∂b ∂Λ

= ηeΛt tbT eΛt

The Moore-Penrose inverse of J maps L2 [0, ∞) to C2n and is given by

Using Feedback to Improve System Identification ∞

J # (·) = F −1 where

0

51

¯

eΛt η T (·)dt ¯ teΛt¯b

W1 W2 ∗ (η T bT ) ¯ W2 ∗ (bη) W3 ∗ (¯bbT )

F =

The matrix F is positive definite if and only if the λi are distinct and all the components of b are nonzero. Remark 1: The matrix F provides a measure of the linear independence of the effects of changing the parameter vectors b and λ and as such might be thought of as an identifiability Gramian, akin to the controllability and observability Gramians of linear control theory. Proof: The derivatives evaluated at (b, Λ) can be identified with the vectors having L2 [0, ∞) components ∂ψ = eλ1 t eλ2 t ... eλn t ∂b

∂ψ = b1 teλ1 t b2 teλ2 t ... bn teλn t ∂Λ

;

Form the 2n-dimensional vector J(t) =

∂ψ ∂ψ ∂b ∂λ

and observe that if we regard J as defining a map from C2n to L2 [0, ∞) then the adjoint is the integral operator ∞

J ∗ (·) =

0

J † (t)(·)dt

Thus the role of A† A in the Moore-Penrose inverse is played by the matrix F =

∞ 0

∞ 0

¯

eΛt η T ¯ teΛt¯b

ηeΛt tbT eΛt

dt

identified in the lemma. The evaluation of F requires only the properties of Wk established above, together with the observation that W1 ∗ (η T η) = W1 By construction, F is Hermitean and nonnegative definite. It will be positive definite unless there is a linear combination of the functions {eλ1 t , eλ2 t , ..., eλn t , b1 teλ1 t , b2 teλ2 t , ..., bn teλn t } that vanishes. It is a familiar fact from the theory of ordinary linear differential equations that these functions are linearly dependent if and only if λi = λj for some i = j, assuming that the bi are all nonzero. As we have seen, the Fisher information matrix determines a lower bound on the error variance associated with identification. If this matrix is poorly conditioned in the sense that the ratio of its largest to its smallest eigenvalue is large, then some aspects of the system will be much less well characterized

52

R.W. Brockett

than others. For example, if the Gramian associated with ψ(b, λ)(t) = b1 eλ1 t + b2 eλ2 t is evaluated at ψ(t) = e−t + e−2t , then F is the four-by-four matrix 1 1 1 1  2

3 4 9 1 1 1 4 9 16 1 2 2 4 9 8 27 1 1 2 2 9 16 27 64

1 3 F = 1

  

The eigenvalues of this matrix range from about 5 × 10−5 to about .88. The smallest eigenvalue has an associated eigenvector v T ≈ [.60, −.58, −.18, −.52]. If we observe y(t) ˙ = b1 e−λ1 t + b2 e−λ2 t + κw˙ on [0, ∞) the rms error in determining b1 − b2 − .5λ1 − λ2 is about 244κ for the given values of the parameters. Clearly, even in such an unexceptional situation, reasonable identification of b and λ requires a very small value of κ. Marginalizing the probability distribution for the error by eliminating the dependence on b increases the size of the smallest eigenvalue, reducing the rms error associated with its eignevector to about 33κ. This is still a significant amplification of the observational noise level. This ill conditioning is consistent with, and even to be expected, given the well documented difficulties with Prony’s problem. As was shown above, the Moore-Penrose inverse of the Jacobian of the C2n to L2 [0, ∞) map y(b, λ)(t) = ηeΛt b involves the integral ¯ ∞ eΛt η T ξ= y(t)dt ¯ teΛt¯b 0 Let yˆ denote the Laplace transform of y. yˆ(s) =

∞ 0

e−st y(t)dt

If y is in L2 [0, ∞) then yˆ is holomorphic in the half-plane Res > 0 and yˆ (s) =

∂ yˆ (s) = − ∂s

∞ 0

e−st ty(t)dt

This leads to an alternative expression for ξ which has both conceptual and typographic advantages, namely ξ= where

and similarly for yˆ .

¯ yˆ(−λ) ¯ −ˆ y (−λ) ∗ ¯b

 ¯1 ) yˆ(−λ ¯2 )   yˆ(−λ  yˆ(−λ) =   ...  ¯n ) yˆ(−λ 


53

Theorem 1: Let b, Λ and Wi be as above and suppose that y is an element of L2 [0, ∞). Then if the choice (b0 , Λ0 ) minimizes the integral ∞

q= we have

0

|y(t) − ηeΛt b|2 dt

¯ ; W2 (Λ0 )b0 = −ˆ ¯ W1 (Λ0 )b0 = yˆ(−λ) y (−λ)

Proof: Taking the partial derivative of q with respect to b and interpreting the derivative as a column vector yields 1 ∂q = 2 ∂b

∞ 0

Likewise,

1 ∂q = 2 ∂λ

Note that

∞ 0

so that

∞ 0

th

0

¯

eΛt η T y(t)dt

¯

−teΛt¯b(y(t) − ηeΛt b)dt

¯

−teΛt¯bbT eΛt dt = W2 ∗ (¯bbT )

0 = W2 ∗ (¯bbT ) η −

If we indicate the ij equation is

∞

¯

−eΛt η T (y(t) − ηeΛt b)dt = W1 b −

∞ 0

¯

teΛt¯by(t)dt

element of W2 as mij then in component notation this n

mij ¯bi bj =

j=1

∞ 0

¯

teλi t¯bi y(t)dt

Assuming that all the components of b are nonzero we can divide the ith of these equations by ¯bi to get W2 b =

∞ 0

¯

teΛt η T y(t)dt

Rewriting this in terms of the Laplace transform gives the final statement in the theorem. Two comments about the theorem are in order. First of all, for some choices of y ∈ L2 [0, ∞) there is no value of (b, Λ) that minimizes the integral. For example, if y = te−t and b is two dimensional there is no minimizing value of (b, Λ), just an infinimizing sequence. Secondly, if a minimum exists and W1 is invertible we can rearrange matters to get an equation for Λ that is independent of b, W2 W1−1

∞ 0

∞

¯

eΛt η T y(t)dt =

0

¯

teΛt η T y(t)dt

54

R.W. Brockett

The last equation in the theorem identifies n complex equations for the n unknown elements in Λ. If there is one solution with bi = 0 and λi = λj there are n! distinct solutions with bi = 0 and λi = λj because the ordering of the λi is arbitrary. However, once Λ has been determined the equations for b are linear and uniquely solvable. Consider two special cases. Example 1: Suppose we observe a complex function of time y(t) = b0 eλ0 t + n(t) and want to estimate the complex numbers b0 and λ0 . Writing λ = σ + iω and b = α + iβ the theorem requires that we solve 2σ =

∞ λt ¯ 0 e y(t)dt ∞ λt te ¯ y(t)dt 0

−

=

¯ yˆ(−λ) ¯ yˆ (−λ)

¯ y (−λ) ¯ must vanish. Because This implies that the imaginary part of yˆ(−λ)/ˆ λ0 t the Laplace transform of b0 e is b0 /(s − λ0 ) the derivative is −b0 /(s − λ0 )2 and we see that when the noise is absent the equation for λ takes the form 2σ = σ − iω + σ0 + iω0 Having found λ, b is given by ¯ b = 2σ yˆ(−λ) When n is small relative to the size of b0 eλ0 t its effect on the solution will be small. It is insightful to reinterpret the equation for σ in the following way. Recall that ln(ˆ y (s)) = ln |ˆ y(s)| + i arg yˆ(s) Rewriting the above equation for σ as ∂ ln(ˆ y (s)) 1 = σ ∂s we see that the Imaginary part of the derivative of ln(ˆ y (s) vanishes at the optimal value of σ+iω. The Cauchy-Riemann conditions applied to ln(ˆ y (s)) assert that we may express the imaginary part of the derivative in the alternative ways Re∂ ln(ˆ y (σ + iω)) Im∂ ln(ˆ y (σ + iω)) =− ∂ω ∂σ Thus, with respect to changes in ω, the logarithm of the magnitude is stationary at (b0 , Λ0 ), and in this case a maximum. ¯ Example 2: Suppose we observe y(t) = b0 eλ0 t + ¯b0 eλ0 t + w˙ and want to estimate b and λ. Let b = α + iβ and let λ = σ + iω. In this case W1−1 =

−1 −1 2σ (2σ−2iω) −1 −1 (2σ+2iω) 2σ

−1

=

2σ 2 (σ 2 + ω 2 ) ω2

1 −1 σ (σ+iω) −1 1 (σ−iω) σ

−1


W2 = −

55

1 1 4σ2 (2σ+2iω)2 1 1 (2σ−2iω)2 4σ2

Although the general solution of these equations is messy, there are important situations for which these equations can be easily interpreted. Suppose that the two eigenvalues lie near the imaginary axis, having negative real parts but with imaginary parts much bigger than the real parts. If the imaginary parts ¯ j ) is small in comparison with both 1/(λi + λ ¯i) are well separated then 1/(λi + λ ¯ and 1/(λj + λj ) and the matrices W1 and W2 are nearly diagonal. In this case 1 I and so λ is approximately the arg the matrix W2 W1−1 is approximately 2σ max of yˆ(s) Remark 2: In lemma one no allowance is made for the fact that one may know that y is real and hence the eigenvalues and the residues must occur in complex conjugate pairs. The effect of restricting consideration to the real case may be analyised as follows. The general Hermitean form, expressed as φ(u + iv) = (u + iv)† (Q + iΩ)(u + iv) with both Q = QT and Ω = −Ω T real matrices, can be rewritten as a real form φ(u + iv) =

[uT , v T ]

Q Ω ΩT Q

u v

The eigenvalues of this matrix coincide with the eigenvalues of Q + iΩ, but now occur with multiplicity two. Remark 3: Lemma 1 applies directly to the partial fraction representation of a transfer function. If the transfer function is described in terms of the coefficients of the numerator and denominator polynomials then there is a mapping (b, λ) → (q, p) defined by n i=1

qn−1 sn−1 + qn−2 sn−2 + ... + q0 bi = s + λi sn + pn−1 sn−1 + ... + p0

that relates the 2n parameters b1 , b2 , ..., bn , λ1 .λ2 , ..., λn to the 2n parameters q0 , q1 , ..., qn−1 , p0 , , p1 , ..., pn−1 . The Jacobian of this map relates small changes in b and λ to small changes in (q, p). This Jacobian can be computed as follows. Let φ1 , φ2 , ..., φn denote the elementary symmetric functions of {λ1 , λ2 , ..., λn }, i.e., φk is the coefficient of sk−1 in the polynomial p(s) = (s+λi ), so that φ1 = p0 = λ1 λ2 · · · λn , φ2 = p1 = λ1 λ2 +...+λn−1 λn ... and φn = pn−1 = λ1 + λ2 + ... + λn . Let S be the matrix whose ij th entry is ∂pi sij = ∂λj A short calculation then shows that S dq = dp 0

∂S ∂λi bi

S

db dλ

56

R.W. Brockett

The determinant of the Jacobian is nonzero unless the eigenvalues are repeated. Note that if the noise is sufficiently small so that linearization applies the variance in (q, p) coordinates are related to the variance expressed in partial fraction coordinates by a congruence transformation Σq,p = JΣb,Λ J ∗ with J being this Jacobian.

5 A Two Variable Identifiability Gramian Against this background, we now turn to the main point of the paper, which is to evaluate the error reduction that is possible by the appropriate use of feedback. In NMR spectroscopy, the values of λi are often close to each other and for some important applications, such as those involving protein structure determination, the information sought relates directly to the amount by which they are separated. In this setting the accurate determination of both the absolute and relative values of nearby eigenvalues is of central importance. However, the matrix F of the previous section becomes singular when eigenvalues approach each other and hence the Cramer-Rao bound approaches infinity. The difficulty of resolving the difference between nearby eigenvalues became apparent early on in the development of NMR and an important technique for improving the situation was proposed by Jenner in 1971. Subsequently this idea was refined and experimentally developed by Ernst and his colleagues, work for which Ernst received the 1991 Nobel prize in physics. (See reference [3] for a detailed account.) The experimental realization involves recording data (effectively impulse responses) from a sequence of experiments, the elements in the sequence being distinguished by the precise time at which an impulsive feedback is applied. The purpose of the feedback is to couple the characteristic responses of different eigenmodes, in effect modulating one set of frequencies with a different set that are more easily identified. The study of the optimal way to apply feedback to achieve such modulation has been undertaken by Khaneja and his collaborators, see e.g., [4-5]. This work uses and extends ideas from Lie Group controllability and trajectory optimization. At present the application of these ideas in NMR spectroscopy represents one of the most important uses of control in the quantum domain. Postponing for now some details, this approach leads to a modified problem of the Prony type, now involving fitting a function of two variables y(t, τ ) with an exponential approximation. The exponents used in the expansion must be shared so the problem takes the form n,n

y(t, τ ) ≈

bij eλi t eλj τ

i=1,j=1

with special constraints on the bij which has the effect of limiting the number of independent variables to 2n, as above. This will become clear in the development below.


57

The measure of the improvement in the accuracy is reflected in the Fisher information matrix which, as we will see, is expressible using Schur products. As remarked above, if Q and H are Hermitian and nonnegative definite their Schur product is also nonnegative definite. This is easily shown by expanding one of the factors, say H, as a sum of the form bi b†i and observing that for † T an arbitrary vector x, x (Q ∗ b1 b1 †)x = (x ∗ b1 ) Q(x ∗ b1 ) ≥ 0 or by observing that bb† ∗ Q is actually congruent to to Q. More explicitly, bb† ∗ Q = BQB † where   b1 0 ... 0  0 b2 ... 0   B=  ... ... ... ...  0 0 ... bn In fact, it is possible to be more specific about the lower bound. If Q and H are n by n Hermitean matrices and if Q − αI ≥ 0 and H − βI ≥ 0 then the Schur product satisfies (Q − αI) ∗ (H − βI) ≥ 0 However (Q − αI) ∗ (H − βI) = Q ∗ H − βdiag(Q) − αdiag(H) + αβI ≥ 0 which leads to the more specific bound Q ∗ H ≥ βdiag(Q) + αdiag(H) − αβI ≥ 0 In connection with the final inequality, note that Q ≥ αI implies diagQ ≥ αI, etc. We need the following two variable version of Lemma 1. Lemma 2: Let η, b and Λ be as in Lemma 1 and let T be a nonsingular matrix. Considered as a mapping from (b, λ) to L2 [0, ∞)×[0, ∞), the Jacobian of ψ(b, Λ)(t, τ ) = ηeΛt T eΛτ b is J(t, τ ) =

∂ψ ∂ψ ∂b ∂Λ

= ηeΛt T eΛτ , tbT eΛτ T T eΛt + τ ηeΛτ T eΛt ∗ bT

Assuming that the λi are distinct and all components of b are nonzero, the Moore-Penrose inverse of this Jacobian maps L2 [0, ∞) × [0, ∞) to C2n and is given by J # (·) = F −1 where

∞ 0

∞ 0

¯

¯

eΛτ T † eΛt η T (·)dt dτ ¯ ¯ τ eΛτ T † eΛt η T ∗ ¯b

¯ ¯ teΛt T¯eΛτ ¯b +

58

R.W. Brockett ∞

∞

F =

0

0

J † (t, τ )J(t, τ )dt dτ

Proof: The arithmetic is an elaboration of that required for Lemma 1. The calculation of the derivative with respect to b is as above. The calculation of the derivative with respect to Λ now yields two terms because Λ enters twice, in ηeΛt T eΛτ b. Writing these as ηeΛt β(τ ) on one hand and as α(t)eΛτ b on the other, facilitates the computation. The derivative that generates the term with the factor t could be written as bT eΛτ T T eΛt t ∗ η T but we can drop the ∗η T because it has no effect. We decompose the evaluation of F as suggested by F =

∞ 0

∞ 0

J † (t, τ )J(t, τ )dt dτ =

W11 W12 † W22 W12

The one-one block is given by W11 =

∞

∞ 0

0

†

†

eΛ τ T † eΛ t η T ηeΛt T eΛτ dt dτ = (T † W1 T ) ∗ W1

(Again, the matrix η T η is the matrix of all ones so that W11 ∗ (η T η) = W11 .) The one-two term ∞

∞

W12 =

0

0

†

†

eΛ τ T † eΛ t η T tbT eΛτ T T eΛt + bT ∗ τ ηeΛt T eΛτ dt dτ

can not be expressed entirely in terms of the Wi because of noncommutivity. It can, however, be simplified to W12 =

∞ 0

∞ 0

†

†

teΛ τ T † eΛ t η T bT eΛτ T † eΛt dt dτ + W2 ∗ T † W1 T ∗ (η T bT )

Finally, W22 is almost completely refractory and is left as ∞ 0

∞ 0

teΛt T eΛτ ¯b + τ eΛt T T eΛτ η T ∗ ¯b

tbT eΛτ T T eΛt + τ ηeΛt T eΛτ ∗ bT dtdτ

The above formulae simplify in an important way if T ΛT −1 commutes with Λ. Because we assume that Λ is diagonal with distinct eigenvalues this implies that T ΛT −1 is diagonal and thus T ΛT −1 is simply a reordering of the original Λ. Theorem 2: Let η, b, Λ and Wi be as in Lemma 1. Assume further that T is orthogonal, that T −1 ΛT commutes with Λ, and that ηT = η. Considered as a mapping from (b, Λ) to L2 [0, ∞) × [0, ∞), the Jacobian of ψ(b, Λ)(t, τ ) = ηeΛt T eΛτ b is


J=

∂ψ ∂ψ ∂b ∂Λ

59

= ηeΛt T eΛτ , b† eΛτ T T eΛt (It + T τ )

Assuming that the λi are distinct and all components of b are nonzero, the Moore-Penrose inverse of this Jacobian maps L2 [0, ∞) × [0, ∞) to C2n and is given by J # (·) = F −1

0

¯

∞

∞ 0

te

¯ Λτ

¯

eΛτ T † eΛt η T (·)dt dτ ¯ ¯ T e b + τ eΛτ T T eΛt η T ∗ ¯b ¯ ¯ † Λτ

where F = with

W11 W12 W21 W22

W11 = W1 ∗ T T W1 T † W12 = W21 = (W2 ∗ (T W1 T T ) + W1 ∗ (T W2 T )) ∗ (η T b)

and

W22 = W3 ∗ T W1 T T + 2W2 ∗ T W2 T T + T W1 T T W3

Proof: This is an obvious specialization of lemma 2 and so we omit the proof. The generation of the term with factor τ is facilitated by using the identities ηeΛt T eΛτ b = ηeΛt T eΛτ T −1 T b = ηT eΛτ T −1 eΛt T b = ηeΛτ T −1 eΛt T b What one sees from the theorem is that the effect of T is to alter the identifiability Gramian, opening up the possibility of increasing the smallest eigenvalue. In section 7 we consider and resolve the question of the best choice of T . Example: To illustrate how these ideas can be applied in an NMR setting we consider a somewhat idealized example which, however, captures many of the essential features of 2D NMR. The system has nine state variables and four resonant frequencies; no simpler model captures all the significant points. The system√is described by x˙ = (A+ uB1 + vB2 )x+ b with output y = cx. The vector b is 2e1 where e1 is the standard basis vector and (A + uB1 + vB2 ) is a nine-by-nine matrix   −1 u 0 u 0 0 0 0 0  −u −σ f 0 0 v 0 0 0     0 −f −σ 0 0 0 0 0 0     −u 0 0 −σ f 0 0 v 0     A + uB1 + vB2 =   0 0 0 −f −σ 0 0 0 0   0 −v 0 0 0 −σ g 0 0     0 0 0 0 0 −g −σ 0 0     0 0 0 −v 0 0 0 −σ g  0 0 0 0 0 0 0 −g −σ

60

R.W. Brockett

and y = x6 + x8 . The identification procedure can be described using a sequence of 5 vectors representing different stages in the evolution of the state x. It involves the repetition of the following steps. √ 1. While √ in the steady state value of xss = 2e1 apply an impulse u = (π/ 2)δ(t). This has the effect of reducing x1 to zero, changing x2 and x4 to minus one, and leaving the remaining components of x unchanged. 2. Let time evolve for τ units and then apply an impulse v = (π/2)δ(τ ). This has the effect of exchanging the values of x2 and x6 and exchanging the values of x4 and x8 while leaving all other components of x unchanged. 3. As time continues to evolve, record y. These steps are summarized by the following sequence. √        0 0 0 2  0  −1   −e−στ cos f τ         −στ   −στ 0   0  0   e   e  sin f τ sin f τ      −στ     0  −1   −e    cos f τ 0      −στ   −στ   0 →  0  →  e → e  sin f τ sin f τ          0  0     −e−σt cos f τ  0          0  0      0        −σt 0   0  0     −e 0 cos f τ  0 0 0 0 followed by



 0  e−στ sin f τ · e−σt sin f t   −στ  −σt  e   −στ sin f τ · e −σt cos f t   e   −στ sin f τ · e−σt sin f t   sin f τ · e cos f t  → e   −e−στ sin f τ · e−σt cos gt   −στ  −σt  e   −στ sin f τ · e −σtsin gt   −e sin f τ · e cos g t  −στ −σt e sin f τ · e sin g t

with the output following the last step being y(t) = −e−στ sin f τ · e−σt cos gt − e−στ sin f τ · e−σt cos g t We can use the development given here to quantitatively evaluate how much mode coupling helps. We focus on the portion of W11 that involves the two modes that are close together. In the case of two complex eigenvalues σ + iω and σ + iν with δ = ω − ν, the corresponding W11 part of the Gramian contains as a sub matrix ˜ 11 = −1 W 2σ

1

−1 1+iδ/2σ

−1 1−iδ/2σ

1


61

˜ 11 is approxFor δ small in comparison with |σ|, the smallest eigenvalue of W imately smin = δ 2 /16σ 3 Thus in this range the smallest eigenvalue of W11 is proportional to δ 2 . By replacing W11 by W11 ∗ T W11 T −1 we effectively replace δ by the difference between two other eigenvalues thus increasing δ and increasing the smallest eigenvalue in the Fisher information matrix. In a situation of the type explored in the example, this means replacing |f − f |, a small difference, with |g − g | which would be much larger. The resulting error variance is then reduced by a factor of |f − f |2 /|g − g |2 .

6 2D Transforms and the Resonant Case In order to express the equations for b and Λ in terms of the Laplace transform we now require a two-dimensional transform. Let yˆ be defined as yˆ(s1 , s2 ) = Defining Y (λ) as

∞ 0

∞ 0

y(t, τ )e−s1 t e−s2 τ y(t, τ )dtdτ



 yˆ(λ1 , λ1 ) yˆ(λ1 , λ2 ) ... yˆ(λ1 , λn )  yˆ(λ2 , λ1 ) yˆ(λ2 , λ2 ) ... yˆ(λ2 , λn )   Y (λ) =    ... ... ... ... yˆ(λn , λ1 ) yˆ(λn , λ2 ) ... yˆ(λn , λn )

we observe that ∞ 0

∞ 0

† † ¯ eΛ τ T † eΛ t y(t, τ )dtdτ = T ∗ Y (−λ)

Defining

∂ yˆ(s1 , s2 ) ∂si gives rise to matrices Yi (Λ), by analogy with the definition of Y . Using these we can express yî (s1 , s2 ) =

ξ=

∞ 0

∞ 0

† † † † teΛ t T † eΛ τ ¯b + τ eΛ τ T T eΛ t η T ∗ ¯b y(t, τ )dt dτ

Thus the transform nicely packages the information needed to compute the Moore-Penrose inverse. Part of the value of this approach is comes from the fact that in the system identification problem we do not know in advance which values of λ will be relevant. In computing the transform we evaluate all the functionals that might be needed, postponing the need to select the particular ones that ultimately matter. It may also be noted that in NMR applications the eigenvalues of interest are lightly damped. In this case the transform can be approximated using methods involving the fast Fourier transform.

62

R.W. Brockett

7 A Matrix Inequality The example of the previous section illustrates a specific situation in which the identifiability Gramian can be improved by means of the coupling distinct modes through a short feedback pulse. In this section we establish a limit on the improvement that can be achieved in this way. In 1953 Alfred Horn proved a remarkable result, providing a converse for a much older result of Issac Schur on the possible values of the diagonal of a Hermitean matrix having a given set of eigenvalues. Schur had provided a short proof that the diagonals dii and the eigenvalues µi are related by a set of inequalities which imply that if we think of the diagonal of the matrix as a point in Rn then they lie within the convex hull of the n! points having coordinates (µπ(1) , µπ(2) , ..., µπ(n) ) where π is an arbitrary permutation of the indices. Schur’s inequalities include, as a special case, the fact that the smallest eigenvalue of a symmetric matrix is upper bounded by the smallest diagonal. Horn established that every point in this convex set is achievable as the diagonal of a Hermitean matrix with the given eigenvalues. The proof of this is more challenging. Later, Bertram Kostant showed that this circle of ideas could be divorced from matrix theory and recast in terms of Lie algebras [7]. In this more general setting the Schur-Horn polytope has come to play a role in a variety of elegant theories involving topics as diverse as completely integrable systems and neural networks. The Schur-Horn polytope associated with a symmetric matrix Q has a well defined center, namely the vector all of whose components are trQ/n. This point qualifies as a center in that it is the midpoint of the intersection of the Schur-Horn polytope with any pencil of diagonal matrices passing through it. If K is a convex set in Rn and T is a linear transformation mapping Rn into itself, then the image of K under T is again convex. We denote this by T (K). Theorem 3: Let Q and H be n by n Hermitean matrices. Let Q have diagonal entries {qii } = {di } with d1 ≥ d2 ≥ ... ≥ dn and let H have eigenvalues µ1 ≥ µ2 ≥ ... ≥ µn . Let K be the Schur-Horn polytope associated with H. The set of possible diagonals of the matrix Q ∗ U † HU as U ranges over the unitary matrices coincides with the set diag(Q)(K) where diag(Q) is the linear transformation   q11 0 ... 0  0 q22 ... 0   Diag(Q) =   ... ... ... ...  0 0 ... qnn Moreover, if Q and H are nonnegative definite then max min λi (Q ∗ U † HU ) ≤ min(di µn−i+1 ) U

i

i

and there exists U such that equality is achieved. Such a U can be chosen from the set of U that diagonalize H while putting the eigenvalues of U † HU in reverse order by size as compared with the diagonals of Q.


63

Proof: The smallest eigenvalue of a Hermitean matrix is less than or equal to the smallest diagonal entry of the matrix. In fact, by the Schur-Horn theorem, the set of possible diagonals of U † HU , considered as the coordinates of points in Rn are bounded by, and fill out, the convex hull of the n! points having coordinates (µπ(1) , µπ(2) , ..., µπ(n) ). Thus the set of points S = {(q11 h11 , q22 h22 , ..., qnn hnn )|(h11 , h22 , ..., hnn ) = (diagU† HU) ; U† U = I}) is convex and we can assert that the smallest eigenvalue of U † HU is less than or equal to the smallest component of any element of this set. Of course the product Q ∗ U † HU is diagonal if U † HU is. Thus we have the inequality of the Theorem. To show that equality can be achieved, we observe that for any U that diagonalizes H we have Q ∗ U † HU ≥ I min qii λi i

Because one can achieve an arbitrary permutation of the diagonals of a matrix by selection of a unitary matrix U and acting on the matrix by U † HU , we can rewrite the right-hand side as Q ∗ U † HU ≥ I(min qii λπ(i) ) i

which gives the result claimed. Remark 4: In the case of two-by-two symmetric matrices there is only one nontrivial permutation and it acts to interchange both the rows and the columns. In this situation it is possible to show that the Schur product of W and Π T W Π represents an improvement over the Schur product of W and W in the sense that λmin

ab ab ∗ bc bc

≤ λmin

ab cb ∗ bc ba

To see this, note that the we are asserting λmin

a2 b 2 b 2 c2

≤ λmin

ac b2 b2 ac

These matrices have the same determinant (product of the eigenvalues) and, by the geometric-arithemetic mean inequality, the matrix on the left has a smaller trace (sum of the eigenvalues). Thus the eigenvalues of W ∗ Π T W Π lie between those of W ∗ W .

8 Beyond NMR Spectroscopy The NMR problem is a linear system identification problem but the emphasis is on the eigenvalue location rather than the accurate determination of a

64

R.W. Brockett

transfer function. Furthermore, physical principles imply that the eigenvalues of interest are lightly damped, giving rise to lengthy transient responses. As we have seen, in the NMR setting these techniques are highly effective and thus it is interesting to explore their wider use, mindful of the fact that the type of additional information to be gained in this way may be more closely related to eigenvalue locations rather than direct improvements in the accuracy of the impulse response. An essential element is the need to generate one or more functions of the form y(t, τ ) = ηeΛt T eΛτ b with T being such that certain modes are made more observable than they would be based on ηeΛt alone. This requires a input channel to generate eΛt b and a mode coupler to generate T . To explore this idea in a general linear system theoretic context consider a multivariable system of the form x˙ = Ax + Bu ; y = Cx and consider the possibility of applying a time varying feedback K(t) to get x˙ = (A − BK(t)C)x(t) + Bu ; y = Cx Fact: Through repeated experiments on x˙ = (A−BK(t)C)x(t)+Bu ; y = Cx we can generate an arbitrarily good approximation to any function of the form y(t, τ ) = CeAt T eAτ B ; 0 ≤ τ < ∞ ; τ + d ≤ t < ∞ where T is any matrix that is reachable from the identity in d time units following solutions of the matrix equation ˙ X(t) = (A + BK(t)C)X(t) ; X(0) = I Explanation: The function y(t, τ ) can be generated by applying an impulse u = δ(t) at time zero, letting time evolve for τ seconds, applying the time varying gain K over the interval [τ, τ + d] and then letting the system evolve without additional input. It is necessary to ignore the output on the interval between τ and τ + d. If the values of T so reachable contains matrices that mix modes effectively one can expect to be able to use two dimensional methods to enhance identifia˙ bility. The characterization of the reachable set for X(t) = (A+ BK(t)C)X(t) with X(0) = I has been studied for many years and naturally begins with a determination of the Lie algebra generated by matrices of the form A+BKi C. In our paper [9] we showed that even in the case where k is a scalar but (A, b, c) controllable and observable, the Lie algebra generated in this way is of one of four types and generically the Lie algebra is all of gl(n). This means that typically the use of time varying feedback will allow us to steer the state of x˙ = Ax + k(t)bcx + bu to an open set in Rn . Thus by linearly combining


65

various functions of the form CeAt T eAτ B, including the obvious possibility CeAt eAτ B, we can achieve any T we like. Of course finding a K which generates a desired value of T may be difficult if the system is completely unknown. These techniques are most likely to be useful if the system is approximately or partially known based on some preliminary experiments.

Acknowledgment. This work was supported in part by the US Army Research Office under Boston University prime GC169369 NGD, the National Science Foundation grant number ECS 0424342, DARPA under Stanford University Prime PY-1606.

References 1. Keith Glover and Jan C. Willems, “Parametrizations of Linear Dynamical Systems: Canonical Forms and Identifiability,” IEEE Trans. Automatic Control, volAC-19, No. 6, 1974 pp. 640-646. 2. M.R. Osborn and G. K. Smyth, “A modified Prony algorithm for Exponential Function Fitting,” J. of Applied Probability,, 1986, pp. 419-430. 3. Richard Ernst, et al. Principles of Nuclear Magnetic Resonance in One and Two Dimensions, Oxford University Press, Oxford, England, 1987. 4. Navin Khaneja et al. “Boundary of Quantum Evolution under Decoherence,” PNAS, vol 100, (2003) pp 13162-13166. 5. Navin Khaneja et al. “Time Optimal Control of Spin Systems,” Physical Review A, vol. 63 (2001). 6. R. Horn, “Doubly Stochastic Matrices and the Diagonal of a Rotation Matrix,” American Journal of Mathematics, Vol. 76,1953, pp. 620–630. 7. Bertram Kostant, “On Convexity, The Weyl Group and the Iwasawa Decomposition,” Ann. Sci. Ecole Norm. Sup., Vol. 6, (1973) pp. 413–455. ¨ 8. I. Schur, “Uber eine Klasse von Mittelbiddungen mit Anwendungen die Determinanten Theorie,” Sitzungsberichte der Berliner Mathematische Gesellschaft, 22, pages 9–20, 1923. 9. Roger W. Brockett, “Linear Feedback Systems and the Groups of Lie and Galois,” Linear Algebra and Its Applications, Vol. 50 (1983) pp. 45-60.

On the Gap Metric for Finite-Dimensional Linear Time-Varying Systems in Continuous Time Michael Cantoni The University of Melbourne, Department of Electrical and Electronic Engineering, Parkville, VIC 3010, Australia – [email protected]

Keywords: non-autonomous differential equations, gap metric, Riccati equations, (J-)spectral factorisation, robust control, approximation

Notation R, R+ , Rn Rp×m AT L2 B πt Lloc 2 L H

G∗ ΠG G⊥

The real line, non-negative reals and n-dimensional Euclidean space R × · · · × R, respectively. The space of (p × m)-dimensional matrices with entries in R. The matrix transpose of A ∈ Rp×m . The Hilbert space of square integrable functions over R+ , with inner product ·, · and norm · ; The Banach algebra of linear operators P : L2 → L2 with induced norm P := sup0=u∈L2 P u / u . The truncating projection on functions over R+ such that (πt f )(τ ) := f (τ ) if τ ≤ t, and 0 otherwise. The extended space of functions over R+ which satisfy πt f ∈ L 2 ∀ t ∈ R+ ; The space of causal (i.e. non-anticipatory) linear operators loc P : Lloc 2 → L2 satisfying πt P πt = πt P ∀ t ∈ R+ . The subspace of all P ∈ L with finite-gain P := supu∈Lloc π P u / π u . For each P ∈ H, the t t 2 , πt u=0, t∈R+ operator P |L2 ∈ B, where P |L2 denotes the restriction of P : Lloc → Lloc to L2 . When P ∈ H, the notational dis2 2 tinction between P and P |L2 is often suppressed and P is identified with the corresponding operator in B. The Hilbert adjoint of G ∈ B – y, Gu = G∗ y, u ∀ u, y ∈ L2 . The orthogonal projection onto the closed subspace G ⊂ L2 ; G ⊥ := {g ∈ L2 | g , g = 0 for all g ∈ G} = (I − ΠG1 )L2 .


68

M. Cantoni

1 Introduction It is well-known that the gap metric, and its variants, provide a natural framework for studying the robustness of feedback interconnections [1–4]. In fact, for arbitrary linear systems, it is known that the gap metric induces the weakest topology in which both feedback stability and closed-loop performance are both robust properties [5]. In this article, we study the gap metric for the class of finite-dimensional, linear time-varying systems, in continuous-time. In particular, it is shown that the gap between two systems is equal to the norm of an operator composed of normalised left and right representations of the system graphs. The development relies on tools familiar from the time-invariant setting. This is in contrast to the abstract results which underpin the discretetime generalisations of the gap framework to the time-varying setting [6, 7]. Using ideas from [8, 9], an alternative characterisation of the gap is also established in terms of a standard linear fractional synthesis problem from robust control theory. This characterisation appears to be useful for approximation in the gap, and might be extendable to other classes of system. The article develops along the following line. Section 2 serves to introduce the class of finite-dimensional systems and to establish the required notions of stability, coprime factors and so-called graph symbols. In Section 3, the gap metric is defined and shown to be equal to the norm of an operator composed of normalised left and right representations of the system graphs. The norm of this operator is then linked to the solution of a differential Riccati equation. This section also includes a discussion of the time-invariant case. The alternative characterisation of the gap is provided in Section 4, and some concluding remarks are made in Section 5.

2 Finite-Dimensional Systems The class of linear time-varying systems considered here are those generated by a finite-dimensional realisation ΣP = (A, B, C, D), of the form x(t) ˙ A(t) B(t) = y(t) C(t) D(t)

x(t) , u(t)

(1)

with initial condition x(0) = 0, where A : R+ → Rn×n , B : R+ → Rn×m , C : R+ → Rp×n and D : R+ → Rp×m are bounded and continuous functions and B and C are not identically zero. Indeed, the realisation (1) generates an operator P : Lloc → Lloc := u → y where, using the so-called variation of 2 2 parameters/constants formula [10], y(t) =

t 0

C(t)X(t)X(s)−1 B(s)u(s) ds + D(t)u(t),

(2)

with X : R+ → Rn×n taken to be a fundamental matrix for the homogeneous part of (1), x(t) ˙ = A(t)x(t) – e.g. the solution to the matrix differential

On the Gap Metric for FDLTV Systems in Continuous Time

69

˙ equation X(t) = A(t)X(t) with initial condition X(0) = I. In particular, P ∈ L (i.e. it is causal) and P is said to admit a finite-dimensional realisation. If D−1 (t) := D(t)−1 is bounded on R+ , then P −1 ∈ L with a realisation ΣP −1 = (A−BD−1 C, BD−1 , −D−1 C, D−1 ). In this case, P and its realisation ΣP , are said to be bi-causal. Given realisations ΣP1 = (A1 , B1 , C1 , D1 ) and ΣP1 = (A1 , B1 , C1 , D1 ) of two systems P1 ∈ L and P2 ∈ L, respectively, it follows directly that A1 0 B , 1 , C1 C2 , D1 + D2 B2 0 A2

ΣP1 +P2 :=

is a realisation of P1 + P2 ∈ L, and A2 B2 C1 B D , 2 1 , C2 D2 C1 , D2 D1 0 A1 B1

ΣP2 P1 :=

is a realisation of P2 P1 ∈ L. 2.1 Exponential Stability, Stabilisability and Detectability Let ΣP = (A, B, C, D) be a finite-dimensional realisation of a system P ∈ L. The homogeneous part, x(t) ˙ = A(t)x(t),

(3)

is said to be exponentially stable if there exist real constants γ > 0 and λ > 0 such that X(t)X(s)−1 ≤ γe−λ(t−s) for all 0 ≤ s ≤ t, where X(t) is any fundamental matrix for (3) [11]. In this case, it follows that the operator P := u → y, defined by (2), is finite-gain stable and hence, that P ∈ H. Furthermore, note that Ξ := X −T is a fundamental matrix for the homogenous equation ˙ = −AT (t)ξ(t). ξ(t) This equation is said to be anti-stable, since Ξ(t)Ξ(s)−1 ≤ γe−λ(t−s) for 0 ≤ t ≤ s. Now, for ν ∈ Lloc 2 the function ω(t) = DT (t)ν(t) +

∞ t

B T (t)Ξ(t)Ξ −1 (τ )C T (τ )ν(τ ) dτ,

t ∈ R+ ,

(4)

is a continuous solution to the system of equations ˙ −AT (t) −C T (t) ξ(t) = B T (t) DT (t) ω(t)

ξ(t) , ν(t)

(5)

without boundary conditions [11, Chapter 3]. From this it can be seen that loc the corresponding operator P : Lloc 2 → L2 := ν → ω is anti-causal in the sense πt P πt = P πt .

70

M. Cantoni

Moreover, since ν, P u = =

∞ 0 ∞ 0

t

ν T (t) ∞

= P ∗ ν, u

t

0

C(t)X(t)X −1 (s)B(s)u(s) ds + D(t)u(t)

B T (τ )Ξ(τ )Ξ −1 (t)C T (t)ν(t) dt + DT (τ )ν(τ )

dt T

u(τ ) dτ

for any ν, u ∈ L2 , we have P |L2 = P ∗ (i.e. the Hilbert adjoint of P .) Given a realisation ΣP = (A, B, C, D), the pair (B, A) is said to stabilisable if there exists a bounded F : R+ → Rm×n such that x ˜˙ (t) = (A + BF )(t)˜ x(t) is exponentially stable. Similarly, the pair (A, C) is said to be detectable if there exists a bounded H : R+ → Rm×n such that x˜˙ (t) = (A + HC)(t)˜ x(t) is exponentially stable. Importantly, when a realisation is both stabilisable and detectable, then exponential (i.e. internal) stability is equivalent to finite-gain (i.e. input-output) stability of the causal operator P defined by (2) [12, 13]. 2.2 Coprime Factors and Graph Symbols A system P ∈ L is said to admit a stable left-coprime factorisation if there ˜, M ˜ , U, V ∈ H, with M ˜ −1 , V −1 ∈ L, such that P = M ˜ −1 N ˜ and exist N ˜U +M ˜ V = I. Similarly, a P ∈ L admits a stable right-coprime factorisation N ˜ , V˜ ∈ H, with M −1 , V˜ −1 ∈ L, such that P = N M −1 if there exist N, M, U ˜ N = I. A left-coprime (resp. right-coprime) factorisation is said and V˜ M + U ˜M ˜∗+N Ñ ˜ ∗ = I (resp. M ∗ M + N ∗ N = I) on L2 . Note to be normalised if M that the superscript ∗ here denotes the Hilbert adjoint of the corresponding operator in B. Remark 1. The normalisation condition specified above is different from the one usually employed in the time-invariant case [2, 14, 15], where A(·) ≡ A ∈ Rn×n , B(·) ≡ B ∈ Rn×m , C(·) ≡ C ∈ Rp×n and D(·) ≡ D ∈ Rp×m . In this case, normalisation is usually imposed in terms of the natural extensions of the factors to the doubly-infinite time axis R. < > The L2 -graph of a system P ∈ L is defined to be G :=

P dom(P ) ⊂ L2 × L2 , I

where dom(P ) := {u ∈ L2 | P u ∈ L2 }. Now suppose P admits stable left˜ and P = N M −1 , respectively. ˜ −1 N and right-coprime factorisations P = M Then with G :=

M ˜ := −N ˜ M ˜ ∈ H, ∈ H and G N

(6)


71

˜ are, respectively, left- and right-invertible in H and it follows that G and G ˜ = 0, which implies GG ˜ = 0. In view of this, G and hence, G = GL2 and GG ˜ are called right and left graph symbols, respectively. Furthermore, since G G is left-invertible, G is closed. The right and left graph symbols are said to be normalised if G∗ G = I ˜G ˜ ∗ = I. That is, when G is an isometry and G ˜ is a co-isometry. In this and G case, G∗ ˜ G

˜∗ = I 0 . GG 0I

(7)

Correspondingly, G∗ ˜ L2 = L2 G

G∗ ˜ = G ⊥ ∩ G = {0}, and ker( ˜ ) = ker(G∗ ) ∩ ker(G) G

where ker(A) := {w ∈ L2 | Aw = 0}. Thus, ˜∗ GG

G∗ ˜ = I. G

(8)

Proposition 1. (c.f. Corollary 4.2 in [13]) Suppose that P ∈ L admits a finite-dimensional realisation. Then the following are equivalent: 1. P is feedback stabilisable, in that there exists a finite-dimensional C ∈ L such that −1 I C ∈ H; P I 2. P admits a stabilisable and detectable realisation; 3. P admits (normalised) right- and left-coprime factorisations and hence (normalised) right and left graph symbols, all with finite-dimensional realisations. Suppose the finite-dimensional realisation ΣP = (A, B, C, D) of P is stabilisable and detectable, and that D ≡ 0, for convenience. Then by construction, given any bounded F : R+ → Rm×n and H : R+ → Rn×p , such that x ˜˙ (t) = (A + BF )(t)˜ x(t) and x ˜˙ (t) = (A + HC)(t)˜ x(t) are exponentially stable, the realisations ΣM := (A + BF, B, F, I), ΣN := (A + BF, B, C, 0), ΣV˜ := (A + HC, −B, F, I), ΣU˜ := (A + HC, H, F, 0), ΣM˜ := (A + HC, H, C, I), ΣN˜ := (A + HC, B, C, 0), ΣV := (A + BF, −H, C, I), ΣU := (A + BF, H, F, 0), ˜, M ˜ , N, ˜ V, U ∈ H with zero initial conditions, generate operators M, N, V˜ , U −1 −1 ˜ ˜ such that P = N M = M N and ˜ V˜ U ˜ ˜ −N M

M −U I0 = . N V 0I

72

M. Cantoni

˜ are right- and left-coprime factorisations of P . ˜ −1 N As such, N M −1 and M Furthermore, by Lemmas 3.3 and 3.4 in [13], the so-called control differential Riccati equation ˙ −X(t) = AT (t)X(t) + X(t)A(t) − X(t)BB T (t)X(t) + C T C(t),

(9)

admits a bounded symmetric differentiable solution X(t) ≥ 0, such that x˜˙ = (A−BB T X)(t)˜ x is exponentially stable, and the so-called filtering differential Riccati equation with initial condition ˙ Z(t) = A(t)Z(t) + Z(t)AT (t) − Z(t)C T C(t)Z(t) + BB T (t), Z(0) = 0, (10) admits a bounded symmetric differentiable solution Z(t) ≥ 0, such that x ˜˙ = T (A − ZC C)(t)˜ x(t) is exponentially stable. Indeed, by Theorem 4.1 in [13], ˜,N ˜ , and the with F = −B T X and H = −ZC T , the coprime factors M, N, M corresponding graph symbols, are normalised. Remark 2. Note that (10) has an initial condition, and as such, even in the time-invariant case, the normalised (in the sense defined above) left-coprime factors and left graph symbols are time-varying. This is avoided in the usual time-invariant analysis by imposing normalisation in terms of the natural extensions to the doubly-infinite time axis R. On the other hand, the normalised (as defined here) right-coprime factors and hence, right graph symbol, can be taken to be time-invariant, because (9) has no boundary conditions. < >

3 The Gap Metric The gap, or aperture, between two closed subspaces G1 , G2 ⊂ L2 , is defined to be [16, Chapter 4] →

gap(G1 , G2 ) := ΠG1 − ΠG2 = max gap(Gi , Gj ), i,j=1,2

(11)

where →

gap(Gi , Gj ) := (I − ΠGj )ΠGi is called the directed gap from Gi to Gj . Now suppose these subspaces are ˜ i Gi = 0, where Gi and G ˜i graphs and, for i ∈ {1, 2}, that Gi = Gi L2 and G are normalised right and left graph symbols for Pi ∈ L. Then the following important identities can be established for i, j ∈ {1, 2}: ΠGi = Gi G∗i ; (I − ˜∗G ˜ ΠGi ) = G i i; → ˜ ∗j G ˜ j Gi G∗i = G ˜ j Gi = G∗i G ˜ ∗j gap(Gi , Gj ) = G

=

1−

inf

0=q∈L2

G∗i Gj q q 2

2

→ ∗ ˜ ˜ i Gj = G ˜ ∗G = G i i Gj Gj =gap(Gj , Gi ).

(12)


73

˜ ∗ is an isometry and G∗ The second and sixth equalities in (12) hold because G i i is a co-isometry, and the fourth and fifth equalities hold by Lemma 1 (in the ˜ ∗ is a co-isometry appendix) and equations (7) and (8), by which G∗i Gj G j

G∗

and G˜i Gj is an isometry. Correspondingly, when the normalised right and i left graph symbols exist, we may conclude that the gap between P1 and P2 is → ˜ 2 G1 = G ˜ 1 G2 . (13) δ(P1 , P2 ) := gap(G1 , G2 ) = max gap (Gi , Gj ) = G i,j=1,2

Remark 3. Proposition 1 establishes the existence of the required graph symbols when P1 , P2 ∈ L are finite-dimensional and feedback stabilisable. Correspondingly, for such systems, the directed gaps (defined in the obvious way) are equal, and hence equal to the gap. < > Remark 4. Observe the similarity between the relationships established above and the standard characterisation of the ν-gap metric for time-invariant systems [3, 17].1 < > Suppose that P1 , P2 ∈ L admit stabilisable and detectable realisations ΣP1 := (A1 , B1 , C1 , D1 ) and ΣP2 := (A2 , B2 , C2 , D2 ), respectively. Furthermore, assume that D1 = D2 ≡ 0, for convenience. Now let Aδ :=

−B2 A2 + H2 C2 H2 C1 − B2 F1 , Bδ := B1 0 A1 + F1 B1

and Cδ := C2 0 , (14)

where F := −B T X, H := −ZC T , X ≥ 0 is a stabilising solution of (9) and Z ≥ 0 is a stabilising solution of (10). Then Σδ = (Aδ , Bδ , Cδ , 0) is an ˜ 2 G1 . exponentially stable (and so stabilisable and detectable) realisation of G Proposition 2. For P1 , P2 as described immediately above, the following are equivalent: 1. δ(P1 , P2 ) < β; ˜ 2 G1 < β; 2. G 3. The differential Riccati equation ˙ + AT (t)S(t) + S(t)Aδ (t) + C T Cδ (t) + 1 S(t)Bδ B T (t)S(t) = 0, S(t) δ δ δ β2 with Aδ , Bδ , Cδ as in (14), has a symmetric differentiable solution S ≥ 0. Proof. The result follows by (13), the fact that (Aδ , Bδ , Cδ , 0) is a realisation ˜ 2 G1 , and direct application of [18, Theorem 5.4] – also see [19]. of G 1

˜ 2 term in the usual ν-gap formula [3, 17] is not normalised in the sense The G used here, so that it can be time-invariant. This goes some way to explaining the additional winding number condition in the definition of the ν-gap.

74

M. Cantoni

Remark 5. Given realisations for the normalised graph symbols, it follows by Proposition 2 that the gap can be computed to any desired accuracy by performing a bisection search, where at each step a differential Riccati equation, without boundary conditions, is solved. < > Remark 6. In the time-invariant case – i.e. Ai (·) ≡ Ai ∈ Rni ×ni , Bi (·) ≡ Bi ∈ Rni ×mi , Ci (·) ≡ Ci ∈ Rpi ×ni – the approach outlined above would involve: 1. Solving the control algebraic Riccati equation, to get the isometric right graph symbol, which is time-invariant; 2. Solving the filtering differential Riccati equation, with a boundary condition, to get the co-isometric left graph symbol, which is time-varying; 3. Conducting a bisection search, where at each step a differential Riccati equation, without boundary conditions, is solved according to Proposition 2. However, as is shown in [15], it is possible to exploit the shift-invariance to obtain the following expression for the directed gap from P1 to P2 : inf

Q∈Q

G1 − G2 Q ,

(15)

where Q is the subspace of shift-invariant operators in H. Computing the value of this expression involves solving one control algebraic Riccati equation to get G1 (G2 does not have to be normalised, and as such, it can be constructed by solving an algebraic Lyapunov equation), and then a bisection search, where at each step we either: 1. Solve one algebraic Riccati equation to obtain a spectral factor and two algebraic Lyapunov equations to compute a Hankel norm [2]; or 2. Solve one algebraic Riccati equation to determine the existence of a solution to a “full-information” H∞ synthesis problem [13, 17]. Furthermore, in view of Rem. 3, conducting this bisection search for only one of the directed gaps yields the gap. < >

4 A Characterisation of the Gap for Approximation In the time-invariant case, the well-known directed gap formula (15) can be used to show the equivalence of operator balls defined in terms of the gap metric and operator balls constructed by perturbing the normalised graph symbols [2]. This is suggestive of the fact that approximation in the gap is closely tied to Hankel norm approximation of the graph symbol, as is confirmed in [20]. On the other hand, it not immediately clear how (14) can be used to establish this type of equivalence here. As such, an alternative characterisation of the gap is presented below, for linear time-varying systems. This


75

characterisation would appear to be useful from the perspective of approximation. In particular, it corresponds to a standard synthesis problem from robust control, which can be solved by established methods [19]. The result is in fact similar to results established in [8, 9]. These papers employ a factorisation consisting of three factors, which implicitly underpin the framework for H2/∞ control problems in the celebrated paper by Doyle, Glover, Khargonekar and Francis [21]. The approach taken here is slightly different. In particular, we use a J-spectral factorisation of the kind employed in [22–24], for linear time-invariant problems. A full development of the result is being prepared for publication elsewhere. Proposition 3. Suppose that P1 , P2 ∈ L are both finite-dimennsional and ˜ 1 denote a normalised left graph symbol of P1 . feedback stabilisable, and let G ¯ Furthermore, define β := supC∈L b(P1 , C), where b(P1 , C) := 1/ ΠK

G1

,

G1 and K are the L2 -graphs of P1 and C ∈ L, respectively, and ΠK G1 := (ΠG1⊥ |K )−1 ΠG1⊥ is the parallel-projection onto K along G1 . Then, for 0 < β < ¯ the following J-spectral factorisation exists β, ˜∗ βI G

I 0 0 −I

βI R11 R12 = ˜ R G 21 R22

∗

I 0 0 −I

R11 R12 =: R∗ JR, R21 R22

−1 0 with R, R−1 ∈ H and R21 ∈ L, where J is the signature matrix ( I0 −I ). Furthermore, the following are equivalent:

1. δ(P1 , P2 ) < β; 2. R[ uy22 ], JR[ uy22 ] > 0 for all [ uy22 ] ∈ G2 , where G2 is the L2 -graph of P2 ; 3. The linear fractional operator F(R, P2 ) := (R11 P2 + R12 )(R21 P2 + R22 )−1 ∈ H and F(R, P2 ) < 1. ˜ the required J-spectral factor can be Remark 7. Given a realisation of G, obtained as described in [18, Corollary 3.26]. This involves solving a differential Riccati equation, without boundary conditions. < > Remark 8. Proposition 3 establishes the equivalence required for Conjecture 1 in [8] to hold. This conjecture is used to described the difference between the so-called ν-gap [3, 17] and gap metrics for time-invariant systems. In a completely time-varying setting, there would appear to be no difference. < >

5 Concluding Remarks The linear fractional characterisation of the gap in Proposition 3 is appealing from the perspective of approximation when designing feedback control systems. Indeed, the analogous result in [9], which only involves the solutions of

76

M. Cantoni

algebraic Riccati equations, is used therein to obtain a procedure for sampleddata approximation of LTI controllers. A similar framework for model order reduction of time-invariant systems is also proposed in [25]. Much of the machinery used above can be extended, to varying degrees, in order to accommodate infinite-dimensional [26], and non-linear [27] systems. It would be interesting to explore the possibility of obtaining similar characterisations of gap-like metrics for these classes of system. Finally, on a more general note, it would be interesting to more closely investigate links between gap-metric results of the type established here within an input-output framework, and ideas about approximation from the behavioural approach [28–30].

Acknowledgements The author is grateful for the financial support of The University of Melbourne, through their Melbourne Research Grant Scheme (MRGS), and several discussions with Dr. C.-Y. Kao. Finally, the author would like to acknowledge the guiding influence of Prof. Keith Glover over the years.

A Technical Lemma Lemma 1. Let H1 , H2 and H3 be Hilbert spaces. If the linear operator [ X Y ] : H1 → H2 × H3 is an isometry, then Y

2

= 1−

inf

0=u∈H1

Xu 2 . u 2

˜ Y˜ : H1 × H2 → H3 is a co-isometry, then Similarly, if the linear operator X Y˜

2

= 1−

inf

0=u∈H1

˜ 2 Xu . u 2

Proof. If [ X Y ] is an isometry, then for any 0 = u ∈ H1 it follows that 1=

X [X Y u, Y u Xu, Xu Y ] u, [ Y ] u = + , u, u u, u u, u

where ·, · denotes the appropriate inner product. As such, the first result follows by the definition of the induced norm. The co-isometry counterpart follows in a similar manner. Remark 9. A similar result, and a slight extension of it, lies at the heart of the ν-gap metric framework of Vinnicombe [17]. < >


77

References 1. G. Zames and A. K. El-Sakkary, “Unstable systems and feedback: The gap metric,” in Proceedings of the Allerton Conference, 1980, pp. 380–385. 2. T. T. Georgiou and M. C. Smith, “Optimal robustness in the gap metric,” IEEE Transactions on Automatic Control, vol. 35, pp. 673–686, 1990. 3. G. Vinnicombe, “Frequency domain uncertainty and the graph topology,” IEEE Transactions on Automatic Control, vol. 38, pp. 1371–1383, 1993. 4. T. T. Georgiou and M. C. Smith, “Robustness analysis of nonlinear feedback systems: An input-output approach,” IEEE Transactions on Automatic Control, vol. 42, pp. 1200–1221, 1997. 5. M. Cantoni and G. Vinnicombe, “Linear feedback systems and the graph topology,” IEEE Transactions on Automatic Control, vol. 47, pp. 710–719, 2002. 6. W. N. Dale and M. C. Smith, “Stabilizability and existence of system representations for discrete-time time-varying systems,” SIAM Journal of Control and Optimization, vol. 31, pp. 1538–1557, 1993. 7. A. Feintuch, Robust Control Theory in Hilbert Space, ser. Applied Mathematical Sciences. New York: Springer, 1998. 8. G. Vinnicombe, “On closed-loop objectives and guaranteed robustness properties,” in Proceedings of the 35th IEEE Conference on Decision and Control, Kobe, 1996, pp. 1869–1874. 9. M. Cantoni and G. Vinnicombe, “Controller discretization: A gap metric framework for analysis and synthesis,” IEEE Transactions on Automatic Control, vol. 49, pp. 2033–2039, 2004. 10. W. Boyce and R. DiPrima, Elementary Differential Equations and Boundary Value Problems, 4th ed. John Wiley & Sons, 1986. 11. W. Coppel, Dichotomies in Stability Theory, ser. Lecture Notes in Mathematics. New York: Springer-Verlag, 1978. 12. B. Anderson and J. Moore, Optimal Control – Linear Quadratic Methods. London: Prentic-Hall, 1989. 13. R. Ravi, A. M. Pascoal, and P. P. Khargonekar, “Normalized coprime factorizations for linear time-varying systems,” Systems and Control Letters, vol. 18, pp. 455–465, 1992. 14. M. Vidyasagar, Control System Synthesis : A Factorization Approach. Cambridge (Mass.): The MIT Press, 1985. 15. T. T. Georgiou, “On the computation of the gap metric,” Systems and Control Letters, vol. 11, pp. 253–257, 1988. 16. T. Kato, Perturbation Theory for Linear Operators. New York: Springer-Verlag, 1966. 17. G. Vinnicombe, Uncertainty and Feedback - H ∞ loop-shaping and the ν-gap metric. London: Imperial College Press, 2001. 18. V. Ionescu and R. Stefan, “Generalized time-varying Riccati theory: A Popov operator based approach,” Integral Equations and Operator Theory, vol. 48, pp. 159–212, 2004. 19. R. Ravi, K. Nagpal, and P. Khargonekar, “H∞ control of linear time-varying systems - A state-space approach,” SIAM Journal on Control and Optimization, vol. 29, pp. 1394–1413, 1991. 20. T. T. Georgiou and M. C. Smith, “Upper and lower bounds for approximation in the gap metric,” IEEE Transactions on Automatic Control, vol. 38, pp. 946–951, 1993.

78

M. Cantoni

21. J. Doyle, K. Glover, P. Khargonekar, and B. Francis, “State-space solutions to the standard H2 and H∞ control problems,” IEEE Transactions on Automatic Control, vol. 34, pp. 831–847, 1989. 22. J. Ball, J. Helton, and M. Verma, “A factorization principle for stabilization of linear control systems,” International Journal of Robust and Nonlinear Control, vol. 1, pp. 229–294, 1991. 23. M. Green, K. Glover, D. Limebeer, and J. Doyle, “A J-spectral factorization approach to H∞ control,” SIAM Journal on Control and Optimization, vol. 28, pp. 1350–1371, 1990. 24. H. Kimura, Chain Scattering Approach to H ∞ Control. Boston: Birkhauser, 1997. 25. M. Cantoni, “On model reduction in the ν-gap metric,” in Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, 2001, pp. 3665–3670. 26. M. Weiss, “Riccati equations in Hilbert spaces: A Popov function approach,” Ph.D. dissertation, Univeristy of Groningen, The Netherlands, 1994. 27. J. Ball and A. van der Schaft, “J-inner-outer factorization, J-spectral factorization, and robust control for nonlinear systems,” IEEE Transactions on Automatic Control, vol. 41, pp. 379–392, 1996. 28. J. W. Polderman and J. C. Willems, Introduction to Mathematical Systems Theory: A Behavioral Approach. New York: Springer Verlag, 1998. 29. B. Roorda and S. Weiland, “Optimal angle reduction: A behavioral approach to linear system approximation,” Linear Algebra and its Applications, vol. 337, pp. 189–235, 2001. 30. A. Sasane, “Distance between behaviours,” International Journal of Control, vol. 76, pp. 1214–1223, 2003.

Robustly Stabilizing Controllers with Internal Loop Ruth F. Curtain Mathematics Institute, University of Groningen, P.O. Box 800, 9700 AV Groningen, the Netherlands [email protected]

1 Introduction The problem of robust stabilization with respect to left coprime factor perturbations was first solved explicity for the rational case in Glover and McFarlane [14]. The irrational case was treated in Georgiou and Smith [12], where it was shown that a solution exists provided that the transfer function has a normalized doubly coprime factorization. Our new contribution is to solve this classic problem for the following very general class of transfer functions G(s) ∈ L(U, Y ), where U, and Y are separable Hilbert spaces and G has the following realization: • G(s) = G(0) + sC(sI − A)−1 A−1 B; • A is a closed, densely defined operator on a separable Hilbert space Z such that ρ(A) contains 0 and a right half-plane and there exist M > 0 and n ∈ Z such that (sI − A)−1 ≤ M |s|n on some right half-plane; • C ∈ L(D(A), Y ); • B ∗ ∈ L(D(A∗ ), U ). This class and even more general ones were introduced in Opmeer [22]. They include all delay systems and most p.d.e. systems, including those that are not well-posed ones (see Staffans [24]). The extension to infinite-dimensional U and Y is entirely new and it supersedes all state-space classes previously studied in the literature. The frequency domain approach in [12] contains an existence result assuming that the transfer function has a normalized doubly coprime factorization, but no sufficient conditions for the existence of such factorizations are given and no formulas for the controllers are derived. Our state-space approach is in the style of [14] which uses an explicit normalized doubly coprime factorization of the transfer function G in terms of the system parameters A, B, C, D and results from the Nehari problem to obtain an explicit parametrization of all robust controllers. Previous results that developed this approach for infinite-dimensional linear systems include Curtain and Zwart [3, Chapter 9.4], Curtain [2] and Oostveen [20]. The class of systems we consider is far larger than any of these B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 79–98, 2006. © Springer-Verlag Berlin Heidelberg 2006

80

R.F. Curtain

classes. This generalization was made possible by the recent results in Curtain and Opmeer [7] on the existence of normalized doubly coprime factorizations and in Curtain and Opmeer [6] on the Nehari problem. A new feature is that we use controllers with internal loop as introduced in Weiss and Curtain [26] and Curtain, Weiss and Weiss [9]. In Section 2 we generalize the results on controllers with internal loop from [9] to allow for transfer functions that are not well-posed. In Section 3 we formulate the problem and reduce it to a Nehari problem for a normalized coprime factorization. The strategy is to first solve the problem for a special class of state linear systems, Σ(A, B, C, D), where A generates a C0 semigroup, B and C are bounded and G(s) = D + C(sI − A)−1 B for some D ∈ L(U, Y ) (see Curtain and Zwart [3]). The relevant results for state linear systems on the Nehari problem from [6] are summarized in Section 4 and those on normalized doubly coprime factorizations from [7] in Section 5. The solution to the robustness problem for state linear systems contained in Section 6 is a nice generalization of the results in [19]: If the system and its dual both satisfy a finite-cost condition, then we obtain the expected parametrization of infinitely many robustly stabilizing conventional controllers. Unlike the rational case, we are not sure that all solutions have this form (c.f. Georgiou and Smith [13]). An interesting corollary is the interpretation of the strange looking Bézout factors [X; Y] found in [7] as those that produce the robustly stabilizing controller K = YX−1 . In Section 7 we extend these results to the very general class of irrational transfer functions specified above using the reciprocal approach as in Curtain and Opmeer, [4], [5], [6] and [7]. If the system and its dual both satisfy a finite-cost condition and 0 ∈ ρ(A), then we can solve the robust stabilization problem by solving one for its reciprocal system that is a state linear system. We obtain an explicit parametrization of infinitely many robustly stabilizing controllers with internal loop for the original system. If either both U and Y are finite-dimensional or the component of ρ(A) that contains a right halfplane also contains the origin, then the controllers with internal loop can be realized as conventional controllers. We remark that, as in Curtain [4], the condition 0 ∈ ρ(A) can be replaced by a condition iω ∈ ρ(A) for some real ω. It is also possible to remove these conditions altogether by using the connection with discrete-time systems via a Cayley transform as in [21] and [22]. The price one pays is much more complicated formulas. We apply the theory to a class of dissipative systems with colocated actuators and sensors that includes the well-posed ones studied in Curtain and Weiss [8].

2 Controllers with Internal Loop In order to formulate our problem we need to define the class of transfer functions, normalized coprime factorizations and especially our concept of a stabilizing controller with internal loop.

Robustly Stabilizing Controllers with Internal Loop

81

The class of stable transfer functions we consider are elements from H∞ (L(Z)), where Z is either the Hilbert space U or Y or elements from H∞ (L(U, Y )), or H∞ (L(Y, U )). For simplicity of notation we denote any of these classes by HH∞ . The class of unstable transfer functions we consider are quotients of the form G = NM−1 , where N ∈ H∞ (L(U, Y )) and M ∈ H∞ (L(U )) has an ˜ −1 N ˜, inverse on some nonempty subset S of C0+ = {s : Res > 0} or G = M ˜ ˜ where N ∈ H∞ (L(Y, U )) and M ∈ H∞ (L(Y )) has an inverse on S. For simplicity of notation we denote any of these classes by HH∞ /H∞ (S). This defines a very general algebra of transfer functions and the factorization approach to controller synthesis developed in Vidyasagar [25] can be applied. The case with finite-dimensional U, Y and the inverses are defined almost everywhere on C0+ is treated in Georgiou and Smith [12]. The well-posed case where the inverses are defined and bounded on some right half-plane is treated in Curtain, Weiss and Weiss [9] and we extend this approach here to allow for systems that are not necessarily well-posed. Our definition of coprimeness is standard, but we use a Matlab notation for row blocks [W, T ] = [W T ] and column blocks [W ; T ] = [W ∗ T ∗ ]∗ . Definition 1 The transfer function G ∈ HH∞ /H∞ (S) has a right-coprime ˜ Y] ˜ ∈ factorization over S if there exist [N; M] ∈ H∞ (L(U, Y ⊕ U )) and [X, H∞ (L(U ⊕ Y, U )) such that M has an inverse on S, G(s) = N(s)M(s)−1 on S and for all s ∈ C+ 0 there holds ˜ ˜ X(s)M(s) − Y(s)N(s) = I. ˜ M] ˜ ∈ H∞ (L(U ⊕ It has a left-coprime factorization over S if there exist [N; ˜ has an inverse on S, Y, Y )) and [X; Y] ∈ H∞ (L(Y, Y ⊕ U )) such that M −1 ˜ ˜ G(s) = M(s) N(s) on S and for all s ∈ C+ 0 there holds ˜ ˜ M(s)X(s) − N(s)Y(s) = I. G has a doubly coprime factorization if there exist a right-coprime factoriza˜ −1 N ˜ over S tion G = NM−1 over S and a left-coprime factorization G = M ˜ ˜ and for certain [X, Y] ∈ H∞ (L(U ⊕ Y, U )) and [X; Y] ∈ H∞ (L(Y, Y ⊕ U )) on C+ 0 there holds ˜ −Y ˜ MY X = ˜ ˜ N X −N M

−1

.

(1)

It is normalized if, in addition, for almost all ω ∈ R ∗ ∗ ˜ ˜ ˜ ˜ M(iω)∗ M(iω) + N(iω)∗ N(iω) = I = M(iω) M(iω) + N(iω) N(iω) . (2)

First we give the natural generalization of the conventional concept of stabilization for transfer functions in our algebra class. Definition 2 Assume that G and C are in HH∞ /H∞ (S). We say that C is an admissible feedback transfer function for G if I − CG (or equivalently

82

R.F. Curtain

I − GC) has an inverse in HH∞ /H∞ (S). An admissible feedback transfer function C for G stabilizes G if I −C −G I

−1

∈ H∞ (L(U× Y )).

While it is straightforward to generalize the factorization theory in [25] for this class, it has fundamental drawbacks that have been pointed out in [26] and [9]. They proposed a new concept of a controller with internal loop for well-posed systems (the transfer function is defined and bounded on some right half-plane S). In fact their theory easily extends to non well-posed systems. We begin with the formal definition. K11 K12 K21 K22 be transfer functions in HH∞ /H∞ (S) with values in L(U, Y ) and in L(Y × G 0 . We say that K is an adR, U × R), respectively, and denote P = 0 K missible feedback transfer function with internal loop for G if I − F P has a well-defined inverse on S, where   0I0 F = I 0 0 , F ∈ L(Y × U× R, U× Y × R). (3) 00I Definition 3 Let U , Y and R be Hilbert spaces. Let G and K =

In particular, we say that K stabilizes G with internal loop if (I − F P)−1 ∈ H∞ (L(U× Y × R)). The intuitive interpretation of Definition 3 is the following: G represents the system to be stabilized and K is the transfer function of the controller in Figure 1 from [yk ; ζi ] to [uk ; ζo ], when all the connections are open. The connection from ζo to ζi is the so-called internal loop. The admissibility part of the definition means that it is possible to close all the connections in Figure 1, and obtain well-defined transfer functions in HH∞ /H∞ (S) from the three external inputs (v, r and ξ) to all the other signals. The stabilization part of the definition means that in fact all these closed-loop transfer functions are stable. If I − K22 has an inverse in HH∞ , then the above stability concept coincides with the conventional one in Definition 2 with the controller K11 + K12 (I − K22 )−1 K21 . In [9] it was shown that with the concept of stabilization with internal loop one obtains a nice generalization of the factorization theory in [25], including a more elegant Youla-Biongiorno factorization. The proofs there were for wellposed systems G, but they work for our more general algebra of transfer functions. In [9] they introduce canonical controllers with the form K=

0 I , K21 K22

with K21 , K22 ∈ H∞ ,

(4)


v

up

+ +

83

yp

G

uk

yk

r

+ +

K

+

ζo

ζi

+

ξ

Fig. 1. The system G connected to a controller K with internal loop

and dual canonical controllers with the form K=

0 K12 , I K22

with K12 , K22 ∈ H∞ .

(5)

We summarize the fundamental properties of the canonical controllers. Theorem 4 Let G be in HH∞ /H∞ (S). a. The canonical controller K stabilizes G with internal loop iff D0 := I − K22 − K21 G is invertible on S and D0−1 , GD0−1 ∈ HH∞ . b. If G has a right-coprime factorization G = NM, then K stabilizes G with internal loop iff D := M − K22 M − K12 N is invertible over HH∞ . c. G has a right-coprime factorization if and only if G is stabilizable with internal loop by a canonical controller. d. If G has a doubly coprime factorization as in Definition 1, then all canonical controllers which stabilize G with internal loop are given by K=

0 ˜ ˜ ˜ E(Y + J M)

I ˜ ˜ ˜ , I − E(X + J N)

˜ ∈ HH∞ and E ˜ is invertible over HH∞ . where J, E We call J appearing in Theorem 4 and in the following dual theorem the Youla parameter of K. Theorem 5 Let G be in HH∞ /H∞ (S). ˜ 0 := a. The dual canonical controller K stabilizes G with internal loop iff D ˜ −1 G ∈ ˜ −1 , D I − K22 − GK12 is invertible on some right half-plane and D 0 0 HH∞ .

84

R.F. Curtain

˜ then K stabilizes G ˜ −1 N, b. If G has a left-coprime factorization G = M ˜ := M ˜ − MK ˜ 22 − NK ˜ 12 is invertible over HH∞ . with internal loop iff D c. G has a left-coprime factorization if and only if G is stabilizable with internal loop by a dual canonical controller. d. If G has a doubly coprime factorization as Definition 1, then all dual canonical controllers which stabilize G with internal loop are given by K=

0 I

(Y + MJ)E , I − (X + NJ)E

where J, E ∈ HH∞ and E is invertible over HH∞ . All stabilizing controllers with internal loop are equivalent to a (dual) canonical controller. To explain what is meant by ”equivalent” let us analyze the connection in Figure 1 in more detail. We use the auxiliary inputs v and r from Figure 1, so that up = uk + v and yk = yp + r, while assuming that ξ = 0. If K stabilizes G with internal loop, then we can define the transfer function TG,K ∈ H∞ by [ˆ up ; yˆk ] = TG,K [ˆ v ; rˆ]. We call TG,K the compensation operator of the pair (G, K) and if K1 and K2 are two stabilizing controllers with internal loop for G, we call them equivalent if their compensation operators (G, K1 ) and (G, K2 ) are equal. Theorem 6. Suppose that G ∈ HH∞ /H∞ (S) and K is a stabilizing controller with internal loop for G. Then there exists a unique J ∈ HH∞ such that K is equivalent to the canonical controller Kl =

0 I ˜ + JM ˜ I − (X ˜ + J N) ˜ , Y

and is also equivalent to the dual canonical controller Kr =

0 Y + MJ . I I − (X + NJ)

The formulae for the parameterization of all stabilizing (dual) canonical controllers in Theorems 4 and 5 are reminiscent of the conventional Youla parameterization. Consequently, we expect a strong relationship between stabilization with internal loop and the usual concept of stabilization. Lemma 1. Let G and C be in HH∞ /H∞ (S). Assume that C has a rightcoprime factorization C = K12 (I − K22 )−1 over S. Then G is stabilized by C iff G is stabilized with internal loop by the dual canonical controller 0 K12 . K= I K22 Of course the above proposition has an obvious dual version.


85

3 Formulation of the Problem We aim to design controllers with internal loop that stabilize transfer functions in HH∞ /H∞ (S) robustly with respect to certain perturbations. We now define the type of perturbations we consider. ˜ M] ˜ Definition 7 Let G ∈ HH∞ /H∞ (S) have a left-coprime factorization [N, over S. ε-left-coprime factor perturbations of G are transfer functions ˜ + ΔM )−1 (N ˜ + ΔN ), GΔ = (M ˜ + ΔM is invertible over S, Δ = [ΔN , ΔM ] ∈ HH∞ and Δ where M

∞

< ε.

Our robust stabilization problem for G is the following. Given ε > 0 find controllers with internal loop K ∈ HH∞ /H∞ (S) that stabilize all ε-left-coprime factor perturbations of G with internal loop. Moreover, find the maximum robustness margin εmax . In Glover and McFarlane [14] it was shown that with this particular class of perturbations one can obtain explicit solutions for the rational case. The following lemma is a generalization of a key result in [14]. As explained in Section 2, we can without loss of generality consider dual canonical controllers. Theorem 8 Suppose that G ∈ HH∞ /H∞ (S) has a normalized doubly coprime factorization over S and K ∈ HH∞ is a dual canonical controller given by (5). Let 0 < ε < 1. If K stabilizes G with internal loop and K12 ˜ −1 ∞ ≤ 1 , (6) D I − K22 ε ˜ = M(I ˜ − K22 )− NK ˜ 12 , then it stabilizes all ε-admissible left-coprime where D factor perturbations of G with internal loop. Proof. From Theorem 5 K stabilizes GΔ with internal loop if and only if ˜ Δ = (M ˜ + ΔM )(I − K22 ) − (N ˜ + ΔN )K12 is invertible over HH∞ . Now D ˜Δ = D

I − −ΔM ΔN

I − K22 ˜ −1 ˜ D D K12

I − K22 ˜ −1 D is in HH∞ K12 and it has norm strictly less than one. So I − Q is invertible over HH∞ and ˜ Δ = (I − Q)D. ˜ so is D So if (6) holds, then the term Q = −ΔM ΔN

In the following theorem we show how the solution to (6) is related to a Nehari problem. Since we only use it in Section 5 for S = SRHP , some right half-plane, it suffices to prove a version for G ∈ HH∞ /H∞ (SRHP ) and for controllers with right-coprime factorizations. We use the notation G† (s) = G(−¯ s)∗ Theorem 9 Suppose that G has a normalized doubly coprime factorization over SRHP and let 0 < ε < 1. The following statements are equivalent.

86

• •

R.F. Curtain

K with a right-coprime factorization over SRHP stabilizes G and (6) holds; there exist W, V ∈ MH∞ such that V is invertible on SRHP and ˜† W −N + ˜ V M†

•

∞≤

1 − ε2 ;

(7)

K has a right-coprime factorization K = WV−1 on some right half-plane, where W, V satisfy (7).

Proof. The proof follows the same arguments as in [20, Theorem 7.2.3], except for one step. First note that all the terms in (1) have boundary functions on iR and we denote their values by M(iω), N(iω), etc., for almost all ω ∈ R. All arguments in this proof are made for a fixed ω and apply almost everywhere M(iω)∗ N(iω)∗ is co-isometric: on R. It is readily verified that W(iω) = ˜ ˜ M(iω) −N(iω) W(iω)W(iω)∗ = I. In [20, Theorem 7.2.3] it is a square matrix, and so trivially we have W(iω)∗ W(iω) = I. It is this isometric property that we need to prove for infinite-dimensional U, Y . We show that W is one-to-one on L2 , i.e., W[u; y] = 0 =⇒ [u; y] = 0, where for brevity we omit the argument ˜ = NM ˜ ˜ M]. ˜ (iω). From (1) we have MN and so Ran[M; N] ⊂ Ker[−N, Let [x; y] ∈ the closure of Ran[M; N]. Then there exist a sequence un ∈ U such that [x; y] = lim→∞ [Mun ; Nun ]. Using (2) we obtain M∗ Mun + N∗ Nun = un and since the left-hand side converges to M∗ x + N∗ y, un has a limit u. Thus [x; y] = [Mu; Nu] and Ran[M; N] is closed, yielding the decomposition [U ; Y ]=Ran[M; N]⊕ Ran[M; N]⊥ = Ran[M; N]⊕ Ker[N∗ , M∗ ]. We now show ˜ M] ˜ by a contradiction argument. that Ran[M; N]=Ker[−N, ˜ ˜ but not in Ran[M; N]. Then, from the above Suppose that x ∈ Ker[−N, M], ˜ M]∩ ˜ ˜ − Nx ˜ = decomposition of [U ; Y ], [x; y] ∈ Ker[−N, Ker[M∗ , N∗ ], i.e., My ∗ ∗ 0 = M x + N y. Using (1) gives ˜ −Y ˜ MY X [x; y] = [x; y] ˜ ˜ N X −N M ˜ − and premultiplying both sides by [M∗ , N∗ ] and using (2) shows that Xx ˜ Yy = 0. Substituting this back into the previous equation shows that [x; y] = 0. So Ker[M∗, N∗ ] ∩ Ran [N; M] = {0} almost everywhere on iR and ˜ ˜ Ker W(iω) = Ker [N(iω)∗ , M(iω)∗ ] ∩ Ker [−N(iω), M(iω)] = {0}. The rest of the proof is as in [20, Theorem 7.2.3].

4 The Suboptimal Nehari Problem In this section we review known results on the Nehari problem from Curtain and Opmeer [6]. It is well-known (see Page [23]) that for G ∈ L∞ (iR; L(U, Y ))


inf

F(−s)∈H∞ (L(U,Y ))

G+F

∞=

HG ,

87

(8)

where HG is the Hankel operator with symbol G. We note that in the following L∞ denotes the class of weakly measurable essentially bounded functions and that all H∞ functions have boundary functions in this class. Definition 10 For G ∈ L∞ (iR; L(U, Y )) we define the Hankel operator with symbol G as the operator HG : H2 (U ) → H2 (Y ) given by HG f = Π(ΛG f− ) for f ∈ H2 (U ), where ΛG is the multiplication map on L2 (iR; U ) induced by G, Π is the orthogonal projection from L2 (iR; U ) onto H2 (U ) and f− (s) := f (−s). From Theorem 9 we see that to obtain formulas for our robustly stabilizing controllers we need an explicit solution to the suboptimal Nehari problem, which is to find for any σ > HG a F(−s) ∈ H∞ (L(U, Y )) such that G + F ∞ ≤ σ. We only need solutions for the special class of system-stable state linear systems Σ(A, B, C, D), which we define below. Definition 11 Let A be the generator of a strongly continuous semigroup T (·) on a separable Hilbert space Z, B ∈ L(U, Z), C ∈ L(Z, Y ), D ∈ L(U, Y ) with U, Y separable Hilbert spaces. We call Σ(A, B, C, D) a state linear system. It is system-stable if • • •

it is input stable: there exists a constant β > 0 such that for all u with compact support in L2 (0, ∞; U ) ∞ ∞ 2 ≤ β 0 u(t) 2 dt; 0 T (t)Bu(t) dt it is output stable: there exists a constant γ > 0 such that for all z ∈ Z ∞ CT (t)z 2 dt ≤ γ z 2; 0 it is input-output stable: the transfer function G = D + C(sI − A)−1 B ∈ H∞ (L(U, Y )).

That these concepts of input and output stability are appropriate ones is clear from the following known results from [15] and [16]. Lemma 2 The state linear system Σ(A, B, C, D) is output stable if and only if the following observability Lyapunov equation has a self-adjoint nonnegative solution L ∈ L(Z) : A∗ Lz + LAz = −C ∗ Cz for all z ∈ D(A). The observability Gramian LC is the smallest bounded nonnegative solution. The state linear system Σ(A, B, C, D) is input stable if and only if the following controllability Lyapunov equation has a self-adjoint nonnegative solution L ∈ L(Z) : ALz + LA∗ z = −BB ∗ z for all z ∈ D(A∗ ). The controllability Gramian LB is the smallest bounded nonnegative solution.

88

R.F. Curtain

These gramians are the key to finding the Hankel norm of a system. Lemma 3 ( [20, Lemma 7.1.5], [6, Theorem 4.4]) Suppose that Σ(A, B, C, D) is a system-stable state linear system with impulse response h(t) = CT (t)B and transfer function G. For u ∈ L2loc ([0, ∞); L(U, Y )) of compact support we define the time domain Hankel operator associated with h by ∞ Γh u(t) = 0 h(t + s)u(s) ds. 1. Γh is bounded from L2 (0, ∞; U ) to L2 (0, ∞; Y ); 1 2. HG = Γh = r 2 (LB LC ), where r denotes the spectral radius and LB , LC are the controllability and observability gramians, respectively, of Σ(A, B, C, D). 3. If σ > Γh , then Nσ := (I − σ −2 LB LC )−1 ∈ L(Z). Moreover, W = + Nσ LB is self-adjoint and nonnegative and ρ(A)∩C+ 0 ⊂ ρ(AW )∩C0 , where −2 ∗ AW = A − σ W C C. First we quote the result concerning the existence of at least one solution to the suboptimal Nehari problem from [7, Theorem 6.5]. Theorem 12 Suppose that Σ(A, B, C, D) is a system-stable state linear system with transfer function G and controllability and observability grami1 ans LB , LC , respectively. If σ > HG = r 2 (LB LC ) and Nσ := (I − −2 −1 σ LB LC ) , then Fc (−s) = Z(s) ∈ H∞ (L(U, Y )) satisfies G+ Fc ∞ ≤ σ, where Z(s) = −D − σ −2 CW (sI − A∗ + σ −2 C ∗ CW )−1 LC B. The following theorem that parametrizes infinitely many solutions to the suboptimal Nehari problem follows easily from the above and from [6, Theorem 7.1]. Theorem 13 Suppose that Σ(A, B, C, D) is a system-stable state linear system with transfer function G and controllability and observability grami1 ans LB , LC , respectively. If σ > HG = r 2 (LB LC ) and Nσ := (I − −2 −1 σ LB LC ) , then solutions F(−s) ∈ H∞ (L(U, Y )) to the suboptimal Nehari problem are given by F(−s) = R1 (s)R2 (s)−1 , where [R1 (s); R2 (s)] = V(¯ s)∗ [Q(−s); IU ], and V is the transfer function of the state linear system Σ(A, σ −2 [−LB C ∗ − BD∗ , B], [C; σ −1 B ∗ LC ]Nσ , Dσ ), where IY 0 and Q(−s) ∈ H∞ (L(U, Y )) satisfies Q ∞ < 1. Dσ = −σ −1 D∗ σ −1 IU The above represents a nice generalization of the finite-dimensional solutions. The differences are that that we are not sure that all solutions have such a parametrization and that parameters Q with norm one are excluded. Under either of the additional assumptions that U, Y are finite-dimensional or σ(A)∩ iR has measure zero parameters Q with norm one are allowed ( [6, Theorem 7.1]).


89

5 Normalized Coprime Factorizations We consider the special class of state linear systems Σ(A, B, C, D). As discovered by Meyer and Franklin [17], there is a strong connection between normalized coprime factorizations and the linear quadratic regulator problem or, more precisely, the optimal control problem minu∈L2 (0,∞;U)

∞ 0

y(t)

2

+

u(t)

2

dt,

where z(t) ˙ = Az(t) + Bu(t); y(t) = Cz(t) + Du(t). It is well-known (see Curtain and Zwart [3, Chapter 6]) that if Σ(A, B, C, D) satisfies the finite-cost condition, i.e., for all initial states z0 ∈ Z there exists an input u ∈ L2 (R+ ; U ) such that the output y ∈ L2 (R+ ; Y ), then for each z0 ∈ Z there exists a unique uopt ∈ L2 (R+ ; U ) for which the minimum is attained and there exists a bounded nonnegative operator Qopt such that the minimal cost is given by Qopt z0 , z0 . The optimal input is a state feedback: uopt (t) = F opt z opt (t), where F opt := −S −1 (D∗ C + B ∗ Qopt ), S := I + D∗ D. Qopt is the smallest bounded nonnegative solution to the control algebraic Riccati equation on D(A) A∗ Q + QA + C ∗ C = (QB + C ∗ D)S −1 (B ∗ Q + D∗ C),

(9)

where F := −S −1 (D∗ C + B ∗ Q) and S := I + D∗ D. Let AQ = A + BF . If the dual system Σ(A∗ , C ∗ , B ∗ , D∗ ) satisfies the finite-cost condition, then then there exists a smallest bounded nonnegative solution P opt to the filter Riccati equation on D(A∗ ) AP + P A∗ + BB ∗ = (P C ∗ + BD∗ )R−1 (CP + DB ∗ ),

(10)

where R := I + DD∗ and L := −(P C ∗ + BD∗ )R−1 . Let AP = A + LC. In [6] the following extra properties were shown to be useful. Lemma 4 If both Σ(A, B, C, D) and its dual satisfy the finite-cost condition, then the left-factor system Σ(AP , R−1/2 [B + LD, L], C, R−1/2 [D, I]), and the right-factor system Σ(AQ , BS −1/2 , [C + DF ; F ], [D; I]S −1/2 ) are both systemstable. The closed-loop generators AQ and AP are related by (I + P Q)AQ z = AP (I + P Q)z for z ∈ D(A),

(11)

+ and so ρ(AQ ) = ρ(AP ). Moreover, ρ(A) ∩ C+ 0 ⊂ ρ(AQ ) ∩ C0 and ρ(A) ∩ iR ⊂ ρ(AQ ) ∩ iR.

Henceforth we choose the smallest solutions to the Riccati equations and drop the suffix “opt”. In this case, P and Q are the controllability, respectively, observability gramians of the left- and right-factor systems. The observability gramian of the left-factor system is Q(I + P Q)−1 and the controllability

90

R.F. Curtain

gramian of the right-factor system is P (I + P Q)−1 (see [7, Lemma 4.9]). So from Lemma 3 we see that these factor systems have the same Hankel norm r1/2 (P Q(I +P Q)−1 ) whisch is strictly less than 1. So by choosing the smallest bounded nonnegative solutions, the transfer functions are normalized which implies that they are coprime. We summarize these new sufficient conditions for Σ(A, B, C, D) to have normalized left- and right-coprime factorizations over H∞ from [7]. The candidates for the coprime factors agree with the known finite-dimensional formulas, but the Bézout factors are surprizingly different. Theorem 14 If both Σ(A, B, C, D) and its dual satisfy the finite-cost condi˜ , M] ˜ of the left-factor system Σ(AP , [B + tion, then the transfer function [N −1/2 −1/2 LD, L], R C, R [D, I]), and the transfer function [N; M] of the rightfactor system Σ(AQ , BS −1/2 , [C + DF ; F ], [D; I]S −1/2 ) form a normalized doubly coprime factorization of G, provided that we choose P and Q to be the smallest nonnegative solutions to their respective Riccati equations. ˜ , M] ˜ and of [N; M] equal r1/2 (P Q(I + P Q)−1 ) < 1. The Hankel norm of [N A Bezout pair corresponding to the left-coprime factor is the transfer function of the system-stable state linear system ˜ = (I + P Q − Σ (A˜F , P C ∗ R−1/2 , [C + DF˜ ; F˜ ], [I, −D∗ ]R−1/2 ), where W −2 −1 ˜ −1 −2 ∗ ∗ ˜ ˜ σ P Q) , F = −S (σ B QW + D C), AF = A + BF˜ and σ is such that r1/2 (P Q(I + P Q)−1 ) < σ < 1. A Bezout pair corresponding to the rightcoprime factor is the transfer function of the system-stable state linear system ˜ L ˜ ], S −1/2 B ∗ Q, S −1/2 [I, −D∗ ]), where L ˜ = −(σ −2 W ˜ P C∗ + Σ(A˜L , [B + LD, ∗ −1 ˜ ˜ BD )R and AL = A + LC . If σ = 1, the formulas for the Bézout factors reduce to the same formulas as in the finite-dimensional case. However, in general we can only show that these σ = 1 Bézout factors are in H2 (modulo a constant) and not necessarily in H∞ . The Bézout factors X, Y are obtained by solving the Nehari problem ˜ , M] ˜ + [F1 , F2 ] ≤ σ, where r1/2 (P Q(I + P Q)−1 ) < σ < 1 and they are [− N given by ˜ − NW ˜ )−1 , Y = W(MV ˜ − NW ˜ )−1 , X = V(MV ∗ ∗ where [W(s), V(s)] = [F1 (−s¯) , F2 (−s¯) ].

6 Robustly Stabilizing Controllers for State Linear Systems In this section we generalize the results in [20] to allow for a larger class of state linear systems that are not necessarily strongly stabilizing and U, Y are allowed to be infinite-dimensional. To obtain formulas for our robustly stabilizing controllers we need an explicit solution to the suboptimal Nehari problem (7). Applying the Page result (8) and Theorems 8, 9 and 14 to our robust stabilization problem, we see that


91

we can only hope to solve (7) for ε < εmax , the maximum robustness margin, which is given by ε2max = 1−

H[−N, ˜ M] ˜

2

= 1 − r(P Q(I + P Q)−1 ).

(12)

The solution to our robust stabilization problem for state linear systems now follows readily from the known results summarized in Sections 2-4. Theorem 15 If the state linear system Σ(A, B, C, D) and its dual satisfy the finite-cost condition, then the robust stabilization problem is solvable for all ε < εmax , where εmax = (1 + r(P Q))−1/2 ,

(13)

and P, Q are the smallest bounded nonnegative solutions to the Riccati equations (10) and (9), respectively. Robustly stabilizing controllers with a robustness margin of ε, 0 < ε < εmax , are given by the following right-coprime factorizations over SRHP C(s) = [φ11 (s)J(s) + φ12 (s)][φ21 (s)J(s) + φ22 (s)]−1 .

(14)

φ is the transfer function of the state linear system Σ(A + BF, T, σ −2 [F ; C + DF ], Dφ ), where F = −S −1 (D∗ C + B ∗ Q), T = (I + P Q − σ −2 P Q)−1 [ε−1 BS −1/2 , −σ −1 P C ∗ R−1/2 ], 1 ε−1 S −1/2 σ −1 D∗ R−1/2 Dφ = −1 , σ = (1 − ε2 ) 2 , −1/2 −1 −1/2 ε DS σ R and J ∈ H∞ (U ) satisfies J

∞

< 1.

Proof. From Theorem 14 we see that a suitable choice for a doubly normalized coprime factorization is given by the transfer functions of the left- and rightfactor systems where P, Q are chosen to be the smallest bounded nonnegative solutions to the Riccati equations (10) and (9), respectively. From Theorem 9 we see that the robust stabilization problem has a solution if H[−N, < 1 and this also holds by Theorem 14. As explained above, ˜ M] ˜ the maximum robustness margin satisfies (12). Note that since both P and Q are nonnegative, I + P Q is boundedly invertible. We show that r(P Q(I + P Q)−1 ) =

r(P Q) . 1 + r(P Q)

For λ = 1 it is readily verified that (λI − P Q) has a bounded inverse if and λ only if 1+λ I − P Q(I + P Q)−1 has. So λ < r(P Q) =⇒

λ < r(P Q(I + P Q)−1 ), and 1+λ

92

R.F. Curtain

r(P Q) ≤ r(P Q(I + P Q)−1 ). 1 + r(P Q) Similarly,

µ < r(P Q(I + P Q)−1 ) =⇒

µ < r(P Q), 1−µ

which implies the reverse inequality and (13) holds. From Theorem 9 the parametrization of the robustly stabilizing controllers with a robustness margin less than εmax proceeds by finding a parametrization of W, V that solve (7). This is done by applying Theorem 13 and finally C = WV−1 on SRHP . These formulas can be simplified as in Curtain and Zwart [3, Theorem 9.4.15]. Finally we show that the central controller (J = 0) corresponds to the controller obtained using the Bézout factors from Theorem 14. Lemma 5 Under the assumptions of Theorem 15, the central controller Cc obtained by substituting J = 0 in (14) coincides with the controller −1 Cb = √ YX , where [X; Y] are the Bézout factors from Theorem 14 with σ = 1 − ε2 . ˆ −1 W ˜ P C∗, Proof. Cc (s) = φ12 (s)φ22 (s)−1 = −D∗ − σ −2 B ∗ Q(sI − A) −2 −1 −2 ˜ ∗ −1 ˜ ˆ where W = (I + P Q − σ P Q) and A = AQ − σ W P C R (C − DB ∗ Q). Cb (s) = Y(s)X(s)−1 = −D∗ + F˜ (sI − A˜F )−1 P C ∗ (I + (C + DF˜ )(sI − A˜F )−1 P C ∗ )−1 ˜ −1 P C ∗ ) = −D∗ + F˜ (sI − A˜F )−1 P C ∗ (I − (C + DF˜ )(sI − A) ˜ (sI − A) ˜ −1 P C ∗ , = −D∗ − σ −2 B ∗ QW ˜ . To prove the equality of the where A˜ = AP − σ −2 (B − P C ∗ D)S −1 B ∗ QW −1 ˆ ˜ ˜ −1 . This is equivalent to two controllers it suffices to prove that W A = A˜W proving (I + P Q − σ −1 P Q)AQ − σ −2 P C ∗ R−1 C = AP (I + P Q − σ −1 P Q) − σ −2 BS −1 B ∗ Q, which is equivalent to proving (I + P Q)AQ − σ −2 (I + P Q)AQ + σ −2 AQ − σ −2 P C ∗ R−1 C = AP (I + P Q) − σ −1 AP (I + P Q) + σ −2 AP − σ −2 BS −1 B ∗ Q. Using (11) we see that this is indeed true. The above gives a nice interpretation of the strange formulas for the Bézout factors in Theorem 14: they are the √ ones that lead to a robustly stabilizing controller with robustness margin 1 − σ 2 . Note that with σ = 1 the controller C = YX−1 reduces to a symmetric LQG-controller.


93

7 Integrated Nodes and Reciprocals First we briefly review the definition of an operator node (see Staffans [24, Section 4.7]) and introduce the concept of an integrated node (this is a specialization of the concept of a distributional resolvent linear system introduced in Opmeer [22]). U , Y , Z are separable Hilbert spaces. An operator node is specified by three operators A, B, C and a characteristic function G. These are assumed to satisfy: • A is a closed densely defined operator on Z with nonempty resolvent set. • C ∈ L(D(A), Y ) is bounded where D(A) is equipped with the graph norm. • B ∗ ∈ L(D(A∗ ), U ) is bounded where D(A∗ ) is equipped with the graph norm. • G : ρ(A) → L(U, Y ) and for α, s ∈ ρ(A) satisfies G(s) − G(α) = (α − s)C(sI − A)−1 (αI − A)−1 B. The dual of an operator node is specified by the operators A∗ , C ∗ , B ∗ , G(¯ α)∗ . Definition 16 An integrated node is an operator node for which ρ(A) contains a right half-plane and there exist M > 0 and n ∈ Z such that (sI − A)−1 ≤ M |s|n on this right half-plane. We recall that the Laplace transform can be defined for certain Banach space valued distributions and that the image of the set of Laplace transformable distributions is exactly the set of functions defined on some right half-plane that are analytic and polynomially bounded. This allows us to define the state and output of an integrated node as Laplace transformable distributions (Z-valued and Y -valued, respectively). Definition 17 For an initial state z0 ∈ Z and a Laplace transformable distribution u the state and output of an integrated node are defined through their Laplace transforms by zˆ(s) = (sI − A)−1 z0 + (sI − A)−1 B u ˆ(s), yˆ(s) = C(sI − A)−1 z0 + G(s)ˆ u(s), with s in some right half-plane that is contained in ρ(A). G is the transfer function and it equals the characteristic function G on any right half-plane contained in ρ(A), but they can differ outside this region (see [6]). The finite cost condition for integrated nodes is defined as for state linear systems: for every z0 ∈ Z there exists a u ∈ L2 (0, ∞; U ) such that the corresponding output y ∈ L2 (0, ∞; Y ). The concept of a reciprocal system was introduced in [4] for well-posed linear system and it easily extends to integrated nodes with 0 ∈ ρ(A).

94

R.F. Curtain

Definition 18 Suppose that the integrated node Σ with generating operators A, B, C and characteristic function G is such that 0 ∈ ρ(A). Its reciprocal system is the state linear system Σ(A−1 , A−1 B, −CA−1 , G(0)). The linear quadratic regulator problem for a system node can be solved via its reciprocal system. The following result was shown in [21] for well-posed linear systems and for the very general class of distributional resolvent systems in Opmeer [22]. Theorem 19 An integrated node Σ with 0 ∈ ρ(A) satisfies the finite cost condition if and only if its reciprocal system does. Moreover, if the finite cost condition is satisfied, then there exists unique optimal controls for Σ and for its reciprocal system and the optimal costs are equal. So we obtain the optimal cost operator for an integrated node as the smallest bounded nonnegative solution to the following reciprocal control Riccati equation A−∗ Q + QA−1 + A−∗ C ∗ CA−1 =

(15)

(QA−1 B − A−∗ C ∗ Dr )Sr−1 (B ∗ A−∗ Q − Dr∗ CA−1 ), where Dr = G(0) and Sr = I + Dr∗ Dr . The closed-loop main generating operator is given by ArQ = A−1 + Sr−1 (Dr∗ CA−1 − B ∗ A−∗ Q). An analogous remark applies to the dual system with the reciprocal filter Riccati equation A−1 P + P A−∗ + A−1 BB ∗ A−∗ = −∗

(P A

∗

−1

C −A

BDr∗ )Rr−1 (CA−∗ P

(16) ∗

−∗

− Dr B A

),

where Rr = I + Dr Dr∗ . The closed-loop main generating operator is given by ArP = A−1 + (−P A−∗ C ∗ + A−1 BDr∗ )Rr−1 CA−1 . This connection provides a normalized doubly coprime factorization of the transfer function of the integrated node. The following result is proven in Curtain and Opmeer [7, Theorem 8.9]. Lemma 6 Suppose that Σ is an integrated node with generating operators A, B, C and characteristic function G such that 0 ∈ ρ(A) and Σ and its dual satisfy the finite cost condition. Denote the transfer functions of the left- and ˜ r, M ˜ r] right-factor systems of its reciprocal system given in Theorem 14 by [N r r and [N ; M ], respectively. Then the transfer function of the integrated node has a left (right)-coprime factorization given on some right half-plane by ˜ r ( 1 )−1 N ˜ r ( 1 ) = Nr ( 1 )Mr ( 1 )−1 . G(s) = M s s s s

(17)

So an idea would be to find the conventional robustly stabilizing controllers Cr for the reciprocal transfer function Gr to show that C(s) = Cr ( 1s ) stabilizes G. However, in general, C need not be well-defined and so we design controllers with internal loop.


95

Theorem 20 Suppose that the integrated node Σ with generating operators A, B, C and characteristic function G is such that 0 ∈ ρ(A). If Σ and its dual satisfy the finite-cost condition, then the robust stabilization problem is solvable for all ε < εmax = (1 + r(P Q))−1/2 , where P, Q are the smallest bounded nonnegative solutions to the Riccati equations (16) and (15), respectively. Robustly stabilizing dual canonical controllers with internal loop and a robustness margin of ε (0 < ε < εmax ) are given by (5), where 1 1 1 1 K12 (s) = φr11 ( )J(s) + φr12 ( ), K22 (s) = I − φr22 ( ) − φr21 ( )J(s). s s s s φr is defined in Theorem 5 applied to the reciprocal system and J ∈ H∞ (U ) satisfies J ∞ < 1. Proof. From Lemma 6 we know that the doubly coprime factorizations of G and its reciprocal transfer function Gr are in a one to one correspondence through (17) on some right half-plane. So we solve the robust stabilization problem for the reciprocal system with transfer function Gr using the results from Section 5. Denote the controllers defined by the formulas (14) applied to the reciprocal system by Cr . From Lemma 1 this is equivalent to the dual canonical controller with open loop with Kr12 (s) = φr11 ( 1s )J( 1s ) + φr12 ( 1s ), Kr22 (s) = I − φr22 ( 1s ) − φr21 ( 1s )J( 1s ). Note that F(s) ∈ HH∞ if and only if F( 1s ) ∈ HH∞ . So from Theorem 4 and Theorem 6 it follows that K(s) = Kr ( 1s ) stabilizes G with internal loop if and only if Kr stabilizes Gr with internal loop. Moreover, since the reciprocal system has the same P and Q operators, the robustness margins are identical. We also give conditions under which stabilizing controllers with internal loop can be realized as conventional controlllers. Corollary 1 Suppose that the integrated node Σ satisfies the assumptions in Theorem 20 and in addition either both U and Y are finite-dimensional or the component of ρ(A) containing a right half-plane also contains the origin. Then conventional robustly stabilizing controllers with a robustness margin of ε (0 < ε < εmax ) are given by C(s) =

1 1 φr11 ( )J(s) + φr12 ( ) s s

1 1 φr22 ( ) + φr21 ( )J(s) s s

−1

,

φr is defined in Theorem 5 applied to the reciprocal system and J ∈ H∞ (U ) satisfies J ∞ < 1. Proof. We seek a controller C ∈ HH∞ /H∞ (S) for some set S in the right half-plane. From Lemma 1 we also need to show that C(s) = Cr ( 1s ) has a right-coprime factorization over S. As in the proof of Theorem 20 we find the robustly stabilizing controllers Cr for the reciprocal system Gr and consider

96

R.F. Curtain

the candidate C(s) = Cr ( 1s ). The coprimeness property follows from that of the reciprocal controller Cr (see Theorem 15). Now C is well-defined for 1/s in some right half-plane, or equivalently for s in the disc D(µ, µ) = {|z − µ| < µ} for some positive µ. Under the assumption that the component of ρ(A) containing a right half-plane also contains the origin, the coprime factorization (17) of G from Lemma 6 extends to this component and this has a nonzero intersection SA with D(µ, µ). So both G and C are in HH∞ /H∞ (SA ) and C is a conventional stabilizing controller. In the case that U and Y are finite-dimensional we note that square matrix+ valued holomorphic functions on C+ 0 are invertible almost everywhere on C0 and so we choose S to be the intersection of this region with the right halfplane where G has its normalized doubly coprime factorization. We remark that our proof for the case of U, Y finite-dimensional coincides with the class considered in [12], [13]. There are, however flaws in this approach. The controller is only defined almost everywhere on some right halfplane and so it does not have a good dynamical interpretation. The controller with internal loop is a more satisfactory approach. The condition that the component of ρ(A) containing a right half-plane should also contain the origin is satisfied by generators of contraction semigroups. In particular, dissipative systems with colocated actuators and sensors have contraction semigroups; the bounded input and output case has been studied in [19]) and the well-posed case in Curtain and Weiss [8]. In these papers the question of when the systems can be strongly or exponentially stabilized by the controller C = −kI for k > 0 is posed. As can be seen in the examples in [8], it is not trivial to verify the well-posedness assumptions. The advantage of our approach is that this is not necessary. However, our approach only looks at the weaker property of input-output stabilization, and we allow a more general class of controllers than static output feedback. Nonetheless, there are sufficient commonalities to make it an interesting comparison. Example 1 Consider the integrated node Σ with generating operators A, B, B ∗ , where A is skew adjoint and dissipative, i.e., D(A∗ ) = D(A) and A = −A∗ , Y = U and C = B ∗ and A generates a contraction semigroup. If 0 ∈ ρ(A), we obtain the reciprocal system Σ(A−1 , A−1 B, B ∗ A−∗ , G(0)). This reciprocal system falls into the class studied in [20, Chapter 2.2] and it is known that it and its dual satisfy the finite cost condition. So we obtain a whole class of systems that satisfy Theorem 20 and can be robustly stabilized. In principle we have explicit formulas for the robustly stabilizing controllers, but in general they do depend on the solutions of the reciprocal Riccati equations and the formulas are complicated. So this result is essentially an existence result. If, however, we consider the special case that G(0) + G(0)∗ = 0, then P = I = Q are solutions and they will be unique provided that the closed-loop


97

operators are strongly stable. Sufficient conditions for the latter to hold are that Σ is well-posed and exactly controllable (or equivalently exactly observable) for then the closed-loop Riccati semigroups will be exponentially stable (see Mikkola [18]). A physical example that satisfies all the above assumptions is the Rayleigh beam model Example 6.3 in [8] if we choose one sensor and one actuator. Under the above assumptions, we obtain the central controller 1 ∗ −∗ ˜ −1 A−1 B, Kc (s) = G(0) − 1−2ε (1/sI − A) 2B A −1 B(I + G(0))−1 B ∗ A−∗ and ε < √12 , the maximum where A˜ = A−1 − 2−2ε 1−2ε2 A robustness margin. Finally, it is interesting to note that if we choose J = σε I, then we obtain the robustly stabilizing controller C = −I as in [8]. 2

References 1. R.F. Curtain, G. Weiss, and M. Weiss, ”Coprime factorization for regular linear systems”, Automatica, vol. 32, pp. 1519–1531, 1996. 2. R.F. Curtain, ”Robust Stabilizability of Normalized Coprime Factors; the Infinite-Dimensional Case”, Int. J. Control, vol. 51, pp. 1173–1190, 1990. 3. R.F. Curtain and H.J. Zwart, An Introduction to Infinite-Dimensional Linear Systems Theory. Springer-Verlag, New York, 1995. 4. R.F. Curtain, ”Regular linear systems and their reciprocals: applications to Riccati equations”, Systems Control Lett., vol. 49, pp. 81–89, 2003. 5. R.F. Curtain, ”Riccati equations for stable well-posed linear systems; the generic case”, SIAM J. Contr. and Optim., vol. 42, pp. 1681–1702, 2003. 6. R.F. Curtain and M.R. Opmeer, ”The suboptimal Nehari problem for stable infinite-dimensional linear systems”, SIAM J. Contr. and Optim., to appear. 7. R.F. Curtain and M.R. Opmeer, ”Normalized doubly coprime factorizations for well-posed linear systems”, MCSS, to appear. 8. R.F. Curtain and G. Weiss. Exponential stabilization of well-posed systems by colocated feedback. SIAM J. Contr. and Optim., to appear. 9. R.F. Curtain, G. Weiss and M. Weiss, ”Stabilization of irrational transfer functions with internal loop”, p.179–208 in Systems, Approximation, Singular integral operators and related topics, Int. workshop on operator theory and applications, IWOTA 2000. Eds., A.A. Borichev and N.K. Nikolski, Vol. 129, in series Advances in operator Theory and Applications, Birkh¨ auser. 10. B.A. Francis, A Course in H ∞ Control Theory. Springer-Verlag, LNCIS, vol. 88, Berlin, 1987. 11. T.T. Georgiou and M.C. Smith, ”Graphs, causality and stabilizability: linear, shift-invariant systems on L2 [0, ∞)”, Math. for Control, Signals and Systems, vol. 6, pp. 195–223, 1993. 12. T.T. Georgiou and M.C. Smith, ”Optimal robustness in the gap metric”, IEEE Trans. Autom. Control, vol. 35, pp. 673–686, 1990. 13. T.T. Georgiou and M.C. Smith, ”Robust stabilization: Controller design for distributed plants”, IEEE Trans. Autom. Control, vol. 37, pp. 1133–1143, 1992. 14. K. Glover and D. McFarlane, ”Robust stabilization of normalized coprime factor plant descriptions with H∞ -bounded uncertainty”, IEEE Trans. Autom. Control, vol. 34, pp. 821–830, 1989.

98

R.F. Curtain

15. P. Grabowski, ”On the spectral-Lyapunov approach to parametric optimization of distributed parameter systems”, IMA Journal of Mathematical Control and Information, vol. 7, pp. 317–338, 1991. 16. S. Hansen and G. Weiss, ”New results on the operator Carleson measure criterion”, IMA Journal of Mathematical Control and Information, vol. 14, pp. 3–32, 1997. 17. D.G. Meyer and G.F. Franklin, ”A connection between normalized coprime factorizations and linear quadratic regulator theory”, IEEE Trans. Autom. Control, vol. 28, pp. 227–228, 1987. 18. K. Mikkola, Infinite-Dimensional Linear Systems, Optimal Control and Riccati Equations, PhD thesis, October 2002, Helsinki University of Technology. Available at http://www.math.hut.fi/reports 19. J.C. Oostveen and R.F. Curtain, ”Robustly stabilizing controllers for disipative infinite-dimensional systems with collocated actuators and sensors”, Automatica, vol. 36, pp. 337–348, 2000. 20. J.C. Oostveen, Strongly Stabilizable Distributed Parameter Systems, SIAM, Philadelphia, 2000. 21. Mark R. Opmeer and Ruth F. Curtain, ”New Riccati equations for well-posed linear systems”, Systems Control Lett., vol. 52, pp. 339–347, 2004. 22. Mark R. Opmeer, ”Infinite-dimensional linear systems: a distributional approach”, Proc. London Math. Soc. (to appear). 23. L.B. Page, ”Bounded and Compact vectorial Hankel operators”, Trans. Amer. Math. Soc., vol. 150, pp. 529–539, 1970. 24. O.J. Staffans, Well-Posed Linear Systems, volume 103, Encyclopedia of Mathematics and its Applications, Cambridge University Press, Cambridge, UK, 2005. 25. M. Vidyasagar, Control Systems Synthesis: A Factorization Approach, M.I.T. Press, 1985. 26. G. Weiss and R.F. Curtain, ”Dynamic stabilization of regular linear systems”, IEEE Trans. Aut. Control, vol. 42, pp. 1–18, 1997.

Model Reduction of Strongly Stable Nonstationary LPV Systems Mazen Farhood and Geir E. Dullerud Department of Mechanical Engineering, University of Illinois, Urbana, IL 61801, USA; [email protected], [email protected]

1 Introduction This paper deals with the model reduction of nonstationary linear parametervarying (NLPV) systems. Our interest in LPV models is motivated by the desire to control nonlinear systems along prespecified trajectories. LPV models arise naturally in such scenarios as a method to capture the possible nonlinear dynamics, while maintaining a model that is amenable to control synthesis. Frequently, when pursuing such an LPV formulation, one ends up with models of relatively large dimension. Accordingly, finding control syntheses for such models, which usually involves solving a number of linear operator inequalities as discussed in [5], requires substantial computation. For this reason, developing a theory that provides systematic methods of approximating such models is beneficial. In the paper, we utilize the theory of generalized gramians to define the notion of balanced realizations for NLPV systems. We also examine the balanced truncation method in detail and derive error bounds for such a reduction process. The contributions of the paper are as follows: • •

•

generalization of the balanced truncation model reduction procedure to the class of NLPV systems; several results on the worst-case balanced truncation error. These results when restricted to the purely time-varying case (i.e., no parameters) provide the least conservative error bounds currently available in the literature; operator theoretic machinery is developed in the context of standard robust control tools for working with NLPV models.

Our paper deploys a combination of recent work on NLPV models in [5] and new work on model reduction using balanced truncation for standard LTV systems in [13, 18]. The basic approach is motivated by the work in [1] on the generalization of balanced truncation to stationary multidimensional systems, and that in [12] on discrete time model reduction of standard LTI B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 99–118, 2006. © Springer-Verlag Berlin Heidelberg 2006

100

M. Farhood and G.E. Dullerud

systems. The basic approach behind balanced truncation originates in [16], and the by-now famous error bounds associated with this method in the LTI case were first demonstrated in [3, 11]. The NLPV models used here are the natural generalization of LPV models, first introduced in [15, 17], to the case of nonstationary systems. The paper is organized as follows: we begin with a section which establishes notation and collects some needed definitions; in Section 3 we introduce NLPV models and discuss two forms of stability; Section 4 provides the balanced truncation procedure and proves the main results of the paper; we conclude with a summary statement.

2 Preliminaries The set of real n × m matrices and that of real symmetric n × n matrices are denoted by Rn×m and Sn respectively. The maximum singular value of a matrix X is denoted by σ ¯ (X). Given two Hilbert spaces E and F , we denote the space of bounded linear operators mapping E to F by L(E, F ), and shorten this to L(E) when E equals F . If X is in L(E, F ), we denote the E to F induced norm of X by X E→F ; when the spaces involved are obvious, we write simply X . The adjoint of X is written X ∗ . When an operator X ∈ L(E) is self-adjoint, we use X 0 to mean it is negative definite; that is there exists a number α > 0 such that, for all nonzero x ∈ E, the inequality x, Xx < −α x 2 holds, where ·, · denotes the inner product and · denotes the corresponding norm on E. We use E⊕F to denote the Hilbert space direct sum of E and F . If Si is a sequence of operators, then diag(Si ) denotes their block-diagonal augmentation. The main Hilbert space of interest in this paper is formed from an infinite sequence of Euclidean spaces (Rn(0) , Rn(1) , Rn(2) , . . .), and is denoted by 2 (Rn(0) , Rn(1) , . . .), or just 2 (Rn ) for short. It is defined as the subn(k) consisting of elements x = space of the Hilbert space direct sum ⊕∞ k=0 R ∞ (x(0), x(1), x(2), . . .), with x(k) ∈ Rn(k) , so that x 2 = k=0 x(k)∗ x(k) < ∞. The inner product of x, y in 2 (Rn ) is hence defined as the sum x, y 2 = ∞ ∗ k=0 x(k) y(k). If the spatial dimensions n(k) are either evident or irrelevant to the discussion, then the notation 2 (Rn ) is abbreviated to 2 . Also, we will n(k) use (Rn ) to denote ⊕∞ . k=0 R A key operator used in the paper is the unilateral shift Z, defined as follows: Z:

2

(Rn(1) , Rn(2) , . . .) → Z

2 (R

n(0)

, Rn(1) , Rn(2) , . . .)

(a(1), a(2), . . .) −→ (0, a(1), a(2), . . .). Clearly this definition is extendable to , and in the sequel, we will not distinguish between these mappings. Given a time-varying dimension n(k), we define the notation I n2 := diag(In(0) , In(1) , In(2) , . . .), where In(k) is an n(k)× n(k) identity matrix.

Model Reduction of Strongly Stable Nonstationary LPV Systems

101

Following the notation and approach in [2], we make the following definitions. First, we say a bounded linear operator Q mapping 2 (Rm(0) , Rm(1) , . . .) to 2 (Rn(0) , Rn(1) , . . .) is block-diagonal if there exists a sequence of matrices Q(k) in Rn(k)×m(k) such that, for all w, z, if z = Qw, then z(k) = Q(k)w(k). Then Q has the representation diag(Q(0), Q(1), Q(2), . . .). A diagonal operator is a block-diagonal operator where each of the matrix blocks is diagonal. Suppose F , G, R and S are block-diagonal operators, and let A be a partitioned operator of the form A=

F G . RS

Then we define the following notation: A := diag

F (1) G(1) F (0) G(0) ,... , R(1) S(1) R(0) S(0)

,

which we call the diagonal realization of A. Clearly for any given operator A of this particular structure, A is simply A with the rows and columns permuted appropriately so that A

k

=

F (k) G(k) . R(k) S(k)

From this definition, it is easy to see that A + B = A + B and AC = A C hold for appropriately dimensioned operators, and similarly that A βI holds if and only if A βI, where β is a scalar. Namely, the • operation is a homomorphism from partitioned operators with blockdiagonal entries to block-diagonal operators.

3 NLPV Systems We now briefly review NLPV models. The reader is referred to [5] for an indepth treatment of the theory. To start, the NLPV models of this paper are of the form x(k + 1) = A(δ(k), k)x(k) + B(δ(k), k)w(k) z(k) = C(δ(k), k)x(k) + D(δ(k), k)w(k), where A(·, ·), B(·, ·), C(·, ·), and D(·, ·) are matrix-valued functions that are known a priori. The variable k is time, and δ(k) := (δ1 (k), . . . , δd (k) ) is a vector of real scalar parameters. In this paper, we are concerned only with the subclass of NLPV models satisfying the condition that the dependence of the matrix functions A, B, C, and D on the parameters δi is rational and given in terms of a feedback coupling. Such models are commonly referred to as LFT systems and are basically the straightforward generalization of the

102


✲

α(k) z(k) ✛

Δ(k)

G

β(k) ✛ ✛ w(k)

Fig. 1. The interconnection of G with Δ(k)

LPV systems first introduced in [15, 17]. We now introduce a model of the said subclass. Let G be a linear time-varying discrete-time system defined by the following state space equation:      x(k + 1) Ass (k) Asp (k) Bs (k) x(k)  α(k)  = Aps (k) App (k) Bp (k)  β(k)  , x(0) = 0, (1) z(k) Cs (k) Cp (k) D(k) w(k) for w ∈ 2 . The vector-valued signals x(k), α(k), β(k), z(k), and w(k) are real and have time-varying dimensions, with the constraint that dim(β(k)) = dim(α(k)). We denote the dimensions of these signals by n0 (k), n(k), n(k), nz (k), and nw (k) respectively. We assume that all the state space matrices are uniformly bounded functions of time. For any scalar sequences δ1 (k), . . . , δd (k) d and associated dimensions n1 (k), . . . , nd (k) satisfying i=1 ni (k) = n(k), we define the diagonal matrix Δ(k) as Δ(k) := diag(δ1 (k)In1 (k) , . . . , δd (k)Ind (k) ) ∈ Rn(k)×n(k) . Also, we also constrain σ ¯ (Δ(k)) ≤ 1 for all k ≥ 0. We will be concerned with the feedback arrangement in Figure 1, where G and Δ(k) are connected in feedback. This system can be expressed formally by x(k + 1) x(k) = H(k) , z(k) w(k)

(2)

where H(k) is given by H(k) =

A (k) Ass (k) Bs (k) A (k) . (3) + sp Δ(k) (I − App (k)Δ(k))−1 ps Bp (k) Cs (k) D(k) Cp (k)

We will refer to the mapping w → z in (2) as the system Gδ . Hence, Gδ is a linear time-varying system with arbitrary rational state-space parameter dependence formulated in an LFT framework, where the time-varying parameters δi act on the system G through the linear fractional feedback channels


103

(α, β). We assume App (k) such that Sk = I − App (k)Δ(k) is invertible for all k ≥ 0 so that the LFT in (3) is well-defined at each time k. This well-posedness condition guarantees that there are unique solutions in to (1). Using the previously defined notation, clearly the matrix sequences Ass (k), Bs (k), Cs (k), and D(k) from (1) define block-diagonal operators. The blocks of the matrix Δ(k) naturally partition α(k) and β(k) into d separate vectorvalued channels, conformably with which we partition the following state space matrices such that Asp (k) = A1sp (k) A2sp (k) · · · Adsp (k)  11  App (k) · · · A1d pp (k)  ..  .. App (k) =  ... . . 

dd Ad1 pp (k) · · · App (k)  1    1 Bp (k) Aps (k) Bp2 (k) A2ps (k)     Bp (k) =  .  Aps (k) =  .   ..   .. 

(4)

Bpd (k)

Adps (k)

Cp (k) = Cp1 (k) Cp2 (k) · · · Cpd (k) , ni (k)×nj (k) where Aisp (k) ∈ Rn0 (k+1)×ni (k) , Aij , Aips (k) ∈ Rni (k)×n0 (k) , pp (k) ∈ R Bpi (k) ∈ Rni (k)×nw (k) , and Cpi (k) ∈ Rnz (k)×ni (k) . The matrix sequence of each of the elements of the state space matrices in (4) defines a block-diagonal operator; and so we construct from the sequence of each of these state space matrices a partitioned operator, each of whose elements is block-diagonal and defined in the obvious way. For instance, the matrix sequences A1sp (k), . . . , Adsp (k) define block-diagonal operators that compose the partitioned operator Asp . With Z being the shift, we rewrite our system equations as      x x ZAss ZAsp ZBs α =  Aps App Bp   β  , (5) Cs Cp D z w

x x x = diag(I n20 , Δ1 , . . . , Δd ) = Δ , β α α where x ∈ (Rn0 ), w ∈ 2 (Rnw ), z ∈ (α1 , . . . , αd ), βi , αi ∈ (Rni ), and

(6)

(Rnz ), β = (β1 , . . . , βd ), α =

Δi = diag(δi (0)Ini (0) , δi (1)Ini (1) , δi (2)Ini (2) , . . .). Before concluding this section, we make some convenient definitions that will be used extensively in the sequel. To start, we define Z˜ := diag(Z, I 2 ) and Bs Ass Asp (7) , C := Cs Cp . , B := A := Bp Aps App

104


Also, we define

(n0 ,...,nd ) 2

Δ := {Δ ∈ L(

:=

2 (R

(n0 ,...,nd ) ) 2

n0

)⊕

2 (R

n1

) ⊕ ···⊕

2 (R

nd

) and

: Δ is partitioned as in (6) and Δ ≤ 1}.

Note that the operator Z˜ = diag(Z, I 2 ) has a conformable partitioning to that of Δ = diag(Δs , Δp ), where Δs = I n20 and Δp = diag(Δ1 , Δ2 , . . . , Δd ). 3.1 Stability of NLPV Models This section tackles the various concepts of stability that are essential to our work. To begin we define a basic notion of stability. Definition 21 An NLPV model is for all Δ ∈ Δ.

2 -stable

˜ has a bounded inverse if I −ΔZA

(n ,...,n )

d Thus, there exists a unique (x, β) ∈ 2 0 satisfying (5) and (6) if the model is 2 -stable. In such a scenario, equations (5) and (6) can be rewritten in the form z = Gδ w, where

˜ −1 ΔZB ˜ + D ∈ L( 2 (Rnw ), Gδ = C(I − ΔZA)

2 (R

nz

)).

(8)

(n ,...,n )

d At this point, we define T as the set of operators T ∈ L( 2 0 ) that have bounded inverses and are of the form T = diag(T0 , T1 , . . . , Td ), where each Ti ∈ L( 2 (Rni )) is block-diagonal so that Ti k = Ti (k) ∈ Rni (k)×ni (k) . Observe that T is a commutant of Δ. Moreover, we define the subset X of T by X = {X 0 : X ∈ T }.

Definition 22 An NLPV model is strongly 2 -stable if there exists P ∈ X satisfying AP A∗ − Z˜ ∗ P Z˜ 0. (9) The following lemma asserts that strongly a subset of the 2 -stable ones.

2 -stable

NLPV models constitute

Lemma 7 A strongly 2 -stable NLPV model is also converse is not true in general.

2 -stable;

however, the

The proof parallels the standard case and is hence omitted. ˜ −1 is bounded for Remark 2 We know that, under 2 -stability, (I − ΔZA) all Δ ∈ Δ. But, this boundedness is not necessarily uniform. On the other hand, strong 2 -stability guarantees that the aforementioned norm is uniformly bounded; this is clearly shown by the following norm condition which is easily derived: ˜ −1 ≤ (I − ΔZA)

1

1

P−2 · P 2 1 1 ˜ 2 1 − P − 2 ZAP

< ∞ for all Δ ∈ Δ,

where P is any solution in X to inequality (9).


105

One of the key features of strongly 2 -stable NLPV models is that they can always be represented by an equivalent balanced realization, as we will show next. But first, we need to define the balanced realizations of an NLPV model. Definition 23 An NLPV system realization is balanced if there exists a diagonal operator Σ ∈ X satisfying AΣA∗ − Z˜ ∗ Σ Z˜ + BB ∗

0,

(10)

˜ − Σ + C ∗C A∗ Z˜ ∗ Σ ZA

0.

(11)

Lemma 8 An NLPV model can be equivalently represented by a balanced realization if and only if it is strongly 2 -stable. Proof . Consider a strongly 2 -stable NLPV model (A, B, C, D; Δ). This is equivalent to the existence of P ∈ X satisfying (9), which in turn is equivalent to the existence of operators X, Y ∈ X solving the generalized Lyapunov inequalities AXA∗ − Z˜ ∗ X Z˜ + BB ∗

0,

˜ − Y + C ∗C A∗ Z˜ ∗ Y ZA

0.

Clearly, these conditions are themselves equivalent. Now we define the operator T ∈ T by 1 1 T := Σ 2 U ∗ X − 2 , where unitary operator U ∈ T and positive definite diagonal operator Σ are 1 1 obtained by performing a singular value decomposition on X 2 Y X 2 , namely 1 1 2 ∗ U Σ U = X 2 Y X 2 . Then, the following holds: T XT ∗ = (T ∗ )−1 Y T −1 = Σ. −1 ˜ ˜ The equivalent realization (Z˜ ∗ T Z)AT , (Z˜ ∗ T Z)B, CT −1 , D; Δ as a result is obviously balanced.

4 Model Reduction of Strongly

2 -Stable

NLPV Systems

This section focuses on the balanced truncation model reduction of strongly 2 -stable NLPV systems. It is divided into three subsections: the first presents a precise formulation of the balanced truncation problem; the second gives upper bounds on the error induced in such a reduction process; and the last deals with eventually periodic NLPV systems and delivers guaranteed finite error bounds for the balanced truncation of such systems.

106


4.1 Balanced Truncation Consider the balanced NLPV realization (A, B, C, D; Δ) with generalized diagonal gramian Σ ∈ X satisfying both of the generalized Lyapunov inequalities (10) and (11). Recall that Σ is of the form     Σi (0) Σ0     Σi (1) Σ1     , , where eachΣ = Σ=   i .. Σi (2)      . .. . Σ d

and Σi (k) is a diagonal positive definite matrix in Sni (k) . We assume without loss of generality that, in each block Σi (k), the diagonal entries are ordered with the largest first. Now given the integers ri (k) such that 0 ≤ ri (k) ≤ ni (k) for all k ≥ 0, we partition each of the matrices Σi (k) into two sub-blocks Γi (k) ∈ Sri (k) and Ωi (k) ∈ Sni (k)−ri (k) so that Σi =

Γi 0 , 0 Ωi

(12)

where Γi and Ωi are block-diagonal operators. Note that, since ri (k) is allowed to be equal to zero or ni (k) at any time k, it is possible to have one of the matrices Ωi (k) or Γi (k) with zero dimension; this corresponds to the case where either zero states or all states are truncated at a particular k. Allowing for matrices with no entries, although a slight abuse of notation, will be very helpful in the manipulations of the sequel. We define the operators Γ and Ω to have a similar structure to that of Σ, namely Γ = diag(Γ0 , Γ1 , . . . , Γd ) and Ω = diag(Ω0 , Ω1 , . . . , Ωd ). The singular values corresponding to the states and parameters that will be truncated are in Ω. At this point, we want to partition A, B and C conformably with the partitioning of Σ. Recall from Section 3 that A, B and C have the following forms:     Ass A1sp · · · Adsp Bs 1d  Bp1  A1ps A11 · · · A pp pp     1 d A= . .  , B =  ..  , C = Cs Cp · · · Cp , .. . .  .   .. . ..  . dd Adps Ad1 pp · · · App

Bpd

where each of the elements of these partitioned system operators is blockdiagonal. Note further that     Ass (k) A1sp (k) · · · Adsp (k) Bs (k) 1d A1ps (k) A11  Bp1 (k) pp (k) · · · App (k)    A k= . B , =   ..  , . . .. k .. ..   ..  .  . (13) dd Bpd (k) Adps (k) Ad1 pp (k) · · · App (k) C

k

= Cs (k) Cp1 (k) · · · Cpd (k) .


107

Let us now focus on the matrices Ass (k), Bs (k), Cs (k) and partition them in accordance with the partitionings of Σ0 (k) = diag(Γ0 (k), Ω0 (k)) and Σ0 (k + 1) = diag(Γ0 (k + 1), Ω0 (k + 1)) so that Ass (k) =

ˆs (k) Aˆss (k) Ass12 (k) B , Bs (k) = , Cs (k) = Cˆs (k) Cs2 (k) , Ass21 (k) Ass22 (k) Bs2 (k)

ˆs (k) ∈ Rr0 (k+1)×nw (k) , and Cˆs (k) ∈ Rnz (k)×r0 (k) . where Aˆss (k) ∈ Rr0 (k+1)×r0 (k) , B Hence, we have Ass =

ˆs Aˆss Ass12 B , Bs = , Cs = Cˆs Cs2 , Ass21 Ass22 Bs2

where each of the elements is block-diagonal. Similarly, the system matrices in (13) are partitioned compatibly of the associated Σi (k) so that  Aˆ1sp A1sp12 Aˆdsp Adsp12 Aˆss Ass12 ···  A 1 1 Adsp21 Adsp22 Asp21 Asp22  ss21 Ass22  ˆ1 1d 11 11 1 ˆ Aˆ1d App App12  Aps Aps12 pp App12 ···  1 11 11 1 1d 1d App21 App22 App21 App22 A A A=  ps21 . ps22 . .. .  .. .. ..  .  d1 ˆdd Add  Aˆdps Adps12 Aˆd1 A A pp pp12 pp pp12 ··· d1 dd Adps21 Adps22 Ad1 Add pp21 App22 pp21 App22 C=

Cˆs Cs2

Cˆp1 Cp12 · · ·

the other elements of with the partitioning 



         , B =           

Cˆpd Cpd2

ˆs B Bs2 ˆp1 B Bp12 .. . ˆpd B Bpd2

      ,    

.

Then a state space realization for the balanced truncation Gδ,r of the system Gδ is (Ar , Br , Cr , Dr ; Δr ) where

Ar Br Cr Dr

 ˆ Ass  Aˆ1  ps  . . =  .  ˆd  Aps Cˆs

Aˆ1sp Aˆ11 pp .. . Aˆd1 pp Cˆp1

Aˆdsp Aˆ1d pp .. . · · · Aˆdd pp ˆ · · · Cpd

··· ··· .. .

ˆs B ˆ Bp1 .. . ˆpd B

    ,   

D

and Δr = diag(I r20 , Δˆ1 , . . . , Δˆd ), with Δî = diag(δi (0)Iri (0) , δi (1)Iri (1) , . . .). Notice that Δr is constructed from the same parameters δi as those in Δ. Lemma 9 Suppose (A, B, C, D; Δ) is a balanced realization of Gδ . Then the corresponding balanced truncation Gδ,r is also strongly 2 -stable and balanced. Proof . To start, there exists a unique permutation Q such that Q∗ ΣQ = diag(Γ, Ω) formally; then we have

108


˜ Ar A¯12 ˜2 A, ¯ Q∗ ZB ˜ = Z = Z ¯ ¯ Z˜ Z˜ A21 A22 ¯ CQ = Cr C¯2 = C,

˜ Q∗ ZAQ =

Z˜

Br ˜ ¯ ¯2 = Z2 B, B

¯2 ), where (Ar , Br , Cr , Dr ; Δr ) is a realization of and Δ¯ = Q∗ ΔQ = diag(Δr , Δ the truncation Gδ,r , and the rest of the operators are defined in the obvious way. As the generalized gramian Σ satisfies both of inequalities (10) and (11), then focusing on (10), and with the aforesaid permutation in mind, the following ensues: Ar A¯12 A¯21 A¯22

Γ

Ω

This clearly gives

B A∗r A¯∗21 Z˜ ∗ Γ Z˜ + ¯r − ∗ ∗ B2 A¯12 A¯22 Z˜ ∗ Ω Z˜ Ar Γ A∗r − Z˜ ∗ Γ Z˜ + Br Br∗

¯∗ Br∗ B 2

0.

0.

Similarly, starting with (11), we can show that ˜ r − Γ + C ∗ Cr A∗r Z˜ ∗ Γ ZA r

0.

Thus directly from the definitions of strong stability and a balanced system we have the desired conclusion. 4.2 Error Bounds This subsection gives upper bounds on the error induced in the balanced truncation model reduction process. We start with the following result. Lemma 10 An NLPV model Gδ is strongly 2 -stable and satisfies the condition Gδ < 1, for all Δ ∈ Δ, if there exists a positive definite operator V in the commutant of Δ such that   −V 0 A∗ C∗  0 −I B∗ D∗    (14)  A B −Z˜ ∗ V −1 Z˜ 0  0. C D 0 −I This is a generalization of the sufficiency part of the Kalman-YakubovichPopov (KYP) Lemma. Its proof is routine and so we do not include it here. Note that the above inequality is necessary and sufficient in the purely timevarying case as proved in [19]; however, in our case, it is in general only sufficient. We will find the following notation convenient: ˜ −1 ΔZB ˜ + D, Gδ = Δ G = C(I − ΔZA) where

G=

˜ ZB ˜ ZA C D

and Δ ∈ Δ.

(15)


109

Theorem 24 Suppose that (A, B, C, D; Δ) is a balanced realization for the NLPV system Gδ , and that the diagonal generalized gramian Σ ∈ X , satisfying both of inequalities (10) and (11), is partitioned as in (12). If Ωi = I 2 for all i = 0, 1, . . . , d, then, for all Δ ∈ Δ, the balanced truncation Gδ,r of Gδ satisfies the following inequality: Gδ − Gδ,r < 2. Proof . As Gδ and Gδ,r are both strongly 2 -stable, then so is 12 (Gδ − Gδ,r ). One realization of the system 12 (Gδ − Gδ,r ) is given in linear fractional form by  ˜ ˜ r ZAr 0 √12 ZB   1 Δr ¯,  0 Z˜2 A¯ √12 Z˜2 B (Gδ − Gδ,r ) = ¯   Δ 2 0 − √12 Cr √12 C¯ ¯ B, ¯ C, ¯ and Δ¯ are as defined in the proof of Lemma 9, and Z˜m = where A, m diag({Ji }i=1 ) where Ji = Z˜ for all i. In the sequel, we will construct a positive ¯ and satisfies inequality definite operator V that commutes with diag(Δr , Δ) 1 (14) for this 2 (Gδ − Gδ,r ) realization. Then, invoking Lemma 10 completes the proof. Given that the diagonal operator Σ ∈ X satisfies inequalities (10) and (11), then direct applications of the Schur complement formula, along with some permutations, guarantee the validity of the following condition: −R1 K∗ ¯ K −Za∗ R2 Z¯a where Z¯a = diag(Z˜2 , I, Z˜2 ),  −1 Γ 0 0  0 Ω −1 0  0 I q2i Ri =   0  0 0 0 0 0 0

0 0 0 Γ 0

 0 0  0 , 0 Ω

0,



0  0  K =  0  Ar A¯21

0 0 0 A¯12 A¯22

0 0 0 Br ¯2 B

Ar A¯21 Cr 0 0

 A¯12 A¯22   C¯2  , 0  0

and q1 = nw , q2 = nz . Define the invertible operators L and S by     −I 0 0 I 0 I I 0 0 0  I  0 0 I 0 I  0 0 I 0   √ 1  1    . L = √  0 I √0 0 I  , S = √  0 0 0 2I 0   2 0 0 2  −I I 0 0 0   2I 0 0  0 −I 0 0 I 0 0 I 0 −I Multiplying the preceding condition on the left by diag(S ∗ , L) and on the right by diag(S, L∗ ) gives the following equivalent inequality:

110


−S ∗ R1 S S ∗ K ∗ L∗ ¯ LKS −Zb∗ LR2 L∗ Z¯b

0,

(16)

˜ Performing the multiplications in this inequality where Z¯b = diag(Z˜3 , I, Z). leads to  1 −1  + Γ ) 12 (Γ −1 − Γ ) 0 0 0 2 (Γ  1 (Γ −1 − Γ ) 1 (Γ −1 + Γ )  0 0 0 2  2 , 1 1 −1 −1 S ∗ R1 S =  0 0 (Ω + Ω) 0 (Ω − Ω)   2 2   0 0 0 I 0 1 1 −1 −1 0 0 − Ω) 0 2 (Ω + Ω) 2 (Ω 1

2 (Γ  1 (Γ 2

LR2 L∗ =        LKS =    

−1

+ Γ)

− Γ −1 ) 0 0 0

Ar 0 0 − √12 Cr A¯21

1 −1 ) 2 (Γ − Γ 1 −1 + Γ) 2 (Γ

0 0 0

0

0

0 + Ω) 0 1 −1 ) 2 (Ω − Ω

0 0 I 0

1 −1 2 (Ω

0 Ar A¯21 √1 Cr 2

0 A¯12 A¯22 ¯2 √1 C

√1 Br 2 √1 Br 2 ¯ √1 B 2 2

0

A¯12 0 0 − √1 C¯2

0

0

¯ √1 B 2 2

A¯22

2

2

0



 0  −1  , − Ω )  0 1 −1 + Ω) 2 (Ω 1 2 (Ω

    =   

M N12 . N21 A¯22

Note that, in the preceding expressions, some of the operators might contain at certain time-instants matrices of zero dimensions. In such scenarios, the rows and columns of which the said matrices are elements would not be present, and the corresponding operator inequalities remain valid. Define the operator V as   1 −1 + Γ ) 21 (Γ −1 − Γ ) 0 2 (Γ . 0 V =  21 (Γ −1 − Γ ) 21 (Γ −1 + Γ ) 1 −1 0 0 (Ω + Ω) 2 Note that, since S ∗ R1 S 0, then V 0. Also, V clearly commutes with the ¯ Recall that, by assumption, Ω = I; hence, 1 (Ω −1 + operator diag(Δr , Δ). 2 Ω) = I and 21 (Ω −1 − Ω) = 0. With this in mind, it is not difficult to see that inequality (16) implies that   V ∗ − M n   I 2w   0. ∗ −1 ˜ ˜   Z V Z3 M − 3 nz I2 Then, invoking Lemma 10, we get

1 2

(Gδ − Gδ,r ) < 1.


111

Theorem 25 Given a balanced NLPV model Gδ , then, for all Δ ∈ Δ, its balanced truncation Gδ,r satisfies the error bound d

ωi,j ,

Gδ − Gδ,r < 2 i=0

j

where ωi,j are the distinct diagonal entries of the block-diagonal operator Ωi . The proof follows from scaling, Lemma 9 and repeated application of the previous theorem. Note that this error bound might involve an infinite summation which in general may not converge to a finite number. In the following, we improve on this result and derive tighter bounds. We will first consider balanced systems where the singular values corresponding to the states and parameters to be truncated are monotonic in time. Before doing this it will be convenient to establish the following terminology. Definition 26 Given a scalar sequence αk defined on a subset W of the nonnegative integers, we define the following hold rule which extends the domain of αk to all k ≥ 0: let kmin = min{k ≥ 0 : k ∈ W} and then set αk =

αkmin , αq , where q := max{q ≤ k : q ∈ W},

if 0 ≤ k ≤ kmin ; if kmin < k.

We now have the following result. Theorem 27 (monotonic case) Suppose that (A, B, C, D; Δ) is a balanced realization for the NLPV system Gδ , and that the diagonal generalized gramian Σ ∈ X , satisfying both of inequalities (10) and (11), is partitioned as in (12). Let si (k) = ni (k) − ri (k) and define the set Fi , for i = 0, 1, . . . , d, by Fi = {k ≥ 0 : si (k) > 0}. Also suppose that for each i = 0, 1, . . . , d the scalar sequence ωi,k satisfies Ωi (k) = ωi,k Isi (k) for all k ∈ Fi . If for each i = 0, 1, . . . , d the sequence ωi,k is monotonic on Fk , then for all Δ ∈ Δ the balanced truncation Gδ,r of Gδ satisfies the following inequality: d

Gδ − Gδ,r < 2

sup ωi,k .

i=0 k∈Fi

Proof . It is sufficient to prove the theorem for the case where only one parameter or state block is being truncated (i.e., si = 0 for all i except for one, say j ∈ {0, 1, 2, . . . , d}), since the general case then follows simply by the standard use of the telescoping series and triangle inequality. Also, we assume without loss of generality that ωj,k ≤ 1 for all k ∈ Fj ; this can always be achieved by scaling inequalities (10) and (11).

112


To begin, we extend the domain of definition of ωj,k to all k ≥ 0 using the hold rule defined in Definition 26; note that the extended sequence is still monotonic. We now split the remainder of our proof into two separate cases, one where this sequence is nondecreasing and the other where it is nonincreasing. Case ωj,k nondecreasing: In this case, we have ωj,k ≤ ωj,k+1 for all k ≥ 0. We define the state space transformation T ∈ T as T

k

− 21

= (ωj,k )

I.

(17)

Note that, since Σ 0, then T is indeed bounded. This gives the following balanced realization for Gδ : ¯ B, ¯ C, ¯ D; Δ := A,

Z˜ ∗ T Z˜ AT −1 , Z˜ ∗ T Z˜ B, CT −1 , D; Δ .

(18)

¯ δ to refer to the system Gδ when the For convenient reference, we will use G realization in use is (18). Our goal now is to show that this new realization is balanced. To this end, given the state transformation T , we use (10) and (11) to arrive at −1 A¯∗ Z˜ ∗ (T ∗ ) ΣT −1

¯ A¯∗ − Z˜ ∗ Σ ¯ Z˜ + B ¯B ¯∗ A¯Σ −1 Z˜ A¯ − (T ∗ ) ΣT −1 + C¯ ∗ C¯

0, 0,

(19)

¯ = T ΣT ∗. Because of the special structure of T and the fact that where Σ ωj,k ≤ ωj,k+1 ≤ 1, it is not difficult to see that C¯ ∗ C¯ = (T ∗ )−1 C ∗ CT −1

T ∗ C ∗ CT,

¯ 2. (T ∗ )2 A¯∗ Z˜ ∗ (T ∗ )−1 ΣT −1 Z˜ AT

¯ Z˜ A¯ = A¯∗ Z˜ ∗ (T ΣT ∗) Z˜ A¯ A¯∗ Z˜ ∗ Σ

Then, pre- and post-multiplying inequality (19) by (T ∗ )2 and T 2 respectively and then using the above inequalities give ¯ Z˜ A¯ − Σ ¯ + C¯ ∗ C¯ A¯∗ Z˜ ∗ Σ

0.

¯ is a diagonal gramian satisfying the generalized Lyapunov inequalHence, Σ ¯ δ . Notice that, by the definition of T , we ities for the system realization G have ¯j (k) = Tj (k)Σj (k)Tj∗ (k) = (ωj,k )−1 Γj (k) Σ =

(ωj,k )

−1

Γj (k)

I

=

Γ¯j (k)

Ωj (k)

¯j (k) . Ω

¯j = I 2 , and so, by invoking Theorem 24, we deduce that the balanced Thus, Ω ¯ δ,r of the system G ¯ δ satisfies the norm condition truncation G


113

¯δ − G ¯ δ,r < 2 G for all Δ ∈ Δ. Now, it is not difficult to see that, because of the special ¯δ − G ¯ δ,r are in structure of T , the error system realizations Gδ − Gδ,r and G fact equivalent, and as a result, we have ¯δ − G ¯ δ,r < 2. Gδ − Gδ,r = G Case ωj,k nonincreasing: A similar argument applies where here the state transformation T ∈ T is 1 defined as T k = (ωj,k ) 2 I. We now consider the more general case, where singular values need not be monotonic in time. But first, we require the following definition from [18]. Definition 28 Given a vector v = (v1 , v2 , . . . , vs ) for some integer s ≥ 1, suppose that v1 cannot be considered as a local maximum and vs cannot be considered as a local minimum. Then vector v has m local maxima vmax,i and m local minima vmin,i for some integer m ≥ 0, and the max-min ratio of v, denoted Sv , is defined as m

Sv = v1

vmax,i , v i=1 min,i

Sv = v1 ,

m>0

m = 0.

Theorem 29 (nonmonotonic case) Given a balanced realization (A, B, C, D; Δ) for the NLPV system Gδ , suppose that a diagonal operator Σ ∈ X satisfies both of inequalities (10) and (11) and is partitioned as in (12), where, for all i = 0, 1, . . . , d and k ∈ Fi = {k ≥ 0 : si (k) > 0} we have Ωi (k) = ωi,k Isi (k) , with si (k) := ni (k) − ri (k). Define the vector ω ˆ i to consist of the elements ωi,k for k ∈ Fi . If for each i = 1, . . . , d we have dim(ˆ ωi ) < ∞, then for all Δ ∈ Δ the balanced truncation Gδ,r of Gδ satisfies the following inequality: d

Sωˆ i .

Gδ − Gδ,r < 2 i=0

The proof below uses the same idea as that of Theorem 27. Basically, we need to define some state space transformation that results in a balanced realization ¯ solving the generalized Lyafor the system Gδ where the diagonal gramian Σ ¯j = I 2 . Then, invoking punov inequalities for this realization is such that Ω Theorem 24 completes the proof. The choice of this state space transformation used is inspired by that of the monotonic case. Proof . As with the proof of Theorem 27 it is sufficient to prove the result for the case where the only Ωi that has non-zero dimension is Ωj for some fixed j in {0, 1, . . . , d}; without loss of generality we assume that ωj,k ≤ 1 for all k.

114


To keep the notation simple, we suppress the subscript j in ωj and ω ˆj . The vector ω ˆ is of the form ˆ min,1 , . . . , ω ˆ max,1 , . . . . . . , ω ˆ max,m , . . . , ωks ), ω ˆ = (ωk1 , ωk2 , . . . , ω corresponding to values of the sequence ωk evaluated at the ordered time points (k1 , k2 , . . . , kmin,1 , . . . , kmax,1 , . . . , kmax,m , . . . , ks ). The denoted local minima and maxima of the vector ω ˆ are as defined in Definition 28. We now use the hold rule of Definition 26 to extend the sequence ωk to all k ≥ 0; the maxima and minima of ωk are illustrated below.

ω ˆ max,m

ω ˆ max,1

ω ˆ max,2

ωk1 ω ˆ min,2

ω ˆ min,1

ωks

We define the state space transformation T ∈ T as  1 for k = 0, 1, . . . , k1 − 1, ωk21 I, 1

ωk2 I,

T

k

=

− 21 ω ω I, min,1 k  1 −1 ωmin,1 ωmax,1 ωk2 I, 1



.. .

for k = k1 , k1 + 1, . . . , kmin,1 , for k = kmin,1 + 1, . . . , kmax,1 , for k = kmax,1 + 1, . . . , kmin,2 , .. .

ρωk2 I,

for k = kmax,m + 1, . . . , ks ,

ρω I,

for k = ks + 1, ks + 2, . . .,

1 2 ks

1

m

−1 . Also, define P, Q ∈ T such that P k = ωk2 I where ρ = i=1 ωmin,i ωmax,i −1 and Q = T P . It is not difficult to see that the constituent scalars of operator T define a nonincreasing sequence, and so do those of operator Q and those ¯ B, ¯ C, ¯ D; Δ = of operator QP 2 . Then, given the equivalent realization A, ∗ −1 ∗ −1 Z˜ T Z˜ AT , Z˜ T Z˜ B, CT , D; Δ of the system Gδ , which we de-

¯ δ , and because of the special structure of T and note for ease of reference by G the assumption that ωj,k ≤ 1, the following ensues: ¯ A¯∗ − Z˜ ∗ Σ ¯ Z˜ + P −2 Q−1 B ¯B ¯ ∗ (Q∗ )−1 (P ∗ )−2 A¯Σ ¯ ¯ Z˜ A¯ − Σ ¯ + Q∗ C¯ ∗ CQ A¯∗ Z˜ ∗ Σ

0, 0,


115

¯ = (P ∗ )−1 ΣP −1 . Notice that P −2 Q−1 where Σ ωk−1 I and Q ρI. Thus, 1 ¯ the diagonal operator Σ satisfies the generalized Lyapunov inequalities (10) ¯ ρC, ¯ D; Δ). As Ω ¯j = I 2 , then, invoking ¯ ω −1 B, and (11) for the realization (A, k1 Theorem 24, we get ¯δ − G ¯ δ,r < 2. ωk−1 ρ G 1 Finally, the special structure of operator T and the fact that Sω = ωk−1 ρ 1 lead to ¯δ − G ¯ δ,r < 2Sω . Gδ − Gδ,r = G

−1

We remark that Theorem 25 generalizes the LTV result in [14] to the NLPV framework. Also, Theorems 27 and 29 are mainly generalizations of their LTV counterparts in [18], with the important exception that the truncations in the theorems need not be restricted connected intervals. To illustrate how to apply these results, we consider the following hypothetical example. Suppose we are to truncate the states corresponding to the sequence Ω0 (k) = ω0,k Is0 (k) for k ∈ [1, 9], where 9

{ω0,k }k=1 = {1, 0.75, 2, 1.25, 3, 1.75, 4, 2.25, 5}. Then the corresponding error bound obtained from Theorem 25 is 2 × (1 + 0.75 + 2 + 1.25 + 3 + 1.75 + 4 + 2.25 + 5) = 42. This is exactly the same bound that the main result of [14] would give assuming a standard LTV system. If we are to apply Theorem 29 to truncate the states in one step, then we obtain the error bound 2×1×

2 3 4 5 × × × ≈ 65. 0.75 1.25 1.75 2.24

This bound is quite conservative and can be significantly improved if we truncate the states in three steps and accordingly divide the sequence ω0,k into the following: {1, 0.75, 2, 1.25}, {3, 1.75, 4, 2.25}, and {5}. Then, applying Theorem 29 recursively, we obtain the improved error bound 2× 1×

4 2 +3× +5 0.75 1.75

≈ 29.

This can also be obtained from the results of [18] if the system in question is a standard LTV system. But, in our case, we can actually further improve on the last bound by dividing the sequence ω0,k into the two monotonic sequences {1, 2, 3, 4, 5} and {0.75, 1.25, 1.75, 2.25} and then applying Theorem 27 twice to get the error bound 2 × (5 + 2.25) = 14.5.

116


4.3 Eventually Periodic LPV Systems This subsection focuses on the balanced truncation of eventually periodic LPV systems. These systems are aperiodic for an initial amount of time, and then become periodic afterwards. One scenario in which they originate is when parametrizing nonlinear systems about eventually periodic trajectories. Such trajectories can be arbitrary for a finite amount of time, but then settle down into a periodic orbit; a special case of this occurs when a system transitions between two operating points. In addition to that, eventually periodic systems naturally arise when considering problems involving plants with uncertain initial states. Note that both finite horizon and periodic systems are subclasses of eventually periodic systems. We refer the reader to [6–8] for some useful results on eventually periodic models. We now give a precise definition of an eventually periodic operator. Definition 30 A block-diagonal mapping P on odic if, for some integers h ≥ 0, q ≥ 1, we have

2

is (h, q)-eventually peri-

Z q ((Z ∗ )h P Z h ) = ((Z ∗ )h P Z h )Z q , that is P is q-periodic after an initial transient behavior up to time h. Moreover, a partitioned operator, whose elements are block-diagonal, is (h, q)eventually periodic if each of its block-diagonal elements is (h, q)-eventually periodic. Theorem 31 Suppose that state space operators A, B, and C are (h, q)eventually periodic. Then solutions X, Y ∈ X satisfying Lyapunov inequalities (10) and (11) exist if and only if (h, q)-eventually periodic solutions Xeper , Yeper ∈ X exist. The outline of the proof is as follows: first, employ a similar averaging technique to that used in [2] to show that the periodic part of any of the generalized Lyapunov inequalities admits a q-periodic solution if feasible, then, having established that, the above result follows from scaling. Thus, if the system is strongly 2 -stable and (h, q)-eventually periodic, then we can construct an (h, q)-eventually periodic balanced realization with an (h, q)-eventually periodic diagonal gramian Σ ∈ X satisfying Lyapunov inequalities (10) and (11). Theorem 32 Suppose that system Gδ is an (h, q)-eventually periodic system with a balanced realization (A, B, C, D; Δ). Then the following hold: (i) There exists an (h, q)-eventually periodic diagonal operator Σ ∈ X , partitioned as in (12), satisfying both of the generalized Lyapunov inequalities (10) and (11); (ii) The balanced truncation Gδ,r of Gδ is balanced and satisfies the finite error bound


117

d

ωi,j < ∞,

G − Gr < Ef h + 2 i=0

j

where the scalar parameters ωi,j are the distinct diagonal entries of the matrix diag(Ωi (h), . . . , Ωi (h + q − 1)), and Ef h is the finite upper bound on the error induced in the balanced truncation of the finite horizon part of Gδ and is derived by applying Theorem 29.

5 Conclusions In this paper we have introduced balanced truncation model reduction for NLPV systems, and derived explicit error bounds for this procedure. Even when restricted to purely time-varying systems the results obtained provide the least conservative bounds currently available in the literature. Although there has been considerable recent achievement in the literature on model reducing nonstationary systems, which are all directly motivated by the original LTI results in [3, 11], the authors conjecture that significantly better bounds may be obtainable.

References 1. C.L. Beck, J.C. Doyle and K. Glover, “Model reduction of multidimensional and uncertain systems,” IEEE Transactions on Automatic Control, 41:1466–1477, 1996. 2. G.E. Dullerud and S.G. Lall, “A new approach to analysis and synthesis of time-varying systems,” IEEE Transactions on Automatic Control, 44:1486– 1497, 1999. 3. D. Enns, “Model reduction with balanced realizations: An error bound and a frequency weighted generalization”, Proc. IEEE Conference on Decision and Control, 1984. 4. M. Farhood, C.L. Beck and G.E. Dullerud, “Model reduction of nonstationary LPV systems,” Proc. IEEE Conference on Decision and Control, 2002. 5. M. Farhood and G.E. Dullerud, “Control of nonstationary LPV systems,” submitted to Automatica. 6. M. Farhood and G.E. Dullerud, “LMI tools for eventually periodic systems,” Sys. and Cont. Letters, 47:417–432, 2002. 7. M. Farhood and G.E. Dullerud, “On the 2 -induced control for eventually periodic systems,” Proceedings of the American Control Conference, 2004. 8. M. Farhood and G.E. Dullerud, “Duality and eventually periodic systems,” to appear in International Journal of Robust and Nonlinear Control. 9. M. Farhood, C.L. Beck, and G.E. Dullerud, “On the model reduction of nonstationary LPV systems,” Proceedings of the American Control Conference, 2003. 10. P. Gahinet and P. Apkarian, “A linear matrix inequality approach to H∞ control,” International Journal of Robust and Nonlinear Control, 4:421–448, 1991.

118


11. K. Glover, “All optimal hankel-norm approximations of linear multivariable systems and their L∞ -error bounds,” International Journal of Control, 39:1115– 1193, 1984. 12. D. Hinrichsen and A.J. Pritchard, “An improved error estimate for reducedorder models of discrete-time systems,” IEEE Transactions on Automatic Control, 35:317–320, 1990. 13. S. Lall and C. Beck, “Error bounds for balanced model reduction of linear timevarying systems,” submitted to IEEE Transactions on Automatic Control. 14. S. Lall and C. Beck, “Model reduction of complex systems in the linear-fractional framework,” Proc. IEEE Conference on Decision and Control, 1999. 15. W.M. Lu, K. Zhou, and J.C. Doyle, “Stabilization of uncertain linear systems: an LFT approach,” IEEE Transactions on Automatic Control, 41:50–65, 1996. 16. B.C. Moore, “Principal component analysis in linear systems: controllability, observability, and model reduction, ” IEEE Transactions on Automatic Control, 26:17–34, 1981. 17. A. Packard, “Gain scheduling via linear fractional transformations,” Systems and Control Letters, 22:79–92, 1994. 18. H. Sandberg and A. Rantzer, “Balanced truncation of linear time-varying systems,” IEEE Transactions on Automatic Control, 49:217–229, 2004. 19. V.A. Yacubovich, “A frequency theorem for the case in which the state and control spaces are Hilbert spaces with an application to some problems of synthesis of optimal controls,” Sibirskii Mat. Zh., 15:639–668, 1975, (English translation in Siberian Mathematics Journal).

Getting Mobile Autonomous Robots to Rendezvous Zhiyun Lin, Bruce Francis, and Manfredi Maggiore Department of Electrical and Computer Engineering University of Toronto 10 King’s College Road Toronto, Ontario Canada M5S 3G4 [email protected], [email protected], [email protected]

1 Introduction Imagine n mobile robots, i.e., rovers, moving in the plane without human supervision. In addition to a motor drive, each robot has onboard a computer and a camera with which it can see the positions of some others relative to itself. The robots are assumed not to possess a common coordinate system— they don’t have GPS receivers, and there are no landmarks in view of all. Also, they are unable to communicate with each other. So it’s problematic if they can meet at a common location by distributed control strategies alone. This is called the rendezvous problem.1 Why take rendezvous to be the task? There are undoubtedly real situations where rendezvous is a goal: The robots should gather for servicing or recharging. In any event, rendezvous is the most basic formation task. It also arises in the notion of consensus [6]: A group of autonomous and distributed automata should come to agree on a piece of information. The robot rendezvous problem lies within a more general subject— distributed control of autonomous agents—where the goal is for a collective intelligence to emerge from relatively simple agents. One interesting aspect of that subject involves the structure of the couplings among the agents; that structure can be modeled by a graph. Returning to robots, let us create a directed graph G with n nodes labeled 1, . . . , n, one for each robot, and an arc from i to j if robot i can see robot j. 1

The Oxford English Dictionary definition of to rendezvous is “to assemble at a place previously appointed; also generally, to assemble, come together, meet.” Without a supervisor or global coordinates known to all, the robots are incapable of assembling at a place previously appointed. The best they can do is meet at some not pre-specified place.


120

Z. Lin, B. Francis, and M. Maggiore

This visibility graph will be state-dependent, and consequently time-varying, if robots may come into and go out of view of each other. Our primary interest is in the connectivity properties of G that permit the existence of distributed control so that the robots rendezvous. Since our interest is in G, we’ll simplify the kinematics of the robots to the extreme and model them as points moving around continuously in the complex plane. Suppose the robots are numbered 1, . . . , n and the position of robot i is denoted zi (t). Suppose the velocity of robot i is directly controllable, so the kinematic equations are z˙i = ui ,

i = 1, . . . , n.

Let Ni denote those robots within view of robot i—its neighbours—and let yi denote the vector composed of the relative positions zj − zi , as j ranges over Ni . Thus yi is the allowed input to the controller on robot i. Example 1. Consider four robots where each can always see the next modulo 4: N1 = {2}, N2 = {3}, N3 = {4}, N4 = {1}. Thus G is the strongly connected graph in Figure 1 (there’s no implication that the robots are standing to form a square) and the sensed variables are

1

2

4

3

Fig. 1. Visibility graph: cyclic structure.

y1 = z 2 − z 1 , y2 = z 3 − z 2 , y3 = z 4 − z 3 , y4 = z 1 − z 4 .

(1)

The block diagram has the structure shown in Figure 2. Notice that the robots are coupled only through the visibility network, defined by (1). Strong connectivity of G is more than sufficient for solvability of the rendezvous problem. A natural distributed control strategy to get the robots to rendezvous is ui = yi . Then the overall state vector z = (z1 , z2 , z3 , z4 ) satisfies z˙ = M z, where M is the cyclic matrix   −1 1 0 0  0 −1 1 0   M =  0 0 −1 1  . 1 0 0 −1 The following facts can easily be established:

Getting Mobile Autonomous Robots to Rendezvous

121

u1 z1 y1 visibility network Fig. 2. Block diagram.

1. The eigenvalues of M lie on a circle centred on the negative real axis; the circle is tangent to the imaginary axis at the origin; there is one eigenvalue at the origin; the vector (1, 1, 1, 1) is an eigenvector corresponding to the zero eigenvalue. From these it follows that the centroid of the robots is stationary, and all robots converge asymptotically to the centroid. 2. As the formation of robots evolves, if you zoom in so that the diameter of the formation is constant, you will see that the formation converges to an ellipse [17]. One could argue that this cyclic pursuit is not truly a distributed control strategy, in that it relies on a numbering of the robots. On the other hand, it is easy to engineer the robots to be distinguishable if there aren’t too many, by colour, for example. Finally, the control law ui = yi assumes instantaneous, noiseless sensor measurements. A more realistic model would allow time delay for image capture, errors in resolution, sampled data, and non-synchroneity in processors from robot to robot. Formation evolution under cyclic pursuit is studied in [13].

2 Fixed Visibility Graph In this section we continue with a fixed G; that is, a robot’s set of visible neighbours is assumed to be fixed over time. Obviously, a special situation since it requires the cameras to have unlimited visibility in terms of distance. We find a necessary and sufficient connectivity condition for solvability of the rendezvous problem. Let mi denote the number of elements in Ni , so dim yi = mi ; robot i can see mi other robots. The permissible decentralized controllers are ui = Fi yi , where Fi is a real matrix of size 1 × mi ; that is, ui is permitted to be a linear combination of the relative positions zj − zi , j a neighbour of i. In the special case that robot i sees no other, then ui = 0.

122


The rendezvous problem is to find, if they exist, permissible controllers so that, for every set of initial positions, zi (0), there exists a (rendezvous) point zr such that every zi (t) converges to zr . As we saw with cyclic pursuit, we have to allow zr to depend on the initial positions for the goal to be feasible. For example, if all the robots are initially placed at a point zi (0) = w, they’ll stay there forever. So it would have to be that zr = w. Example 2. Let there be four robots with G as Figure 3. The graph isn’t

1

2

3

4

Fig. 3. Visibility graph of four robots.

strongly connected, but, for example, node 2 is reachable from all others. We say node 2 is globally reachable. (So are nodes 3 and 4.) We have y1 = z 2 − z 1 , y2 = z 3 − z 2 , y3 =

z2 − z3 , y4 = z 3 − z 4 . z4 − z3

As before, let us take F1 = 1, F2 = 1, F3 = 1 1 , F4 = 1. Then the overall state equation is z˙ = M z, where   −1 1 0 0  0 −1 1 0   M =  0 1 −2 1  . 0 0 1 −1 The eigenvalues of M are {−3, −1, −1, 0} and (1, 1, 1, 1) is again an eigenvector for the eigenvalue 0. It follows that the robots rendezvous. Bringing in algebraic graph theory, introduce the adjacency matrix   0100 0 0 1 0  A= 0 1 0 1, 0010 the out-degree matrix (diagonal matrix of row-sums of A)   1000 0 1 0 0  D= 0 0 2 0, 0001


123

and the Laplacian L = D − A. Then M = −L. The key property of L (and hence M ) is that 0 is a simple eigenvalue. And this is because G has a globally reachable node.

Here’s the main result for a fixed G: Theorem 1. [14] The rendezvous problem is solvable iff G has a globally reachable node. When the problem is solvable, one solution is ui =

(zj − zi ),

(2)

j∈Ni

or ui = 0 if robot i has no neighbours. A related result in discrete time (the controllers (2) solve the rendezvous problem iff G has a globally reachable node) is in [21]. The proof of Theorem 1 uses classical algebraic graph theory [2]: For any digraph G, the eigenvalues of L are in Re λ ≥ 0; 0 is an eigenvalue of L, but there are no others on the imaginary axis; if G is strongly connected, then 0 is a simple eigenvalue. It follows that every trajectory of z˙ = −Lz converges to a point on ker L, the subspace of stationary points. If 0 is a simple eigenvalue, that is, dim ker L = 1, then, since (1, . . . , 1) is a spanning vector for ker L, the robots rendezvous. A trivial example where 0 isn’t simple is two blind robots: L is the 2 × 2 zero matrix. Thus the key to the proof of Theorem 1 is the following result derived independently in [21], [14], and [20]. Lemma 1. G has a globally reachable node iff 0 is a simple eigenvalue of L. If G is symmetric, in the sense that if i sees j then j sees i (the adjacency matrix is symmetric), then we may as well consider G to be undirected. Then existence of a globally reachable node is equivalent to connectivity.

3 Limited Visibility Cameras In a seminal paper [8], Jadbabaie et al. study a system of “boids” moving at constant speed in the plane. Each boid has a heading angle, θi , evolving in discrete time. Also, at time t boid i can see a set Ni (t) of neighbours, and this leads to a time-varying graph G(t). Jadbabaie et al. assume if boid i can see boid j, then j can see i. Thus, G(t) is undirected. The paper studies a single control strategy, namely, at time t + 1 boid i changes its heading to the average heading at time t of itself and its neighbours. The paper proves that all the heading angles converge to a common value (similar to rendezvous) provided G(t) has a connectedness property over time, namely, there exists T > 0 such that the union graph t0 ≤t≤t0 +T G(t) is connected for all t0 .

124


Here, the union of graphs with the same node set is obtained by taking the union of the edges. Unfortunately, the condition on G(t) is not checkable—it would require an infinite time simulation. The proof in [8] uses a theorem of Wolfowitz on ergodicity [27]. In the theorems in [8] the vector θ(0) of initial heading angles is fixed. This allows the visibility graph to be a function of t alone, and not both t and θ(0). In actuality, for a sensible model of limited visibility, the graph is state dependent, not time dependent. To see this, let us return to our robots. Suppose robot i has an omnidirectional camera of range Ri . The set of visible neighbours of robot i is Ni (z) = {j : j ∈ {1, . . . , n}, |zj − zi | ≤ Ri }. Then there’s an edge in the visibility graph from i to j iff j ∈ Ni (z), and therefore the visibility graph is a function of z , G(z). If the control strategy is given and if the state z(t) evolves uniquely from z(0), the visibility graph is a function of time, G(z(t)). Interesting insight into the rendezvous problem is presented by Moreau in [20]. The framework is nonlinear, discrete-time, and the rendezvous analysis is based on convexity instead of Wolfowitz’s theorem. The idea is, roughly speaking, that if every robot moves toward the interior of the convex hull of the set of neighbour robots at each step, rendezvous will result. Now we turn to the circumcentre control law, first proposed for robot formations by Ando et al. [1]. Example 3. Consider six robots, with omnidirectional cameras of identical ranges, positioned at t = 0 as in Figure 4. The discs show the fields of view for robots 1 and 2. Since each camera has the same range, the visibility graph is undirected—Figure 5. Thus the neighbour sets at t = 0 are

3 2

4

3 5

6

1 Fig. 4. Six robots; the fields of view of robots 1 and 2 are shown.

2 1

4

6 5

Fig. 5. Visibility graph.


3

3 2 1

4

125

2

6 5

1

Fig. 6. Direction of u1 .

4

5

6

Fig. 7. Direction of u2 .

N1 = {2}, N2 = {1, 4}, N3 = {4}, N4 = {2, 3, 5}, N5 = {4, 6}, N6 = {5}. The circumcentre control law is defined as follows: Robot 1 has one neighbour, robot 2. Let Z1 = {z1 , z2 } and let c1 denote the circumcentre of Z1 —the centre of the smallest circle containing Z1 . Then set u1 = c1 − z1 ; see Figure 6. (In the picture, the little arrow is u1 translated from the origin to z1 .) So robot 1 moves towards the centre at t = 0: z˙1 = c1 − z1 . Actually, in this case where 1 sees only 2, clearly c1 = (z1 + z2 )/2, so at t = 0 z˙1 =

1 (z2 − z1 ). 2

Similarly, let c2 denote the circumcentre of the set {z2 } ∪ {zj : j ∈ N2 } and define u2 = c2 − z2 ; see Figure 7. And so on. These control laws can actually be implemented using onboard cameras, that is, relative positions, by translation. For example, for robot 2, the relative positions {z1 − z2 , z4 − z2 } are sensed. Let Z2 denote the set of points {0, z1 − z2 , z4 − z2 } (the translate of {z2 , z1 , z4 } by −z2 ), and let c2 denote the circumcentre of Z2 . Then define u2 = c2 . Let’s look at u1 again. The set Z1 equals {z1 , z2 } and so the circumcentre c1 of Z1 is a function of z; write c1 (z ). It turns out that c1 (z) is continuous in z, but not Lipschitz continuous—see below. In this way, the robots’ motions are governed by the coupled equations z˙1 = u1 (z) = c1 (z) − z1 .. . z˙6 = u6 (z) = c6 (z) − z6 , or in aggregate form z˙ = u(z), where the vector field u(z) is only continuous, not Lipschitz. Thus uniqueness of a solution is not guaranteed. In what follows, a statement about a solution should be interpreted as applying to all solutions if indeed there is more than one.

126


The fact that the circumcentre control law isn’t a Lipschitz continuous function causes difficulty in its use, as we’ll see later. So it’s perhaps of interest to see a proof. Lemma 2. [3] The circumcentre control law isn’t Lipschitz continuous. Proof Construct three points {p1 , p2 , p3 } and their circumcentre c, and three perturbed points {p1 , p2 , p3 } and their circumcentre c , as shown in Figure 8. Define the vectors p = (p1 , p2 , p3 ), We’ll show that the ratio

p = (p1 , p2 , p3 ).

|c − c | p−p

is not bounded by a constant. This proves the mapping p → c isn’t Lipschitz. Let the radii of the circles be 1 and define x = |c − c |, y = |p2 − p2 |. Since p1 didn’t move and |p2 − p2 | = |p3 − p3 |, √ (3) p − p = 2y. Look at Figure 9. Define the angle θ. Then we have the lengths p1 q = cos θ, qc = sin θ. Thus qc = 1 − cos θ, so by Pythagoras on the small triangle qcc

p1

p1

θ

p2 p3 p2

c

q c

c

c

x 1

1

p3

Fig. 8. Three points, their displacements, and their circumcentres.

p2

1

y

p2

Fig. 9. Figure for analysis.


127

x2 = (1 − cos θ)2 + sin2 θ = 2(1 − cos θ), and therefore qc = x2 /2. By Pythagoras again on the triangle qcc , the length x2 of qc equals x 1 − . Finally, apply Pythagoras to triangle qp2 c : 4 (y + 1)2 =

1+

2

x2 2

Thus we have x= so from (3)

+ x2 1 −

x2 4

= 2x2 + 1.

1 2 y + y, 2 2 1+ . y

1 x |c − c | =√ = p−p 2 2y

However, the great thing about the circumcentre law is that it preserves connectivity of the visibility graph. Thus we will only have to assume that the visibility graph is connected at t = 0. In fact, under the circumcentre control law, no links are dropped (though the distances between some neighbours may increase), so if G(z(0)) is connected, then G(z(t)) is connected for all t > 0. Of course, new links may form: As the robots rendezvous, eventually the graph becomes complete. Lemma 3. Under the circumcentre control law, over time no links are dropped in the visibility graph. Outline of Proof [16] For this proof it’s more convenient to view a robot position zi as a vector in R2 instead of a complex number. Let t ≥ 0 be arbitrary. Let Vij (z(t)) denote the distance-squared between two neighbour robots i and j, and let V (z(t)) denote the maximum distancesquared between any two neighbours: V (z(t)) = max i

max

j∈Ni (z(t))

Vij (z(t)).

Let I(z(t)) denote the set of pairs of indices where the maximum is attained; that is, (i, j) ∈ I(z(t)) iff robots i and j are neighbours of maximum distance apart among all neighbours. Thus V (z(t)) =

max

(i,j)∈I(z(t))

Vij (z(t)).

We would like to show that d/dtV (z(t)) ≤ 0. Unfortunately, V (z(t)) is not differentiable. We need some non-smooth analysis—the upper Dini derivative:

128


D+ V (z(t)) = lim supτ →0+

V (z(t + τ )) − V (z(t)) . τ

Then, it is a fact that D+ V (z(t)) =

max

(i,j)∈I(z(t))

d Vij (z(t)). dt

(4)

In this way we get D+ V (z(t)) = = = = ≤

max

d zi (t) − zj (t) 2 dt 2 zi (t) − zj (t), z˙i (t) − z˙j (t)

max

2 zi (t) − zj (t), ui (t) − uj (t)

max

2{ zi (t) − zj (t), ui (t) + 2 zj (t) − zi (t), uj (t) }

max

2 zi (t) − zj (t), ui (t)

max

(i,j)∈I(z(t)) (i,j)∈I(z(t)) (i,j)∈I(z(t)) (i,j)∈I(z(t)) (i,j)∈I(z(t))

+

max

(i,j)∈I(z(t))

To conclude that we’ll show that

D+ V (z(t)) ≤ 0,

(5)

max

zi (t) − zj (t), ui (t) ≤ 0

(6)

max

zj (t) − zi (t), uj (t) ≤ 0.

(7)

(i,j)∈I(z(t))

and

2 zj (t) − zi (t), uj (t) .

(i,j)∈I(z(t))

To illustrate the argument, suppose (1, j) ∈ I(z(t)) for some j, that is, the maximum separation between robot neighbours occurs for robot 1 (and perhaps others). Suppose the neighbours of robot 1 are robots 2, 3, 4, 5; Figure 10 shows the circumcentre control vector u1 (translated to z1 ). The figure shows three neighbour robots—2, 3, and 4—on the smallest encompassing circle. Now in Figure 11 construct the line as shown through z1 perpendicular to u1 , and using this line as diameter, draw a second circle. In the shaded crescent there must be a neighbour of robot 1, for otherwise the encompassing circle in Figure 10 would be smaller (in fact it would be the unshaded circle). Consider the robot in the shaded crescent that is maximum distance from robot 1; in the figure it is robot 3. The angle between the vectors u1 and z1 − z3 is greater than π/2. Therefore z1 (t) − z3 (t), u1 (t) ≤ 0 and so

max

(1,j)∈I(z(t))

z1 (t) − zj (t), u1 (t) ≤ 0.

This proves (6), and (7) follows from this.


z4

z4

z1 u1 z5

129

z1 u1

z3

z2 Fig. 10. Circumcentre law.

z5

z3

z2 Fig. 11. Construction for proof.

Finally, from (4) and (5), if two neighbours i and j are maximum distance apart (among all neighbours), then d/dtVij (z(t)) ≤ 0 and so the distance between them is non-increasing. Here’s the main result that the circumcentre control law solves the rendezvous problem: Theorem 2. [15] Suppose z(0) is such that G(z(0)) is connected. Under the circumcentre control law, the robots rendezvous. The proof [12] uses LaSalle’s theorem. Here we want to discuss the ideas without the details. Ideas for a Proof We’re given that G(z(0)) is connected. By Lemma 3, G(z(t)) is connected for all t > 0. Now G(z(t)) is either fixed or it’s not. Suppose not. Then at some time a new link appears (no link is dropped). After this, G(z(t)) is either fixed or it’s not. Suppose not. Then at some time, another new link appears. Since there are only finitely many nodes, this process must stop. Thus we may assume without loss of generality that G(z(t)) is fixed and connected for all t ≥ 0. (We don’t assume the graph is complete, but it must actually be so, since the robots rendezvous.) Bring in Example 3 for illustrative purposes. Figure 12 shows the constellation at t = 0, its convex hull co{z(0)}, and the instantaneous velocities ui (z(0)) of the robots at the vertices. Even though the vector fields ui (z(0)) point into co{z(0)}, we can’t conclude that zi (t) ∈ co{z(0)} because we don’t have Lipschitz continuity. So it’s problematic to prove even that a solution z(t) is bounded. Let a be an arbitrary point in the plane and define the function V a (z) to be the distance squared from a to the farthest zi . (Again, we take the plane R2 .) Assume for simplicity that the farthest-away robot doesn’t

130


3 2 1

u3

z2

4

6 5

Fig. 12. The convex hull and the initial velocities of the vertex robots.

z3

z4

z1

z6 z5 z3 − a a

Fig. 13. The point a.

change, that it’s always robot 3; see Figure 13. Then V a (z) is differentiable and d d a V (z(t)) = z3 (t) − a 2 dt dt = 2 u3 (z(t)), z3 (t) − a . From the vector orientations in Figure 13, u3 (z(t)), z3 (t) − a ≤ 0. Thus V a (z(t)) is nonincreasing, and this kind of argument shows that z(t) is defined for all t > 0 and is bounded. Now invoke LaSalle’s theorem. The solution converges to the largest invariant manifold M in {z : V˙ a (z) = 0}. To see what this manifold is, let z(0) ∈ M. Continuing with the assumption that the farthest-away zi (0) from a is z3 (0), we have that u3 (z(0)) and z3 (0) − a are orthogonal. Looking at Figure 13 we conclude that u3 (z(0)) = 0; for if u3 (z(0)) = 0 then z4 (0), the only neighbour of z3 (0), is farther from a than is z3 (0). Since u3 (z(0)) = 0, then z3 and z4 must be collocated at t = 0. If z2 (0) and z5 (0), the neighbours of z4 (0), are not also collocated with z4 (0), then z4 (t) will move away from z3 (0), which is impossible since M is invariant. Using this kind of argument, one can prove that for z ∈ M, all zi are equal. A rigorous proof is considerably more complicated since V a (z) is not actually differentiable. In [26] Tanner et al. study a group of autonomous mobile continuous-time boids. They propose distributed control laws, involving potential functions, that achieve convergence to a common heading while avoiding collisions. Persistent connectivity of the visibility graph is assumed rather than guaranteed.


131

4 Related Work Here we look at the rendezvous problem, and extensions, in a few other contexts. Discrete-Event Robots Versions of the robot rendezvous problem have been studied extensively in computer science (where it is usually called the gathering problem). Let us look at [7] as an interesting example. Each robot is viewed as a point in the plane. The robots have limited visibility: Each can see only the other robots within a fixed radius. Moreover, the robots are modeled as asynchronous discrete-event systems having four possible states: Wait, that is, not moving and idle; Look, during which the robot senses the relative positions of the other robots within its field of view; Compute, during which it computes its next move; and Move, during which it moves at some pre-determined speed to its computed destination. There are soft timing assumptions, such as, a robot can be in Wait for only a finite period of time. The robots have local coordinate frames and these are assumed to have a common orientation, e.g, they may each have a compass; see Figure 14. The paper proposes the following control law, in the form of four if-then rules: 1. If in the Look state a robot sees a another robot to its left or vertically above, then it does not move. 2. If a robot sees robots only below on its vertical axis, then it moves down toward the nearest robot. 3. If a robot sees robots only to its right, then it moves horizontally toward the vertical axis of the nearest robot.

Fig. 14. Common orientation.

132


4. If a robot sees robots both below on its vertical axis and on its right, then it computes a certain destination point and performs a diagonal move down and to the right. It is proved that, assuming the initial visibility graph is connected, the robots rendezvous after a finite number of events. For example, starting as in Figure 14, the lower-right robot will not move, and the other three will become collocated with it. The proof is quite complicated, because, although each robot goes through a sequence of event cycles Wait-Look-Compute-Move, the robots are entirely unsynchronized, and so a robot may start to move before another has finished moving. Discrete-Time Robots Let us look in more detail at the model of Jadbabaie et al. [8] mentioned already. For simplicity, suppose there are only two boids, each a neighbour of the other. They move at unit speed with heading angles θ1 , θ2 with respect to the global frame. The model in [8] is 1 [θ1 (k) + θ2 (k)] 2 1 θ2 (k + 1) = u2 (k) = [θ1 (k) + θ2 (k)]. 2

θ1 (k + 1) = u1 (k) =

The heading angles converge in just one time step. It may not be obvious that these steering laws are feasible by onboard sensors, but they are. Rewrite the equations as 1 [θ2 (k) − θ1 (k)] 2 1 θ2 (k + 1) − θ2 (k) = [θ1 (k) − θ2 (k)]. 2

θ1 (k + 1) − θ1 (k) =

Thus the heading angles can be updated from the measured relative heading angles. Cortés et al. [5] and Lin et al. [11] take the discrete-time robot model zi (k + 1) = zi (k) + ui (k),

(8)

that is, the position update zi (k + 1) − zi (k) is directly controllable. If only local onboard cameras are available, then again ui must be a function of the relative positions zj − zi , j ∈ Ni . The circumcentre law is applied in [5] to solve the rendezvous problem. Lin et al. propose properties for a distributed control law and prove rendezvous under these properties—with the additional assumption of graph connectivity, of course. The circumcentre law satisfies their properties, as does the following: Suppose robot 1 has two visible neighbours, 2 and 3. Let D1 , D2 , D3 denote the closed disks of radius R, the camera


133

range, centred at z1 , z2 , z3 , respectively. Then robot 1 moves toward the centroid of D1 ∩ D2 ∩ D3 . Theorem 2 is the continuous time equivalent of these results. From our point of view of onboard sensors and distributed control, there’s an interesting point about discrete-time models. Consider, again for simplicity, just two robots, each the neighbour of the other. Suppose the robots head for each other according to the equations z˙1 = u1 = z2 − z1 z˙2 = u2 = z1 − z2 . Now suppose the onboard controllers are digital: The sensed signals z2 − z1 , z1 − z2 are sampled via a periodic sampler S with sampling period T , then converted from discrete time to continuous time via a zero-order hold H. See Figure 15. Then the model at the sampling instants is z1 [(k + 1)T ] = z1 (kT ) + T [z2 (kT ) − z1 (kT )]

(9)

z2 [(k + 1)T ] = z2 (kT ) + T [z1 (kT ) − z2 (kT )].

(10)

Note that it requires the two digital controllers in Figure 15 to be synchronized somehow, by a centralized clock and communication system. So the system isn’t really distributed. Lack of synchroneity would lead to jitter, which could alternatively be modeled. (It’s interesting to note that system (9), (10) is unstable for large enough T .) Is the continuous-time setup in Section 3 asynchronous, that is, are the onboard clocks of the robots independent? No, because t = 0 is the common starting time for all robots. However, one can show that if the initial times for the robots are independent, the robots use the circumcentre law, and the visibility graph is connected before all the initial times, then rendezvous occurs. Wheeled Robots The simplest wheeled robot is a unicycle. With respect to a global coordinate system, it has coordinates (x, y ) and a heading angle θ measured positively

−

S z2

H u2 H

u1

z1 S

−

Fig. 15. Two robots with digital control.

134


from the horizontal axis. The model is x˙ = v cos(θ) y˙ = v sin(θ) θ˙ = ω, where v is the forward speed input and ω the turning rate input. The rendezvous problem is much more challenging for unicycles. Rendezvous means that unicycles converge to a common point in the plane—there is no requirement of convergence to a common heading. The first solution of the rendezvous problem for unicycles, proving global convergence, is [14]. There it is assumed that the visibility graph is fixed, and it is proved that, remarkably, the rendezvous problem is solvable iff the visibility graph has a globally reachable node—the same condition as for point robots! The controllers used for the proof are due to Yamaguchi and Burdick [28]. Suppose there are n unicycles and the forward speeds are all equal and constant. Then the control inputs are the turning rates only. Since the unicycles must keep moving, rendezvous is not possible, but certain moving formations are feasible. An equilibrium formation is one where all the unicycles appear stationary to each other. Justh and Krishnaprasad [10] proved that there are only two equilibrium formations: circular (all the unicycles move around a fixed circle) and rectilinear (all the unicycles move in straight lines in the same direction). Cyclic pursuit of unicycles is treated in [18], [19]. In particular, local controllers are designed to achieve stable circular formations. Coupled Oscillators There are fascinating examples in nature of synchronized oscillations [25]. One example is where thousands of fireflies flash in unison after a transient period. A seminal study to understand synchronization was done by Kuramoto in 1975; the work is reviewed in [24]. Kuramoto’s model can be depicted as in Figure 16. There are n radial lines rotating, say, counterclockwise. Line i has phase θi and is rotating at a nominal angular speed of ωi . But the lines are coupled according to the equations θ˙i = ωi + k

sin(θj − θi ), j=i

where k > 0 is the coupling coefficient. Synchronization means that all the rotating lines are merged into one. Notice in the Kuramoto model that the coupling is all-to-all—each line “sees” all the others. For identical local frequencies, i.e., ωi = ω, the transformation xi = θi − ωt yields the model x˙ i = k sin(xj − xi ). j=i


135

θ2 θ1

Fig. 16. Two Kuramoto oscillators.

Then synchronization means that x1 (t), . . . , xn (t) all converge to a common value. This problem is studied, for example, in [24], [23], [16], [9].

5 Conclusions Much recent research on the rendezvous problem has resulted in non-checkable conditions on a time-varying visibility graph. What is needed instead is a control strategy that guarantees sufficient connectivity that rendezvous will result. The circumcentre law is one such strategy for point robots. Distributed control of mobile robots falls under the general heading of decentralized control. That subject, though quite old, has only a few main results. Consider the stabilization problem, for example: Stabilize a largescale system using only local control. There is a solution [4], but it’s not distributed; it says to make the overall system controllable from one local input, and then stabilize from there. By contrast, the solutions reviewed in this paper are distributed. Endnote by B.F. It is a great pleasure to contribute to this Festschrift on the occasion of Keith’s 60th birthday. My friendship and professional relationship with Keith began in 1976 at the University of Cambridge. Keith was a new Lecturer and I a Postdoctoral Fellow. We collaborated on an optimal control problem (“Bounded peaking in the optimal linear regulator with cheap control,” IEEE Trans. Aut. Control. 1978). Then in the 1980s we were both active in H∞ control theory and collaborated on the paper “State-space solutions to standard H2 and H∞ control problems,” IEEE Trans. Auto. Control, 1989. I have the fondest memories of both collaborations.

136


References 1. H. Ando, Y. Oasa, I. Suzuki, and M. Yamashita, “Distributed memoryless point convergence algorithm for mobile robots with limited visibility,” IEEE Trans. on Robotics and Automation, vol. 15, pp. 818–828, 1999. 2. A. Berman and R. J. Plemmons, Nonnegative Matrices in the Mathematical Sciences. Philadelphia: SIAM, 1994. 3. S. Bespamyatnikh, B. Bhattacharya, D. Kirkpatrick, and M. Segal, “Mobile facility location,” Fourth Int. ACM Workshop on Discrete Algorithms and Methods for Mobile Computing and Communication, pp. 46-53, 2000. 4. J. P. Corfmat and A. S. Morse, “Decentralized control of linear multivariable systems,” Automatica, pp. 479-495, 1976. 5. J. Cortés, S. Martinez, and F. Bullo, “Robust rendezvous for mobile autonomous agents via proximity graphs in d dimensions,” to appear IEEE Trans. on Automatic Control. 6. M. H. DeGroot, “Reach a consensus,” J. of the American Statistical Association, vol. 69, pp. 118-121, 1974. 7. P. Flocchini, G. Prencipe, N. Santoro, and P. Widmeyer, “Gathering of asynchronous robots with limited visibility,” Proc. Int. Symp. Theoretical Aspects of Computer Science, Lecture Notes in Computer Science, vol. 2010, pp. 247-258, 2001. 8. A. Jadbabaie, J. Lin, and A. S. Morse, “Coordination of groups of mobile autonomous agents using nearest neighbor rules,” IEEE Trans. on Automatic Control, vol. 48, pp. 988-1001, 2003. 9. A. Jadbabaie, N. Motee, and M. Barahona, “On the stability of the Kuramoto model of coupled nonlinear oscillators,” Proc. American Control Conference, pp. 4296-4301, 2004. 10. E. Justh and P.. S. Krishnaprasad, “Steering laws and continuum models for planar formations,” Proc. IEEE Conf. on Decision and Control, pp. 3609-3615, 2003. 11. J. Lin, A. S. Morse, and B. D. O. Anderson, “The multi-agent rendezvous problem - part 1: the synchronous case,” submitted for publication, 2005. 12. Z. Lin, Coupled Dynamic Systems: From Structure Towards Stability, PhD thesis, in progress, University of Toronto, 2005. 13. Z. Lin, M. Broucke, and B. Francis, “Local control strategies for groups of mobile autonomous agents,” IEEE Trans. on Automatic Control, vol. 49, pp. 622-629, 2004. 14. Z. Lin, B. Francis, and M. Maggiore, “Necessary and sufficient graphical conditions for formation control of unicycles,” IEEE Trans. on Automatic Control, vol. 50, pp. 121-127, 2005. 15. Z. Lin, B. Francis, and M. Maggiore, “On the state agreement problem for multiple nonlinear dynamical systems,” Proc. IFAC World Congress, Prague, 2005. 16. Z. Lin, B. Francis, and M. Maggiore, “State agreement for coupled nonlinear systems with time-varying interaction,” submitted for publication, 2005. 17. J. Marshall, Coordinated Autonomy: Pursuit Formations of Multivehicle Systems, PhD thesis, University of Toronto, 2005. 18. J. A. Marshall, M. E. Broucke, and B. A. Francis, “Formations of vehicles in cyclic pursuit,” IEEE Trans. on Automatic Control, vol. 49, pp 1963-1974, 2004.


137

19. J. A. Marshall, M. E. Broucke, and B. A. Francis, “Pursuit formations of unicycles,” Automatica, to appear. 20. L. Moreau, “Stability of multiagent systems with time-dependent communication links,” IEEE Trans. on Automatic Control, vol. 50, pp. 169-182, 2005. 21. W. Ren, R. W. Beard, and T. W. McLain, “Coordination variables and consensus building in multiple vehicle systems,” in Cooperative Control: A Post-Workshop Volume 2003 Block Island Workshop on Cooperative Control, Springer-Verlag, pp. 171-188, 2004. 22. W. Ren, R. W. Beard, and E. Atkins, “A survey of consensus problems in multiagent coordination,” Proc. American Control Conf., pp. 1859-1865, 2005. 23. R. Sepulchre, D. Paley, and N. Leonard, “Collective motion and oscillator synchronization,” in Cooperative Control: A Post-Workshop Volume 2003 Block Island Workshop on Cooperative Control, Springer-Verlag, pp. 189-205, 2004. 24. S. H. Strogatz, “From Kuramoto to Crawford: exploring the onset of synchronization in populations of coupled oscillators,” Physica D, vol. 143, pp. 1-20, 2000. 25. S. H. Strogatz, Sync: How Order Emerges from Chaos in the Universe, Nature, and Daily Life, Hyperion, 2003. 26. H. Tanner, A. Jadbabaie, and G. J. Pappas, “Stable flocking of mobile agents, part II: dynamic topology,” Proc. IEEE Conf. on Decision and Control, pp. 2016-2021, 2003. 27. J. Wolfowitz, “Products of indecomposable, aperiodic, stochastic matrices,” Proc. Amer. Mathematical Soc., vol. 15, pp. 733-736, 1963. 28. H. Yamaguchi and J. Burdick, “Time-varying feedback control for nonholonomic mobile robots forming group formations,” Proc. IEEE Conf. on Decision and Control, pp. 4156-4163, 1998.

Experiment Design for Robust Control: Why Do More Work Than Is Needed? M. Gevers1 , X. Bombois2 , G. Scorletti3 , P. Van den Hof2 and R. Hildebrand4 1 2 3 4

CESAME, Université Catholique de Louvain, Belgium [email protected] Delft Center for Systems and Control, Delft University of Technology, The Netherlands GREYC Equipe Auto, Caen, France IMAG, Université J. Fourier, Grenoble, France

1 Introduction Experiment design for open-loop identification Optimal input design for system identification was an active area of research in the 1970’s, with different quality measures of the identified model being used for this optimal design [1–3]. The questions at that time addressed open-loop identification and the objective functions that were minimized were various measures of the parameter covariance matrix Pθ , where θ is the parameter vector of the model structure. Let the “true system” be given by: v(t)

G0 (z)

S : y(t) = G(z, θ0 ) u(t) + H(z, θ0 )e(t)

(1)

for some unknown parameter vector θ0 ∈ Rk , where e(t) is white noise of variance σe2 , while G(z, θ0 ) and H(z, θ0 ) are stable discrete-time transfer functions, with H(z, θ0 ) a monic and minimum-phase transfer function5 . In the optimal input design literature, it is assumed that this system is identified with a model structure M = {G(z, θ), H(z, θ)}, θ ∈ Rk , that is able to represent the true system; we shall call such structure a “full order model structure”. When Prediction Error identification is used with a full order model structure, the estimated parameter vector θˆN is known to converge, under mild assumptions, to a Gaussian distribution: N →∞

(θˆN − θ0 ) −→ N (0, Pθ ),

(2)

where the asymptotic parameter covariance matrix Pθ can be estimated from the data. Important examples of optimal design criteria developed in the 5

By monic is meant that H(z) = 1 +

P∞

k=1

hk z −k .


140

M. Gevers et al.

1970’s are D-optimal design which minimizes det(Pθ ), E-optimal design which minimizes λmax (Pθ ), and L-optimal design which minimizes tr(W Pθ ), where W is a nonnegative weighting matrix. In open-loop identification, the dependence of the covariance matrix on the input spectrum is made apparent by the following expression: Pθ−1 = + Here, Fu (z, θ0 ) =

N 1 σe2 2π 1 N 2π

π −π π −π

ΛG (z,θ0 ) H(z,θ0 ) ,

Fu (ejω , θ0 )Fu (ejω , θ0 )∗ Φu (ω)dω

Fe (ejω , θ0 )Fe (ejω , θ0 )∗ dω

Fe (z, θ0 ) =

ΛH (z,θ0 ) H(z,θ0 ) ,

ΛG (z, θ) =

(3) ∂G(z,θ) ∂θ

and

ΛH (z, θ) = ∂H(z,θ) . The formula shows that the data length N and the input ∂θ spectrum Φu (ω) appear linearly in the expression of the information matrix Pθ−1 , and that, for a given data length N , the input spectrum is the only

design quantity that can shape the parameter covariance matrix. Zarrop used Tchebycheff system theory to parametrize the input spectrum in terms of its so-called “trigonometric moments” with respect to the system [2]. These π Φu (ω) 1 trigonometric moments are defined as mk = 2πσ 2 −π |H(ejω ,θ0 )|2 cos(kω)dω. e

The information matrix Mθ Pθ−1 can then be expressed as a finite linear combination of these moments, m0 , m1 , . . . , mn , which express the effect of the filtered input spectrum Φu (ω) on that information matrix. The number n depends on the degree of H(z, θ). These moments have to obey some positivity constraint in order for them to be generated by a genuine spectral density function. The optimal input design problem can then be reformulated as a constrained optimization problem in terms of these trigonometric moments. Once the optimal moment sequence has been obtained, it is an easy step to compute a corresponding quasistationary input signal u(t) that will match this optimal moment sequence. A solution can always be obtained using multisines. We refer the reader to [2–5] for background material on Tchebycheff systems, trigonometric moments, and their use in input design problems. One should also note that another way of obtaining a finite parametrization of the information matrix is to restrict the admissible input signals to those that generate a finite dimensional parametrization of the spectrum Φu (ω), or to approximate the input spectrum by a finite dimensional parametrization: see e.g. [6]. The classical experiment design results of the 1970’s were limited to openloop identification with full order model structures, and they were based on parameter covariance formulas. In the mid-eighties, Ljung and collaborators produced bias and variance formulas [7, 8] directly for the transfer function estimates, rather than for the parameter estimates which only serve as auxiliary variables in the representation of these transfer functions. The asymptotic variance formulas were derived under the assumption that the model order n tends to infinity in some appropriate way when the data length N tends

Experiment Design for Robust Control

141

to infinity. Thus, for the variance of the vector of transfer function estimates G(z, θˆN ), H(z, θˆN ), the following approximation was obtained in [7] under an assumption of high model order: Cov

G(ejω , θˆN ) H(ejω , θˆN )

n Φu (ω) Φue (ω) ∼ = Φv (ω) Φeu (ω) σ 2 N

−1

(4)

where n is the model order, N is the number of data, Φu (ω) is the input spectrum, Φv (ω) is the output disturbance spectrum, and Φue (ω) is the crossspectrum between u and e: see (1). When identification is performed in open loop, we have Φue (ω) = 0. These variance formulas for the transfer function estimates paved the way for the formulation of goal-oriented experiment design problems. In [7] a number of open-loop optimal input design problems were formulated and solved. Experiment design for closed-loop identification The first closed-loop optimal design problems, including control-oriented problems, were formulated in [9]. In particular, it was shown in that paper that, when the model is identified for the purpose of designing a minimum variance controller, the optimal design is to perform the experiment in closed loop with the minimum variance controller in the loop. These results were later extended to other control design objectives in [10, 11]. The optimal design criterion used in all these contributions was the average performance degradation, namely the mean squared error between the output of the optimal loop (i.e. the loop that would be obtained if the optimal controller, dependent on the unknown true system, were applied to the system), and the output of the achieved loop (i.e. the loop in which the controller obtained from the estimated model is applied to the true system). The results were all based on the transfer function variance formulas of [7], derived under the assumption that the model order tends to infinity, and it was observed in recent years that the use of these formulas for finite order models can sometimes lead to erroneous conclusions. This observation triggered a revival of interest in optimal design formulations based on variance expressions for finite order models. Experiment design for robust control Robust stability and robust performance criteria are typically expressed as constraints on frequency weighted expressions of the variance of the transfer function error, rather than as L2 performance criteria. For example, a robust stability constraint is typically formulated as V ar G(ejω , θˆN ) ≤ W −1 (ejω ) ∀ω

(5)

where W (ejω ) is a frequency weighting function that takes account of closedloop properties (e.g. robust stability condition). In order to formulate optimal input design problems in terms of control-oriented quality measures on G(ejω , θˆN ) such as in (5), using the finite model order formula (3) rather than

142

M. Gevers et al.

the asymptotic (in model order) variance formulas, several approaches can be taken. One commonly used approach to go from parameter covariance to transfer function covariance is to use the following first order Taylor series approximation: σ 2 ∂G∗ (ejω , θ0 ) ∂G(ejω , θ0 ) V ar G(ejω , θˆN ) ≈ e Pθ (6) N ∂θ ∂θ This approach was adopted in [12] where it is shown that several useful H∞ design criteria can be reformulated as weighted trace optimal input design problems subject to LMI constraints. A sensible open-loop optimal input design problem can then be formulated as follows: min tr[W (θ0 )Pθ ] subject to

Φu (ω)

π −π

Φu (ω)dω ≤ α,

and Φu (ω) ≥ 0 ∀ω, (7)

where α is some positive constant. This is still a difficult, infinite dimensional optimization problem. However, by the use of Schur complement, the problem can be reformulated as a convex optimization problem under Linear Matrix Inequality (LMI) constraints. The numerical solution of such problems became possible in the nineties with the advent of interior point optimization methods [13, 14]. The problem becomes finite dimensional if the input spectrum Φu (ω) can be finitely parametrized. There are various ways of doing this, as noted earlier. Another approach to optimal input design for robust control is based on the use of the ellipsoidal uncertainty set Uθ centred on θˆN : Uθ = {θ|(θ − θˆN )T Pθ−1 (θ − θˆN ) < χ2 }.

(8)

It follows from the property (2) that the true parameter vector θ0 ∈ Rd belongs to Uθ with probability α(d, χ2 ) = P r(χ2 (d) ≤ χ2 ), where χ2 (d) denotes the χ2 distribution with d degrees of freedom. The results in [15, 16], which connect robust stability and robust performance measures directly to the ellipsoidal uncertainty region Uθ , allow one to formulate experiment design problems for robust control in terms of the minimization of some appropriate function of Uθ (or of Pθ ) without the need for the intermediate step of transfer function variance estimation, which typically requires both a Taylor series approximation and/or a conservative step of overbounding of the uncertainty set. The first open-loop optimal input design problem for robust control based on the direct use of the uncertainty ellipsoid Uθ was formulated in [5]. The robust stability measure minimized in that paper, with respect to the input spectrum Φu (ω), was the worst-case ν-gap δW C (G(z, θˆN ), D) between the identified model G(z, θˆN ) and all models in the Prediction Error uncertainty set D {G(z, θ)|θ ∈ Uθ }: δW C (G(z, θˆN ), D) = sup δν (G(z, θˆN ), G(z, θ)) θ∈Uθ

(9)


143

where the ν-gap is defined in [17]. One of the merits of the worst-case ν-gap is that it is directly related to the size of the set of its stabilizing controllers: the smaller the worst-case ν-gap of the uncertainty set D, the larger is the set of controllers that stabilize all models in D. The optimal input design problem solved in [5] was min δW C (G(z, θˆN ), D) subject to Φu

π −π

Φu (ω)dω ≤ α, and Φu (ω) ≥ 0 ∀ω.

(10) The solution proposed in [5] is based on Tchebycheff system theory (see above); the optimal solution can always be obtained as a multisine. Why do more work than is needed ? The traditional approach to optimal input design, as exemplified by the problem formulations (7) or (10), has been to optimize some measure of the resulting uncertainty, subject to a constraint on the input signal power. However, in an identification for robust control setting, one should not spend more effort on the identification than what is needed for the design of a robust controller, under the constraint that this controller must achieve stability and a prespecified level of performance with all models in the uncertainty set. This idea has led to the recent concept of “least costly identification for control”, which was first proposed in [18]. Instead of minimizing some measure of the uncertainty set, the objective is to deliver an uncertainty set that is just within the bounds required by the robust control specifications, and to do so at the smallest possible cost. In [19] open-loop identification is considered and the cost is then defined as the total input signal power. The idea of least costly (or minimum energy) identification experiment for control has been further developed in an open-loop framework in [20]. Here we present the formulation of this optimal experiment design objective in a closed-loop disturbance rejection setup, where no reference excitation is applied in normal operation6 . The identification cost is then defined as the additional penalty that occurs in the control performance cost when an excitation signal is added for the purposes of doing the identification. The full technical details can be found in [21]. We first recall some basic results and tools about Prediction Error identification for closed-loop systems.

2 Prediction Error Identification Aspects We consider the identification of the closed-loop system (1) using the full order model structure M = {G(z, θ), H(z, θ)}, θ ∈ Rk . In addition, we 6

This setup is very reminiscent of M. Gevers’ first practical experience with experiment design in a closed-loop disturbance rejection framework on the lakes of Canberra, Australia in 1985, when he and Keith Glover had jointly bought their first windsurfer and were identifying its dynamics.

144

M. Gevers et al.

assume throughout the paper that the model structure is globally identifiable at θ0 [22]. This means that θ0 is the only value of θ for which G(z, θ0 ) and H(z, θ0 ) represent the true system. Conditions for global identifiability of the commonly used model structures (ARX, ARMAX, BJ, etc) can be found in Theorem 4.2 of [22]. Note that they do not strictly require that all polynomial orders be known exactly. This true system is assumed to be operated in closed loop with a controller Cid : C : u(t) = r(t) − Cid (z)y(t) (11) to be replaced by a better controller: see Fig. 1. In normal operation, the external excitation signal r(t) is assumed to be zero. The objective is to perform a closed-loop identification experiment in order to estimate a model that must lead to a new robust controller with better performance. This is typically achieved by applying an external excitation signal r, even though we shall see that this is not always necessary. The closed-loop system can be written as: yr (t)

y(t) = Sid v(t) + G0 Sid r(t)

(12)

ur (t)

u(t) = −Cid Sid v(t) + Sid r(t) with Sid = 1/(1+CidG0 ). Under normal operation, only the first terms (driven by v) appear in the input and output signals. During an identification experiment with an external excitation signal r, the controlled output and input of the closed-loop system contain added contributions yr and ur due to the excitation signal r(t); these are perturbations with respect to the normal operating conditions.

r(t) +

u(t)

v(t) + G0

+

y(t)

− Cid Fig. 1. Block-diagram of the closed-loop system

ˆ ˆ Consider now that a model G(z) = G(z, θˆN ), H(z) = H(z, θˆN ) of the true system is identified using a direct Prediction Error identification method on the basis of N input-output data collected on the actual closed loop system with the controller Cid in the loop. If an external excitation signal r(t) is


145

applied during this identification experiment, we assume that it is a sample taken from a quasi-stationary signal (see [22]) for which a power spectrum Φr (ω) exists. The parameter vector estimate θˆN is defined by: 1 Δ θˆN = arg min θ N

N

2

(t, θ)

(13)

t=1

Δ

where (t, θ) = H(z, θ)−1 (y(t) − G(z, θ)u(t)). Note that (t, θ) depends on the chosen signal r(t) via (12). We introduce the following cost function: ¯ 2 (t, θ) V¯ (θ) = E

1 N →∞ N

N

lim

E 2 (t, θ).

(14)

t=1

As shown in [22], if the identification experiment is “informative enough”, then θˆN tends w.p.1 to a minimum of the cost function V¯ (θ); by our standing assumption that the system is in the model set and that the model structure is globally identifiable at θ0 , this cost function has θ0 as its unique minimum. The easiest and most common way to make a closed-loop experiment informative enough is to choose a reference signal that is persistently exciting of sufficient order (in closed-loop identification, the required order of the excitation is related to the degree of the sensitivity function Sid ); however, a closed-loop experiment can also be informative enough even without any external excitation, provided the controller Cid is sufficiently complex, i.e. provided its degree is sufficient: see [21, 23]. In this paper, we shall assume throughout that the experimental conditions are informative enough. Thus we make the following Standing Assumption. Assumption 2.1 With reference to the closed-loop identification experiment presented above (see (12)-(13)), assume that the system is in the model set, that the model structure is globally identifiable at θ0 , and that the experimental conditions are informative enough so that the true parameter vector θ0 is the only global minimum of the cost function V¯ (θ) of (14). We observe that this assumption is restrictive only in its requirements on the model structure. The requirements on the experimental conditions can always be satisfied. Assumption 2.1 ensures that the uncertainty region constructed around θˆN contains θ0 (and not another minimum of V¯ (θ)) and that this uncertainty region is not infinitely large. When Assumption 2.1 holds, we have the following result. Lemma 2.1 Consider the closed-loop identification experiment described above (see (12)-(13)) together with Assumption 2.1. Then the identified parameter vector θˆN is asymptotically normally distributed around the true parameter vector θ0 , i.e. θˆN −θ0 converges in distribution to N (0, Pθ ), and the covariance matrix Pθ has the following expression [22]:

146

M. Gevers et al.

Pθ = with ψ(t, θ) = − ∂

σe2 ¯ E ψ(t, θ0 )ψ(t, θ0 )T N

−1

(15)

(t,θ) ∂θ .

The covariance matrix Pθ depends on the experimental conditions (i.e. the data length N and the spectrum Φr (ω) used during the identification), as evidenced by the following expression of Pθ−1 , which is easily deduced from (12) and (15): Pr−1 (Φr (ω),θ0 ,σe2 )

Pθ−1 = N

1 1 σe2 2π

π −π

Fr (ejω , θ0 )Fr (ejω , θ0 )∗ Φr (ω)dω Pv−1 (θ0 )

+N

1 2π

π −π

Fe (ejω , θ0 )Fe (ejω , θ0 )∗ dω

G (z,θ0 ) , Fe (z, θ0 ) = Here, Fr (z, θ0 ) = Sid ΛH(z,θ 0)

∂G(z,θ) ∂θ

∂H(z,θ) . ∂θ

ΛH (z,θ0 ) H(z,θ0 )

(16)

− Cid Sid ΛG (z, θ0 ),

and ΛH (z, θ) = Note that Pθ−1 is made up of ΛG (z, θ) = a part depending on Φr (ω) and a part which does not depend on Φr (ω). Both parts are linear in N and both depend on the controller Cid . Note also that, in the vector Fr , the entries corresponding to the parameters that are only present in H(z, θ) are identically zero. The vector Fe has no identically zero entries; Fe is made up of the sum of two components: one component pertains to the parameters in H(z, θ) and one pertains to the parameters in G(z, θ). Lemma 2.2 ( [22]) Consider the closed-loop identification experiment described above (see (12)-(13)) and assume that Assumption 2.1 holds. Then Pθ−1 is strictly positive definite: Pθ−1 0. Using Lemmas 2.1 and 2.2, it is possible to define an uncertainty region D(θˆN , Pθ ) around the identified model which contains the unknown true system G(z, θ0 ) at any desired probability level β [15, 24]: ZN (z)θ |θ ∈ U = {θ|(θ − θˆN )T Pθ−1 (θ − θˆN ) < χ} 1 + ZD (z)θ (17) where χ is a real constant dependent on the chosen probability level β and ZN , ZD are row vectors containing powers of z −1 and zeros. The size of the uncertainty region D(θˆN , Pθ ) is a function of the covariance matrix Pθ and thus, by (16), a function of the design parameters N and Φr (ω) used during the identification experiment. D(θˆN , Pθ ) = G(z, θ) =


147

3 Control Design Objectives and Control Design Method As stated before, our aim is to replace the present controller Cid in the loop of ˆ Fig. 1 by a new controller C(z) = C(G(z, θˆN )) that has better performance. We adopt the following control performance measure for a stable closed-loop system [C G]: ¯ G, C, Wl , Wr ) J(G, C, Wl , Wr ) = sup J(ω, ω

(18)

where ¯ G, C, Wl , Wr ) = σmax (Wl (ejω )F (G(ejω ), C(ejω ))Wr (ejω )) J(ω, Δ

F (G, C) =

(19)

G GC 1+GC 1+GC C 1 1+GC 1+GC

Here σmax (A) denotes the largest singular value of A, and Wl (z), Wr (z) are given diagonal filters which reflect the performance specifications that we want to achieve with the true system. The performance measure (18) is quite general: J(G, C, Wl , Wr ) ≤ 1 ensures that the four entries of Wl (z)F (G, C)Wr (z) have an H∞ norm smaller than one. Simpler H∞ criteria can be chosen as special cases. A controller C will be deemed satisfactory for the system G0 if [C G0 ] is stable and if J(G0 , C, Wl , Wr ) ≤ 1. As mentioned in the introduction, we want to design the new controller ˆ = G(z, θˆN ) of G0 . For this Cˆ = C(G(z, θˆN )) using an identified model G purpose, we use a pre-selected nominal control design method. Assumption 3.1 We have pre-selected a fixed nominal control design method which maps the identified model G(z, θˆN ) to one controller C(G(z, θˆN )) which stabilizes G(z, θˆN ) and achieves with this model a nominal performance level J(G(z, θˆN ), C(G(z, θˆN )), Wl (z), Wr (z)) ≤ γ < 1,

(20)

where γ is a fixed scalar, strictly smaller than 1. One possible control design choice that satisfies Assumption 3.1 is to choose for C(G(z, θˆN )) the central controller of the four-block H∞ control design method with performance objective (20). If Assumption 3.1 holds, then the controller Cˆ = C(G(z, θˆN )) designed ˆ = G(z, θˆN ) will achieve J(G, ˆ C, ˆ Wl , Wr ) ≤ γ < 1. from an identified model G ˆ When this controller C is applied to the true system G0 , the achieved performance will generically be poorer than the designed performance. However, by choosing the design criterion (20) with γ < 1, we ensure that there is a whole set of systems G(z) around G(z, θˆN ) that are also stabilized by Cˆ and that ˆ Wl , Wr ) ≤ 1. In the sequel, we will denote by Dadm (θˆN ) the achieve J(G, C, largest set of systems G(z) having these properties.

148

M. Gevers et al.

4 Demands on the Identification Experiment Our objective is to determine the experimental conditions (N and Φr (ω)) of the identification experiment on the loop [Cid G0 ] in such a way that the ˆ identified through this experiment, delivers a controller Cˆ which model G, ˆ Wl , Wr ) ≤ 1. Since G0 is stabilizes the unknown G0 and achieves J(G0 , C, unknown but lies (with probability β) in the uncertainty region D(θˆN , Pθ ) ˆ this performance constraint will be replaced by the identified along with G, following checkable constraint7 . Constraint 4.1 The experimental conditions (N and Φr (ω)) of the identification experiment on the loop [Cid G0 ] (see Section 2) must be such that the ˆ = G(z, θˆN ) and the identified uncertainty region D(θˆN , Pθ ) identified model G ˆ Wl , Wr ) ≤ 1 for all G(z) ∈ D(θˆN , Pθ ), where have the property that J(G, C, ˆ ˆ using the control design method presented C is the controller designed from G in Assumption 3.1. ˆ Wl , Wr ) ≤ 1 for all G(z) in the set Dadm (θˆN ) defined in the Since J(G, C, last paragraph of Section 3, Constraint 4.1 imposes that D(θˆN , Pθ ) is a subset of this set Dadm (θˆN ). We now discuss the requirements on the identification design that will guarantee this. Remember that Dadm (θˆN ) is the largest set of systems G(z) around ˆ Wl , Wr ) ≤ 1. By Assumption 2.1 and Lemma 2.2, G(z, θˆN ) for which J(G, C, ˆ ˆ D(θN , Pθ ) ⊆ Dadm (θN ) is always achievable if N and/or Φr (ω) are chosen large enough: see (16). Moreover, we have the following trade-offs: the larger N is chosen, the smaller Φr (ω) can be while still verifying Constraint 4.1; conversely, the larger Φr (ω) is chosen, the smaller N can be while still verifying this constraint. The sets Dadm (θˆN ) and D(θˆN , Pθ ) are both a function of the parameter vector θˆN that we want to identify. Moreover, besides being a function of Φr (ω) and N , Pθ is also a function of the unknown quantities θ0 and σe2 . Consequently, whatever the method we use to determine experimental conditions Φr (ω) and N satisfying Constraint 4.1, it will need to be based on some 2 initial estimates θo,est , θˆN,est and σe,est of those unknown quantities. In this context, we will state that Constraint 4.1 is satisfied for given N and Φr (ω) when D(θˆN,est , Pθ,est ) ⊆ Dadm (θˆN,est ) where Pθ,est is computed using (16) 2 with θ0 and σe2 replaced by θo,est and σe,est . Note that, in the methods we present in the sequel, we will not determine the set Dadm explicitly, but implicitly (see Theorems 7.1 and 7.2). 7

ˆ Wl , Wr ) ≤ 1 with all G(z) ∈ D(θˆN , Pθ ), then, under mild assumptions, If J(G, C, ˆ C(z) = C(G(z, θˆN )) also stabilizes all G ∈ D(θˆN , Pθ ).


149

5 The Cost of a Closed-Loop Identification Experiment It is clear that many possible choices of experimental conditions allow one to fulfill Constraint 4.1. Among those, we seek to determine an identification experiment with the smallest possible cost. In this section, we give a precise definition of the cost of an identification experiment in the context where the closed-loop system operates with an acting controller Cid and with a disturbance rejection performance objective. As mentioned in Section 2, in normal operation the signals u(t) and y(t) are given by: y(t) = Sid v(t), u(t) = −Cid Sid v(t). (21) By applying an external signal r(t) to the loop during the identification, we introduce additional disturbances yr (t) and ur (t) on top of the normal operation signals: see (12). Those disturbances represent the cost of the identification experiment, since they entail a performance degradation. The ideal closedloop identification experiment would be one in which the normal operation signals u(t) and y(t) are used for a certain length N without any external excitation, i.e. with r(t) = 0. We show in Section 6 that such costless identification experiment can, in certain circumstances, lead to fulfillment of the Constraint 4.1. We also show how to compute the minimum number Nmin of measurements that are necessary to reach this objective. In the cases where Constraint 4.1 can not be achieved with r(t) = 0, the application of a nonzero external signal r(t) for a certain amount of time is unavoidable, but we show how N and Φr (ω) can be chosen in order to achieve Constraint 4.1 with minimal cost. This cost can be a function of either the experiment time N , the power of the perturbations yr and ur , or a combination of both. In the sequel, we consider three different situations which are representative of practical situations, and determine for each of them how we can optimally choose the experimental conditions. Situation 1. The cost of the identification is mainly determined by the duration N of the identification experiment. Based on the trade-off discussed in Section 4 between excitation power and duration of the experiment, the power spectrum Φr (ω) of the to-be-applied signal r(t) is in this case chosen at each frequency as large as the constraints on the actuators allow. For such fixed Φr (ω), the optimal experiment time can subsequently be determined via an optimization problem yielding the smallest identification time Nmin satisfying Constraint 4.1. Situation 2. Situation 2 is the converse situation: the cost of the identification is mainly determined by the power of the perturbations yr (t) and ur (t) due to the excitation signal r(t). Based on the same trade-off, the experiment time N is in this case chosen as large as is allowed. For such fixed N , the optimal power spectrum Φr (ω) can then be determined via an optimization problem

150

M. Gevers et al.

whose objective is to minimize the following cost function Jr representing the total disturbance power, subject to satisfaction of the Constraint 4.1: Jr = αy =

1 2π

1 2π π

−π

π −π

Φyr (ω) dω

+ αu

1 2π

π −π

Φur (ω) dω

αy |G0 (ejω )Sid (ejω )|2 + αu |Sid (ejω )|2 Φr (ω) dω

(22)

where αy and αu are scalars chosen by the designer to reflect the relative importance of the costs due to each of the perturbation signals, and where Φyr (ω) and Φur (ω) are the power spectra of these disturbance signals yr (t) and ur (t). Situation 3. Situation 3 is the situation where N and Φr (ω) are both important in the cost of the identification. In this situation, we can determine the optimal spectrum Φr (ω) such as in Situation 2 for different values of the length N . Since, for increasing values of N , the optimal cost function Jr decreases, such approach allows one to find the “optimal” combination for the duration of the identification experiment and the induced disturbance on the input and output signals.

6 Identification Experiments Without External Excitation In this section we examine the situation where Constraint 4.1 can be achieved using an identification experiment on the closed loop [Cid G0 ] without any external excitation signal r(t), i.e. using only the excitation due to the noise v(t) (see (12) with r = 0). We have the following result. Theorem 6.1 Consider a closed-loop identification experiment as presented in Section 2 with r(t) = 0 and assume that Assumption 2.1 holds. Then, Constraint 4.1 can always be verified by using for the identification a set of input-output data (21) of sufficient length N . Proof. When r = 0, expression (16) of Pθ−1 becomes: Pθ−1 = N Pv−1 (θ0 ).

(23)

By Assumption 2.1 and Lemma 2.2, we have Pv−1 (θ0 ) 0. Therefore, with N sufficiently large, Pθ−1 = N Pv−1 (θ0 ) can be made such that D(θˆN , Pθ ) ⊆ Dadm (θˆN ) for any set Dadm (θˆN ) around G(z, θˆN ), which implies that Constraint 4.1 holds. Theorem 6.1 shows that, if Assumption 2.1 holds with r = 0, the identification leading to a new and satisfactory controller Cˆ for G0 can be achieved without applying any external excitation: we just need to measure the input


151

and output signal in normal operation for a sufficient amount of time. For this theorem to be of practical use, we need to examine under what conditions Assumption 2.1 holds when r = 0, and which smallest data length Nmin is required for the robustness Constraint 4.1 to be satisfied when r = 0. The computation of Nmin for the case r = 0 is a special case of the computation of Nmin for a given reference excitation spectrum Φr (ω); it will be presented in the next section. As for the first question, a necessary and sufficient condition for Assumption 2.1 to hold is that (t, θ) = (t, θ0 ) =⇒ θ = θ0 [23]. In the case where r = 0, this condition specializes to the following result. Lemma 6.1 ( [23]) Consider the closed-loop identification configuration of Section 2 with r = 0. Then, Assumption 2.1 holds if and only if, for any θ, H −1 (z, θ)(1 + Cid G(z, θ)) = H −1 (z, θ0 )(1 + Cid G(z, θ0 )) =⇒ θ = θ0 .

(24)

A necessary condition for this identifiability condition to hold, in the case considered here of a linear time-invariant regulator Cid , is that the regulator be sufficiently complex (i.e. of sufficiently high order): see Section 13.4 of [22]. One can make this statement more precise by considering specific model structures. This has been done in [23] (see Complement C10.1) for the case of an ARMAX or ARX model structure, and in [21] for the case where an OE or BJ model structure is used. The conditions are essentially degree constraints on the structure of the controller Cid .

7 Least Costly Identification Experiments for Control An identification experiment without external excitation may be impossible for two reasons: i) the initial controller Cid , which is often not chosen by the user, is of lower complexity than required, or ii) the data length required to satisfy Constraint 4.1 (see Theorem 6.1) is unrealistic (e.g. Nmin corresponds to one year of data). We now address the problem of computing the least costly identification experiment for control, as has been defined by the problems presented at the end of Section 4. Those problems involve the computation under Constraint 4.1 of either the smallest data length for a given Φr (ω) (Situation 1) or of the power spectrum Φr (ω) minimizing Jr for a given N (Situations 2 and 3). 7.1 Shortest Identification Experiment for Control with Fixed Φr(ω) The first experiment design problem can be formulated as follows. Experiment Design Problem 1. Consider the closed-loop identification experiment of Section 2. Consider also that the power spectrum Φr (ω) of the

152

M. Gevers et al.

excitation signal r(t) is given. Determine then the smallest length N of an excitation signal r(t) with power spectrum Φr (ω) that must be applied to [Cid G0 ] in order to fulfill Constraint 4.1. We show that this problem can be expressed as an LMI-based optimization problem [14]. For this purpose, we first express the robust performance ¯ G, C, ˆ Wl , Wr ) ≤ 1 ∀G ∈ D(θˆN , Pθ ) at one particular frequency constraint J(ω, ˆ Wl , Wr ) ≤ 1 ω as an LMI, linear in Pθ−1 . Note that, according to (18), J(G, C, ˆ ¯ ˆ ∀G ∈ D(θN , Pθ ) ⇐⇒ J(ω, G, C, Wl , Wr ) ≤ 1 ∀ω and ∀G ∈ D(θˆN , Pθ ). Proposition 2 Consider the controller Cˆ = C(G(z, θˆN )) designed from the model G(z, θˆN ) using the control design method presented in Assumption 3.1. Consider also the set D(θˆN , Pθ ) defined in (17). Then Cˆ achieves ¯ G, C, ˆ Wl , Wr ) ≤ 1 with all G in D(θˆN , Pθ ) if and only if ∃ τ (ω) > 0 J(ω, (τ (ω) ∈ R) and a skew-symmetric matrix L(ω) ∈ R(k+1)×(k+1) (i.e. L(ω) = −L(ω)T ) such that √ with j = −1 and

τ (ω)E(ω, θˆN ) − R(θˆN ) + j L(ω) ≤ 0

R(θˆN ) =

„

Ik T −θˆN

«

Pθ−1

„

Ik T −θˆN

«T

„ +

0 0 0 −χ

(25) «

« I4 0 Ω(ejω ) 0 −1 „ ««« „ 1 10 0„ ˆ ZN 0 C W I ⊗ W 0 2 r l A@ ZD 1A Ω(z) = @ 1 ˆ N 1 ZD + CZ 0 1 E(ω, θˆN ) = Ω ∗ (ejω )

„

The symbol ⊗ denotes the Kronecker product. Proof. See [21]. The previous proposition shows that Constraint 4.1 can be replaced by LMI’s at each frequency, linear in Pθ−1 . Expression (16) shows that Pθ−1 is linear in the decision variable N of Experiment Design Problem 1. By combining these two facts, it is easy to see that Experiment Design Problem 1 would be solvable exactly if Pθ−1 was not a function of θ0 and σe2 , and if condition (25) was not a function of the to-be-identified θˆN . This difficulty is inherent to all experiment design problems [22] and is generally circumvented 2 by using a-priori estimates for those quantities: θo,est , σe,est and θˆN,est . The problem can then be solved using the LMI optimization problem of Theorem 7.1 below. Note that θˆN,est will often be chosen equal to θo,est . Theorem 7.1 Consider the approximations θ0 ≈ θo,est , θˆN ≈ θˆN,est and 2 2 σe2 ≈ σe,est and the shorthand notations: Pr−1 (Φr (ω)) = Pr−1 (Φr (ω), θo,est , σe,est ),


153

Pv−1 = Pv−1 (θo,est ) (see (16)) and E(ω) = E(ω, θˆN,est ) (see (25)). Then, the minimum duration N which solves Experiment Design Problem 1 is the solution (rounded up to the nearest integer) of the following LMI optimization problem: min N under the constraint that ∃ a frequency function τ (ω) valued in R and a frequency-dependent skew-symmetric matrix L(ω) valued in R(k+1)×(k+1) such that τ (ω)E(ω) − −

Ik T ˆ −θN,est 0 0 0 −χ

N (Pr−1 (Φr (ω)) + Pv−1 ) + j L(ω) ≤ 0

Ik T ˆ −θN,est

∀ω

T

(26)

Proof. Direct consequence of Proposition 2 and the expression of Pθ−1 in (16).

Comment 1. Condition (26) must be considered at every frequency. This is impossible in practice. The optimal N can nevertheless be approximated by using a finite frequency grid. An exact but more cumbersome solution consists of using the Kalman-Yakubovitch-Popov (KYP) lemma [25]; see [21] for details. Comment 2. As stated in Section 6, the minimal data length for an identification experiment without external excitation can be determined via Theorem 7.1 as well, by setting Φr (ω) = 0 in (26). 7.2 Least costly identification experiment with fixed data length We now examine the second situation presented at the end of Section 4, in which the data length N is fixed and one seeks to satisfy Constraint 4.1 with a power spetrum Φr (ω) that minimizes the identification cost Jr defined in (22). We restrict our search to signals whose power spectrum Φr (ω) can be written as [26]: m

Rr (i)cos(iω) ≥ 0

Φr (ω) = Rr (0) + 2

∀ω

(27)

i=1

where m is a positive integer selected by the user. The parameters Rr (i) (i = 0...m) can be interpreted as the auto-correlation sequence of a signal that has been generated by a white noise passing through an FIR filter of length m + 1. If we select Rr (i) = σ 2 NNc −i and m = Nc , then Φr (ω) represents the c power spectrum of a Random Binary Signal with clock period Nc and maximal

154

M. Gevers et al.

amplitude σ. Such a parametrization can be very useful if the amplitude of the time domain signal is constrained. An important property of the parametrization (27) is that Pθ−1 and Jr (see (16) and (22)) are affine functions of the design variables Rr (i) (i = 0...m), as we show in the following two propositions. Note that other parametrizations of Φr (ω) have the same property and could therefore also be considered here: m e.g. Φr (ω) = i=1 Rr (i)δ(ω − ωi ) corresponding to a multisine signal r(t) m [27], or Φr (ω) = i=0 Rr (i) Bi (ejω ) + Bi∗ (ejω ) where Bi (ejω ) are preselected basis functions [28]. Proposition 3 Consider the expression (16) of Pθ−1 and let Φr (ω) be parametrized ˜ k (θ0 ) be the sequence of Markov parameters of Fr F ∗ i.e. by (27). Let M r ∞ Fr (ejω , θ0 )Fr (ejω , θ0 )∗ = k=−∞ M˜k (θ0 )e−jkω with Fr (z, θ0 ) as defined in (16). Then, Pθ−1 ∈ Rk×k can be written as: Pθ−1

¯ (θ0 ) + =M

m

Mi (θ0 , σe2 ) Rr (i)

i=0

¯ (θ0 ) = N P −1 (θ0 ), M0 (θ0 , σ 2 ) = where M v e N T ˜ ˜ 2 (Mi (θ0 ) + M (θ0 )) for i = 1...m. σe

N σe2

˜ 0 (θ0 ), and Mi (θ0 , σ 2 ) = M e

i

Proof. Direct consequence of Result 5.6 of [26] applied to the closed-loop expression for Pθ−1 as given in (16). Proposition 4 Consider the cost function Jr defined in (22) and let Φr (ω) be parametrized by (27). Then Jr can also be written as: m

[αy ci (θ0 ) + αu di (θ0 )]Rr (i),

Jr = [αy c0 (θ0 ) + αu d0 (θ0 )]Rr (0) + 2 i=1

∗ where the coefficients ci (θ0 ) and di (θ0 ) are the Markov parameters of G0 G∗0 Sid Sid ∗ and Sid Sid respectively, i.e.

G0 (ejω )G0 (ejω )∗ Sid (ejω )Sid (ejω )∗ =

∞

ck (θ0 )e−jkω

k=−∞

and Sid (ejω )Sid (ejω )∗ =

∞ k=−∞

dk (θ0 )e−jkω .

Proof. Direct consequence of Result 5.4 of [26] applied to yr (t) = G0 Sid r(t) and ur (t) = Sid r(t). With the parametrization (27) for Φr (ω), the experiment design problem corresponding to Situation 2 can then be formulated as follows.


155

Experiment Design Problem 2: Consider the closed-loop identification experiment of Section 2 with a fixed number N of data. Determine the parameters Rr (i) (i = 0...m) of the spectrum Φr (ω) in (27) which minimize Jr , subject to satisfaction of the Constraint 4.1. The experiment design problem described above would be solvable exactly, using the results of Propositions 3, 4 and 2, if the parametrizations of Pθ−1 and Jr with respect to the design variables Rr (i) were not functions of the unknown θ0 and σe2 , and if condition (25) was not a function of the to-beidentified θˆN . This difficulty is again circumvented by using a-priori estimates 2 and θˆN,est . The solution is then obtained by for those quantities: θo,est , σe,est solving the LMI optimization problem described in Theorem 7.2 below. Theorem 7.2 Consider Experiment Design Problem 2. Consider also the ap2 proximations θ0 ≈ θo,est , θˆN ≈ θˆN,est and σe2 ≈ σe,est and the shorthand no2 ¯ ¯ tations: ci = ci (θo,est ), di = di (θo,est ), M = M (θo,est ), Mi = Mi (θo,est , σe,est ) ˆ and E(ω) = E(ω, θN,est ). Then the auto-correlation sequence Rr (i) (i = 0...m) which solves Experiment Design Problem 2 is the solution of the following LMI optimization problem: min

Rr (i)(i=0...m)

[αy c0 + αu d0 ]Rr (0) + 2

m X

[αy ci + αu di ]Rr (i)

i=1

under the constraint that there exists a symmetric matrix Q of appropriate dimension, a frequency function τ (ω) valued in R and a frequency-dependent skew-symmetric matrix L(ω) valued in R(k+1)×(k+1) such that τ (ω)E(ejω ) − −

m

Ik −θˆT

¯ + M

0 0 0 −χ

+ j L(ω) ≤ 0

N,est

Mi Rr (i) i=0

Ik −θˆT

N,est

∀ω

Q − AT QA C T − AT QB T C − B QA D + DT − B T QB

and that

T

(28) ≥0

(29)

with the following definitions of A, B, C, D: 0 0 B = 1 0 ... 0 Im−1 0 C = Rr (1) Rr (2) ... Rr (m) D = Rr2(0) A=

The optimal spectrum Φr (ω) can thereafter be computed using (27). Proof. The existence of a symmetric matrix Q such that (29) holds is a necm essary and sufficient condition for Rr (0) + 2 i=1 Rr (i)cos(iω) to be positive at each ω and thus for (27) to represent a spectrum, as shown in [26] via the Positive Real Lemma. Consequently, the result in this theorem is a direct consequence of Propositions 3, 4 and 2.

156

M. Gevers et al.

Comment. Condition (28) must be considered at every frequency, which is impossible in practice. The optimal Φr (ω) can nevertheless be approximated by using a finite frequency grid. An exact but more cumbersome solution consists of using the Kalman-Yakubovitch-Popov (KYP) lemma [25], as shown in [21]. 7.3 Dealing with the Unknown Quantities θ0 , θˆN and σe2 Theorems 7.1 and 7.2 provide solutions to the optimal identification experiment design problems defined above. However, these solutions require that an approximation of θ0 , θˆN and σe2 be used. If those approximations are not accurate, this could lead to poor results. Here we present a procedure for dealing with those unknown variables. Let us first recall the problem setup. One wants to replace the existing controller Cid , which operates on the true system, by a new controller via a new identification of the true system. In order to design the experimental conditions of this identification experiment, one needs reliable estimates of the unknown quantities θ0 , θˆN and σe2 . It is very often the case that estimates for θ0 and σe2 are already available, because the initial controller Cid has typically been computed from an initial identified model. This initial identification typically delivers estimates not only of θ0 and σe2 , but also of uncertainty regions for those quantities. Moreover, it is also possible to deduce from this initial identification a (truncated) Gaussian probability density function which defines the likelihood of each element of these uncertainty regions. If this is not possible, the density functions are then chosen uniform over the uncertainty regions. The estimate, the uncertainty region, and the probability density function of the to-be-identified θˆN are typically chosen equal to those of θ0 . To summarize, from the initial identification, one can assume that q0 = T T ˆT lies in a set Q and that the likelihood of the event q = q0 is given θ0 θN σe2 by a probability function p(q). Based on this information, one can robustify the procedure that consists in adopting a unique and possibly poor estimate of q0 for the design of the experimental conditions by adopting instead an approach based on randomized algorithms (see e.g. [29, 30]). We briefly describe such approach. In the case of Experiment Design Problem 1, one wants to determine the smallest duration N for which the Constraint 4.1 is verified, whatever the value of q0 . For this specific problem, since one assumes that q0 lies in Q, ˆ of sup this is equivalent to computing an estimate8 N q∈Q Nq where Nq is the solution obtained by Theorem 7.1 with the approximation q. Considering Nq as a function of q, this can be done [30] with accuracy and confidence9 δ by generating n ≥ ln(δ −1 )/ln((1 − )−1 ) estimates qj of q0 according to the 8 9

The exact computation is NP-hard. ˆ ) ≤ ) ≥ 1 − δ. This means that P r(P r(Nq > N


157

probability density function p(q), and by determining Nqj for each of these ˆ of supq∈Q Nq is then given estimates qj using Theorem 7.1. The estimate N by supqj (j=1...n) Nqj . In the case of Experiment Design Problem 2, the approach above can not be considered since one determines the parameters Rr (i) (i = 0...m) of Φr (ω) rather than Φr (ω) itself. The so-called scenario-approach can then be considered (see e.g. [29, Chapter 12]). This approach is also based on a randomized algorithm which uses the probability density function p(q). The main difference between the two approaches is that, for Experiment Design Problem 1, Theorem 7.1 is applied a fixed number of times for different estimates of q0 that are randomly generated with p(q); while for Experiment Design Problem 2, the optimization problem is solved only once but with several robust performance constraints (28), each of them evaluated at a different estimate of q0 randomly generated with p(q).10

8 Simulation Results In order to illustrate our results, we consider the following ARX system [31] as the true system: y(t) =

1 z −3 B0 (z) u(t) + e(t) A0 (z) A0 (z)

(30)

with B0 (z) = 0.10276 + 0.18123z −1, A0 (z) = 1 − 1.99185z −1 + 2.20265z −2 − 1.84083z −3 + 0.89413z −4, and e(t) a realization of a white noise signal of variance σe2 = 0.5. The control performance criterion J(G, C, Wl , Wr ) focuses on the sensitivity function. It is defined as in (18) with the filters: Wl (z) = diag (0, W (z)) Wr (z) = diag (0, 1) W (z) =

0.5165−0.4632z −1 1−0.999455z −1 .

The true system initially operates in closed loop with a controller Cid which has been designed using an initial estimate of the true system θo,est = (−1.9755, 2.1965, −1.8495, 0.8881, 0.0817, 0.172)T and the 4-block H∞ control design method of [32] that satisfies Assumption 3.1: Cid =

10

0.036249(z + 0.9244)(z 2 − 1.951z + 1.101) (z − 0.9995)(z 2 − 1.002z + 0.3641) (z 2 − 0.5109z + 0.8248)(z 2 − 0.1828z + 0.9416) × (z 2 − 1.279z + 0.835)(z 2 − 0.1746z + 0.9229)

(31)

For each of these robust performance constraints (28), the frequency functions τ (ω) and L(ω) are different.

158

M. Gevers et al. 1

10

0

10

−1

10

−2

10

−3

10

−2

10

−1

ω

10

0

10

Fig. 2. Identification without external excitation, with N = 4901: ˛ ˛ ˛ ˛ −1 −1 1 ˆ ˆ supG∈D(θˆN ,Pθ ) ˛ 1+C(e (solid), |(1 + C G) | (dashdot) and |W | (dotted) ˆ jω )G(ejω ) ˛

Identification without external excitation. The complexity of the initial controller Cid is sufficient for Assumption 2.1 to hold with r(t) = 0. Thus, Constraint 4.1 can be verified with an identification experiment that uses noise excitation only (see 21) provided the experiment is of sufficient length. The minimal length required can then be determined using Theorem 7.1 with Φr (ω) = 0 ∀ω. This theorem is applied here using the approximations θ0 ≈ θo,est and θˆN ≈ θo,est (an estimate of σe2 is not necessary since Pv−1 (θ0 ) is not a function of σe2 ). This delivers a minimal length Nmin = 4901. In order to verify the validity of this result, we have measured 4901 samples of the signals y(t) and u(t) obtained in normal operation on the loop [Cid G0 ] ˆ = G(z, θˆN ) along with its uncertainty region and we have identified a model G ˆ ˆ D(θN , Pθ ). From G, we have then designed a controller Cˆ using the method ˆ Wl , Wr ) ≤ 1 with of [32] and we have verified whether Cˆ achieves J(G, C, 1 jω −1 for all G all G in D(θˆN , Pθ ), or equivalently 1+C(e ˆ jω )G(ejω ) ≤ |W (e )| ˆ in D(θN , Pθ ). This is indeed the case as can be seen in Figure 2. Moreover, 1 jω −1 we also observe in Figure 2 that supG∈D(θˆN ,Pθ ) 1+C(e ˆ jω )G(ejω ) = |W (e )| in the low frequencies. Consequently, N = 4901 is indeed the smallest N for which Constraint 4.1 holds with Φr (ω) = 0 ∀ω.

Sensitivity to the initial estimates. In our example, the initial estimate θo,est chosen to approximate the unknown quantities θ0 and θˆN has delivered accurate results, as shown by Figure 2. This may not always be the case. Thus, it is safer to compute the minimal data length N using the method proposed in Section 7.3. We illustrate the application of this method to our example. In order to generate multiple estimates of θ0 (which are then used to approximate both the true θ0 and θˆN ), we have used the information provided by the initial


159

identification which had delivered θo,est . This was an open-loop identification with Φu (ω) = 1 ∀ω and N = 500.11 Using the covariance matrix of θo,est , we have randomly generated 46 parameter vectors θi (i = 1...46) around θo,est ; 46 samples correspond to a confidence of 80%. For each of these estimates, we have applied Theorem 7.1 and we have thus obtained 46 different lengths Nθi . A more robust choice of the length N is then (see Section 7.3): maxθi Nθi = 5897 The standard deviation of these 46 Nθi was 383. 25

20

Jr

15

10

5

0

0

500

1000

1500

2000

2500 N

3000

3500

4000

4500

5000

Fig. 3. Cost of the least costly experiment that satisfies Constraint 4.1 (circles) and of the white noise experiment that satisfies the same constraint (crosses), for different values of the data length.

Least costly identification with external excitation. For the same example, we have also studied the effect of applying least costly excitation signals r for a range of data lengths that were too short to lead to identifiability using only the noise excitation. Thus, we have computed the optimal signal spectra Φr (ω) resulting from Theorem 7.2 for data lenghts N ranging from 500 to 4,500 by steps of 500. For each of these data lenghts, we have then compared the identification cost Jr resulting from the application of the optimal excitation signal (when m = 10 in (27)) with the cost that would result by applying a white noise reference excitation (i.e. m = 0 in (27)) with a variance that is sufficient to satisfy the robust performance Constraint 4.1. The comparison between the cost of the least costly experiment and the cost of a corresponding identification experiment with white noise excitation is shown in Figure 3. As can be seen from this figure, the use of an optimally designed excitation signal r reduces the identification cost by a factor of 2 to 3 whatever the data length. Similar comparisons leading to similar conclusions can be found for the case of open-loop identification in [6]. 11

This initial identification was too cheap to verify Constraint 4.1.

160

M. Gevers et al.

9 Conclusions We have presented a new paradigm for optimal experiment design in an identification for robust control context, where the objective is to design an identification experiment at the smallest possible cost. The identification cost must be as small as possible while still delivering a model uncertainty set that just meets the robust control performance constraints. The cost of the identification experiment is expressed either as the experiment time, or in terms of a measure of the deterioration of the closed-loop performance, during the identification experiment, with respect to the closed-loop performance under normal (non perturbed) operation. This paradigm was initially proposed in [18], and subsequently elaborated upon and extended in a number of other papers [19, 21, 28]. The underlying theme of this new paradigm is “Why do more (identification) work than is needed?” One might wonder whether it pays to apply the heavy mathematical machinery required to compute optimal input designs, given that the optimal solution necessarily depends on the unknown system, which means that a preliminary model estimate must be obtained first before an approximately optimal input signal can be computed. This is sometimes referred to as adaptive (or iterative) optimal input design. In [33] the benefits of optimal input design for control have been demonstrated for two benchmark problems. In both cases, significant savings are obtained by the application of a two-step identification procedure, where the second step uses an optimally designed input signal computed from a preliminary model estimate. From a practical point of view, the cost of identification is an issue of major importance. It is often estimated that 75% of the cost associated to an advanced control project goes into model development. Even though the definition of the cost used in the present work on “least costly identification for control” does by no means cover all the practical costs of modelling, the disruption caused to normal operation and the time required to arrive at a satisfactory model are considered to be very significant elements of this total modelling cost. These two costs are incorporated in the “least costly” criterion developed in the present paper. Acknowledgements This paper presents research results of the Belgian Programme on Interuniversity Attraction Poles, initiated by the Belgian Federal Science Policy Office. The scientific responsibility rests with its authors.

References 1. R. Mehra, “Optimal input signals for parameter estimation in dynamic systems - survey and new results,” IEEE Trans. on Automatic Control, vol. AC-19, no. 6, pp. 753–768, December 1974.


161

2. M. Zarrop, Optimal experiment design for dynamic system identification. Berlin, New York: Lecture Notes in Control and Information Sciences, Vol. 21, Springer Verlag, 1979. 3. G. Goodwin and R. Payne, Dynamic System Identification: Experiment Design and Data Analysis. New York: Academic Press, 1977. 4. S. Karlin and W. Studden, Tchebycheff systems with applications in analysis and statistics. New York: Pure Appl. Math. 15, Wiley-Interscience, 1966. 5. R. Hildebrand and M. Gevers, “Identification for control: optimal input design with respect to a worst-case ν-gap cost function,” SIAM Journal on Control and Optimization, vol. 41, no. 5, pp. 1586–1608, March 2003. 6. H. Jansson, “Experiment design with applications in identification for control,” Ph.D. dissertation, Royal Institute of Technology (KTH), December 2004, tRITA-S3-REG-0404. 7. L. Ljung, “Asymptotic variance expressions for identified black-box transfer function models,” IEEE Trans. Automatic Control, vol. AC-30, pp. 834–844, 1985. 8. B. Wahlberg and L. Ljung, “Design variables for bias distribution in transfer function estimation,” IEEE Trans. Automatic Control, vol. AC-31, pp. 134–144, 1986. 9. M. Gevers and L. Ljung, “Optimal experiment designs with respect to the intended model application,” Automatica, vol. 22, pp. 543–554, 1986. 10. H. Hjalmarsson, M. Gevers, and F. D. Bruyne, “For model-based control design, closed-loop identification gives better performance,” Automatica, vol. 32, pp. 1659–1673, 1996. 11. U. Forssell and L. Ljung, “Some results on optimal experiment design,” Automatica, vol. 36, pp. 749–756, 2000. 12. H. Jansson and H. Hjalmarsson, “A framework for mixed H∞ and H2 input design,” in Proc. MTNS, Leuven, Belgium, 2004. 13. Y. Nesterov and A. Nemirovskii, Interior Point polynomial algorithms in convex programming. Philadelphia: SIAM Studies in Applied Mathematics, 1994, vol. 13. 14. S. Boyd, L. E. Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory. Philadelphia: SIAM Studies in Applied Mathematics, 1994. 15. X. Bombois, M. Gevers, G. Scorletti, and B. Anderson, “Robustness analysis tools for an uncertainty set obtained by prediction error identification,” Automatica, vol. 37, no. 10, pp. 1629–1636, October 2001. 16. M. Gevers, X. Bombois, B. Codrons, G. Scorletti, and B. Anderson, “Model validation for control and controller validation in a prediction error identification framework - Part I: theory,” Automatica, vol. 39, no. 3, pp. 403–415, March 2003. 17. G. Vinnicombe, “Frequency domain uncertainty and the graph topology,” IEEE Trans Automatic Control, vol. AC-38, pp. 1371–1383, 1993. 18. X. Bombois, G. Scorletti, P. Van den Hof, and M. Gevers, “Least costly identification experiment for control: a solution based on a high-order model approximation,” in CD-ROM Proc. American Control Conference, Boston, MA, USA, July 2004, pp. 2818–2823. 19. X. Bombois, G. Scorletti, M. Gevers, R. Hildebrand, and P. Van den Hof, “Cheapest open-loop identification for control,” in CD-ROM Proc. 33rd IEEE Conf on Decision and Control, The Bahamas, December 2004, pp. 382–387.

162

M. Gevers et al.

20. H. Jansson and H. Hjalmarsson, “A general framework for mixed H∞ and H2 input design,” submitted for publication to IEEE Trans. Auto. Control, 2004. 21. X. Bombois, G. Scorletti, Gevers, P. Van den Hof, and R. Hildebrand, “Least costly identification experiment for control,” submittted to Automatica, 2005. 22. L. Ljung, System Identification: Theory for the User, 2nd Edition. Englewood Cliffs, NJ: Prentice-Hall, 1999. 23. T. S¨ oderstr¨ om and P. Stoica, System Identification. Hemel Hempstead, Hertfordshire: Prentice-Hall International, 1989. 24. X. Bombois, “Connecting prediction error identification and robust control analysis: a new framework,” Ph.D. thesis, CESAME, Université Catholique de Louvain, Tech. Rep., 2000. 25. V. Popov, Hyperstability of Control Systems. New-York: Springer-Verlag, 1973. 26. K. Lindqvist, On experiment design in identification of smooth linear systems. Stockholm, Sweden: PhD Thesis, Royal Institute of Technology, 2001. 27. E. V. den Eijnde and J. Schoukens, “On the design of optimal excitation signals,” in Proc. 9th IFAC/IFORS Symp. on Identification and System Parameter Estimation, Budapest, Hungary, 1991, pp. 827–832. 28. H. Jansson and H. Hjalmarsson, “Mixed H∞ and H2 input design for identification,” in CD-ROM Proc. 43rd IEEE Conf on Decision and Control, The Bahamas, 2004, pp. 388–393. 29. R. Tempo, G. Calafiore, and F. Dabbene, Randomized algorithms for analysis and control of uncertain systems. New York: Springer Verlag, 2004. 30. R. Tempo, E. Bai, and F. Dabbene, “Probabilistic robustness analysis: explicit bound for the minimum number of samples,” Systems and Control Letters, vol. 30, pp. 237–242, 1997. 31. I. Landau, D. Rey, A. Karimi, A. Voda, and A. Franco, “A flexible transmission system as a benchmark for robust digital control,” European Journal of Control, vol. 1, no. 2, pp. 77–96, 1995. 32. G. Ferreres and V. Fromion, “H∞ control for a flexible transmission system,” in CD-ROM Proc. European Control Conference, Brussels, Belgium, 1997. 33. M. Barenthin, H. Jansson, and H. Hjalmarsson, “Applications of mixed H2 and H∞ input design in identification,” in 16th IFAC World Congress on Automatic Control, paper 03882, Prague, July 2005.

Past, Present and Future of Automotive Control Lino Guzzella and Christopher Onder1 Swiss Federal Institute of Technology (ETH), Zurich, Switzerland [email protected], [email protected]

This paper contains a survey of the application of control systems theory to automotive systems. For this application field, the most useful modeling approaches and control design methods are described. Some open problems are mentioned and possible trends for future developments are listed. A case study shows what role control systems may play for maximizing the efficiency of future vehicle propulsion systems.

1 Objectives of Automotive Control The design of feedforward and feedback control systems for automotive applications (here automotive control for short) is one of the most important application areas of classical and advanced control systems theory. The key aspect that radically differentiates the automotive field from other areas where control systems play a vital role is the cost issue. While in aerospace and defense applications this argument comes much later in the priority list, in the automotive industry the designer is always faced with very stringent cost limitations. The request to include a new sensor that adds 0.1% to the total manufacturing cost can be the reason to stop that development project. On the other hand, a project that aims at developing an improved control algorithm that meets the specifications with a reduced number of sensors or actuators will easily receive substantial funding from management. The reasons for this are, of course, the fierce competition in the automotive industry and the economies of scale. The value of electronic hardware and computer software built into automobiles by all OEMs is estimated to be of the order of 100 billion Euros per year. This figure is very likely to increase in the future. In particular the upcoming very demanding pollution emission limits and the increasing concerns regarding greenhouse effects and fossil fuel depletion will cause an increased demand for embedded control systems. This will lead to the situation where


164

L. Guzzella and C. Onder

approximately one fourth of the manufacturing cost of a passenger car will be induced by mechatronic components and the corresponding software. Figure 1 shows the typical complexity of a modern gasoline engine system. Control systems theory (feedforward and feedback control, system identification, monitoring and diagnosis, etc.) may be a hidden technology (Astrom), but in this problem area (as in many others) it is an enabling technology as well without which a systematic and cost-efficient problem solution is not possible. ECU

ET

MA

TA

VE

PM

CP

FP

IC

λ1

AK

CCV

CCV

AK CP IC MA SE FP

CC

TWC

SA TE

Tank

knock sensor camshaft sensor ignition command air mass-flow sensor engine speed sensor fuel pressure control

DP

λ2 TWC

SE

PM ET TA TE CC λ1,2

manifold pressure sensor electronic throttle intake air temperature sensor cooling water temperature sensor active carbon canister air/fuel ratio sensors

VE SA TWC ECU CCV DP

EGR valve secondary air valve 3-way catalyst controller CC control valves driver pedal

Fig. 1. Schematic representation of the sensors and actuators present in modern gasoline engines.

In automotive applications there are three main areas where control systems play a vital role: 1. pollution abatement, 2. active safety, and 3. fuel consumption. Looking back, the pollution abatement problem, in particular the feedforward and feedback control of the air/fuel ratio necessary to achieve the desired conversion rates of three-way catalytic converters, was the entry door by which embedded control systems were accepted by the traditionally rather conservative automotive industry. This process, which started in the seventies, was inexorably accelerated by more stringent pollution legislation and the development of affordable active safety systems (automatic braking, vehicle stability enhancement, collision detection and appropriate passenger protection systems, etc.). The fact that by the year 2000 almost 50% of all new cars sold in Europe were equipped with electronic stability systems (ESP, ASC, etc.) may be taken as a token for that development.

Past, Present and Future of Automotive Control

165

The pollutant emissions of modern gasoline engines have been reduced to almost negligible levels. It is clear that there is no real need to further reduce these emissions, in particular beyond the planned EURO V legislation. This is not true for Diesel engines1 and for novel combustion methods that are discussed at the moment. Without sophisticated control systems these approaches will not work properly. Huge progress will be made in active safety systems as well. Radar and video sensors will be used to observe the traffic situation in which a vehicle is operated. Appropriate data interpretation algorithms will provide information to safety systems that can indirectly (driver information) or directly (active steering and braking) react to impending dangers. Finally, the fuel economy of road vehicles must be substantially improved as well. Automotive control systems will be necessary to cope with the increasing complexity of such fuel-efficient power trains. Downsized and supercharged IC engines, automated gear boxes, hybrid-electric and fuel cell power trains, navigation and communication systems, path planning and inter-vehicle communication and coordination are a few of the key words that describe the plethora of opportunities that arise in this area.

2 Engine System Models 2.1 Model Classes Because of their reciprocating behavior, internal combustion engines are inherently discrete-event systems. The single events (ignition, injection, valve closing, etc.) are triggered by the crank angle which is the independent variable of this process. However, on very short and on very long time scales, several effects are time-based such that a very detailed description should use a hybrid systems approach. Examples of the former class of effects are the combustion delays, the flame propagation, and the torque peaks. The latter class includes manifold dynamics, turbocharger speed, and the engine’s thermal behavior. In the models proposed so far the single-combustion events were mostly disregarded since a mean-value description proved to be sufficient in most cases (see below). In future, novel combustion processes such as the controlled auto ignition (or homogeneous charge compression ignition) approach will require a very precise cycle-by-cycle process control. In particular the rate of recirculated (hot) exhaust gas must be precisely controlled in such systems. Novel actuators and sensors and appropriate control algorithms will be needed for that. 1

Particularly in heavy-duty and off-road applications, Diesel engines are unlikely to be replaced by any alternative prime mover. Due to their superior fuel economy, Diesel engines have recently gained a big market share in Europe in the passenger car segment as well.

166


Several authors have proposed discrete-event models of IC engines (see for instance [10] and [11]). A common simplification is obtained by assuming constant engine speeds. In this case, the engine dynamics may be described using a discrete-time approach. It was shown in [18] that this simplification is usually well justified and that the time constants of all other relevant dynamic effects are much smaller than those of the engine-speed dynamics. In the mean-value paradigm, the engine is assumed to be working as a continuous-flow machine such as a gas turbine. This approximation is equivalent to the assumption that the engine has infinitely many infinitely small cylinders, with the total displacement remaining constant. 2.2 Some Important Examples In gasoline engines one of the most salient dynamic effects is caused by the wall-wetting phenomenon in the intake part (see Fig. 2). The dynamics of this effect can be described by 1 1 d mf (t) = − mf (t) + β u(t), y(t) = mf (t) + (1 − β) u(t), dt τ τ

(1)

where u(t) is the amount of fuel to be injected (in a mean-value setting this is a mass flow) and y(t) is the amount of fuel entering the cylinder. The correct modeling of this effect and the identification of the parameters τ and β is still one of the most important problems in automotive control [2]. In particular during cold starts or after fuel-cut-off events, nonlinear effects cause substantial deviations from the description (1) [1], [18].

intake

injector exhaust wall-film mass mf (t)

Fig. 2. Wall wetting in port-injected gasoline engines; the variable mf (t) represents the total mass of fuel stored on the walls of the intake ports.

For efficiency reasons, in all modern Diesel engines the fuel is directly injected into their combustion chamber. Since this process is very fast, the main dynamic effects in such engines are not caused by the fuel path. On


167

the other hand, since all modern Diesel engines are turbocharged2 this device renders this type of engine systems hard to control.3 up

p2

ui injection

ωtc

p1

ϑ1

compressor (fluid- and thermodynamics) .

mc

Tc

.

ϑc

mα

intake receiver p2

p3

ϑ2 gas exchange

.

ωe

.

ϑ2

mϕ

mα

p3

combustion and torque generation

ϑe

.

.

.

mt

mα+mϕ outlet receiver

Tl

Te

engine inertia

ωe

uvnt

ϑ3

p3

p4

ωtc

turbine (fluid- and thermodynamics) .

mt

Tt

Tc

inertia turbocharger

ωtc

Fig. 3. Causality diagram of a turbocharged Diesel engine (exhaust gas recirculation not shown).

Figure 3 shows the causality diagram of such a turbocharged Diesel engine. Obviously, the turbocharger introduces a feedback loop from the exhaust to the intake side of the engine. The mechanical inertia of the turbocharger is part of this loop. Not surprisingly, this causes substantial delays and requires advanced automatic control methods for a correct operation. 2 3

Without turbocharging, the power density would be too low, whereas using larger naturally aspirated engines would lead to higher costs and a reduced fuel economy. The situation is exacerbated by the additional exhaust gas recirculation (EGR) loop that is required to lower the N Ox emissions.

168


Both nonlinear and linear mean-value models have been proposed [3] to represent such engine systems. The behavior of this system is difficult to model at very low turbocharger speeds [4] where large parameter variations and uncertainties must be taken into consideration. Unfortunately, this situation is often encountered in average driving conditions. Accordingly, the design of controllers for turbocharged engine systems must be very robust [5].

3 Control Systems 3.1 Introduction

amplifier, relays, etc.

event controller (TPU)

D/A converter, digital output

microcontroller

input signals from engine

A/D converter, digital input

From the beginning, internal combustion engines had to be designed to be homeostatic systems. In steady-state conditions they nominally operate on periodic orbits. Unavoidable disturbances (combustion fluctuations caused by turbulence effects, chaotic piston/wall slapping, load torque disturbances, etc.) are rejected by feedback loops that in older engines were realized using hardware components. In gasoline engines the main stabilizing effect is the feedback loop from the crank angle to the ignition event. Traditionally, this loop was realized using mechanical ignition distributors; in modern engines two magnetic pickups sense the crank-angle position while solid-state semiconductor amplifiers actuate the ignition coil. Similarly, in Diesel engines the main stabilizing effect is the feedback loop from the crank angle to the injection event. Traditionally, this loop was realized using in-line pumps that were mechanically connected to the crank shaft; in modern engines the same sensors as in gasoline engines are used to control the electromagnetically actuated fuel injectors. In this sense “burn-by-wire” systems have become the standard approach.

command signals to engine

crank-angle pulses

Fig. 4. Hardware structure of an engine control unit including time-based (slow) and event-based (crank-angle triggered) control channels.

Due to the engine’s reciprocating behavior, all engine control systems contain what is known as a time processing unit (TPU). These circuits (often on


169

the same chip as the microcontroller itself) interface the several asynchronous processes present in engine control loops. The adaptation of, say, the ignition angle to varying engine speeds is a time-based and relatively slow process. However, the ignition itself has to be triggered in each engine cycle at precisely the desired crank-angle location. This task is accomplished by the TPU. In addition to the basic loops mentioned above, all modern IC engines contain many higher-level control systems. The main emphasis is on feedforward systems while feedback, if used at all, is usually rather simple (SISO PI loops). The feedforward control systems often use tabulated values for the main system input commands (injection, ignition, EGR rate, etc.) with engine speed and load (torque) as main scheduling variables. These maps (see the example of a spark-advance map shown in Fig. 5) are calibrated off line. In these calibrations the manufacturers minimize the fuel consumption and optimize the comfort while ensuring that the pollutant legislation are satisfied. Some maps are partially adapted during operation, but in general the factory settings are kept constant.

spark advance [° ca]

40

20

0 100

6000

80 4000

60 relative engine torque [%]

40

2000

20 0

engine speed [rpm]

0

Fig. 5. Spark advance as a function of engine speed and load (torque).

These map-based control loops produce a static controller behavior. In other loops dynamic feedforward control systems are used as well. For instance the wall-wetting dynamics (1) are (partially) compensated for by choosing the injection command as follows u(t) = ∗

1 1 ∗ mf,d (t) − m ˆ f (t) , 1−β τ

(2)

where mf,d (t) is the desired mean-value fuel mass flow entering the cylinder and m ˆ f (t) the estimated mass of fuel in the wall film. Of course the parameters

170


τ and β depend on the engine speed, load, temperature, etc. These dependencies must be experimentally identified by the manufacturer and are stored in appropriate tables for later use. This process is very cumbersome and its complexity increases with each additional interacting loop. As mentioned above, feedback loops are realized using rather direct approaches. For instance the feedforward control loop for the mass of fuel to be injected described by Eq. (2) must be complemented with a feedback loop to attain the required precision of the air/fuel ratio (see below). However, the bandwidths of these loops are rather modest and feedback is only used to reject very low frequency disturbances and, of course, modeling errors in the feedforward path. Simple PI controllers are mainly used whose parameters are scheduled according to the main engine operating conditions (speed, load, temperature, etc.). Several authors have proposed and tested advanced design methods such as H2 or H∞ -optimal designs [6]. These approaches showed clear improvements over the traditional solutions. However, because the specifications for the feedback loops are relatively easy to meet, so far few of these advanced approaches have been applied in practice. This situation will change when the specifications become tighter and when the increasing plant complexity enforces MIMO control systems to be used. 3.2 Some Important Examples All IC engines include several feedback control loops. In addition to the basic loops mentioned in the last section, several additional loops may be identified. Almost all gasoline engines have an air/fuel ratio control loop, which is necessary for the correct operation of the catalytic converter [14]. Figure 6 shows the conversion efficiency of such a device. To achieve a simultaneous reduction of all three of the main pollutant species N Ox (oxides of nitrogen), HC (hydrocarbons), and CO (carbon monoxide), the air/fuel ratio has to be kept on the average very close to its stoichiometric value with an error of less than 0.1%. Fortunately, the catalyst is able to smooth out any fast transient deviations [13] such that the air/fuel ratio feedback loop can be chosen to have a rather small bandwidth. Moreover, periodic excitations around that value may improve the system behavior [12]. Another feedback control system that is present in most IC engines is the idle-speed control loop. Despite unobservable disturbances, such systems permit the engine’s idling speed to be kept at low values and, thus, fuel savings [7]. Due to the large nonlinearities and uncertainties always present in engine systems, these control loops must be very robust [9]. Moreover, the transfer from regular to idling operation (and vice versa) requires sophisticated transition strategies [8]. As mentioned above, novel approaches to combustion hold the promise of combining excellent fuel economy with very low engine-out pollutant emissions. The key idea is to very rapidly burn highly diluted, homogeneous mix-


171

conversion efficiency % 80

HC

60 40

CO

20 NOx 0.9

1

1.1

λ

Fig. 6. Conversion efficiency of a warmed-up three-way catalytic converter (100% complete removal of harmful species . . .) as a function of normalized air/fuel ratio λ (λ = 1 stands for stoichiometric conditions).

tures. The fast combustion yields excellent thermodynamic efficiencies and the high dilution reduces peak temperatures and hence the formation of N Ox . In addition, since the mixture is premixed (homogenous), almost no particulate matter is formed during the combustion. Unfortunately, the reaction kinetics of such combustion processes are very difficult to control. Moreover, it is impossible to use this approach in the full range of engine operation. Therefore, in automotive systems, these controlled auto-ignition operation modes must be smoothly connected with other combustion modes. Fully controllable valves will be a corner stone of such systems. Only such actuators will allow for a precise management of the amount of recirculated exhaust gases that will play a very important role in controlling the reaction kinetics. The modeling and control of these devices is a challenging design problem [15] and [16]. 3.3 Future Trends and Open Problems The most important developments in automotive control are expected to happen in the following areas: • • • •

hybrid control systems (discrete and continuous dynamics), automated and model-assisted control system calibration and verification, adaptive and “learning” systems, and model-based optimal feedforward control.

The first development will be needed in order to realize novel combustion systems that must be controlled on a cycle-by-cycle basis while time-based processes have a big influence as well.4 The challenge here will be the un4

Auto-ignition combustion engines lack the stabilizing effect of a crank-angle-based feedback provided in traditional engines by the ignition (gasoline) or injection (Diesel) events. For this reason, the kinetics of the auto-ignition combustion must be carefully controlled by imposing the gas temperature and composition and the flow conditions inside the cylinder.

172


derstanding of the basic dynamic phenomena and the formulation of feedback algorithms that are able to react within one engine cycle to reject disturbances. As mentioned above, the number of interacting control loops is inexorably increasing. The design, the experimental calibration, and the verification of these systems on a purely experimental basis is very time consuming, better solutions would be very welcome. Key points here are software reusability and guaranteed performance conditions that must be satisfied under all circumstances. Because of the ever increasing system complexity, the third development is an absolute must. Adaptive feedforward and feedback control could help to cope with relaxed manufacturing tolerances and thus could lower the system price. These algorithms must be guaranteed to work properly and to be very robust. Some must be able to quickly adapt to new situations, while others will be allowed to require long adaptation intervals (for instance, the continuous improvement of feedforward controls). Finally, the potential offered by the increasing computational power of modern microprocessors must be tapped for the optimization of the engine and power train management. Advanced vehicles will include powerful communication and navigation devices. Model-based control systems, capable of optimizing the fuel consumption while taking into consideration the planned trip and the actual traffic situation are feasible within the next few years.

4 Case Study – The PAC Car II Project 4.1 Introduction The vehicle analyzed in this case study, called “PAC Car II,” was designed for the Shell Eco-marathon events, which are competitions that reward the lowest fuel consumption. The PAC Car II vehicle is equipped with a fuel cell electric power train. To compare its hydrogen consumption to cars which use liquid hydrocarbons, the lower heating value of those fuels is taken as the reference. Besides the optimization of each component of the vehicle and its power train, the driving strategy had to be carefully chosen in order to achieve the best possible fuel economy. This optimization is the topic of the analysis presented below. First a short description of the main features of the vehicle is given. Then, starting from the formulation of the optimal control problem (OCP) various aspects of the optimization of the driving strategy are discussed in this case study. With such an optimized driving strategy PAC Car II won the Nogaro (F) Shell Eco-marathon race in May 2005. A fuel consumption of 7 g of hydrogen per 100 km was recorded in that race (using the lower heating values of hydrogen and gasoline, this corresponds to an equivalent mileage of 3836 km per liter gasoline). In the Shell Eco-marathon in Ladoux (F), one month later the same vehicle set a new world record for fuel efficient ground vehicles by


173

achieving an equivalent mileage of 5385 km per liter gasoline (5 g of hydrogen per 100 km). 4.2 Description of PAC Car II The PAC Car I and II vehicles were developed in the years 2000 to 2005 by a team of mechanical engineering graduate and undergraduate students at ETH Zurich. Figure 7 shows a picture of PAC Car II taken during the Nogaro 2005 event.

Fig. 7. The PAC Car II vehicle at the 2005 Shell Eco-marathon in Nogaro (F).

The vehicle was designed and manufactured according to the rules defined by the Shell Eco-marathon organizers. The main parameters are listed in Table 1. The vehicle is driven by two optimized direct current motors. The electric energy is produced by a fuel cell system fed by compressed hydrogen. The high-level control of the powertrain, i.e., the control of the fuel cell system, the monitoring of all important data in the vehicle, and the realization of the driving strategy are performed by an on-board computer system. Highefficiency DC/DC converters that distribute the electric power to the highperformance electric motors are connected to the control system via an optical link. The total force Ftot which acts on a road vehicle is the sum of the propulsion force u, the aerodynamic resistance Fa , the rolling resistance Fr , and the gravitational force Fg Ftot = u − Fa − Fr − Fg ρ = u − Af cd v 2 − cr m g cos(γ) − m g sin(γ), 2

(3) (4)

174

L. Guzzella and C. Onder Table 1. Main parameters of the PAC Car II vehicle frontal area Af aerodynamic friction coefficient cd mass of the vehicle m rolling friction coefficient cr

0.254 0.075 29 0.001

m2 kg -

where the variable γ stands for the inclination of the track at the actual position x (g = 9.81 m/s2 is the earth gravitational acceleration). Since only small inclinations must be considered, this force may be approximated by ρ Ftot ≈ u − Af cd v 2 − m g cr − m g γ. (5) 2 More detailed models are known [19], but such descriptions are not needed for the purposes discussed in the following sections. 4.3 Optimal Control Problem The problem of driving on a specific track while minimizing the fuel consumption can be formulated as an optimal control problem [17]. In a first step, the force at the wheel is minimized. Using Newton’s law and (5), the following equations define the dynamics of the OCP x˙ = v v˙ = c1 u − c2 v 2 − c3 − c4 γ(x)

(6) (7)

As mentioned above, the travelled distance is symbolized by x, the vehicle speed by v, and the slope of the track by γ(x). The coefficients ci immediately follow from (5) dividing all constant terms by the vehicle mass m. Braking the vehicle always results in a loss of energy and thus must be avoided under all circumstances. Accordingly, the force at the wheels has to be between zero and a maximum value u ∈ [0, umax ]. The objective function, which has to be minimized, is the total work at the wheel over the demanded time tb J(u) =

tb

0

u(t) v(t) dt,

v > 0 ∀t.

(8)

The Hamiltonian associated to this OCP can be written as H = u v + λ1 v + + λ2 (c1 u − c2 v 2 − c3 − c4 γ(x)).

(9)

Since the problem is time-invariant, the Hamiltonian has to be constant. The differential equations that define the dynamics of the co-state vector are ∂γ λ˙1 = λ2 c4 ∂x

(10)

λ˙2 = −u − λ1 + 2 λ2 c2 v.

(11)


175

According to the minimum principle, the Hamiltonian has a global minimum for the optimal control input H(x◦ (t), v ◦ (t), u◦ (t), t, λ◦ (t)) ≤ H(x◦ (t), v ◦ (t), u(t), t, λ◦ (t)),

(12)

where ◦ indicates the optimal trajectory. Taking into account the constraints for the input, the following control strategy can be defined  0 if v ◦ + λ◦2 c1 > 0  u◦ = using =? if v ◦ + λ◦2 c1 = 0 (13)  u max

if v ◦ + λ◦2 c1 < 0

Below, the variable h(t) = v ◦ + λ◦2 c2 will be referred to as the switching function. When h(t) ≡ 0 for a finite interval of time, the optimal control input cannot be derived directly from the switching function and the system is on a singular arc. In this case the derivatives of h(t) with respect to time have to be used to find the input u(t). The first derivative of h(t) is given by ˙ h(t) = v˙ + c1 λ˙ 2

(14)

= 2 λ2 c1 c2 v − c1 λ1 − c2 v 2 − c3 − c4 γ(x). Inserting h = 0 results in ˙ h(t) = −3 c2 v 2 − c1 λ1 − c3 − c4 γ(x).

(15)

Unfortunately, the input u does not explicitly appear in this expression. Therefore, an additional derivation is necessary ¨ = −6 c2 v (c1 u − c2 v 2 − c3 − c4 γ(x)). h(t)

(16)

With this equation the optimal control input using on the singular arc can be found 1 (c2 v 2 + c3 + c4 γ(x)). (17) using = c1 Using (7) it is easy to show that using results in v = constant. This result is somewhat surprising because the slope of the track is not constant. 4.4 Optimization of the Driving Strategy The discussion of the OCP showed that only three types of control signals need to be considered, viz. full load (boost), keeping the speed constant (sustain) and no power (coast). Using these three possibilities, some subproblems are now to be discussed in order to find a solution to the full problem.

176


Small Elevations, Full Cycle Since the aerodynamic resistance is quadratic in the velocity it is obvious that as long as the downhill forces do not exceed the sum of rolling and aerodynamic frictions, the smallest constant speed which exactly satisfies the average-speed boundary condition will be the solution for the infinite-horizon OCP. For a finite horizon it might be preferable to sustain a slightly higher speed and to recover some of the kinetic energy by adding a coasting phase at the end. This problem is analyzed first because it can be solved without being forced to solve a two-point boundary-value problem; a simple parameter optimization suffices to find the optimal driving strategy. 8

velocity [m/s]

6

4

2

0

0

50

100

150

200

2800

2850

2900

2950

3000

time [s]

Fig. 8. Speed traces for the two different strategies to solve the “small elevations” problem.

Figure 8 shows the traces of the velocities for the infinite-horizon strategy and the optimized finite-horizon strategy for a specific average-speed situation. Figure 9 illustrates the demanded energy at the wheel for varying sustain speeds. The smallest sustain speed corresponds to the infinite-horizon strategy. Thus, in this specific case, reducing the end speed to approximately half of the sustain speed results in a 2.8% reduction of the demanded energy. 4

energy at wheel [J]

4.2

x 10

4.15 4.1 4.05 4 3.95 6.7

6.8

6.9

7

7.1 7.2 vhold [m/s]

7.3

7.4

7.5

Fig. 9. Integrated energy at the wheel for varying sustain speeds.


177

Large Elevations, Infinite Horizon For a small and aerodynamically optimized vehicle even small elevations can result in an acceleration when driving downhill. Thus the force at the wheels can be set to zero and the control strategy has to be changed. Here, the infinite-horizon problem for a specific race track is discussed. The race track chosen is the motor speedway in Rockingham (UK). Its elevation profile is illustrated in Fig. 10. An average minimum speed of vavg,des = 6.7 m/s is required by the race regulations. The length of one lap on the track is slap = 2244 m. velocity [m/s] , traction force on wheel [N] , elevation [m]

14 12 10

velocity

8

powertrain force

6 4

elevation

t1

2

t2 tend

0

0

500

1000 1500 distance [m]

2000

2500

Fig. 10. Elevation profile, optimal vehicle speed, and force at the wheel calculated for the Rockingham race track.

Knowing the only possible form of the optimal control signals, the following driving strategy is proposed: • • •

starting with an initial velocity v0 , a coasting phase is chosen first since the slope of the trace at the start line is known to be negative, at t1 the control is switched on to keep a constant speed vsust , and at t2 the motors are shut off again until tend is reached.

The unknown variables v0 , t1 , t2 , and tend are chosen such that the energy at the wheel is minimized and that the boundary conditions: • • •

v(tend ) = v0 , s(tend ) = slap , and vavg,dem = s(tend )/tend

all are satisfied.

178


The solution of a numerical optimization problem with four unknown variables and three constraints poses quite some problems to standard numerical solvers. Therefore another method is proposed: • • • • • •

choose v0 as the independent variable of the optimization problem, use vsust to satisfy the boundary conditions, integrate the differential equations forward starting with t = 0, s = 0, and v = v0 until vsust is reached with decreasing velocity; this yields the two uknown variables t1 and s(t1 ), integrate the system equations backwards starting with t = slap /(vavg,dem ), s = slap , and v = v0 until vsust is reached with increasing velocity (with decreasing time) resulting in t2 and s(t2 ), vary vsust until vsust = (s(t2 ) − s(t1 ))/(t2 − t1 ), and vary v0 in order to minimize energy at wheel.

Energy at wheel [J]

5640 5630 5620 5610 5600 5590 2.7

2.8

2.9

3

3.1 3.2 v0 [m/s]

3.3

3.4

3.5

3.6

Fig. 11. Energy at the wheel for varying initial speeds.

The driving strategy for the Rockingham race track obtained with this approach is illustrated in Fig. 10. The sensitivity of the optimum to varying initial speeds is shown in Fig. 11, whereas Fig. 12 shows the dependency of the sustain speed for varying initial speeds. The sustain speed strongly depends on the initial speed, whereas the energy at the wheel varies only by approximately 1% in the range investigated. Thus it is important to choose the correct combination of initial and sustain speeds in order to meet the average-speed constraint. However, the proposed control strategy is robust with respect to variations in these parameters, i.e., the energy demand will not substantially increase even if the chosen initial and sustain speeds are suboptimal. So far the behavior of the subsystem providing the propulsion energy has not been included in the optimization. All results obtained so far remain valid if this subsystems has a constant efficiency when transforming chemical energy stored in the hydrogen to mechanical energy that is absorbed by the traction wheel. Fortunately, this condition is (approximately) satisfied for the fuel cell propulsion system of PAC Car II, as the efficiency map of Fig. 13 shows. The


179

vsustain [m/s]

6.5 6 5.5 5 4.5 2.7

2.8

2.9

3

3.1 3.2 v0 [m/s]

3.3

3.4

3.5

3.6

Fig. 12. Sustain speed for varying initial speeds.

gear ratio between motors and traction wheel must be chosen such that the operation limits are not violated and such that the propulsion system operates most of the time in the best-efficiency region (approximately at an efficiency of 0.45 in this example). The limits of operation are defined by the motor characteristics (current and power limits) and the polarization curve of the fuel cell. These limits are shown in Fig. 13 as well.

c)

0.35

0

5

0

0.4

5

0.4

0.2 0.1

a)

0.35

0.1

0.45

5

0.4

0.4

0 0 .1 0.25 .15 0.3 0.2

0.2

0.3 0.25

motor torque [Nm]

b)

0.3 0. 5 0.253 0.2 0. 1 0.05 2000

0.4

0.45

0. 0.3 35 0.2 5 0.2 0.15 0.15 0.1 0.05 4000 motor speed [rpm]

0.45 0.4 0.35 0.3 0.25 0.1 0.05 0.2 6000

Fig. 13. Tank-to-wheel efficiency map of the PAC Car II propulsion system.

Since PAC Car II is built for best aerodynamic performance, at narrow curves the vehicle may roll over if the speed is too high. Therefore, for a race track with curves with small radii, the speed in the turns has to be limited as well. The track of Nogaro (F), whose main characteristics are shown in Fig. 14, belongs to this class. If this restriction is taken into account, the space of possible solutions to the OCP is significantly reduced.

180


NOGARO MOTOR CIRCUIT

15 14 13

16

17 18

12

TOP

21

BOTTOM

20

19

8

11

9 10

7 6 22

5 27 4 3 2

26

28 29 30

25

31

1

24

23

32

33 34 35

Fig. 14. Layout and elevetaion information of the race track in Nogaro (F).

In Fig. 14 positive elevations are indicated with a square, negative ones with a circle. The two critical curves are in the segments 16 and 23. In these curves the speed must not be higher than 10m/s. Thus, the following strategy is chosen: • • • • •

drive with constant speed until the vehicle reaches that point from which, by shutting off the motor, the vehicle coasts to the appropriate speed at the boundary between segments 10 and 11, continue coasting until the sustain speed is again reached in the next ascent, since in this phase the grade is so steep that the speed cannot be held constant, use a short boost (maximum torque) phase here, after that, a coasting phase follows, such that the vehicle does not violate the top speed limitation at the trasition point from segment 22 to 23, drive with constant speed to the end of the lap.

The elevation, the speed of the vehicle, the traction force, and the external force to the wheel are indicated in Fig. 15. If the the torque that the motors are supposed to produce is very low, a bad efficiency of the powertrain results (see Fig. 13). In such situations, the motors are shut off until the vehicle speed drops to 2% below the demanded value. Then, a short high-torque phase is commanded until the demanded speed is again reached. This duty-cycle mode is continued as long as necessary.


181

velocity [m/s] , traction force on wheel [N] , ext. forces [N] , elevation [m] 15 velocity 10 powertrain force

5

0 external force −5

0

500

1000

elevation 1500 2000 distance [m]

2500

3000

3500

Fig. 15. Driving strategy for Nogaro (F).

Outlook The strategies presented in this case study utilize the insight about the form of the optimal control signals provided by the minimum principle. With this information and using heuristic arguments, the OCP is simplified to a nonlinear programming problem, which then can be solved easily. Another method to find optimal control strategies is to use a dynamicprogramming approach. This method is particularly well suited to optimization problems in which the elevation and the speed profile are known a priori. This information can be estimated using modern communication and navigation systems, once the driver informs the system about the desired destination. Combining this information with the available mathematical models of the vehicle and the propulsion system and utilizing fast numerical optimization algorithms will permit to find feedback control algorithms that are able to adapt to varying traffic situations.

References 1. Davison D E, Cornelius S J, Collings N, Glover K (2000) Observations of Transstoichiometric AFR spikes in UEGO Sensors. SAE International Fall Fuels and Lubricants Meeting and Exposition 2. Genc U, Ford F, Glover K, Collings N (2002) Experimental Investigation of Changing Fuel Path Dynamics in Twin-Independent Variable Camshaft Timing Engines. SAE Powertrain and Fluid Systems Conference

182


3. Jung M, Glover K (2003) Control-Oriented Linear Parameter Varying Modelling of a Turbocharged Diesel Engine. Proceedings of the IEEE Conference on Control Applications 4. Jung M, Ford R, Glover K, Collings N, Christen U, Watts M (2002) Parameterization and Transient Validation of a Variable Geometry Turbocharger for Mean-Value Modeling at Low and Medium Speed-Load Points. SAE Powertrain and Fluid Systems Conference 5. Jung M, Glover K, Christen U (2005) Comparison of Uncertainty Parameterisations for H-infinity Robust Control of Turbocharged Diesel Engines. Control Engineering Practice, 13:15–25 6. Jung M (2003) Mean-Value Modelling and Robust Control of the Airpath of a Turbocharged Diesel Engine. University of Cambridge PhD Thesis 7. Ford R, Glover K (2000) Spark Ignition Engine Idle Speed Control Utilising a Novel Framework and Facilitating Control of the Tradeoff Between Fuel Consumption and Disturbance Rejection Capability. AVEC International Symposium on Advanced Vehicle Control 8. Ford R, Glover K (2000) An application of coprime factor based anti-windup and bumpless transfer control to the spark ignition engine idle speed control problem. IEEE Conference on Decision and Control 9. Ford R (2000) Robust Automotive Idle Speed Control in a Novel Framework. University of Cambridge PhD Thesis 10. Christen U, Vantine K J, Glover K, Collings N (2001) Event-based MeanValue Modelling of DI Diesel Engines for Controller Design. SAE International Congress, Detroit 11. Vantine K J, Christen U, Glover K, Collings N (2001) Analysis of an EventBased Diesel Engine Model for Control Purposes. 3rd IFAC Workshop on Advances in Automotive Control 12. Davison D E, Cornelius S J, Glover K (2000) A Control Scheme to Produce a Catalyst Feed-gas with Arbitrary Periodic Air-to-fuel Ratio. ASME International Congress and Exposition Symposium on Advanced Automotive Technologies 13. Cornelius, S J, Collings N, Glover K (2000) The Role of Oxygen Storage in NO Conversion in Automotive Catalysts. CAPoC5, Brussels 14. Brandstetter M (1996) Robust Air-Fuel Ratio Control for Combustion Engines. University of Cambridge PhD Thesis 15. Genc U, Glover K, Ford R (2001) Nonlinear Control of Hydraulic Actuators in Variable Cam Timing Engines. International Workshop on Modeling, Emissions and Control in Automotive Engines, MECA’01, Salerno 16. Genc U (2003) Linear Parameter-Varying Modelling and Robust Control of Variable Cam Timing Engines. University of Cambridge PhD Thesis 17. Bryson E, Ho YC (1975) Applied Optimal Control. Taylor & Francis, New York, NY 18. Guzzella L, Onder C (2004) Introduction to Modeling and Control of IC Engine Systems. Springer, Berlin Heidelberg New York 19. Guzzella L, Sciarretta A (2005) Vehicle Propulsion Systems — Introduction to Modeling and Optimization. Springer, Berlin Heidelberg New York

Review of Multivariable Control Applied to the VAAC Harrier and Remaining Open Questions Rick A. Hyde Department of Mechanical Engineering, University of Bristol, University Walk, Bristol BS8 1TR, U.K. [email protected]

1 Introduction In the early nineties, a multivariable flight control law, designated FCL005, was developed for the Defence Evaluation and Research Agency (DERA), now QinetiQ, Vectored thrust Aircraft Advanced flight Control (VAAC) research Harrier XW175, and has been previously reported [1–3]. It was developed within the Cambridge University Engineering Department (CUED) Control Group, and much of its success depended on parallel theoretical developments within the group. The design itself used H∞ loop-shaping [4], with increased understanding of how to select weights being deduced from the application. Gain scheduling was initially done using switching between controllers, but in piloted simulation the switching points were evident, even when using bumpless transfer techniques. At around this time it was shown that the H∞ loop-shaping controller could be written in observer form (see for example [5]), providing an improved scheduling method. When it came to implementation of the control law on the aircraft, the lack of processing power necessitated discrete time implementation at a relatively slow rate compared to the bandwidth. To support this, the discrete time solution for the optimal H∞ controller in observer form developed in [6] was used. Later research within the CUED Control Group looked at model (in)validation [7] using the Harrier as a vehicle for testing its practicality. The H∞ loop-shaping controller developed for the VAAC Harrier is but one of a number of control laws studied and sponsored, at that time, by DERA [2] under the MOD Applied Research Programme. The primary VAAC programme aim was to look at requirements for the Future Carrier-Borne Aircraft (FCBA) that would ultimately replace the Sea Harrier. A key research objective was to reduce pilot workload, one way of achieving this being to move from a three-inceptor (throttle, nozzle, stick) to a two-inceptor (for example, airspeed and flight path demands) control system. Balancing powered and aerodynamic lift (the third degree of freedom) in semi-wingborne flight


184

R.A. Hyde

Fig. 1. The VAAC Harrier - Crown Copyright, reproduced with permission

presents a challenging design problem, particularly as there is strong cross coupling between vectored thrust and aerodynamic forces. Given the successful implementation and testing of FCL005 and many other applications of H∞ including the Lynx rotorcraft [8] and the Bell 205 helicopter [9], it might reasonably be concluded that multivariable control design is now a mature technology. This paper explores this based on experience of the VAAC Harrier multivariable control law. Section 2 sets the scene by providing a short history of the development of FCL005 and highlighting challenges faced during its design and implementation. Open issues are then highlighted and developed in Sections 4, 5 and 6. In addition to this, Section 3 presents previously unpublished flight test results and applies an analysis method proposed and tested in [7] that provides a measure, in the paradigm of normalized coprime factor uncertainty, of whether the model and its assumed uncertainty can explain the flight test data. Conclusions are then drawn in Section 7.

2 Review of Control Law 005 2.1 Control Law Development History Table 1 shows the development of the flight control law from its inception in 1988 through to latest test flights in 2001. Changes made to the control law were mainly to improve the transition between the wing-borne and powered lift regimes, design in other flight regimes being relatively straightforward. The main challenge at transition speeds is the large pitching moment induced as the nozzles swing down to increase powered lift. On approach, these large pitching moments cause rapid changes in angle-of-attack and hence aerodynamic lift thereby adversely affecting the flight path control. A second challenge is to manage the ratio of powered-lift to aerodynamic lift in a way that is both efficient and gives good pilot handling qualities. The most mature and successful solution has previously been outlined in [10]. The control law has always exhibited good robustness with no signs of the onset of instabilities at any speed. This may, in part, be due to the fidelity of the aircraft model used, but could as equally well be testament to

Review of Multivariable Control Applied to the VAAC Harrier

185

Table 1. Development history of FCL005 Period

Activities

1988 – 1991

Application of loop-shaping & selection of weighting functions. Scheduling of controller gains by switching and interpolation based on observer form. Piloted simulation on DERA Large Motion Simulator. Re-design for implementation on DERA VAAC Harrier. Improved implementation based on hybrid measurements of form [stabilization variable + λ× hold variable]. Direct discrete time design using discrete solution of H∞ controller. Successful first flight trial. Review of additional challenges posed by multivariable controllers when preparing a clearance case. Fidelity of model reviewed against flight data making use of the method of [7] and conventional methods. Identification of nozzle-tailplane interference effects using non-linear identification tools. Re-design of controller at transition speeds to support further flight tests and understanding of model uncertainty.

1992 – 1993

1995 – 1996 1999 – 2001

the appropriateness of the coprime factor uncertainty measure to capture differences between aircraft and model. In order to try and gauge the level of stability achieved, post-flight stability analysis was carried out by Davis and Glover [7, 11]. Only limited testing was possible at this time due to aircraft availability, but the feasibility of the method was demonstrated. Here, in 1999, the approach was applied again to the transition regime, and then in 2000 the idea of calculating bounds on achieved stability in real-time during flight explored. This work has not been previously reported, and is presented in Section 3. The control law has never been extensively refined and tested in order to optimize handling qualities, the main interest in FCL005 for the VAAC programme being to understand the merits and practicalities of using H∞ feedback controllers. It can be reasonably assumed that if the feedback controller achieves a robust and well decoupled closed-loop, then pilot command conditioning can be implemented in much the same way as for the other control laws studied. 2.2 Open Questions The VAAC Harrier has provided an ideal test bed on which to evaluate the practicalities of the H∞ loop-shaping method. Issues addressed and answered include [3] selecting inputs and outputs, scaling, weights selection (based largely on the earlier work of McFarlane and Glover [4, 12], gain scheduling and anti-windup. In parallel with this, there have been many other similar

186

R.A. Hyde

studies using loop-shaping for airframe control, most notably the work of Walker and Postlethwaite [8] on the Westland Lynx helicopter model and in piloted flight simulation. The selection of weights has also been separately studied, with an investigation of the use of non-diagonal weights [13, 14]. The net result is that, for multivariable systems, the control designer has access to a very comprehensive toolbox. However, there remain some unanswered questions that are now described. What stability requirement should the controller be designed to? For aircraft, Nichols exclusion regions [15] are often used to define uncertainty on the primary surfaces to which the closed-loop must remain stable. It will be shown in Section 4.3 that generalisation of this approach to multivariable systems and controllers can be problematic, even if the use of H∞ based uncertainty does provide some answers. How significant is the choice of feedback architecture and how should it be chosen? For the earlier FCL005 designs [3], the H∞ controllers were three-input threeoutput and with a range of open-loop cross-over frequencies at each of the three primary actuators. With experience, it was found that much better closed-loop dynamics could be achieved with an inner-loop pitch rate controller plus outer-loop 3-input 3-output multivariable controller [10]. The implications of architecture choice are considered in more detail in Sections 5.1 and 5.2. Is it sensible to invert plant dynamics in the weights, and if so, under what conditions? FCL005 only uses diagonal weights on the assumption made at the time that inverting system dynamics can result in poor stability margins. Subsequently, in [14], it was shown that if non-diagonal weights are used, their condition number should be small. In [16] results from designing Non-linear Dynamic Inversion (NDI) and H∞ loop-shaping autopilots for a highly cross-coupled missile suggest (but do not prove) that better robustness but similar performance is achieved when not directly cancelling plant cross-terms. The implications of non-diagonal weights in the context of H∞ loop-shaping is considered further in Section 5.3. Is the H∞ controller unnecessarily complex? The H∞ controllers flown on the VAAC Harrier have 21 states following model reduction. This looks high compared to a conventional controller consisting of single-input single-output feedback loops, where the number of states might be typically two or three per feedback path. This is compounded when it is considered how many state-space matrix entries must be gain scheduled with flight condition. This is explored further in Section 6.


187

3 Estimation of Achieved Stability Margins 3.1 Method Conventionally stability margins are estimated from flight data by performing stick doublets, and then estimating the loop gain and phase by means of an FFT of the response. An alternative is proposed in [11] that can handle multi-input multi-output systems, and is based on normalized coprime factor uncertainty as is used for the FCL005 design. Consider the block diagram ˜ and M ˜ represent the normalised left coprime shown in Figure 2. Here N factorisation of the linearised plant model G for a given flight condition i.e. ˜ . The blocks Δ ˜ and Δ ˜ represent the uncertainty on the nominal ˜ −1 N G=M N M coprime factors. In other words, the assumption is that the true plant is given by: ˜ + Δ ˜ )−1 (N ˜ + Δ ˜) Gtrue = (M M N where

ΔN˜ ΔM˜

∞

≤ γ.

-

Δ N~

u

~ N

Δ M~

~ M −1

y

n

Fig. 2. Left coprime factor model of plant uncertainty

The analysis approach proposed in [11] introduces an additional noise or exogenous disturbance input n shown in Figure 2 and poses the question: given an assumed maximum amplitude for each element of the vector n, how large does γ need to be to explain the measured output y in response to the measured input u? The linear model can be either treated as Linear Time Invariant (LTI) or Time Varying (TV). These two cases impose different restrictions on the class of ΔN˜ and ΔM˜ , and result in a different value of γ required to accommodate the flight test data. The TV case allows a larger class of uncertainty, and hence results in a smaller value of γ. The notation introduced in [11] for the two values of γ are γmin (T I) and γmin (T V ) respectively. A straightforward way of calculating γmin (T V ) is given in Lemma 5.2 of [7], and is as follows. Let plant input and output sampled data be denoted T T by sampled sequences u0 u1 . . . . . . and y0 y1 . . . . . . . Then define w and b as:

188

R.A. Hyde T

w = u 0 y0 u 1 y1 . . . . . .     u0 y0  u1   y1      .  .  b = TM˜   ..  − TN˜  ..  = TM˜ y¯ − TN˜ u¯     .. .. . . where TN˜ and TM˜ are the Toeplitz matrices mapping discrete-time system ˜ and M ˜ . Then the model-set is validated if and only inputs to outputs for N if there exists a real vector n ∈ Rny , with n 2 ≤ δ, satisfying πk (b − n)

2

≤ γ π2k w

2

where • 2 denotes the standard Euclidean norm and πk denotes the k-step truncation operator: πk : u0 u1 . . . → u0 u1 . . . uk−1 0 . . . . The minimum value of γ required to validate the model set can calculated by minimising the left-hand side of this inequality. Given an infinity norm bound of the form n ∞ < δ, the smallest value of γ is achieved by setting:   bi , if |bi | < δ ni = δ, if bi ≥ δ  −δ, if bi ≤ −δ Calculation of γmin (T I) involves finding the minimum of a linear function subject to Linear Matrix Inequality (LMI) constraints. This quickly becomes impractical for relatively short lengths of flight data. An upper bound on γmin (T I) is proposed in [7], denoted γub (T I). Its calculation involves solving a positive definite quadratic programming problem. Even this is computationally intensive, and it was found that for the three-input three-output system considered here that at most 15 seconds of flight data could be handled when sampling at 25Hz. 3.2 Application In order to gain confidence that the aircraft dynamic response can be explained by the model and assumed uncertainty set, it is important to exercise all dynamics and signal paths through the system. To achieve this, swept sine waves were applied during flight to each of the shaped plant’s inputs (d1 of dimension 3) and outputs (d2 of dimension 3) in turn and u and y recorded. Each sweep took 15 seconds, giving a total data capture time of 90 seconds. All data was taken at approximately 130 knots airspeed and 2000 feet. At this flight condition good modelling and/or stability margins are crucial as the aircraft is partly jet-borne, and powerplant-airframe interactions are very significant. The overall amplitude of the disturbances was set with the aid of the nonlinear aircraft model. Too low a disturbance will be insignificant compared to


189

background noise and turbulence, and too large a disturbance risks plant nonlinearity driving up the value of γ required to explain results. Following [7], each sweep has the form: (

3.8 ωmax − ωmin 4 t + 0.2) × sin(ωmin t + t ) 15 13, 500

where 0 < t < 15, ωmin = 0.2 rad/s and ωmax = 15 rad/s. The effect of the gain term is to increase the amplitude of the disturbance at higher frequencies in order to improve signal-to-noise ratio. This term was designed by trial and error on the non-linear model. The gain at lower frequencies was influenced mainly by the aircraft incidence limit and engine max rpm limit. 3.3 Results Results presented here are for VAAC flight test F1593 on 12th October 1999. The data was taken under good meteorological conditions resulting in a reasonably good internally-generated disturbance to external disturbance ratio. Ultimately this should result in a tighter bound on the stability margin calculation than if the tests were carried out in turbulent air. To determine bounds on the exogenous turbulence and noise (n), the measurements of u and y for the 10 seconds prior to the start of the swept sine −3

10

x 10

n1 n2

8

n

3

Noise (normalized by [N M])

6 4 2 0 −2 −4 −6 −8 0

1

2

3

4

5 6 Time (seconds)

7

Fig. 3. Equivalent noise levels

8

9

10

190

R.A. Hyde

˜ M ˜ . This gives an equivalent noise vector waves were passed as inputs to N defined by: u n = TN˜ TM˜ y Figure 3 shows a plot of n. From this figure the L1 noise bound is taken to T T be −0.007, −0.010, −0.005 < n < 0.007, 0.010, 0.005 . The variation on this signal is at a relatively low frequency, suggesting that its primary source is low-level turbulence. Figures 4 and 5 show part of the flight data collected. Constant offsets are removed from the raw data using the MATLAB detrend.m command. In total six frequency sweeps are applied, each taking 15 seconds. Given the sample time of 0.02 seconds this gives a total of 4500 points. The calculated value of γmin (T V ) for this data set is 0.0664. What the result precisely says is, that given the assumed turbulence/noise values, the bound on the time-varying coprime factor uncertainty required to explain the differences between the model and the aircraft is γmin (T V )=0.0664. Of course, in practice, the timeinvariant uncertainty required to explain the data could be higher than this. Also, during the test, higher levels of turbulence than measured during the pre-test ten seconds of flight will act to give a misleadingly high value of γmin (T V ). The value of γmin (T V ) should be compared with the bound on the smallest normalized coprime factor uncertainty [4] that will destabilize the nominal shaped plant and controller K. Loop shapes were chosen in order to ensure that was at least 0.4. The value of γmin (T V ) is small in comparison to this, suggesting more than adequate robustness of the design. Additional insight can be obtained from calculating γmin (T V ) for a subset of the data. Calculating γmin (T V ) for the first 15 seconds of data only gives γmin (T V )=0.013, perhaps indicating that the modelling in pitch is better than in other loops given that the first sweep is on the tailplane demand. 3.4 Using Pilot Commands The use of swept sine wave generators integrated into the control law gives control over the frequency content and the paths excited. This gives more confidence in the results than if an unspecific excitation signal, such as general pilot-invoked manoeuvring, is used. However, pilot manoeuvring could be used to monitor aircraft stability margins in-service as part of a fault-detection system. This idea was explored by calculating γmin (T V ) for specific manoeuvres. Figure 6 shows a flight path change from 0 to +5 degrees at around 130 knots. For this set of flight data, the calculated value of γmin (T V ) is 0.0837, which again is low compared to the linear-time invariant nominal robustness value of ε =0.40. The difficulty with applying this method in general is that the analysis, as presented, must start from steady initial conditions. Without steady initial

Review of Multivariable Control Applied to the VAAC Harrier 0.04 0.03 0.02

Longitudinal thrust Normal thrust Tailplane

Umeasured

0.01 0 −0.01 −0.02 −0.03 −0.04 −0.05 0

100

200

300 400 sample number

500

600

700

Fig. 4. Flight F1593 plant input data (first 15 seconds only)

0.04 Forward motion 0.03

Normal motion Pitching motion

Ymeasured

0.02

0.01

0

−0.01

−0.02

−0.03 0

100

200

300 400 sample number

500

600

700

Fig. 5. Flight F1593 plant output data (first 15 seconds only)

191

192

R.A. Hyde

conditions, the transients due to the initial state will look like an external disturbance. In order to address this, predicted initial condition transients derived from the nominal plant model and controller can be subtracted from the aircraft dynamic response. The initial plant state x0G and controller state x0K are calculated by simulating the following two equations for a few seconds of flight prior to the start of the analysis: x0G = [ TNG I − TMG ] x0K = [ TN I − TM ]

um ym

ym um

Here [N M ] is a normalized left coprime factorization of the controller K∞ , [NG MG ] is a normalized left coprime factorization of the nominal shaped plant model W2 GW1 and u¯m and y¯m are the sequences of measured aircraft inputs and outputs. With this architecture γmin (T V ) can be calculated in real-time during flight, giving the pilot or fault monitoring system a continuous readout of how the aircraft is behaving in comparison to a set of linearized models. To date this has been tested only during piloted simulation.

6 5 4

Response (see key)

3 2 1 0 −1 Flight path hold flag

−2

Flight path (degrees) −3 −4 1460

Stick position (inches) 1461

1462

1463

1464 1465 1466 Time (seconds)

1467

1468

1469

Fig. 6. Flight path change from 0 to +5 degrees

1470


193

4 Retrospective Stability Analysis 4.1 Measuring Stability This section aims to review the stability properties of FCL005 in the light of recent theoretical developments. But before doing this, it is worth exploring how the manufacturers assess the stability of military aircraft. One of the challenges for the early work on FCL005 was to relate the chosen target normalised coprime factor robustness level of > 0.4 to conventional measures such as gain and phase margins or Nichols exclusion regions. Only with this would the method find wider application. The dashed line in Figure 7 shows a Nichols exclusion region as used by the manufacturers [15]. It defines a range of gain and phase offsets which, when applied to each of the actuators simultaneously, should not result in instability. This region is made larger for the case where it is applied only to one actuator at a time. Note that this multi-loop stability test is only applied at the actuators and not at the plant outputs. The original design target of > 0.4 was defined based on experience of a number of design examples, and then later supported by the bounds on gain and phase margin derived in [17], namely that gain margin > (1 + ) / (1 − ) and phase margin > 2 sin−1 . A direct relationship between and equivalent Nichols exclusion regions was derived much later in [18], but the underlying approach of approximating the exclusion region as an ellipse in the 10

(1+δ)/(1−δ), |δ| Ni for some i. (8) It is called finite probability assumption. The finite probability assumption is justified if the deterministic version (1) is stable. In that case, the solution subject to the assumption (8) will give a good approximation to the solution of the original master equation (2). Due to the assumption (8), we can eliminate all the components of the population vector corresponding xi > Ni for some i and retain only the components corresponding to xi ≤ Ni for each i. Then, we obtain the truncated population vector xf of dimension N = N1 N2 · · · NK . Writing pf = P (xf ), the truncated master equation is now represented as dpf = L f pf , dt

(9)

where Lf is a part of the original L in (4) corresponding to pf in p. It is important to notice that Lf in (9) is not a stochastic matrix in the sense that it does not satisfy (5), because some of the off-diagonal elements of L is excluded in Lf so that 1Tf Lf ≤ 0, (10) which implies that each component of 1Tf Lf is non-negative. Here, 1f is a truncated version of 1, i.e., 1f = 1 1 · · · 1

T

.

230

H. Kimura

In order to make Lf stochastic, we must assume that if any specy reaches at the upper bound, then no reaction occurs which increases the specy. This is equivalent to assume wj (x) = 0, if xi = Ni and aji > 0 for some i.

(11)

Under this additional constraint, it is shown that Lf becomes stochastic, i.e., 1Tf Lf = 0.

(12)

3 Toggle Switch Now, we discuss a gene regulatory system which acts as a biological switch depicted in Figure 1. This system was constructed artificially in a cell of Esherichia coli as a synthetic addressable memory unit of the living organism [5]. It consists of two proteins which are called repressors 1 and 2 in Figure 1, respectively. They have their promotors (the areas of the gene that control the start of the transcription) adjacent to each other. Each repressor inhibits the transcription (and thus the expression) of the other. Thus, this regulatory system forms bilateral inhibition network (double negative feedback loops) which is common in biological networks.

promotor 2

repressor 1 promotor 1

repressor 2

Fig. 1. Biological Switch

The system is composed of the following four reactions: α

(13)

δ

(14)

φ −→ R2

β

(15)

R2 −→ φ2

(16)

φ −→ R1 R1 −→ φ

where Ri denotes the repressor i (i = 1, 2), and φ denotes void. The relation (13) represents the synthesis of the repressor 1 with rate α, while (14) represents its degradation with rate δ. Equations (15) and (16) have the same meaning for the repressor 2. The synthesis rates α and β depend crucially on the rate of transcriptions, and as we assumed in Figure1, they depend on the

Intrinsic Uncertainty in Gene Regulation Network

231

amount of the other repressor already synthesized. Let x and y be the numbers of molecules of the repressors 1 and 2, respectively. Since synthesis of the repressor 1 is inhibited by the repressor 2, synthesis rate α is a decreasing function of y. In enzymology, this function is of the form α(y) =

α , 1 + yρ

(17)

where α denotes the maximum synthesis rate and ρ the cooperatively index. Likewise, β is of the form β β(x) = . (18) 1 + xµ The functions (17) and (18) play the role of propensity functions in the stochastic analysis of the system. Now, the equation (1) is represented in this system as α − δx 1 + yρ β − y y˙ = 1 + xµ

x˙ =

(19) (20)

The system (19)(20) was fully analysed in [5], and it was shown that for some parameter values, the system (19)(20) has two stable equilibrium points, each of which synthesis of one repressor dominates the other. They showed that the outer stimuli can transform the state from one equilibrium point to the outer, thus the system (19)(20) can act as a toggle switch which can memorize the state before the stimulus comes.

4 CME for Toggle Switch Now, we deal with stochastic version of the system. We first assume that during the infinitesimal time interval, only one molecular is synthesized or degradated of each repressor. In other words, only the transitions x −→ x ± 1 y −→ y ± 1 occur in the infinitesimal time interval. Under this assumption, we can write the master equation for the switch as dP (x, y; t) = α(y)(x − 1, y; t) + β(x)P (x, y − 1; t) dt + δ · (x + 1)P (x + 1, y; t) + · (y + 1)P (x, y + 1; t) − α(y) + β(x) + δx + y P (x, y; t)

(21)

232

H. Kimura

Now we rewrite the master equation in a more tractable form based on the probability matrix   .. P (0, 0) P (0, 1) · · · P (0, m) · · · .      P (1, 0) P (1, 1) · · · P (1, m) · · · ...     .  .. P (t) =  (22) ······ ,   .  P (n, 0) P (n, 1) · · · P (n, m) · · · ..   .. .. .. . . . where P (n, m) = P rob. x(t) = n, y(t) = m . Introduce the following infinite matrices U, V, U1 , V1 and Σ1 as follows:   −α(i − 1) − (i − 1) , i = j (U )ij = (i − 1) , (23) i=j+1  0 otherwise   −β(i − 1) − (i − 1)δ, i = j (V )ij = (i − 1)δ, (24) j =i+1  0 otherwise U1 = diag [α(0), α(1), α(2), · · · ]

(25)

V1 = diag [β(0), β(1), β(2), · · · ]

(26)

To help the visual conception, we write down the explicit forms of U and V as   −α(0) 0 0 ···  −α(1) − 0 · · ·   U = 0 (27) 2 −α(2) − 2 · · ·   .. .. .. . . .   −β(0) δ 0 ···  0 −β(1) − δ 2δ · · ·   . (28) V = 0 0 −β(2) − 2δ · · ·   .. .. .. . . . It is not difficult to see that the master equation (21) can be represented as dP (t) = P (t)U + V P (t) + Σ1 P (t)U1 + V1 P (t)Σ1T , dt where Σ1 is the operator of column shift, i.e.,

(29)




0 0  1 0  Σ1 =  0 1  .. .. . . It is clear that

(U + U1 )1 = 0,

233



··· ···  . ··· 

(30)

···

1T (V + V1 ) = 0,

(31)

where 1 is the vector defined in (7).

5 Truncation and Reduction to Standard Form Now, we consider the truncation of the master equation under the assumption P (x, y, t) = 0, if either x > N or y > N,

(32)

where N is chosen sufficiently large to be beyond the upper bound of population numbers of the repressors. Since all the solutions of the deterministic equation (7) is assured to be bounded, the assumption (32) is considered to hold. It is important to note that the assumption (32) implies that the transitions x(t) = N −→ x(t + Δt) = N + 1 y(t) = N −→ y(t + Δt) = N + 1 cannot occur. In order to meet this condition, the master equation (21) should be rewritten as dP (x, y; t) = α(y)P (x − 1, y; t) + β(x)P (x, y − 1; t) dt + δ · (x + 1)P (x + 1, y; t) + · (y + 1)P (x, y + 1; t) − α1 (x, y) + β1 (x, y) + δx + y P (x, y; t),

(33)

where α1 (x, y) =

α(y) if 0 ≤ x ≤ N − 1 0 if x ≥ N

(34)

β1 (x, y) =

β(x) if 0 ≤ y ≤ N − 1 0 if y ≥ N.

(35)

This modification corresponds to the assumption (11). Under this truncation, the equation (29) is rewritten as dP N (t) =P N (t)U N + V N P N (t) + Σ1N P N (t)U1N + V1N P N (t)(Σ1N )T dt + eN eTN P N (t)U1N + V1N P N (t)eN eTN , (36)

234

H. Kimura

where the superscript N denotes the truncations of matrices (27)(28)(25)(26) and eN is the (N + 1)- dimensional vector whose elements are all zero except the last one, i.e., T eN = 0 0 · · · 0 1 . The last two terms in the right-hand side of (36) corresponds to the modification (34)(35). We can further rewrite (36) as (“t”is omitted) dP N = P N U N + V N P N + Σ N P N U1N + V1N P N (Σ N )T , dt where



Σ N = Σ1N + eN eTN

0 0 1 0   0 1 = 0 0   ··· 0 0

··· ··· ··· ··· ··· ···

0 0 0 0 ··· 1

 0 0   0 . 0   1

(37)

(38)

The relations (31) hold in the truncated version: (U N + U1N )1N = 0,

(1N )T (V N + V1N ) = 0.

(39)

Now, we rewrite (37) in the form (9). To do so, we introduce the Kronecker product ⊗ between matrices A and B as 

a11 B  a21 B A⊗B = 

a12 B a22 B ··· am1 B am2 B

 · · · a1n B · · · a2n B  .  ··· · · · amn B

(40)

The size of the matrix A ⊗ B is mr × np if A is m × n and B r × p. Now, we introduce the vectorization Pˆ of a matrix P = (pij ) as Pˆ = p11 p21 · · · pn1 p12 · · · pn2 · · · pnm

T

(41)

Then, it is easily seen that AP = (I ⊗ A)Pˆ

(42)

P B = (B T ⊗ I)Pˆ

(43)

Let us write the vectorization of the probability matrix P N as pN := Pˆ N Then, repeated use of (42) and (43) in the vectorization of (37) yields

(44)


dpN = ((U N )T ⊗ I)pN + I ⊗ V N pN dt T + U1N ⊗ (Σ N ) pN + Σ N ⊗ V1N pN

235

(45)

N

= LN p , where LN = (U N )T ⊗ I + I ⊗ V N + (U1N )T ⊗ Σ N + Σ N ⊗ V1N .

(46)

Here, we used an obvious identity (A ⊗ B)(C ⊗ D) = AC ⊗ BD subject to the size consistency. Let 1N be the N-dimensional vector with all elements being 1, i.e., 1N = T 1 1 · · · 1 . Then, from (31) and the relation Σ N 1N = 1N , we have (1N )T ⊗ (1N )T LN = 0

(47)

Here, we used the identity f1 ⊗ g + f2 ⊗ g = (f1 + f2 ) ⊗ g. The implies that LN defined by (46) is a stochastic matrix. Now, we can write down an explicit form of LN , which can be seen an array of N × N cells, each of which is an N × N matrix. Let Lij denote the (i, j) cell, i.e.,   L11 L12 · · · L1N  L21 L22 · · · L2N   , Lij ∈ RN ×N . LN =  (48)   ··· ··· LN 1 LN 2 · · · LN N Then, Lij is calculated to be  (1) uii I + ui Σ N + V N ,  uji I Lij = V1N  0,

i=j j =i+1 i=j+1 otherwise.

(49)

(1)

where uij denotes the (i, j)-element of U N and ui the i-th diagonal element of U1N . Thus, LN is a block tridiagonal matrix. From the structure of Σ N and V N , the diagonal blocks Lii are tridiagonal with only the diagonal elements being negative.

6 Irreducibility The time evolution of the truncated probability distribution pN follows the linear differential equation (45), i.e., dpN = L N pN , dt

(50)

236

H. Kimura

where LN is given by (48) and (49). We are interested in the convergence of pN (t) to a stationary distribution pN ∞ as t goes to infinity. According to the stochastic time-evolution theory [10], the differential equation (50) has the unique stationary solution pN ∞ satisfying L N pN ∞ = 0,

(1N )T pN ∞ = 1

and any solution of (50) tends to pN ∞ as t goes to infinity, irrespective of initial distribution pN (0), provided that LN is irreducible, i.e., LN cannot be represented in a form LN =

A B , 0 C

A, C; square matrices

(51)

through permutations of rows and columns. Now, we shall prove that LN given by (48)(49) is really irreducible. The following lemma is crucial for our purpose. Lemma 1. The matrix LN given in the block form (48) is irreducible, if each diagonal block Lii is irreducible and each super-diagonal and sub-diagonal blocks Li i+1 and Li+1 i are non-zero. Proof . From the definition of a reducible matrix (51), LN is reducible if there exists an index set I such that lik = 0,

¯ i ∈ I, k ∈ I.

(52)

¯ Lii If there is a diagonal block Lii such that its columns are split in I and I, itself is obviously reducible. Hence, each column of Lii belongs to either I or I¯ for any i. Assume, contrary to the assumption, that LN is reducible. Then, there exists a index set I such that Lik = 0,

i ∈ I, k ∈ I¯

It is obvious that there exists a boundary of I and I¯ such that i ∈ I and ¯ This obviously implies that either Li i+1 = 0 or i + 1 ∈ I¯ or i + 1 ∈ I, i ∈ I. Li+1 i = 0 holds, which contradicts the assumption. In view of Lemma 1, we can establish the irreducibility of LN , if we show that Lii given in (49) are irreducible. From (49) and the forms of Σ N and V N , Lii is tri-diagonal given by  uii , k=j  (1) k =j+1 ui (Lii )kj = (k − 1)δ j = k + 1  0 otherwise


237

The assertion follows immediately, if we can show that any tri-diagonal matrix is irreducible, provided that all of its super-diagonal and sub-diagonal elements are non-zero. The assertion is verified using the same argument as the proof of Lemma 1 with Ljk being replaced by (Lii )jk . Thus, we have establised the following result: Theorem 1. The truncated CME (36) has the unique stationary probability distribution p∞ (x, y), to which every solution of (21) converges irrespective of the initial probability distribution p(x, y; 0).

7 Simulation Results We have performed numerical simulations of (21). Figures 2-4 shows the simulation results. The system parameters are selected such that the deterministic dynamics (19)(20) has two stable equillibrium points. Actually, without loss of generality, parameters are chosen symmetrically, i.e., α = β, δ = , ρ = µ, in (19)(20). The upper bound N of molecular numbers are chosen to be N = 200, 400, 600. We used Euler method to solve the equation (50). Figure 2 shows a simulation result for the case where the stable equillibrium points are (x, y) = (40, 7) and (x, y) = (7, 40). The initial state is chosen to be x(0) = 60, y(0) = 30. Initially, the peak probability gradually moves to (40, 7) which is the attractor for the initial condition. However, as time proceeds, another peak gradually grows which corresponds to the other stable equillibrium point. Finally, approximately at 500,000 sec from the initial time, the probability distribution forms two peaks of almost equal height. These peaks are generated almost at the points (x, y) = (7, 40) and (x, y) = (40, 7) as was expected. Figure 3 demonstrates another simulation result for the same system with a different initial condition. In this case, the peak probability distribution approaches to the other equillibrium point (7, 40), then another peak gradually grows and finally the probability distribution tends to the same probability distribution as in Figure 2. Figure 4 illustrates another simulation result where the initial condition is at (5, 10) which has almost equal distances to the two stable equillibrium points. In this case, the transient behavior is different from those shown in Figures 2 and 3, but eventually, the probability distribution tends to the same one as in Figures 2 and 3. These results are perfectly consistent with the theoretical results we have shown. In order to check the validity of the finite probability assumption we have performed simulation choosing N = 600. The result is totally identical to the case N = 200. From this observation, we can conclude that the finite probability assumption is justified at least in this case.

238

H. Kimura

25sec Start at (60, 30) 250sec The fluctuation grows 2500sec The peak is

formed around as stable equilibrium

6000sec (70, 4) Stable state is kept

15000sec Gradual flow out the probability

Equillibrium Points are (40, 7) and (7, 40) Initial State (60, 30)

50000sec Gradual increase 100000sec The formation 30000sec No changes after the beginning of probability of the probability at another of small peak at another EP EP flow out

250000sec Gradual growth of peak at another EP

500000sec The stationary distribution

Fig. 2. Snapshots of probability distributions for the initial condition (60,30)


100sec (50, 75) starting at

2500sec

15000sec

100000sec

500sec

100sec

5000sec

7500sec

30000sec

200000sec

239

50000sec

500000sec

The final distribution does not depends on the initial state! Fig. 3. Snapshots of probability distributions for the initial condition (60,70)

240

H. Kimura

50sec

2000sec

300sec

4000sec

1000sec

6000sec sudden Growth of Fluctuation

Initial State (5, 10)

10000sec

15000sec

100000sec

250000sec

30000sec

Fig. 4. Snapshots of probability distributions for the initial condition (5,10)

8 Conclusion A gap between the deterministic and the stochastic frameworks has been demonstrated. It is reduced to the fact that the master equation which is a representative of the stochastic framework is almost always monostable, while the deterministic dynamics may not be monostable. In that case, the two frameworks might lead to different conclusions which are contradictory to each other in the limit of the time evolution. Since bio-chemical processes taking place inside cells are essentially stochastic, this gap may create serious problems.


241

We take the genetic toggle switch as a typical example of such cases. In deterministic description, its dynamics is bistable, i.e., it has two stable equillibrium points. In stochastic formulation, however, dynamics is mono-stable, i.e., all the probabilistic states tend to a unique stationary distribution, irrespective of the initial distribution. We have demonstrated the unique convergence of the master equation theoretically and numerically by showing that the dynamics corresponding to the genetic toggle switch is irreducible through the conversion of a two dimensional stochastic array to one dimensional distribution. Through this result, we have shown that the genetic toggle switch does not work eventually. This suggests that the long-run breakdown of biological function due to stochasticity is a reason for the finiteness of the life. Since our method is universal, it is applicable to many other problems. It is an interesting problem to see how the life manages to solve the intrinsic uncertainty to properly execute the normal functions. Finally, we have actually validated theoretical results experimentally for the engineered toggle switch formed by Gardners et. al. [5]. The results are discussed in a different paper [21].

References 1. A. Becskei, B. Ser´ aphin and L. Serrano, Positive feedback in eukaryotic gene network: cell differentiation by graded to lineary response conversion,” The EMBO J., vol. 20, pp. 2528-2535, 2001. 2. A. Becskei and L. Serrano, Engineering stability in gene networks by autoregulation,” Nature, vol. 405, pp. 590-593, 2000. 3. W.J. Blake, M. Kaern, C.R. Cantor and J.J. Collins, Noise in enkaryotic gene expression,” Nature, vol. 422, pp. 633-637, 2003. 4. M.B. Elowitz and S. Leibler, A synthetic oscillatory network of transcriptional regulators,” Nature, vol. 403, pp. 335-338, 2000. 5. T.S. Gardner, C.R. Cantor and J.J. Collins, Construction of a genetic toggle switch in Esherichia coli,” Nature, vol. 403, pp. 339-342, 2000. 6. D.T. Gillespie, Approximate accelerated stochastic simulation of chemically reacting systems,” J. of Chem. Phys., vol. 115, pp. 1716-1733, 2001. 7. D.T. Gillespie, Exact stochastic simulation of coupled chemical reactions,” J. of Chem. Phys., vol. 81, pp. 2341-2361, 1977. 8. B. Houchmandzadeh et al., Establishment of developmental precision and proportions in the early Drosophila embryo,” Nature, vol. 415, pp. 798-802, 2002. 9. F. Jacob and J. Monod, Genetic and regulatory mechanisms in the synthesis of proteins,” J. of Molecular Biology, vol. 3, pp. 318-356, 1961. 10. N.G. Van Kampen, Stochastic Processes in Physics and Chemistry, Elsevier, 1992. 11. T.B. Keller and T.C. Elston, Stochasticity in transcriptional regulation: Origin, consequences, and mathematical representations, ” Biophysics J., vol. 81, pp. 3116-3136, 2001.

242

H. Kimura

12. J. Monod and F. Jacob, General conclusions: Teleonomic mechanisms in cellular metabolism, growth and differentiation,” Cold Spring Harbor Symp. Quant. Biology, vol. 26, pp. 389-401, 1962. 13. N. Le Novére and T.S. Shimizu, STOCHSIM: modelling of stochastic biomolecular processes,” Bioinformatics, vol. 17, pp. 575-576, 2001. 14. E.M. Ozubdak et al., Regulation of noise in the expression of a single gene,” Nature Genet., vol. 31, pp. 69-73, 2002. 15. C.V. Rao, D.M. Wolf and A.P. Arkin, Control, exploitation and tolerance of intracellular noise,” Nature, vol. 420, pp. 231-237, 2002. 16. J. Ross, K.L.C. Hunt and P.M. Hunt, Thermodynamics far from equilibrium: Reactions with multiple stationary states,” J. Chem. Phys., vol. 88, pp. 27192728, 1988. 17. J. Ross, K.L.C. Hunt and P.M. Hunt, Thermodynamic and stochastic theory for nonequilibrium systems with multiple reactive intermediates: The concept and role of excess work,” J. Chem. Phys., vol. 96, pp. 618-629, 1992. 18. P. Smolen, D.A. Boxter and J.H. Byrne, Modern transcriptional control in gene networks—methods, recent results, and future directions,” Bull. Mathematical Biology, vol. 62, pp. 247-292, 2000. 19. Y. Tao, Intrinsic and external noise in an auto-regulatory genetic network,” J. Theoretical Biology, vol. 229, pp. 147-156, 2004. 20. R. Tomioka, H. Kimura, T.J. Kobayashi and K. Aihara, Multivariable analysis of noise in genetic regulatory networks,” J. Theor. Biology, vol. 229, pp. 501521, 2004. 21. H. Tozaki, R. Yamamoto, H. Okano, K. Aihara and H. Kimura, A long-run effect of noise in genetic toggle switch,” submitted for publication. 22. Q. Yang, P.A. Lindahl and J.J. Morgan, Dynamic responses of protein homeostatic regulatory mechanisms to perturbations from steady state,” J. Theor. Biology, vol. 222, pp. 407-423, 2003.

An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems Randy Cogill1 , Michael Rotkowitz2 , Benjamin Van Roy3 , and Sanjay Lall4 1 2 3

4

Department of Electrical Engineering, Stanford University. [email protected] Department of Aeronautics and Astronautics, Stanford University. [email protected] Departments of Management Science and Engineering and Electrical Engineering, Stanford University. [email protected] Department of Aeronautics and Astronautics, Stanford University. [email protected]

Summary. We consider the problem of computing decentralized control policies for stochastic systems with finite state and action spaces. Synthesis of optimal decentralized policies for such problems is known to be NP-hard [1]. Here we focus on methods for efficiently computing meaningful suboptimal decentralized control policies. The algorithms we present here are based on approximation of optimal Qfunctions. We show that the performance loss associated with choosing decentralized policies with respect to an approximate Q-function is related to the approximation error.

1 Introduction The study of decentralized control is motivated by the fact that, in many practical control applications, control decisions must be made using incomplete information about the system state. In particular, it is common that multiple interacting decision makers must make control decisions at each time period, and each decision must be made using a different limited observation of the overall system state. The complicating factor in such problems is that the impact of individual decisions is system-wide. We consider the problem of determining decentralized control policies for stochastic systems. Decentralized control problems have been well studied over the past several decades [2], and it is generally recognized that decentralized problems are often considerably more complex than their centralized counterparts. For example, it has been shown in [1] that computing optimal


244

R. Cogill et al.

policies for a simple single stage control problem is NP-hard, whereas the corresponding centralized problem is trivial. This single-stage problem arises as a special case of the problems considered here. With this in mind, our focus is on efficient computation of meaningful suboptimal decentralized control policies. In stochastic control problems, the relevant information required for decision making can be captured by a Q-function. When a Q-function has special structure, we can easily compute a decentralized policy which is greedy with respect to this function. The approach taken here can be interpreted as approximating the optimal Q-function for a problem by one with this special structure. We show that the performance loss associated with choosing policies with respect to an approximate Q-function is related to the approximation error. A related approach for computing suboptimal policies for specially structured multi-agent control problems can be found in [3]. The problems considered there have transition probabilities modeled as a dynamic Bayesian network and structured single-stage costs. The authors show that by using specially structured approximations to the cost-to-go function, the required computations can be simplified. The approach discussed here resembles a method commonly used to synthesize decentralized control policies for linear time-invariant systems [4]. Semidefinite programming conditions can be formulated for these problems which produce stabilizing decentralized control laws when they are satisfied. These conditions involve a simultaneous search for a stabilizing control law and a Lyapunov function proving stability. The Lyapunov functions used are restricted to have special structure, where the structure is chosen to facilitate the search for a decentralized control law.

2 Dynamic Programming Background Here we consider discrete-time stochastic control problems. The systems considered here have a finite state space X , and a finite set U of actions available at each time step. Taking action u ∈ U when in state x ∈ X incurs a cost g(x, u). After taking action u in state x, the system state in the next time period is y ∈ X with probability p(y |x, u). The goal is to find a rule for choosing actions which minimizes some measure of the overall cost incurred. A rule for choosing actions is commonly referred to as a policy. A static state-feedback policy µ : X → U is a rule which chooses an action based on the current system state. Here we consider the problem of choosing a policy to minimize the expected total discounted cost : J µ (x) = E

∞ t=0

αt g(xt , µ(xt )) x0 = x ,

An Approximate Dynamic Programming Approach

245

where 0 ≤ α < 1. For any policy µ, we can compute the cost-to-go J µ by solving the linear equations J µ (x) = g(x, µ(x)) + α

p(y|x, µ(x))J µ (y).

(1)

y∈X

We can write these equations in matrix-vector notation as J µ = gµ + αPµT J µ . Here, gµ is the cost vector evaluated for the policy µ, Pµ is the state transition matrix associated with the policy µ, and PµT is its transpose. We can further simplify this equation using the shorthand J µ = Tµ J µ , where the operator Tµ maps a vector J to the vector gµ + αPµT J. This notation will be convenient in later sections. For the expected total discounted cost criterion, there is a single policy which minimizes the cost-to-go for all initial states. Also, the minimum costto-go J ∗ is unique and satisfies Bellman’s equation   J ∗ (x) = min g(x, u) + α u∈U

p(y|x, u)J ∗ (y) .

(2)

y∈X

We can simplify notation by writing this equation as J ∗ = T J ∗ , where the operator T maps a vector J to the vector T J defined as   (T J)(x) = min g(x, u) + α u∈U

p(y|x, u)J(y) . y∈X

It is often useful to work directly with the expression in the right hand side of Bellman’s equation. This is often referred to as a Q-function, and the optimal Q-function Q∗ is given by Q∗ (x, u) = g(x, u) + α

p(y|x, u)J ∗ (y). y∈X

The optimal policy µ∗ is obtained from Q∗ by taking µ∗ (x) = argminu Q∗ (x, u). Like cost-to-go functions, we can compute the Q-function associated with a particular policy µ by solving the linear equations Qµ (x, u) = g(x, u) + α

p(y|x, u)Qµ (y, µ(y)). y∈X

We can introduce an operator Fµ and write the equations above simply as Qµ = Fµ Qµ . Also, Bellman’s equation can be expressed directly in terms of Q functions as Q∗ (x, u) = g(x, u) + α

p(y|x, u) min Q∗ (y, w) , y∈X

w

(3)

246

R. Cogill et al.

with Q∗ being the unique solution. As before, we can express this equation in the notation Q∗ = F Q∗ , where the operator F maps a vector Q into a vector F Q satisfying p(y|x, u) min Q(y, w)

(F Q)(x, u) = g(x, u) + α y∈X

w

Q-functions will play an important role in the upcoming material. We will conclude this section by mentioning that the operators Tµ , T , Fµ , and F have some important properties [5]. The following properties, presented only in terms of F for convenience, hold for Tµ , T , Fµ , and F : • •

Contractiveness: For any Q1 and Q2 , F Q1 − F Q2 ∞ ≤ α Q1 − Q2 ∞ . As a result of the contraction mapping theorem, F has a unique fixed point Q∗ which is obtained as the limit of the sequence {F n Q} for any Q. Monotonicity: For any Q1 and Q2 , if Q1 ≤ Q2 , then F Q1 ≤ F Q2 .

These properties will be used in the proofs of some upcoming theorems.

3 Computing Decentralized Policies The problem of computing decentralized policies is considerably more complex than the corresponding problem of computing centralized policies. For example, we can consider the simple special case where α = 0. In this case, the optimal centralized policy is simply µ∗ (x) = argminu g(x, u). On the other hand, we can consider a simple decentralized variant of this problem where the state is described by two state variables x1 and x2 and each action is described by two decision variables u1 and u2 . The desired decentralized policy µ chooses u1 based only on x1 and chooses u2 based only on x2 . Unlike the centralized problem, there may not be one decentralized policy which achieves lower cost than all other decentralized policies at all states. Therefore, the problem of minimizing expected cost may not be well defined until we specify an initial probability distribution on the states. Once an initial distribution is specified, the resulting optimization problem is NP-hard [1]. This example highlights the importance of efficient algorithms for computing meaningful suboptimal policies for decentralized control problems. In the systems we consider, the action taken at each time period can be described by a collection of decision variables as u = (u1 , . . . , uk ). The action space for this system is U = U1 × · · · × Uk . Likewise, the state of the system is described by a collection of state variables and the corresponding state space is X = X1 ×· · ·×Xp . In general, the policies produced by dynamic programming determine each action ui based on the entire state x = (x1 , . . . , xp ). However, we are interested in computing policies where each decision is only based on a subset of the state variables. A decentralized policy is a policy in which each action ui depends only on some specified subset of the state variables x1 , . . . , xp .


247

The required dependencies of actions on state variables is typically referred to as the information structure of the desired policy. We will use the following notation to describe the information structure of a particular policy. The set of variables which may be used to decide action i is denoted as Ii = {j | ui depends on xj }. In particular, Ii gives the indices of the state variables that action i depends on. The Cartesian product of the spaces of the variables used to decide action i is denoted as Xj . Xi =

×

j∈Ii

Using this notation, a decentralized policy is a set of functions µi : Xi → Ui for i = 1, . . . , k. Recall that the optimal centralized policy is µ∗ (x) = argminu Q∗ (x, u), where Q∗ is obtained as the unique solution to Bellman’s equation. Consider a k policy µ(x) = argminu Q(x, u), where Q is a function of the form Q = i=1 Qi and each Qi : Xi × Ui → R. Since ui is the only decision variable appearing in the argument of Qi , choosing u to be argminu Q(x, u) is equivalent to choosing each ui to be argminui Qi (x, ui ). Also, since the only state variables appearing in the argument of Qi are those in Xi , the choice of ui only depends on the values of the variables in Xi . Therefore, the policy µ is decentralized since each decision ui is determined only in terms of the state variables in Xi . This suggests an algorithm, in which an appropriately structured Q is chosen to approximate Q∗ , and a decentralized policy is obtained from Q. Such an algorithm will be presented in the next section. Theorem 1 justifies this approach by showing that the suboptimality of the performance achieved by µ is related to the approximation error between Q and Q∗ . Before presenting Theorem 1, we will introduce some notation. Let Qµ denote the function Q evaluated for the policy µ. That is, Qµ (x) = Q(x, µ(x)). We must be careful to make the distinction between Qµ : X → R, the function Q evaluated for the policy µ, and Qµ : X × U → R, the Q-function associated with following the policy µ from time t = 1 onward. Also, let · 1,ν denote the weighted 1-norm, defined as J

1,ν

=

ν(x)|J(x)|, x

where the weights ν are positive. For the upcoming theorem, we will restrict ourselves, without loss of generality, to weights which satisfy x ν(x) = 1. Theorem 1. Suppose Q satisfies Q ≤ F Q. If µ is the policy defined as µ(x) = argminu Q(x, u), then the following bound holds Jµ − J∗

1,ν

≤

1 Q∗ − Qµ 1−α µ

1,ω

,

248

R. Cogill et al.

where ν is an arbitrary probability distribution on X and ω is the probability distribution defined as ω = (1 − α)(I − αPµ )−1 ν. Proof. First we will show that ω is a probability distribution. Let e denote the vector with 1 for each entry. Since since eT ν = 1 and eT Pµt = eT for all t, eT ω = (1 − α)eT (I − αPµ )−1 ν ∞

= (1 − α) t=0 ∞

= (1 − α)

αt eT Pµt ν αt eT ν

t=0

=1

Also, since Pµ ≥ 0 and ν ≥ 0 componentwise, ω ≥ 0. Recall that we require Q ≤ F Q. By monotonicity of F , this implies Q ≤ F n Q for all n. Since F n Q → Q∗ for any Q, this implies Q ≤ Q∗ . Also, the policy µ is greedy with respect to Q, so Qµ ≤ J ∗ ≤ J µ . Therefore, Jµ − J∗

1,ν

≤ J µ − Qµ 1,ν = ν T ((I − αPµT )−1 gµ − Qµ ) = ν T (I − αPµT )−1 (gµ − (I − αPµT )Qµ ) 1 ω T ((gµ + αPµT Qµ ) − Qµ ) = 1−α

Since µ is greedy with respect to Q, we have (F Q)µ = gµ +αPµT Qµ . Therefore, the inequality above can be expressed as Jµ − J∗

1,ν

≤

1 (F Q)µ − Qµ 1−α

1,ω

Since Qµ ≤ (F Q)µ ≤ Q∗µ , we have the result Jµ − J∗

1,ν

≤

1 Q∗ − Qµ 1−α µ

1,ω

A theorem similar to Theorem 1, based only on cost-to-go functions, can be found in [6].

4 The Linear Programming Approach It is well known that the solution to Bellman’s equation J ∗ can be obtained by solving a linear program [7]. We can determine Q∗ (x, u) = g(x, u) +


249

α y∈X p(y|x, u)J ∗ (y) once J ∗ is known. In the previous section, we outlined a method for computing a decentralized policy based on an approximation of Q∗ . In this section, we present an algorithm for computing such an approximation. This algorithm is based on an extension of the linear programming approach to solving Bellman’s equation. The standard linear programming formulation is extended to explicitly include Q. The need for including Q will become clear shortly. Theorem 2. The solutions to the equations (2) and (3) can be obtained by solving the linear program maximize:

x∈X

u∈U

Q(x, u)

subject to: Q(x, u) ≤ g(x, u) + α

y∈X

p(y|x, u)J(y)

J(x) ≤ Q(x, u)

for all x ∈ X , u ∈ U for all x ∈ X , u ∈ U

Proof. Q∗ satisfies Q∗ (x, u) = g(x, u) + α

p(y|x, u) min Q∗ (y, w) . y∈X

w

Also, J ∗ (x) = minu Q∗ (x, u) ≤ Q∗ (x, u) for all x, u. Therefore, Q∗ and J ∗ are feasible for this LP. Also, since J(x) ≤ Q(x, u) for any feasible J, any feasible Q satisfies Q(x, u) ≤ g(x, u) + α

p(y|x, u)J(y) y∈X

≤ g(x, u) + α

p(y|x, u) min Q(y, w) , y∈X

w

or more succinctly Q ≤ F Q. By monotonicity of F , this implies Q ≤ F n Q for all n. Since F n Q → Q∗ for any Q, any feasible Q satisfies Q ≤ Q∗ . Therefore Q∗ is the optimal feasible solution. Note that the usual LP approach to solving Bellman’s equation only involves J, and therefore contains fewer variables and constraints. The explicit inclusion of Q in the linear program is required because we will require Q to have special structure in order to obtain a decentralized policy. In general, requiring J to have some special structure is not sufficient to guarantee that Q will have the required structure. Using this linear program, we have the following algorithm for computing a decentralized policy.

250

R. Cogill et al.

Algorithm 1. k

1. For any information structure, let Q = i=1 Qi and J = Qi : Xi × Ui → R and Ji : Xi → R. 2. For arbitrary positive values ω(x), solve the linear program maximize:

x∈X

u∈U

k i=1

Ji , where

ω(x)Q(x, u)

subject to: Q(x, u) ≤ g(x, u)+α

y∈X

p(y|x, u)J(y) for all x ∈ X , u ∈ U

J(x) ≤ Q(x, u)

for all x ∈ X , u ∈ U,

3. Let µ(x) = argminu Q(x, u). This policy is decentralized with the desired information structure. It is not immediately clear how the optimization problem solved in Algorithm 1 relates to the cost-to-go achieved by µ. We will show that this optimization problem is equivalent to minimizing a weighted norm of the error between Q∗ and Q subject to some constraints. From Theorem 1, this error is directly related to the performance achieved by µ. Theorem 3. The linear program in Algorithm 1 is equivalent to minimize: Q∗ − Q

1,Ω

subject to: Q ≤ F Q, where Ω(x, u) = ω(x) for all x ∈ X and u ∈ U. Proof. From the proof of Theorem 2, Q is feasible if and only if Q ≤ F Q, which implies Q ≤ Q∗ . Therefore, for any feasible Q Q − Q∗

1,Ω

ω(x)(Q∗ (x, u) − Q(x, u))

= x∈X u∈U

=−

ω(x)Q∗ (x, u).

ω(x)Q(x, u) + x∈X u∈U

x∈X u∈U

Since x∈X u∈U ω(x)Q∗ (x, u) is constant, minimizing this quantity is equivalent to maximizing x∈X u∈U ω(x)Q(x, u), the objective in the original LP. From the previous theorem, we see that Algorithm 2 seeks to minimize the approximation error Q − Q∗ 1,Ω subject to the constraint Q ≤ F Q. For any policy µ ˆ and any probability distribution ω, the bound Qµb − Q∗µb 1,ω ≤ Q − Q∗

1,Ω

holds. This, together with Theorem 1, assures us that Algorithm


251

1 attempts to produce a decentralized policy µ with performance close to that of an optimal centralized policy. When no structure is imposed on Q (i.e., the centralized case), the weights ω in the objective function are unimportant since we obtain Q∗ for any positive weights. However, when Q has structure, the weights chosen will affect the quality of the policy obtained. A good choice of weights may be guided by the interpretation of these weights in Theorem 1. In particular, these weights impose a tradeoff in the quality of the approximation across the states. Ideally, we would like to closely approximate states that are visited most often under any policy. The role of these weights are discussed in greater detail in [6]. For highly decentralized information structures, the number of variables in the linear program increases linearly with the number of state variables. Therefore, the number of variables appearing in this linear program is often significantly less than the number of variables required for computation Q∗ . However, the number of constraints appearing in the linear program is still on the order of the number of state-action pairs. The application of Algorithm 1 to large-scale problems may still be prohibitive due to the large number of constraints. Similar problems are encountered in the application of the approximate linear programming methods of [6] to large problems, however constraint sampling has been shown to be an effective method for overcoming this problem [8]. We believe that the analysis and application of constraint sampling as in [8] will carry over to the problems here, although we have not explored the details at this point.

5 Example Here we will apply the algorithm developed in Section 4 to an example problem. The problem considered is the following. Two robots are placed in opposite corners of a region, and calls for services to be performed by the robots originate from various points within the region. This is depicted in Figure 1. Calls for service are sent to both robots. Each robot, if idle, must make a decision of whether or not to service the current location. Each robot must make this decision without knowledge of the other robot’s status (idle or busy), resulting in a decentralized decision making problem. For this problem, it is not clear which strategy the robots should follow. If both robots, when idle, respond to all calls for service, then we will often have both robots responding to calls when only one is required to service the location. On the other hand, if we partition the set of locations and assign each location to a single robot, then locations will often go unserviced when their assigned robot is busy at another location. We will see that both of these strategies are typically outperformed by strategies which assign a single robot to some locations, and both robots to others. In our model, events occur at discrete time periods t = 0, 1, . . .. The system state has three components x = (x1 , x2 , x3 ), where the compo-

252

R. Cogill et al.

Fig. 1. Robots are stationed in the lower left and upper right corners, and respond to calls for service at the locations shown by circles.

nents are described as follows. At time t, robot i is in state xi (t) ∈ Xi = {robot i idle, robot i busy}, depending on whether or not it is idle or currently serving a location. For a problem with N locations, the call for service at time t is x3 (t) ∈ X3 = {location 1, . . . , location N, no service requested}. We make the restriction that only one call for service may be active at a time. At time t, if robot i is idle and there is an active call for service, robot i chooses an action ui (t) ∈ Ui = {respond, don’t respond}. The overall state space for this system is X = X1 × X2 × X3 . The overall action space is U = U1 × U2 . At each time period, calls for service are independent and identically distributed. Therefore, if there is no response to a call, it does not persist and is replaced by a new call (or no call) in the following time period. Let p3 (y3 ) give the probability of calls for service in each location. If a robot is busy servicing a location in any time period, it remains busy in the following period with probability 0.9. Let pi (yi |xi , ui ) give the probability that robot i is busy or idle in the following time period given its current state and action. The dynamics of this system are described by the transition probability function p(y|x, u) = p1 (y1 |x1 , u1 )p2 (y2 |x2 , u2 )p3 (y3 ). The control objective is expressed through costs that are incurred in each time period. Costs incurred for servicing a location are based on the distance required to reach the location and the time required to complete servicing. Upon responding to a location, a cost of 2(distance to location) is incurred by each robot which responds (i.e., the cost of getting there and back). For our example, the region considered is the unit square, and the distance used is the Euclidean distance from the the robot’s corner to the call location. For each time period that a robot is busy, it incurs an additional cost of 0.25 units. These servicing costs for each robot are given by a function gi (xi , ui ).


253

Fig. 2. The decentralized policy computed in Example 1. Lighter shaded circles are served only by robot 1, darker shaded circles are served only by robot 2, and circles with both shades are served by both robots. 7380 computed respond closest respond all

7370

7360

7350

7340

7330

7320

7310

7300

0

5

10

15

20

25

30

35

40

45

Fig. 3. The total discounted cost as a function of state for the policies used in Example 1.

Additionally, if neither robot responds to a call, a large penalty is incurred. In this case, we use a penalty of 8 units. These penalties are given by a function g3 (x1 , x2 , u1 , u2 ). The overall per-period costs for this problem are given by g(x, u) = g1 (x1 , u1 ) + g2 (x2 , u2 ) + g3 (x1 , x2 , u1 , u2 ). Our goal is to determine control policies µ1 : X1 × X3 → U1 and µ2 : X2 × X3 → U2 which make the total discounted cost as small as possible (with α = 0.999).

254

R. Cogill et al.

Fig. 4. The decentralized policy computed in Example 2. Lighter shaded circles are served only by robot 1, darker shaded circles are served only by robot 2, and circles with both shades are served by both robots. 7250 computed respond closest respond all

7240 7230 7220 7210 7200 7190 7180 7170 7160 7150

0

5

10

15

20

25

30

35

40

45

Fig. 5. The total discounted cost as a function of state for the policies used in Example 2.

We applied Algorithm 1 to several instances of this problem with N = 10. The results of the first example are shown in Figures 2 and 3. In this example, a decentralized policy is computed and compared against two heuristics, ‘respond closest’ and ‘respond all’. In the ‘respond closest’ policy, robots only respond to the locations closest to them. In the ‘respond all’ policy, each robot responds to every location. The results of the second example are shown in Figures 4 and 5. It is interesting to note the difference between the perfor-


255

mance of the heuristic policies in the two examples. In particular, relative performance of the two heuristic policies depends on the particular problem instance. The computed policies outperform both heuristics in both examples.

6 Conclusions Here we considered the problem of designing decentralized control policies for stochastic systems. An algorithm based on linear programming was presented. This algorithm obtains a decentralized policy from a function with special structure that approximates the optimal centralized Q-function. The performance loss associated with the resulting decentralized policy was shown to be related to the approximation error.

References 1. J. Tsitsiklis and M. Athans, “On the complexity of decentralized decision making and detection problems,” IEEE Trans. Automatic Control, vol. 30, no. 5, pp. 440–446, 1985. 2. N. Sandell, P. Varayia, M. Athans, and M. Safonov, “Survey of decentralized control methods for large-scale systems,” IEEE Trans. Automatic Control, vol. 26, pp. 108–128, 1978. 3. C. Guestrin, D. Koller, and R. Parr, “Multiagent planning with factored MDPs,” Advances in Neural Information Processing Systems, vol. 14, pp. 1523–1530, 2001. 4. J. Bernussou, P. Peres, and J. Geromel, “Robust decentralized regulation: a linear programming approach,” Proc. IFAC Symposium on Large Scale Systems, vol. 1, pp. 133–136, 1989. 5. D. Bertsekas and J. Tsitsiklis, Neuro-dynamic Programming. Athena Scientific, 1996. 6. D. de Farias and B. V. Roy, “The linear programming approach to approximate dynamic programming,” Operations Research, vol. 51, no. 6, pp. 850–865, 2003. 7. A. Manne, “Linear programming and sequential decisions,” Management Science, vol. 6, no. 3, pp. 259–267, 1960. 8. D. de Farias and B. V. Roy, “On constraint sampling in the linear programming approach to approximate dynamic programming,” to appear in Mathematics of Operations Research, submitted 2001. 9. M. Aicardi, F. Davoli, and R. Minciardi, “Decentralized optimal control of Markov chains with a common past information set,” IEEE Trans. Automatic Control, vol. 32, no. 11, pp. 1028–1031, 1987. 10. R. Cogill, M. Rotkowitz, B. V. Roy, and S. Lall, “An approximate dynamic programming approach to decentralized control of stochastic systems,” Proceedings of the 2004 Allerton Conference on Communication, Control, and Computing, pp. 1040–1049, 2004. 11. Y. Ho, “Team decision theory and information structures,” Proceedings of the IEEE, vol. 68, no. 6, pp. 644–654, 1980.

256

R. Cogill et al.

12. K. Hsu and S. Marcus, “Decentralized control of finite state Markov processes,” IEEE Trans. Automatic Control, vol. 27, no. 2, pp. 426–431, 1982. 13. C. Papadimitriou and J. Tsitsiklis, “Intractable problems in control theory,” SIAM Journal on Control and Optimization, vol. 24, no. 4, pp. 639–654, 1986. 14. ——, “The complexity of Markov decision processes,” Mathematics of operations research, vol. 12, no. 3, pp. 441–450, 1987. 15. M. Puterman, Markov decision processes. John Wiley and Sons, New York, 1994. 16. R. Srikant and T. Ba¸sar, “Optimal solutions in weakly coupled multiple decision maker Markov chains with nonclassical information,” Proceedings of the IEEE Conference on Decision and Control, pp. 168–173, 1989.

An H∞ Loop-Shaping Approach to Steering Control for High-Performance Motorcycles Simos Evangelou1 , David J. N. Limebeer1 , Robin S. Sharp1 , and Malcolm C. Smith2 1

2

Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK. {s.evangelou; d.limebeer; robin.sharp}@imperial.ac.uk Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, UK. [email protected]

Summary. A fixed-parameter active steering compensation scheme that is designed to improve the dynamic behaviour of high-performance motorcycles is introduced. The design methodology is based on the Glover-McFarlane H∞ loop-shaping procedure. The steering compensator so designed, is seen as a possible replacement for a conventional steering damper, or as an alternative to the more recently introduced passive mechanical compensation networks. In comparison with these networks, active compensation has several potential advantages including: (i) the positive-reality of the compensator is no longer a requirement; (ii) it is no longer necessary for the device to be low-order; (iii) in a software implementation, it is easy to adjust the compensator parameters and (iv) an adaptive, or parameter varying version of this scheme is a routine extension. The study makes use of computer simulations that exploit a state-of-the-art motorcycle model whose parameter set is based on a Suzuki GSX-R1000 sports machine. The results extend further the significant improvements achieved in the dynamic properties of the primary oscillatory modes (‘wobble’ and ‘weave’) obtained previously by replacing the conventional steering damper with passive mechanical steering compensation schemes.

1 Introduction The lightly damped modes associated with high-performance motorcycles have a limiting effect on their performance. In extreme cases one or more of these modes may even be unstable. These lightly damped modes can also lead to an undesirable response to road undulations [1]. This citation refers to several accidents in which resonant forcing type phenomena played a role in their initiation. Early theoretical work on bicycle and motorcycle dynamics was confined to the study of small oscillations around a straight-running trim condition [2–5]. In later work these studies, and the models on which they were based, were extended to include the analysis of small perturbations


258

S. Evangelou et al.

from steady-state cornering conditions [6–11]. In addition to this theoretical work, motorcycle oscillations have been widely studied via measurement programs [12–27]. The main lateral oscillations in two-wheeled vehicles are ‘wobble’ and ‘weave’. In straight running, the weave mode is well damped at moderate speeds, but becomes less so as the machine’s forward speed increases. The natural frequency rises from zero at very low speed to somewhere in the range 2-4 Hz, depending on the mass and size of the machine—the lower frequencies corresponding to heavier motorcycles. The only properly documented wobble oscillations involve moderate speeds, although there are many anecdotal accounts of wobble at high speeds [28]. Theoretical results indicate that the torsional stiffness of the motorcycle frame at the steering head determines whether a machine will be prone to wobbling at medium speeds (compliant frame), or at high speeds (stiff frame) [29, 30]. The frequency of the wobble mode is insensitive to speed variations, and is governed primarily by the mechanical trail, the front tire cornering stiffness and the front frame steer inertia. The wobble mode’s frequency is normally in the range 6-9 Hz. Stiff framed machines, being prone to wobbling at high speed, often depend on a steering damper for satisfactory wobble-mode damping. Normally, however, a steering damper will destabilize the high-speed weave mode. In cornering, the lateral modes and the in-plane modes associated with tire deflections and suspension motions become coupled, as was first shown in any detail by Koenen [6]. The motorcycle becomes prone to resonant forcing via regular road undulations when the displacement forcing they produce is tuned to the lightly damped modal frequencies of the machine. Moderate roll angles are likely to represent the worst case conditions [1]. Mechanical steering compensation networks were introduced in [31] as a possible replacement for conventional steering dampers. These networks consist of springs, dampers and inerters, a component recently introduced in [32]. The inerter made it possible to obtain a complete analogy between passive electrical and mechanical components. The force-current mechanical-electrical analogy has the following correspondences: spring↔inductor, damper↔resistor and inerter↔capacitor. The use of the inerter instead of the mass element allows electrical circuit synthesis theory to be applied to mechanical networks with equal facility. These ideas were used in conjunction with classical BodeNyquist ideas to design mechanical compensation schemes that can improve substantially the stability and performance properties of motorcycles [31]. Motivated by this success, a new question arises: “what additional improvement is possible if active control schemes are entertained?”. The primary aim of this research is to answer this question by introducing an active steering control scheme that makes use of H∞ synthesis theory. We begin our study with a review of the essential ingredients of the particular H∞ control synthesis problem that will be used in this study. In Section 3 the background to the motorcycle model is described. Some of the important characteristics of the reference motorcycle-rider system are described in Sec-

An H∞ Loop-Shaping Approach

259

tion 4. The controller design is given in Section 5.1 and the performance of the base motorcycle fitted with the designed controller is evaluated in Section 5.2. Conclusions are drawn in Section 6.

2 Normalized Coprime Factor H∞ Synthesis The active control system design approach used in this paper is based on the robust stabilization of the perturbed normalized coprime factors of a linearized plant model. This method incorporates a loop shaping design procedure to obtain the requisite performance and robust stability features. The key step is the solution of an H∞ optimization problem that guarantees the robust stability of the closed-loop. The theoretical basis of this approach was developed in [33–35]. Robustness optimization for normalized coprime factor perturbations is closely related to robustness optimization in the gap metric [36–38]. 2.1 The Optimization Problem The normalized coprime factor modeling error is used to represent uncertainty in the system model. As compared with the usual additive or multiplicative uncertainty representations, the nominal and perturbed models need not have the same number of right-half-plane poles [39, 40]. In the following analysis, boldface letters are used to represent transfer functions, while normal letters and symbols represent signals. The definitions of H∞ spaces and the norm · ∞ are standard [39, 40]. The small gain theorem and coprime factor representations will also be used [39, 40]. For a given plant with transfer function G, a normalized left coprime factorization of G is given by G = M −1 N in which M , N ∈ H∞ are coprime and satisfy M M ∼ + N N ∼ = I;

(1)

the symbol ∼ denotes the conjugate transpose. The left coprime factor perturbed plant is given by GΔ = (M − ΔM )−1 (N + ΔN ) where ΔM , ΔN ∈ H∞ are unknown modeling perturbations. Assuming that δ = GΔ Ts ,

(2)

with δ and Ts representing the motorcycle’s steering angle and steering torque respectively, one obtains M δ = N Ts + φ (3)

260

S. Evangelou et al.

with φ = ΔN ΔM

Ts . δ

(4)

The influence of external disturbances is introduced by modifying (3) so that:  M δ = N Ndr Ndf

 Ts  dr  + φ df

(5)

in which dr and df are road-induced vertical displacement forcing signals applied to the rear and front wheels respectively of the vehicle. The transfer functions Gdr = M −1 Ndr and Gdf = M −1 Ndf map dr and df to the steering angle δ. In the context of the problem being studied here, the nominal plant transfer function G is obtained by linearizing a nonlinear motorcycle dynamic model. A feedback loop is closed using the controller K such that Ts = Kδ. The resulting situation is illustrated diagrammatically in Figure 1. The aim is to design a controller that attenuates road displacement forcing disturbances and stabilizes the perturbed closed-loop system. With the loop broken at φ, direct calculation shows that   φ Ts K −1 −1  (I − GK) = M Gdr Gdf dr  . I δ df Now consider the problem of finding K to minimize γ such that K (I − GK)−1 M −1 Gdr Gdf I

< γ.

(6)

∞

Assuming (6) holds, (1) implies that K (I − GK)−1 I G I

-pi & ph 0, there exists a finite dimensional relaxation that results in an -suboptimal solution for the original infinite dimensional problem. In Section 4, we give an algorithm to design a suboptimal filter using convex optimization. An example is given in Section 6, and conclusions are drawn in Section 7. The notation is standard. The space RHm×n is all proper, real-rational ∞ functions with poles outside the unit disk. For A ∈ Cm×n , σ ¯ (A) and σ(A) denote the maximum and minimum singular value of matrix A. In a normed space (X, · ), for x ∈ X and r > 0, define B(x, r) := {y ∈ X : y − x < r}. B11 B12 Suppose B and T are linear operators, with B partitioned as B = . B21 B22 If T is dimensioned correctly and I − B11 T is invertible, define Fu (B, T ) := −1 B22 + B21 T (I − B11 T ) B12 .

2 Problem Formulation and Preliminaries We pose a generalized robust “control” problem that does not, in fact, involve feedback around the designed controller. The absence of feedback simply means that genuine feedback control problems are not addressed by this work, but problems such as robust input design/shaping and robust filtering/estimation are covered. The lack of feedback also means that the resulting minimax optimization is convex (though infinite dimensional). The focus of this paper is a design algorithm for such problems, as well as the application to robust H∞ linear estimation. The system considered is shown in Figure 1. All signals are multivariable, though for clarity we will not be overly specific in their dimensions. Nothing is assumed to be scalar, so care in manipulating the noncommuting multivariable operators will be maintained. For ease of expression, we denote the dimensions of y and u as ny and nu , respectively. The system consists of a known, stable system G, in feedback with an unknown matrix Δ and a to-be-designed linear controller F . The unknown matrix Δ takes values in

Optimal, Worst Case Filter Design via Convex Optimization

295

✲ Δ p e ✛

G y

✛ q ✛ ✛

d

u

✲ F Fig. 1. General Robust Design Problem r IrN , δ1c Ik1 , . . . , δSc IkS , Δ1 , . . . , ΔF ] : δir ∈ R, Δ := {diag[δ1r Ir1 , . . . , δN

δic ∈ C, Δi ∈ Cli ×li

(1)

with σ ¯ (Δ) ≤ 1 as an additional constraint. We denote {Δ ∈ Δ : σ ¯ (Δ) ≤ 1} by BΔ . There is an obvious 3-by-3 block partition of G into individual relations between its 3 sets of inputs and 3 sets of outputs. With G already assumed stable, it follows that each Gij ∈ RH∞ (of appropriate dimension). We make two main assumptions: Assumption 1 For all Δ ∈ BΔ , (I − G11 Δ)−1 is stable, i.e., in RH∞ . Assumption 2 For all Δ ∈ BΔ , G33 + G31 Δ (I − G11 Δ) cally zero.

−1

G13 is identi-

The first assumption is standard for most robust filter formulations. The second assumption is the “no feedback around F ” assumption, forcing the problem we consider to be an open-loop problem, which accounts for the persistent use of the word “filter” as opposed to “controller.” Consequently, the closedloop map from d to e, denoted Td→e (G, Δ, F ), is Td→e (G, Δ, F ) = RΔ + UΔ F VΔ , where

−1

RΔ := G22 + G21 Δ (I − G11 Δ) G12 −1 UΔ := G23 + G21 Δ (I − G11 Δ) G13 −1 VΔ := G32 + G31 Δ (I − G11 Δ) G12 .

In most cases we use R (and V , U ) for simplicity, however to emphasize the dependence on Δ or even frequency, we use RΔ or RΔ,θ (and similar for U and V ). Assumption 1 implies that there exist positive constants r¯, u ¯ and v¯ such that for all Δ ∈ BΔ and θ ∈ [0 2π], σ ¯ (RΔ,θ ) ≤ r¯, σ ¯ (UΔ,θ ) ≤ u ¯ and σ ¯ (VΔ,θ ) ≤ v¯. We also make the modest assumption that at some value of Δ ∈ BΔ , U and V are bounded below for all θ ∈ [0 2π], namely Assumption 3 There exists Δ ∈ BΔ and u > 0 and v > 0 such that for all θ ∈ [0 2π], σ(UΔ,θ ) ≥ u and σ(VΔ,θ ) ≥ v.

296

K. Sun and A. Packard

An example of a relevant problem without feedback, consider the robust estimation problem depicted in Figure 2.

d✲

y✲ PΔ

zˆ

F

z

−✲ ✲❄ W +

e ✲

Fig. 2. Uncertain Plant and Filter

Here, an uncertain plant PΔ is driven by a generalized disturbance d. Variable y is measured and processed with the intent to estimating z. The estimation error zˆ − z is weighted by a (frequency-dependent) W , giving the error signal which defines the performance objective. A goal of estimation would be to minimize (by choice of F ) the worst-case (over all Δ ∈ BΔ ) H∞ gain from d → e. In general, the robust design problem is to minimize, by choice of F , the worst-case (over BΔ ) gain of Td→e (G, Δ, F ). In terms of RΔ , UΔ and VΔ , this is inf max RΔ + UΔ F VΔ ∞ (2) F ∈RH∞ Δ∈BΔ

Let Λopt denote the infimum. Given > 0, the objective is to find an F ∈ RH∞ whose cost is within of Λopt . Such an F is called an -suboptimal solution. Definition 1 ( -suboptimal solution). For > 0, a given F ∈ RH∞ is an -suboptimal solution for (2) if maxΔ∈BΔ RΔ + UΔ F VΔ ∞ ≤ Λopt + . The uncertainty model in this paper consists of uncertain real parameters (which are intuitive, and natural to include in an uncertain model) and uncertain complex matrices. Theorems, proofs and algorithms are presented for that set. Nevertheless, some discussion as to the relevance of constant, complexvalued uncertainty is in order. In the case of H∞ performance bounds, uncertain complex matrix uncertainty also addresses more realistic unmodeled dynamics uncertainty. In fact, when considering robustness of stability, and robustness of H∞ performance, norm-bounded, complex matrix uncertainty is mathematically equivalent to linear, norm bounded, time-invariant realrational dynamic uncertainty. Roughly, an uncertain system is robustly stable to constant, complex-valued, norm-bounded (by 1, using singular value) uncertain parameters, if and only if it is robustly stable to linear, time-invariant, real-rational, dynamic uncertainty with H∞ norm less than 1. Similar statements hold true for robust and worst-case performance as well. Detailed theoretical statements can be found in [28] and references therein.


A important observation is that max RΔ + UΔ F VΔ Δ∈BΔ

297

∞

is convex on F ∈ RH∞ . Indeed, F appears affinely within the norm, so for any fixed uncertainty, the objective is a convex function of F . Furthermore, the pointwise supremum of a family of convex functions is convex ( [7]). Therefore, the robust filtering problem in (2) is a convex optimization problem. Though convex, the optimization problem is infinite dimensional: sets BΔ and [0, 2π] are infinite; RH∞ is infinite dimensional. Practically, we can only obtain suboptimal solutions using finite and finite dimensional approximations. Let FΔ denote a finite subset of BΔ , FΘ a finite subset of [0 2π] and Φ := {φ1 , φ2 , . . . , φK } a finite, linear independent set in RH∞ . Define a finite dimensional subspace QΦ K

QΦ :=

F : F (z) =

Qk φk (z), Qk ∈ Rnu ×ny

.

(3)

k=1

For ρ > 0, let Bρ QΦ := {F ∈ QΦ : F ∞ ≤ ρ}, and Qρ,Φ := {F ∈ QΦ : σ ¯ (Qk ) ≤ ρ, k = 1, . . . , K}. A finite dimensional approximation of problem (2), parameterized by ρ, Λρ,Φ,FΔ ,FΘ :=

inf

max max σ ¯ (RΔ,θ + UΔ,θ F (θ)VΔ,θ ) .

F ∈Qρ,Φ Δ∈FΔ θ∈FΘ

(4)

Since Qρ,Φ is compact, and FΔ and FΘ are finite, the infimum is achieved. ∗ Denote a minimizer by Fρ,Φ,F . In Section 4.2, we show that the minΔ FΘ imization can be carried out with semidefinite programming. For this finite dimensional relaxation to be theoretically useful, we need to show that for any > 0, there exists ρ and finite sets FΔ , FΘ and Φ such that the associated ∗ Fρ,Φ,F is -suboptimal for (2). Δ FΘ For the remainder of the paper, we always take φk (z) := z k−1 , and denote ∗ Φ by its cardinality K, using QK , Bρ QK , Qρ,K , Λρ,K,FΔ ,FΘ and Fρ,K,F Δ ,FΘ ∗ (or even more compactly, Λρ,K,F and Fρ,K,F ) in place of the general notation above. Lemma 1. Bρ QK ⊂ Qρ,K ⊂ B√Kρ QK . The proof is given in Appendix 8.1. A continuity result will be used in Section 3. Lemma 2. The mapping σ ¯ R(·,·) : BΔ × [0, 2π] → C is uniformly continuous on BΔ × [0, 2π], with norm σ ¯ (Δ) + |θ|. Moreover, for fixed ρ > 0 and K, the family of mappings {¯ σ (U(·,·) F (·)V(·,·) ) : F ∈ Qρ,K } is uniformly equicontinuous. Proof . To show RΔ,θ is uniformly continuous on BΔ × [0, 2π], it is enough to show that RΔ,θ is continuous on this set, since BΔ × [0, 2π] is compact and σ ¯ (·) is a continuous mapping. Recall that RΔ,θ = G22 (θ) +

298


G21 (θ)Δ (I − G11 (θ)Δ)−1 G12 (θ). By Assumption 1, each individual term is continuous, so R is continuous. Similar arguments show that UΔ,θ and VΔ,θ are uniformly continuous. For given K, we know the family of functions {ej(k−1)θ , k = 1, 2, . . . , K} is equicontinuous. By definition, for any F ∈ Qρ,K , each Qk is uniformly bounded, so the family of functions Qρ,K is also uniformly equicontinuous. So we conclude that the family of functions {UΔ,θ F (θ)VΔ,θ : F ∈ Qρ,K } is uniformly equicontinuous. The following simple lemma gives a procedure for generating finite covers for a compact set in a metric space with a given radius. The proof is omitted. Lemma 3. In a metric space X, given any compact set A ⊂ X and δ > 0. 0 0 does not ⊂ A that contains a finite number of points. If FA Pick any set FA satisfy A ⊂ ∪x∈FA B(x, δ), then find any y ∈ A that y ∈ ∪x∈FA B(x, δ), and k+1 k update FA := FA ∪ {y}. This procedure stops in finite steps and generates a finite cover of A with balls of radius δ. Finally, we make assumptions concerning our algorithmic performance needs of uncertain systems gain analysis. Uncertain systems are represented in linear fractional form, as in Figure 3. We require worst-case gain algorithms with the following convergence property: given S ∈ RH∞ , an uncertain set Δ and > 0, there exist algorithms L and U (which return real numbers) such that L (S, Δ) ≤ max Fu (S, Δ) ≤ U (S, Δ) Δ∈BΔ

and U (S, Δ) − L (S, Δ) < . Algorithm L must also produce a Δ ∈ BΔ and θ ∈ [0, 2π] such that σ ¯ Fu (S(ejθ ), Δ ) ≥ L (S, Δ).

✲ Δ p e ✛

S

✛ q ✛

d

Fig. 3. General Linear Fractional Form

In Appendix II, we outline (tersely and implicitly) a conceptual generalized branch & bound algorithm based on sequences of finite coverings of BΔ with decreasing radii, to compute such bounds. Actual algorithms, combining upper bounds based on LMIs, lower bounds using problem-specific search methods, and practical branch & bound schemes are the subject of other work and not the focus here. Refer to [2], [16], [17] and [31] for more details on this subject.


299

3 -Suboptimal Filters via Finite Dimensional Relaxation In this section, we show that the finite dimensional approximations of equation (4) can be used to get suboptimal solutions of (2). First, modify the original problem in (2) to only consider filters in QK , ΛK := inf

max RΔ + UΔ F VΔ

F ∈QK Δ∈BΔ

(5)

∞

An -suboptimal solution to (5) is any F ∈ QK which achieves a cost within of the infimum, (just as Definition 1 defines suboptimal solutions to (2)). Theorem 1. The infimum for ΛK is achieved. 2¯ r , then using the lower bounds on U Proof . If F ∈ QK satisfies F ∞ > uv and V and the upper bound on R (assumptions 3 and 1), we have

max

Δ∈BΔ

RΔ + UΔ F VΔ

∞

> max RΔ Δ∈BΔ

∞

and hence such an F is “worse” than simply taking F = 0. Consequently, for 2¯ r , all ρ > uv ΛK =

inf

max RΔ + UΔ F VΔ

F ∈Bρ QK Δ∈BΔ

∞

(6)

Therefore, with Bρ QK compact, it follows that the infimum is achieved. ∗ Denote any such minimizer by FK ∈ Bρ QK . Next we show that, given ∗ > 0, there exists K0 < ∞, such that when K > K0 , FK is an -suboptimal solution for (2). To do this, recall a classic problem of uniformly approximating functions in H∞ by functions in QK .

Theorem 2. (from [22, 25]) Suppose r > 1 and η > 0 are given. Let U := {z : |z| < r} ⊂ C denote an open disk. For any X that is analytic on U , and supz∈U σ ¯ (X(z)) ≤ η, then inf F ∈QK X − F ∞ ≤ rηK . Using this result, we can obtain an error bound for the robust filtering by using QK rather than RH∞ : Theorem 3. For every K, Λopt ≤ ΛK . Moreover ΛK → Λopt as K → ∞. Proof . Λopt ≤ ΛK is trivial because RH∞ ⊃ QK . For any > 0, let F 0 ∈ RH∞ be an 2 -suboptimal solution for (2). Since F 0 ∈ RH∞ , it has only finitely many poles outside the unit disk D := {z : |z| < 1}. Hence there exists r > 1 and η > 0 such that F 0 is analytic on rD, and bounded by η. Choose K0 so that rKη 0 < 4¯uv¯ . Then for any K ≥ K0 , by Theorem 2, there exists FK ∈ QK , such that F 0 − FK ∞ < 4¯uv¯ . So, for any Δ ∈ BΔ , we have

300


RΔ + UΔ FK VΔ

∞

= RΔ + UΔ F 0 VΔ + UΔ FK − F 0 VΔ ≤ RΔ + UΔ F 0 VΔ ≤ Λopt + ≤ Λopt +

2

∞

+ UΔ FK − F 0 VΔ ∞

+u ¯v¯ FK − F

0

∞

∞

3 . 4

This is true for all Δ ∈ BΔ , so for all K ≥ K0 ΛK ≤ max RΔ + UΔ FK VΔ Δ∈BΔ

Since

∞

< Λopt + .

is arbitrary, the result follows.

Next, we show that given and K, we can find ρ > 0 and finite sets ∗ FΔ ⊂ BΔ and FΘ ⊂ [0, 2π], such that Fρ,K,F is an -suboptimal solution for (5). Theorem 4. For any and FΘ such that max

Δ∈BΔ

> 0 and K > 0, there exists ρ > 0 and finite sets FΔ ∗ RΔ + UΔ Fρ,K,F VΔ

∞

< Λρ,K,F + .

Remark 1. Additional inequalities (which always hold) are useful to record. Λρ,K,F ≤ ΛK ≤ max

Δ∈BΔ

∗ RΔ + UΔ Fρ,K,F VΔ

∞

< Λρ,K,F + ≤ ΛK +

Theorem 4 ∗ (to show Λρ,K,F ≤ ΛK , notice that FK ∈ Bρ QK , and by Lemma 1, Qρ,K ⊃ Bρ QK , furthermore, fewer constraints are used in obtaining Λρ,K,F ). 2¯ r Proof . Choose ρ > uv . By Lemma 2, there exists δ1 > 0, such that if (Δ1 , θ1 ) − (Δ2 , θ2 ) < δ1 , then σ ¯ (RΔ1 ,θ1 − RΔ2 ,θ2 ) < /2. Similarly, there exists δ2 > 0, such that when (Δ1 , θ1 ) − (Δ2 , θ2 ) < δ2 , then σ ¯ (UΔ1 ,θ1 F (θ1 )VΔ1 ,θ1 − UΔ2 ,θ2 F (θ2 )VΔ2 ,θ2 ) < /2 for all F ∈ Qρ,K . Take M δ := min(δ1 , δ2 ). Consider any finite sets FΔ = {Δi }i=1 ⊂ BΔ , and N δ FΘ = {θj }j=1 ⊂ [0, 2π], such that. 2 -balls around these finite sets cover BΔ and [0 2π],

BΔ ⊂

B(Δi , δ/2), 1≤i≤M

[0, 2π] ⊂

B(θj , δ/2). 1≤j≤N

Clearly, for any F ∈ Qρ,K , max max σ ¯ (RΔ,θ + UΔ,θ F (θ)VΔ,θ ) ≤ max RΔ + UΔ F VΔ

Δ∈FΔ θ∈FΘ

Δ∈BΔ

∞

.

Then, for every Δ ∈ BΔ and every θ ∈ [0 2π], there exist Δi ∈ FΔ , θj ∈ FΘ such that Δ ∈ B(Δi , δ2 ) and θ ∈ B(θj , 2δ ). It follows that for all F ∈ Qρ,K ,


301

σ ¯ RΔ,θ + UΔ,θ F (θ)VΔ,θ − RΔi ,θj − UΔi ,θj F (θj )VΔi ,θj =σ ¯ RΔ,θ − RΔi ,θj + UΔ,θ F (θ)VΔ,θ − UΔi ,θj F (θj )VΔi ,θj ≤σ ¯ RΔ,θ − RΔi ,θj + σ ¯ UΔ,θ F (θ)VΔ,θ − UΔi ,θj F (θj )VΔi ,θj < . (7) Summarizing (and rearranging) – for every Δ ∈ BΔ and θ ∈ [0 2π], there exist Δi ∈ FΔ , θj ∈ FΘ such that for all F ∈ Qρ,K σ ¯ [RΔ,θ + UΔ,θ F (θ)VΔ,θ ] < σ ¯ RΔi ,θj + UΔi ,θj F (θj )VΔi ,θj + . Maximizing over each set yields that for all F ∈ Qρ,K max RΔ + UΔ F VΔ

Δ∈BΔ

∞

< max max σ ¯ [RΔ,θ + UΔ,θ F (θ)VΔ,θ ] + . Δ∈FΔ θ∈FΘ

(8)

∗ Use F = Fρ,K,F to obtain the result.

To conclude the section, combine Theorem 3 and 4, along with the proof of Theorem 4, yielding Theorem 5. For any > 0, there exists ρ > 0 and K0 < ∞, and for any K > K0 , there exists δ > 0, such that for any finite sets FΔ ⊂ BΔ and FΘ ⊂ [0, 2π] satisfying ∪Δ∈FΔ B(Δ, δ) ⊃ BΔ and ∪θ∈FΘ B(θ, δ) ⊃ [0, 2π], it follows that ∗ max R + U Fρ,K,F V ∞ ≤ Λopt + . Δ∈BΔ

Based on , this gives a criterion for choosing filter order K and finite sets ∗ is -suboptimal. In practice though, this criterion FΔ and FΘ such that Fρ,K,F is conservative (regarding δ and K0 ) and not constructive. In Section 4, we propose a successive finite dimensional approximation to carry out the design.

4 Design Algorithms and Some Error Bounds In this section, a successive design procedure is proposed, and sets FΔ and FΘ will be refined in each iteration. 4.1 Fixed Order Filter Design In this section, we propose a robust filter design algorithm for a given filter order K. For any given > 0, the algorithm yields an -suboptimal filter to the problem in equation (5). The basic idea is to start with FΔ and FΘ having ∗ a few elements, and design Fρ,K,F via the solution to a finite dimensional convex optimization problem. Then check the error bounds. If they are close to zero, stop. Otherwise, add some points to FΔ and FΘ , and design the filter again. This process is repeated until the desired tolerance is reached.

302


Algorithm 1 (Fixed order robust filter design algorithm) Initialization: Filter order K is given, pick > 0 as the desired 2¯ r suboptimal tolerance; set ρ > uv ; pick finite sets FΔ ⊂ BΔ and FΘ ⊂ [0, 2π]; ∗ ; Step 1: Solve problem (4) with current FΔ and FΘ , yielding Fρ,K,F ∗ Step 2: Apply worst case gain analysis on RΔ +UΔ Fρ,K,F VΔ with a tolerance ∗ VΔ , Δ , and define Γ := of 0 := 4 . Let γK,F := U 0 RΔ + UΔ Fρ,K,F γK,F − Λρ,K,F . If Γ ≤ , exit; otherwise, add the points (from L 0 ) Δ 0 and θ 0 to FΔ and FΘ , and repeat Step 1; The explicit formulation of Step 1 as a finite-dimensional SDP, and convergence of this algorithm are addressed in the following subsections. This algorithm can be thought of as a generalization of the cutting plane method, see [23] and [20]. Its convergence is guaranteed, but there are no bounds on the rate-of-convergence. 4.2 Casting the Finite-Dimensional Problem (Equation 4) as an SDP The finite dimensional problem of equation (4) is convex, and can be formulated as a semidefinite program, [6], using standard linear algebra to convert σ ¯ constraints into matrix definiteness constraints. The finite dimensional space QK is defined in (3). Let QK := {Q1 , . . . , QK } ∈ RK×nu ×ny denote the “vector” of coefficients. Enumerate the elements of FΔ as {Δ1 , . . . , ΔM } and the elements of FΘ as {θ1 , . . . , θN }. Problem (4) can be written as an SDP, min

t,QK ∈R

t

s.t. Gmn (t, QK ) QTk

ρI Qk ρI

(9) 0 ∀ m = 1, 2, . . . , M and n = 1, 2, . . . , N 0

∀ k = 1, 2, . . . , K.

The matrix Gmn (t, QK ) is given by Gmn (t, QK ) :=

tI Pmn (QK )∗ K Pmn (Q ) tI

where Pmn is Pmn (QK ) := RΔm ,θn + UΔm ,θn

K

Qk φk (θn ) VΔm ,θn . k=1

∗ . Solving (9) yields t∗ = Λρ,K,FΔ ,FΘ , along with the associated Fρ,K,F Δ ,FΘ

Remark 2. This conversion of an affine σ ¯ (·) minimization into an SDP is common in robust control. The SDP in equation (9) is readily solved ( [29]).


303

4.3 Bounds via Worst Case Analysis We next verify that the stopping criteria of Step 1 and 2 will eventually be met. Lemma 4. For any iterations.

> 0, Algorithm 1 will terminate in a finite number of

Proof . We need to show Γ ≤ 2 in finite iterations as steps 1-2 of Algorithm 1 are repeated. Notice that (dropping arguments for clarity) Γ = (γK,F − L 0 )+ (L 0 − ΛK,F ) . The first term U

0

∗ VΔ , Δ − L RΔ + UΔ FK,F

0

∗ VΔ , Δ RΔ + UΔ FK,F

0, TM 2∞ < γ for all M ∈ M, if there exists Pi 0, i = 1, . . . , N , and G, such that   Pi Ai G Bi 0  GT ATi G + GT − Pi 0 GT CiT    0. (12)  BT 0 I DiT  i 0 Ci G Di γI N

N

Proof . Let P (ξ) := i=1 ξi Pi , where i=1 ξi = 1 and ξi ≥ 0. Since P (ξ) 0, we have (P (ξ) − G)T P (ξ)−1 (P (ξ) − G) 0, and hence GT P (ξ)−1 G G+ T G − P (ξ). So when (12) is true, we have   P (ξ) A(ξ)G B(ξ) 0  GT A(ξ)T GT P (ξ)−1 G 0 GT C(ξ)T     B(ξ)T 0 I D(ξ)T  0 C(ξ)G D(ξ) γI   P (ξ) A(ξ)G B(ξ) 0  GT A(ξ)T G + GT − P (ξ) 0 GT C(ξ)T    0  B(ξ)T 0 I D(ξ)T  0

C(ξ)G

D(ξ)

γI

Multiplying above inequality by T := diag(I, P (ξ)G−T , I, I) from left and T T from right, we get   P (ξ) A(ξ)P (ξ) B(ξ) 0  P (ξ)A(ξ)T P (ξ) 0 P (ξ)C(ξ)T    0.  B(ξ)T 0 I D(ξ)T  0 C(ξ)P (ξ) D(ξ) γI By Lemma 6, we conclude that TM

2 ∞

< γ for all M ∈ M.

8.4 Filtering Result In this section, the goal is to transform the robust H∞ filtering problem into LMIs. To this end, some change of variables are necessary:


G :=

311

Y ? Z −1 ? . , G−1 := V ? U ?

Notice that we can always calculate blocks “?” in order to have GG−1 = I. Also introduce the following: Af Bf Cf 0

VT 0 := 0 I

−1

QF H 0

UZ 0 0 I

−1

.

Let R := V T U Z, and we are ready for the main result: Theorem 6. For γ > 0, the estimation error transfer function TM 2∞ < γ for all M ∈ M, if the following LMI is satisfied   Pi Ji Z T Ai Z T Ai Z T Bi 0  ∗∗ S Y T A + F C + Q  Y T Ai + F Ci Y T Bi + F D i 0 i i i    ∗∗ ∗ T T T T T  Z + Z − Pi Z + Y + R − Ji 0 L −H     0,  ∗∗ ∗  ∗ Y + Y T − Si 0 LT    ∗∗ ∗  ∗ ∗ I 0 ∗∗ ∗ ∗ ∗ ∗ γI where matrices Q ∈ Rn×n , H ∈ Rl×n , F ∈ Rn×r , R, Z, Y ∈ Rn×n and Pi = PiT , Si = SiT , Ji ∈ Rn×n for i = 1, . . . , N . And the filter parameters are given by Af = QR−1 , Bf = F, Cf = HR−1 . The proof of Theorem 6 can be done by mimicing that of Theorem 5.1 in [11]. 8.5 Worst-Case Analysis Consider an uncertain linear system as shown in Figure 3. With S fixed, a state-space model of the uncertain relation between d and e is of the form 

    xk+1 A B1 B2 xk  pk  =  C1 D11 D12   qk  ek C2 D21 D22 dk

qk = Δpk

where the entries A, B1 , . . . , D22 depend on the state-space models of S. We assume that this uncertain system is well-posed (unique solutions to xk+1 , pk , ek , qk for any xk , dk ) and exponentially stable for all Δ ∈ BΔ . Define a constant matrix H by the partition implied below   A B1 B2 H11 H12   :=  C1 D11 D12  H21 H22 C2 D21 D22

312


and Δe := {diag [δc I, Δ] : δc ∈ C, Δ ∈ Δ}. Then well-posedness and stability are equivalent to (see [18]) det (I − H11 Δe ) = 0, ∀Δe ∈ BΔe (note: this is equivalent to the spectral radius of A, ρ(A) being less than 1, and I − −1 D11 + C1 ejθ I − A B1 Δ being nonsingular for all 0 ≤ θ ≤ 2π, Δ ∈ BΔ ; or, equivalently, ρ(A) < 1 and I − D11 + C1 (zI − A) for all Δ ∈ BΔ ). Furthermore, max

Δ∈BΔ

Fu (G, Δ)

∞

−1

B1 Δ

−1

∈ RH∞

= max σ ¯ (Fu (H, Δe )) Δe ∈BΔe

So, the worst case H∞ gain of an uncertain dynamic system is the worst case gain of an uncertain constant matrix. Since Δe is also of the general form for Δ in equation (1), we simply focus on a general problem of the form: given a set Δ as in (1), and a complex matrix M , with I − M11 Δ nonsingular for all Δ ∈ BΔ , develop computable, converging lower and upper bounds for the ¯ (Fu (M, Δ)). Note that since BΔ is compact, it worst-case gain, maxΔ∈BΔ σ follows that (I − M11 Δ)−1 and Fu (M, Δ) are bounded on Δ ∈ BΔ . 8.6 Branch and Bound Algorithm We outline the necessary continuity arguments which ensure that “branch & bound”-like algorithms will succeed in tightly estimating the worst-case gain. For c ∈ BΔ and r > 0, define Qc,r := {x : x = c + rΔ, Δ ∈ BΔ } to denote a ball in Δ (possibly spilling out of BΔ ) with center c and radius r > 0. With M fixed, define Lc,r := σ ¯ (Fu (M, c)) and Uc,r :=

r + inf α 2 α>0 1

1

subject to σ ¯

1

r 2 M11 (I − cM11 )−1 r 2 r 2 (I − M11 c)−1 M12 1 1 −1 21 r α M21 (I − cM11 ) α Fu (M, c)

< 1. (13)

Since (I − M11 c)−1 is bounded on c ∈ BΔ , it follows that for small enough r, Uc,r is finite for all c ∈ BΔ , and moreover Lc,r ≤

max

Δ∈Qc,r ∩BΔ

σ ¯ [Fu (M, Δ)] ≤ max σ ¯ [Fu (M, Δ)] ≤ Uc,r . Δ∈Qc,r

To see this, note that if α > 0 is feasible in the definition of Uc,r , then for all Δ ∈ BΔ it follows (eg., [18]) that σ ¯ [Fu (M, c + rΔ)] < α. The bound on Lc,r is trivial. In a branch & bound approach, we cover BΔ with a finite union of these balls, BΔ ⊂ ∪N i=1 Qci ,ri , and define bounds associated with the particular covering as L := max Lci ,ri ≤ max σ ¯ [Fu (M, Δ)] ≤ max Uci ,ri =: U 1≤i≤N

Δ∈BΔ

1≤i≤N


313

(the element of BΔ that achieves L is simply the ci value associated with the maximization). If it is true that Uc,r − Lc,r → 0 uniformly on c ∈ BΔ , as r → 0, then one guarantees ( [2]) that L and U will converge as the covering becomes finer (ie., as the covering radii shrink). A proof verifying the uniform convergence of Uc,r − Lc,r in our situation is given below. Theorem 7. Given > 0, there exists δ > 0, such that if r < δ, c ∈ BΔ , then Uc,r − Lc,r < . Proof . Since σ ¯ (Fu (M, c)) is bounded on c ∈ BΔ , given any > 0, there exist γ < 1 such that for all c ∈ BΔ , σ ¯ (Fu (M, c)) ≤ σ ¯ (Fu (M, c)) +

2

γ = Lc,r +

2

γ.

(14)

It is always true that 1

1

1

r 2 M11 (I − cM11 )−1 r 2 r 2 (I − M11 c)−1 M12 1 1 −1 12 r Lc,r + M21 (I − cM11 ) Lc,r + Fu (M, c)

σ ¯

2

2

1 2

−1

≤σ ¯ r M11 (I − cM11 ) σ ¯

r

1 2

1 2

+σ ¯ r (I − M11 c)−1 M12 +

1 1 M21 (I − cM11 )−1 r 2 Lc,r + 2

+σ ¯

1 Fu (M, c) . (15) Lc,r + 2

Since σ ¯ ((I − M11 c)−1 ) and σ ¯ ((I − cM11 )−1 ) are bounded on c ∈ BΔ , and 2 1 Lc,r + 2 is bounded on c ∈ BΔ by , there exists δ > 0 (and without loss of generality, δ < ), such that if r ≤ δ, 1

1

1

¯ r 2 (I − M11 c)−1 M12 σ ¯ r 2 M11 (I − cM11 )−1 r 2 + σ +¯ σ

1 1 M21 (I − cM11 )−1 r 2 Lc,r + 2

< 1 − γ.

Substituting above and (14) into (15), we have 1

1

σ ¯

1

r 2 M11 (I − cM11 )−1 r 2 r 2 (I − M11 c)−1 M12 1 1 −1 12 r Lc,r + M21 (I − cM11 ) Lc,r + Fu (M, c) 2

< (1 − γ) + γ = 1.

2

So Lc,r + 2 ≥ Uc,r − 2r , hence Uc,r −Lc,r ≤

r 2+2

< and the proof is complete.

References 1. B. Anderson and J. B. Moore. Optimal Filtering. Prentice–Hall, 1979. 2. V. Balakrishnan, S. Boyd, and S. Balemi. Branch and bound algorithm for computing the minimum stability degree of parameter-dependent linear systems. Technical report, ISL, Stanford Univ, 1991.

314


3. G. J. Balas, J. C. Doyle, K. Glover, A. K. Packard, and R. Smith. µ-Analysis and Synthesis Toolbox: For Use with MATLAB. The MathWorks, Inc., 3 edition, 2001. 4. D. P. Bertsekas and I. B. Rhodes. Recursive state estimation for a setmembership description of uncertainty. IEEE Trans. Auto. Contr., AC16(2):117–128, April 1971. 5. S. Boyd, C. Barratt, and S. Norman. Linear controller design: limites of performance via convex optimization. Proceedings of the IEEE, 78(3):529–574, March 1990. 6. S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities in System and Control Theory. SIAM, 1994. 7. S. Boyd and L. Vandenberghe. Convex Optimization. 2002. Available via: http://www.stanford.edu/~boyd. 8. M. Dahleh and I. Diaz-Bobillo. Control of Uncertain Systems: A Linear Programming Approach. Prentice Hall, NJ, 1995. 9. M. Fu, C.E. de Souza, and Zhi-Quan Luo. Finite-horizon robust Kalman filter design. IEEE Trans. Signal Proc., 49(9):2103–12, 2001. 10. J. C. Geromel, J. Bernussou, G. Garcia, and M. C. De Oliveira. H2 and H∞ robust filtering for discrete-time linear systems. SIAM J. Contr. Optim., 38(5):1353–1368, 2000. 11. J. C. Geromel, M. C. De Oliveira, and J. Bernussou. Robust filtering of discretetime linear systems with parameter dependent lyapunov functions. SIAM J. Contr. Optim., 41(3):700–711, 2002. 12. A. Ghulchak and A. Rantzer. Robust control under parametric uncertainty via primal-dual convex analysis. IEEE Trans. Auto. Contr., 47(4):632–636, April 2002. 13. A. Ghulchak and A. Rantzer. Robust control under parametric uncertainty via primal-dual convex analysis. SIAM J. Control Optim., 41(1):253–277, January 2002. 14. J. L¨ ofberg. Yalmip: Yet another LMI parser, 2003. Available via: http://www.control.isy.liu.se/~johanl/yalmip.html. 15. K. M. Nagpal and P. P. Khargonekar. Filtering and smoothing in an H∞ setting. IEEE Trans. Auto. Contr., 36(2):152–166, February 1991. 16. M.P. Newlin and M.P. Young. Mixed mu problems and branch and bound techniques. Int. J. Rob. Nonlinear Contr., 7(2):145–64, 1997. 17. A. Packard, G. Balas, R. Liu, and J. Shin. Results on worst-case performance assessment. In Proc. Amer. Conf. Contr., pages 2425–2427, Chicago, Il, 2000. 18. A. Packard and J. C. Doyle. The complex structured singular value. Automatica, 29:71–109, 1993. 19. F. Paganini and A. Giusto. Robust synthesis of dynamic prefilters. In Proc. Amer. Conf. Contr., pages 1314–1318, June 1997. 20. P. Parrilo. On the numerical solution of LMIs derived from the KYP lemma. In Proc. Conf. Decision Contr., pages 2334–2338, Phoenix, Ar, 1999. 21. I. R. Petersen and A. V. Savkin. Robust Kalman Filtering for Signals and Systems with Large Uncertainties. Birkhauser, Boston, 1999. 22. A. Pinkus. n-Widths in Approximation Theory (Ergebnisse der Mathematik und ihrer Grenzgebiete). Berlin: Springer-Verlag, 1985. 23. E. Polak. Optimization: Algorithms and Consistent Approximations. Springer, 1997.


315

24. A. H. Sayed. A framework for state-space estimation with uncertain models. IEEE Trans. Auto. Contr., 46(7):998–1013, July 2001. 25. C. W. Scherer. Multiobjective H2 /H∞ control. IEEE Trans. Auto. Contr., 40(6):1054–1062, June 1995. 26. J. F. Sturm. Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optimization Methods and Software, 11–12:625–653, 1999. Version 1.05 available from http://fewcal.kub.nl/sturm. 27. K. Sun and A. K. Packard. Optimal worst-case H∞ filtering via convex optimization. In Proc. Conf. Decision Contr., pages 1380–1385, 2003. 28. A. Tits and M. Fan. On the small-µ theorem. Automatica, 31(8):1199–1201, 1995. 29. L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review, 38(1):49–95, March 1996. 30. P.M. Young. Controller design with real parametric uncertainty. Int. Jour. Contr., 65(3):469–509, 1996. 31. P.M. Young, M.P. Newlin, and J.C. Doyle. Computing bounds for the mixed mu problem. Int. J. Rob. Nonlinear Contr., 5(6):573–590, 1995.

Distance Measures, Robust Stability Conditions and Robust Performance Guarantees for Uncertain Feedback Systems George Papageorgiou1 and Alexander Lanzon2 1 2

Honeywell Laboratories, 3660 Technology Drive, Minneapolis, MN 55418, USA. Research School of Information Sciences and Engineering, The Australian National University, Canberra ACT 0200, Australia and National ICT Australia Ltd., Locked Bag 8001, Canberra ACT 2601, Australia.

Summary. Given a nominal plant, a perturbed plant, an uncertainty structure and performance weights, we use robust model validation ideas to define and compute a measure of the distance between the nominal and perturbed plants. We also define a stability margin for a feedback system that is related to robust stability and nominal performance, and derive conditions for the stability and bounds for the performance degradation of the perturbed feedback system in terms of the distance measure. These robust stability and robust performance results give the distance measure a feedback interpretation. The simplicity and power of our procedure for computing the distance between two systems is illustrated using a normalized coprime factor uncertainty model to derive results that have already been published in the literature using different techniques. All systems considered in this paper are linear timeinvariant.

Keywords: Robust stability, robust performance, distance measures, model validation, feedback systems, gap metric, ν-gap matric, H∞ control, stability margin

1 Introduction When asked to design a controller for a plant, a control engineer will typically identify a number of plant models that approximate the true plant dynamics, and design a controller that achieves the desired robustness and performance objectives with all the plant models. Let us consider an example. The current standard for designing flight controllers is to divide the flight envelope of an aircraft into a number of flight conditions, design a linear time-invariant (LTI) controller for each flight condition and schedule the gains of the LTI controllers to give a full envelope flight controller. The LTI controllers are typically not scheduled on weight and B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 317–344, 2006. © Springer-Verlag Berlin Heidelberg 2006

318

G. Papageorgiou and A. Lanzon

center-of-gravity location, at least for civil aircraft. At each flight condition, the engineer is given a number of aircraft models that correspond to different combinations of weight and center-of-gravity location, and is asked to design a LTI controller for all combinations. A control engineer can handle more than one plant models in at least two different ways. The first way is current practice in industry: 1. Use engineering experience to choose a nominal plant model; 2. Design a controller that achieves the desired robustness and performance objectives with the nominal plant model; 3. Check robustness and performance with all other plant models; and 4. Iterate if necessary. This approach relies on the engineer’s experience for the choices of a nominal plant model and a measure of robustness. An alternative approach that can be more systematic, is to embed all the plant models into a family of plants and then design a controller for the family. A family of plants is characterized by a nominal plant model, an uncertainty structure and one or more weighting functions that specify the desired performance. It must be emphasized that even though our definition of a family of plants has been taken from H∞ control, the controller for the family does not have to be designed using H∞ synthesis. In this paper, given a nominal plant model, an uncertainty structure, weighting functions and a perturbed plant model (all systems are assumed to be LTI), we develop a generic procedure for computing the size of the smallest family that contains both the nominal and perturbed plants. We define the distance between the nominal and perturbed plants to be the size of the smallest family. We believe that such a distance measure is of practical interest since it can help an engineer choose a suitable family of plants (nominal plant, uncertainty structure and weighting functions). Given a number of plant models, the idea is to choose the smallest family that contains all models, the presumption being that it will be easier for a controller to achieve the desired closedloop objectives with a small family. Indeed, every distance measure comes with a dual quantity, the so-called stability margin, that captures essential information about the robustness and performance of a feedback system. It is shown in this paper, that the smaller the distance between two plants, the larger the guaranteed residual stability margin and the smaller the guaranteed difference in closed-loop performance when both plants are in feedback with the same controller. The high-level ideas outlined above are not new. Vidyasagar [1], ElSakkary [2], Georgiou and Smith [3], Qiu and Davison [4], and Vinnicombe [5, 6] have all defined and worked on similar ideas but specifically for normalized

Distance Measures, RS Conditions and RP Guarantees

319

coprime factor or 4-block uncertainty structures. 3 What is new is the generic procedure for computing the distance between two plants and the robust stability and robust performance guarantees which can handle many different uncertainty structures including normalized coprime factor uncertainty that was studied by Georgiou and Smith, and Vinnicombe. In fact, in this paper we show that our procedure captures the relevant results of Vinnicombe. Our approach has been inspired by robust model validation theory (see Davis [8], Poolla et al. [9], Chen and Gu [10], and Newlin and Smith [11]). The objective of robust model validation is, given a family of plants and data from the real plant, to verify that there exists a plant model in the family that interpolates the data. If there doesn’t, then we have invalidated the family and any controller designed with the family is not guaranteed to work. In our case, we do not have real plant data but a perturbed plant model; that is, infinite data. Given a nominal plant, an uncertainty structure, weighting functions and a perturbed plant, the first step of our generic procedure is to solve a socalled consistency equation. For example, in the case of input multiplicative uncertainty, we would solve (if a solution exists) PΔ = P (I − W Δ) for all perturbations Δ ∈ RL∞ that satisfy the equation, where P is the given nominal plant, PΔ is the given perturbed plant and W is a given weighting function (the uncertainty structure is implied by the structure of the equation). Solving the consistency equation can be thought of as a model validation problem. Once we have parameterized all admissible Δ, the second step is to compute the L∞ -norm of the smallest perturbation which satisfies consistency and this is our distance measure. Given a controller that stabilizes the nominal plant, we also determine conditions for stability and bound the degradation in performance of the perturbed plant in feedback with the same controller, all in terms of our distance measure. This step is based on a powerful lemma by Vinnicombe which essentially generalizes the small gain theorem [6, p. 45, Lemma 1.22]. In fact, much of our work has been influenced by the ”Cambridge view of feedback control”. In summary, we have used robust model validation ideas to develop a generic procedure for computing the distance between two LTI systems. The procedure works with different uncertainty structures and the defined distance measure has a feedback interpretation because we have also derived robust stability and robust performance guarantees in terms of this distance measure. Compared to other work reported in the literature, although the philosophical 3

The works of Georgiou and Smith, and Vinnicombe have been used to design a number of experimental controllers, the most notable one being for a Harrier aircraft that was successfully flight tested (see Hyde [7]).

320


idea is not new, we derive our distance measures in a different way and our procedure can systematically handle different uncertainty structures. What follows is an outline of the paper. In Section 2, we define our distance measure, the stability margin, show how to compute the distance measure via a systematic procedure and derive robust stability and robust performance results all generic. In Section 3, we apply our tools to a specific uncertainty structure that has been published in the literature. In Section 4, we indicate that our procedure works for many uncertainty structures and in Section 5 we conclude the paper. 1.1 Notation Let R denote the set of proper real-rational transfer functions. 4 Also, let P ∗ (s) denote the adjoint of P (s) ∈ R defined by P ∗ (s) = P (−s)T . Let RL∞ denote the space of proper real-rational functions bounded on jR including ∞ and RH∞ denote the space of proper real-rational functions bounded and analytic in the open right half complex plane. Denote also the space of functions that are units in RH∞ by GH∞ (i.e. f ∈ GH∞ ⇔ f, f −1 ∈ RH∞ ). Let Fl (·, ·) (resp. Fu (·, ·)) denote a lower (resp. upper) linear fractional transformation. Also, let · · denote the Redheffer star product of two transfer functions with respect to some partition. For a scalar p(s) ∈ R, its winding number wno p(s) is defined as the number of encirclements of the origin made by p(s) as s follows the standard Nyquist D-contour, indented into the right half plane around any imaginary axis poles or zeros of p(s). The ordered pair {N, M }, with M, N ∈ RH∞ , is a right-coprime factorization (rcf ) of P ∈ R if M is invertible in R, P = N M −1 and N and M are right-coprime. Furthermore, the ordered pair {N, M } is a normalized rcf of P if {N, M } is an rcf of P and M ∗ M + N ∗ N = I. Likewise, the ordered pair ˜ M ˜ }, with M ˜,N ˜ ∈ RH∞ , is a left-coprime factorization (lcf ) of P ∈ R if {N, ˜ ˜ −1 N ˜ and N ˜ and M ˜ are left-coprime. FurtherM is invertible in R, P = M ˜ ˜ ˜ M ˜ } is an lcf more, the ordered pair {N, M } is a normalized lcf of P if {N, ∗ ∗ ˜ ˜ ˜ ˜ ˜ ˜ and M M + N N = I. Let {N, M } be a right and {N, M } be a left coprime ˜ , V˜ } be a left factorization of a plant P . Also, let {U, V } be a right and {U coprime factorization of a controller C. Define G :=

N ˜ := −U ˜ := −M ˜ V˜ , ˜ N ˜ , K := V , K ,G U M

˜ will be referred to as the graph symbols of P and K and K ˜ where G and G will be referred to as the inverse graph symbols of C. 5 4 5

P ∈ R implies that σ(P (∞)) < ∞. Also, P ∈ R is invertible in R if, and only if, P is square and det P (∞) = 0. For normalized graph symbols,


321

2 A Systematic Procedure for Robust Stability and Robust Performance Analysis 2.1 Objective Given a nominal plant P , an uncertainty structure with weights, and a controller C, derive conditions for the stability of the feedback interconnection of a perturbed plant PΔ with C, and bounds for the degradation in performance when P is replaced by PΔ that depend only on the given data. 2.2 Definitions Let [P, C] denote the standard feedback interconnection illustrated in Fig. 1. From Fig. 1 and after some algebra, P y (I − CP )−1 −C I = I u

v1 . v2

(1)

Definition 1. Given a plant P ∈ R and a controller C ∈ R. [P, C] is said to be well-posed if the transfer function in (1) belongs to R, that is, if (I − CP )−1 ∈ R. Also, [P, C] is said to be internally stable if, in addition, the transfer function in (1) belongs to RH∞ . Let H, C denote the standard H∞ synthesis interconnection depicted in Fig. 2. From Fig. 2 and after some algebra,         z H11 0 H12 w H12 y  = H21 0 H22  + H22  C(I − H22 C)−1 H21 −I H22  v1  . (2) u 0 0 I v2 I "

# " # G∗ h ˜ ∗ i h ˜ ∗ i G∗ GG = GG ˜ ˜ = I. G G

y

P

✛

u

✛ v2 ✻

v1 ✲ ❄ ✲ C Fig. 1. The standard feedback interconnection.

322

G. Papageorgiou and A. Lanzon z ✛ y v1 ✲ ❄ -

" # H11 H12 ✛ H21 H22 ✛ u

w

✛ v2 ✻

✲ C

Fig. 2. The standard H ∞ synthesis interconnection.

Definition 2. Given a generalized plant H ∈ R and a controller C ∈ R. H, C is said to be well-posed if the transfer function in (2) belongs to R, that is, if (I − H22 C)−1 ∈ R. Also, H, C is said to be internally stable if, in addition, the transfer function in (2) belongs to RH∞ . Finally, H is said to be stabilizable if there exists a C such that H, C is internally stable. 6 It will be shown in the sequel, that if [H22 , C] is stable, then the larger the size of Fl (H, C) −1 ∞ , the larger the size of the set of plants that C is −1 guaranteed to stabilize. Therefore, it can be argued that Fl (H, C) ∞ is a measure of the robust stability of a feedback system. Also, the nominal performance of a feedback system, that is the performance with no plant −1 uncertainty, can be related to H and Fl (H, C) ∞ (by performance it is typically meant reference tracking, disturbance rejection and noise rejection). This is because typically the size of any closed-loop transfer function can be bounded above in terms of the size of any weighting functions absorbed in H −1 −1 and Fl (H, C) ∞ . 7 Therefore, Fl (H, C) ∞ is a very important quantity that summarizes the robustness and performance of a feedback system. Definition 3. Given a plant P ∈ Rp×q , a generalized plant H ∈ R with H22 = P , and a controller C ∈ Rq×p . Define the stability margin bH (P, C) of the feedback interconnection H, C by: bH (P, C) :=

Fl (H, C) 0,

−1 ∞ ,

if Fl (H, C) ∈ RL∞ and [P, C] is internally stable otherwise.

The choice of generalized plant H and the required size of the stability margin are application specific. Now, we define a distance measure between a nominal plant and a perturbed plant. # A B2 is the state-space realization of H22 inherited from a stabilizable and C2 D22 detectable state-space realization of H, then this definition of ”H is stabilizable” is equivalent to (A, B2 ) being stabilizable and (C2 , A) being detectable [12]. For an example of how nominal performance can be related to H and Fl (H, C) −1 ∞ , see [13, p. 493, Theorem 18.11]. "

6

7

If


323

Definition 4. Given a plant P ∈ Rp×q , a generalized plant H ∈ R with H22 = P , and a perturbed plant PΔ ∈ Rp×q . Let the set of all admissible uncertainties that satisfy consistency of equations be given by: Δ = Δ ∈ RL∞ : (I − H11 Δ)−1 ∈ R, PΔ = Fu (H, Δ) . Define the distance measure dH (P, PΔ ) between plants P and PΔ for the uncertainty structure Fu (H, Δ) by: 8 dH (P, PΔ ) :=

minΔ∈Δ Δ ∞,

∞

,

if Δ = ∅ otherwise.

Note that dH (P, P ) = 0 since P = Fu (H, 0). It will be shown in the sequel, that the smaller the size of dH (P, PΔ ), the smaller the worst-case degradation of the stability margin due to a plant perturbation. Therefore, we will say that dH (P, PΔ ) can be interpreted as the distance between two plants from a feedback perspective. 9 Definition 5. Given a plant P ∈ Rp×q , a generalized plant H ∈ R with H22 = P , and a perturbed plant PΔ ∈ Rp×q . Define Δmin :=

{Δ ∈ Δ : Δ ∅,

∞

= dH (P, PΔ )},

if Δ = ∅ otherwise.

The set Δmin is the set of the smallest-sized Δ ∈ RL∞ that satisfy the consistency equation PΔ = Fu (H, Δ). Note that Δmin ⊂ Δ and that Δ = ∅ if, and only if, Δmin = ∅. 2.3 Robust Stability The following theorem gives a necessary and sufficient condition for the stability of a perturbed feedback system given a bound on the distance between the nominal and perturbed plants. Theorem 1 (Robust Stability). Given a plant P ∈ Rp×q , a stabilizable generalized plant H ∈ R with H22 = P , a perturbed plant PΔ ∈ Rp×q and a controller C ∈ Rq×p such that dH (P, PΔ ) < bH (P, C), then [PΔ , C] is internally stable ⇔ ∃Δ ∈ Δmin : η(PΔ ) = η(P ) + wno det(I − H11 Δ), (3) 8

9

We are using min instead of inf in the definition of dH (P, PΔ ) as we know that the minimizer belongs to the set Δ for our use in Section 3. In general, one should use inf and adjust a few technical issues that arise. We have not yet explored whether dH (·, ·) is a metric on Rp×q or some subset of R for any uncertainty structure H, though it can be shown that in some cases it is a metric.

324


where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . Proof . If dH (P, PΔ ) < bH (P, C), then dH (P, PΔ ) < ∞ and bH (P, C) > 0, and therefore by definition, Δ = ∅ and [P, C] is internally stable. Δ = ∅ implies that Δmin = ∅. dH (P, PΔ ) < bH (P, C) ⇒

Δ

⇒

Δ Fl (H, C)

∞

< Fl (H, C) ∞

< 1,

−1 ∞

,

∀Δ ∈ Δmin

∀Δ ∈ Δmin .

Therefore, for all Δ ∈ Δmin • • • •

Δ ∈ RL∞ ; Δ Fl (H, C) ∞ < 1; (I − H11 Δ)−1 ∈ R; and PΔ = Fu (H, Δ).

The proof follows from [6, p. 45, Lemma 1.22]. (⇒) Since H is stabilizable and [P, C] is internally stable, from Lemma 1.22 in [6] [PΔ , C] is internally stable ⇒ η(PΔ ) = η(P ) + wno det(I − H11 Δ),

∀Δ ∈ Δmin,

where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . (⇐) Also from Lemma 1.22 in [6], [PΔ , C] is internally stable ⇐ ∃Δ ∈ Δmin : η(PΔ ) = η(P ) + wno det(I − H11 Δ), where the winding number is evaluated on the aforementioned contour. The robust stability theorem also holds if, instead of supposing that dH (P, PΔ ) < bH (P, C), we suppose that Fl (H, C) ∈ RL∞ , that [P, C] is internally stable and that for all Δ ∈ Δmin σ(Δ(jω))
(≥ 0) and H is stabilizable, then [PΔ , C] is internally stable for all PΔ that belong to the set P = PΔ = Fu (H, Δ) : dH (P, PΔ ) ≤ , Δ ∈ RL∞ , (I − H11 Δ)−1 ∈ R, η(PΔ ) = η(P ) + wno det(I − H11 Δ) .


325

Therefore, a controller C that achieves a stability margin greater than with a nominal plant P , is guaranteed to stabilize all the plants in the set P. The robust stability theorem will be of practical use if, given a nominal plant P , a generalized plant H and a perturbed plant PΔ , we can readily compute dH (P, PΔ ) and rewrite the winding number condition in terms of only P , H and PΔ , that is, independent of the perturbation Δ ∈ Δmin . What follows is a step-by-step procedure for computing dH (P, PΔ ) and rewriting the winding number condition. Given P ∈ Rp×q , H ∈ R with H22 = P , and PΔ ∈ Rp×q : 1. 2. 3. 4.

Simplify PΔ = Fu (H, Δ) assuming that (I − H11 Δ) is invertible in R; Parameterize the set Δ; Compute dH (P, PΔ ); and Parameterize the set Δmin and rewrite the winding number condition independent of Δ ∈ Δmin .

Our experience from working with the generalized plants described in the next section suggests, that steps 1 and 3 are fairly straightforward but steps 2 and 4 can be much harder. 10 In Section 3, we will apply our procedure for a particular choice H. 2.4 Generalized Plants Considered Given a plant P ∈ R and a controller C ∈ R. Consider the feedback interconnection illustrated in Fig. 3 that will be referred to as the left 4-block interconnection. From Fig. 3,      w1 I −P P z1  z2  =  0 0 I   w2  . y I −P P u In order to pin down the technical machinery, in the remainder of this paper, we will only consider generalized plants that can be derived from Fig. 3. Let 10

There is evidence that dH (P, PΔ ) can also be computed if Δ is a structured uncertainty set. This will be reported in a subsequent paper. z 1 w1

✻ y

w2 z 2

❄ ✛

P

✛- ❄ ✛

✻ u

✲ C Fig. 3. The left 4-block interconnection.

326


z1 , z2

z = Sz

w1 = Sw w, w2

where Sw , Sz ∈ R select and possibly filter the inputs and outputs of interest, 

H=

Sz

 I −P P  0 0 I  Sw I I I −P P

and

w = Δz,

where Δ ∈ R. If

(4)

I − Sz

I −P Sw Δ 0 0

−1

∈ R, then PΔ ∈ R and

PΔ = Fu (H, Δ) ⇔

PΔ = P + I −P Sw Δ I − Sz

I −P Sw Δ 0 0

⇔

PΔ − P = I −P Sw Δ I − Sz

I 0

PΔ − P =

I 0

⇔ ⇔

I − I −P Sw Δ Sz

I − I −P Sw Δ Sz

I 0

⇔

PΔ − P = I −P Sw ΔSz

⇔

PΔ − P = I −P Sw Δ Sz

−1

Sz

P I

−1

Sz

P I

I −P Sw ΔSz

P I

I −P Sw Δ −1

(PΔ − P ) = I −P Sw ΔSz

P I

P PΔ − P + I 0 PΔ . I

Also, if (I − P C)−1 ∈ R, then Fl (H, C) ∈ R and

(5)


327

Fl (H, C) = Sz

P I −P C(I − P C)−1 I −P Sw Sw + Sz I 0 0

= Sz

I 0

I −P Sw + Sz

P C(I − P C)−1 I −P Sw I

= Sz

P I C(I − P C)−1 + I 0

= Sz

I − PC PC + 0 C

= Sz

I −P Sw

(I − P C)−1 I −P Sw

I (I − P C)−1 I −P Sw . C

(6)

˜, M ˜ } is a normalized left coprime Table 1. Uncertainty structures considered. {N factorization of P and R ∈ GH ∞ . Uncertainty structure

"Sw# I 0 " # I 0 " # 0 I " # 0 I " # I 0 " # 0 I

Output inverse multiplicative Additive Inverse additive Input multiplicative Output inverse multiplicative and additive Inverse additive and input multiplicative

Sz h i I 0 h i 0I h i I 0 h i 0I I I

Output inverse multiplicative and inverse additive

I

h i I 0

Additive and input multiplicative

I

h i 0I

Left 4-block Left coprime

"

I # ˜ (RM)−1 0

I I

328


Table 1 lists the uncertainty structures that will be considered. The theory and tools developed in this paper readily apply to any uncertainty structure that can be derived from the left and right 4-block interconnections. 11 2.5 Robust Performance The robust stability theorem gives a necessary and sufficient condition for the stability of a perturbed feedback system. In addition to checking whether [PΔ , C] is stable, an engineer typically wishes to know the residual stability margin bHΔ (PΔ , C), where HΔ is the generalized plant that corresponds to the perturbed plant PΔ . We will now derive an upper and a lower bound for the stability margin of a perturbed feedback system, and an upper bound for Fl (HΔ , C) − Fl (H, C) ∞ which is a measure of the difference in performance between the nominal and perturbed feedback systems. To derive the bounds, we will need the following lemma. Lemma 1. Given a nominal plant P ∈ Rp×q , a generalized plant 

H=

Sz

 I −P P  0 0 I  Sw , I I I −P P

where Sw , Sz ∈ R, a controller C ∈ Rq×p and a perturbation Δ ∈ R such that • • •

(I − P C)−1 ∈ R; (I − Fl (H, C) Δ)−1 ∈ R; and (I − H11 Δ)−1 ∈ R.

Let PΔ = Fu (H, Δ), SwΔ = Sw (I − kΔSz Sw )−1 ∈ R for a given k ∈ {0, 1}, 

HΔ =

Sz

 I −PΔ PΔ SwΔ 0 0 I  I I I −PΔ PΔ

and S = (1 − k)Sz Sw , then Fl (HΔ , C) = Fl (H, C) + Fl (H, C) Δ(Fl (HΔ , C) − S).

3 0 P P 7 6 The generalized plant of the right 4-block interconnection is equal to 4 0 I I 5 −I P P " # " # y z1 . Hence, it is a dual problem and = and can be derived from Fig. 1 with z2 u can be treated easily using similar techniques. 2

11


329

Proof . First consider the case k = 0. We will only prove the case when Sw = I and Sz = I (that is, left 4-block uncertainty) for simplicity, as the proof in the general case is identical (with Sw and Sz carried on either side to obtain the required result).  I −PΔ HΔ =  0 0 I −PΔ  0 I = Fu  0 I  

I 0   = Fu  I  0  I

     I I00 PΔ I  = 0 0 I  + 0 PΔ 0 −I I PΔ I I00     0 0 −I 0 −I I I I 0 I 0 0  , Fu (H, Δ) = Fu H   0 0 0  0 0 I I 0 0 I I 0  −P 0 −P 0 −P

0 0 I 0 I

−P −I −P 0 −P

  I 0 ,Δ I  0

 P I   P , Δ .  I P 

J

By expanding

I −H

det I − H

Δ 0 0 C

Δ 0 0 C

in terms of its Schur complements,

= det(I − H11 Δ) det(I − PΔ C) = det(I − P C) det(I − Fl (H, C) Δ).

Since (I −P C), (I −Fl (H, C) Δ) and (I −H11 Δ) are invertible in R, it follows that det(I − PΔ C)(∞) = 0 and thus (I − PΔ C) is also invertible in R.

330


Fl (HΔ , C) = Fl (Fu (J, Δ) , C) = Fu (Fl (J, C) , Δ)      0 −I I = Fu Fl  0 0 I  H, C  , Δ I I 0      0 −I I = Fl Fu  0 0 I  , Δ , Fl (H, C) I I 0 = Fl

0 I , Fl (H, C) I −Δ Δ

= Fl (H, C) (I − ΔFl (H, C))−1 (I − Δ) = (I − Fl (H, C) Δ)−1 Fl (H, C) (I − Δ). Therefore, Fl (HΔ , C) ∈ R and ⇔

Fl (HΔ , C) = (I − Fl (H, C) Δ)−1 Fl (H, C) (I − Δ) (I − Fl (H, C) Δ)Fl (HΔ , C) = Fl (H, C) (I − Δ)

⇔

Fl (HΔ , C) = Fl (H, C) + Fl (H, C) Δ(Fl (HΔ , C) − I).

This concludes the proof for the first case. Now consider the case k = 1. From the first case,    I −PΔ PΔ Sz F l   0 0 I  , C  Sw = Fl (H, C) + I −PΔ PΔ      I −PΔ PΔ Fl (H, C) Δ Sz Fl  0 0 I  , C  Sw − Sz Sw  I −PΔ PΔ which can be rearranged as    I −PΔ PΔ Sz F l   0 0 I  , C  Sw = Fl (H, C) (I − ΔSz Sw )+ I −PΔ PΔ    I −PΔ PΔ Fl (H, C) ΔSz Fl  0 0 I  , C  Sw . I −PΔ PΔ The result follows by multiplying both sides of the above equation by (I − ΔSz Sw )−1 and noting that in this case    I −PΔ PΔ Fl (HΔ , C) = Sz Fl  0 0 I  , C  Sw (I − ΔSz Sw )−1 . I −PΔ PΔ


331

The following theorem bounds the stability margin of a perturbed feedback system and the performance difference between the nominal and perturbed feedback systems. Theorem 2. [Robust Performance] Given a nominal plant P ∈ Rp×q , a stabilizable generalized plant   I −P P Sz  0 0 I  Sw H= , I I I −P P where Sw , Sz ∈ R, a perturbed plant PΔ ∈ Rp×q and a controller C ∈ Rq×p such that dH (P, PΔ ) < bH (P, C). Assume furthermore that there exists a Δ ∈ Δmin that satisfies η(PΔ ) = η(P ) + wno det(I − H11 Δ), where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . Let SwΔ = Sw (I−kΔSz Sw )−1 ∈ R for a given k ∈ {0, 1},   I −PΔ PΔ Sz SwΔ 0 0 HΔ = I  I I I −PΔ PΔ and S = (1 − k)Sz Sw . Then the following results hold when S ∈ RL∞ : 1. Fl (HΔ , C) ∈ RL∞ and [PΔ , C] is internally stable 2. bHΔ (PΔ , C) − bH (P, C) ≤ Fl (HΔ , C) − S 3. Fl (HΔ , C) − Fl (H, C)

∞

≤

∞

bHΔ (PΔ , C) dH (P, PΔ )

Fl (HΔ , C) − S ∞ dH (P, PΔ ) . bH (P, C)

Proof . We will only consider the case k = 0, as the proof for the other case is identical. If dH (P, PΔ ) < bH (P, C), then dH (P, PΔ ) < ∞ and bH (P, C) > 0, and therefore by definition, Δ = ∅, Fl (H, C) ∈ RL∞ and [P, C] is internally stable. Δ = ∅ implies that Δmin = ∅. dH (P, PΔ ) < bH (P, C) ⇒

Δ Fl (H, C)

∞

< 1,

∀Δ ∈ Δmin

⇔

0 < 1 − σ(Δ Fl (H, C))(jω),

∀ω, ∀Δ ∈ Δmin

⇒

0 < σ(I − Δ Fl (H, C))(jω),

∀ω, ∀Δ ∈ Δmin

⇔

det(I − Δ Fl (H, C))(jω) = 0,

∀ω, ∀Δ ∈ Δmin .

332


Thus, (I − Δ Fl (H, C))−1 ∈ RL∞ for all Δ ∈ Δmin. Since internal stability of [P, C] implies that (I − P C)−1 ∈ RH∞ and Δ ∈ Δmin implies that (I − H11 Δ)−1 ∈ R, from Lemma 1 and for all Δ ∈ Δmin , (7)

⇔

Fl (HΔ , C) = Fl (H, C) + Fl (H, C) Δ(Fl (HΔ , C) − Sz Sw ) (I − Fl (H, C) Δ)Fl (HΔ , C) = Fl (H, C) (I − ΔSz Sw )

⇔

Fl (HΔ , C) = Fl (H, C) (I − ΔFl (H, C))−1 (I − Δ Sz Sw ).

(8)

Since Δ, (I − Δ Fl (H, C))−1 ∈ RL∞ for all Δ ∈ Δmin and bH (P, C) > 0 implies that Fl (H, C) ∈ RL∞ , if Sz Sw ∈ RL∞ , then from (8), Fl (HΔ , C) ∈ RL∞ for all Δ ∈ Δmin. From Theorem 1 and because by supposition there exists a Δ ∈ Δmin that satisfies the winding number condition, [PΔ , C] is internally stable. This concludes the proof of part (a). Inequality (b) follows from the properties of the L∞ -norm and (7). Fl (HΔ , C) Fl (H, C) ⇔

Fl (H, C)

−1 ∞

∞

≤ Fl (H, C) Δ

∞

∞

∞

+

Fl (HΔ , C) − Sz Sw

≤ Fl (HΔ , C)

−1 ∞

∞

+ −1

Fl (HΔ , C) − Sz Sw ∞ Δ ∞ Fl (HΔ , C) ∞ (dividing both sides by Fl (H, C) ∞ Fl (HΔ , C) ⇔

− Fl (HΔ , C) − Sz Sw Fl (HΔ , C)

−1 ∞

−

∞ Fl −1 Fl (H, C) ∞ . ∞

Δ

(HΔ , C)

−1 ∞

∞)

≤

Similarly, Fl (HΔ , C)

−1 ∞

− Fl (H, C)

−1 ∞

≤


∞

Δ

∞

Fl (HΔ , C)

−1 ∞ ,

∞

Δ

∞

Fl (HΔ , C)

−1 ∞ .

and therefore, Fl (HΔ , C)

−1 ∞

− Fl (H, C)

−1 ∞

≤


Since bH (P, C) > 0, Fl (H, C) ∞ = 1/bH (P, C) and since Δ ∈ Δmin , Δ ∞ = dH (P, PΔ ). The inequality follows by noting that since Fl (HΔ , C) ∈ RL∞ and [PΔ , C] is internally stable, Fl (HΔ , C) ∞ = 1/bHΔ (PΔ , C). We will now prove inequality (c). Again from the properties of the L∞ norm and (7), Fl (HΔ , C) − Fl (H, C) The result then follows.

∞

≤ Fl (H, C)

∞

Δ

∞


∞

.


333

Note that it is straightforward to derive analogues of both inequalities that are valid at each frequency. If Fl (HΔ , C) − S ∞ bHΔ (PΔ , C) ≤ 1, as is the case for additive, inverse additive, left 4-block and coprime factor uncertainties, we can state the following corollary which is a more concise version of the robust performance theorem. Corollary 1. For any nominal plant P ∈ Rp×q with a stabilizable and detectable state-space realization, perturbed plant PΔ ∈ Rp×q and controller C ∈ Rq×p . Let •

• • •

W1 , Sz = 0 W2 and SwΔ = Sw , where W1 , W2 ∈ RL∞ (that is, 0 additive uncertainty); or 0 , Sz = W2 0 and SwΔ = Sw , where W1 , W2 ∈ RL∞ (that is, Sw = W1 inverse additive uncertainty); or Sw = I, Sz = I and SwΔ = Sw (that is, left 4-block uncertainty); or ˜ )−1 (RM , Sz = I and SwΔ = Sw (I − Δ Sz Sw )−1 (that is, coprime Sw = 0 factor uncertainty),   I −P P Sz  0 0 I  Sw H= I I I −P P Sw =

and



HΔ = If

Sz

 I −PΔ PΔ SwΔ 0 0 . I  I I I −PΔ PΔ

∃Δ ∈ Δmin : η(PΔ ) = η(P ) + wno det(I − H11 Δ),

where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ , then bHΔ (PΔ , C) ≥ bH (P, C) − dH (P, PΔ ).

(9)

Furthermore, when bH (P, C) > dH (P, PΔ ), Fl (HΔ , C) − Fl (H, C)

∞

≤

dH (P, PΔ ) . bH (P, C)bHΔ (PΔ , C)

(10)

Proof . First suppose that bH (P, C) ≤ dH (P, PΔ ). Then the right-hand-side of inequality (9) is ≤ 0. But from Definition 3, bHΔ (PΔ , C) ≥ 0 and hence (9) is automatically true.

334


Now suppose that bH (P, C) > dH (P, PΔ ). The proof follows from Theorem 2, noting that if P has a stabilizable and detectable state-space realization, then H is stabilizable, and that in the case of coprime factor uncertainty −1

SwΔ = Sw (I − ΔSz Sw )

˜ )−1 (RM = 0

˜ )−1 (RM I −Δ 0

−1

belongs to R for all Δ ∈ Δmin because by definition (I − H11 Δ)−1 =

I−

˜ )−1 (RM Δ 0

−1

belongs to R for all Δ ∈ Δmin . Inequalities (9) and (10) immediately follow from conditions (b) and (c) of Theorem 2 on noting that for each of the cases considered Fl (HΔ , C) − S ∞ bHΔ (PΔ , C) ≤ 1.

3 Left 4-Block Uncertainty Characterization with a Left 4-Block Performance Measure In this section, we illustrate the novel generic procedure described in Section 2 on a left 4-block uncertainty characterization with a left 4-block performance measure and derive results specific for this case. The results for this particular (and only this particular) case have also been derived via other technical machinery in [5, 6]. Consequently, in this section, we are illustrating that the generic procedure of Section 2 reproduces existing results in a simple systematic manner. We shall indicate in Section 4 that new results, generated by our novel procedure of Section 2 on different uncertainty structures and performance measures, are also possible. Towards this end, consider Figure 3 again and note that for this particular case Sw = I and Sz = I, as also seen in Table 1. 3.1 Define the Robust Stability Margin b(P, C) Plugging in Sw = I and Sz = I in equation (4) gives   I −P P H = 0 0 I  I −P P and hence Fl (H, C) =

I (I − P C)−1 I −P . C

Consequently, straight from Definition 3, we see that the stability margin b(P, C) for left 4-block uncertainty characterizations is given by:


 b(P, C) :=

I (I − P C)−1 I −P C

 

335

−1

if [P, C] is internally stable, ∞

0

otherwise.

(11) In this specific case, Fl (H, C) ∈ RL∞ is dropped in (11) since [P, C] is internally stable automatically guarantees this. 3.2 When Is the Given Data Consistent and What Are All Solutions? Given a nominal plant P and a perturbed plant PΔ , we would now like to derive necessary and sufficient conditions for there to exist a Δ ∈ RL∞ satisfying PΔ = Fu (H, Δ) . That is, we would like to know when the given P , PΔ data is consistent with the uncertainty characterization considered, expressed via generalized plant H. Furthermore, provided these necessary and sufficient conditions are satisfied, we would like to parameterize all solutions Δ that fit the given P , PΔ data. For this specific case, plugging in Sw = I and Sz = I in equation (5) gives PΔ = Fu (H, Δ) ⇔

PΔ − P = I −P Δ

PΔ I

which can then be equivalently rearranged into PΔ I

⇔

P − PΔ = −I P Δ

⇔ ⇔

˜ ˜ (P − PΔ )MΔ = GΔG M Δ ˜ Δ = GΔG ˜ GG Δ

⇔

˜ Δ=G ˜ G ˜∗ G GG

⇔

˜ Δ= I0 GG

⇔

˜∗ G Δ= G

(12)

˜ G ˜ ∗ GΔ Δ G Δ G∗

˜ G ˜ ∗ GΔ Δ G Δ G∗ ˜Δ ˜ Δ G Q1 GG

Q2 Q3

G∗Δ

˜Δ G GΔ G∗Δ

0 I (13)

where Qi ∈ RL∞ are arbitrary. Consequently, for this specific case, the given P , PΔ data is always consistent with the uncertainty characterization considered, as there always exists

336


at least one solution for Δ (assuming that (I − H11 Δ)−1 ∈ R). Also, given any P , PΔ pair, there in fact always exist multiple solutions for Δ ∈ RL∞ , and the non-uniqueness in the solution Δ is parameterized above by the arbitrary objects Qi ∈ RL∞ . 3.3 Define the Solution Set Δ Since PΔ = Fu (H, Δ) is well-posed when det(I − H11 Δ)(∞) = 0, we need to express det(I − H11 Δ)(∞) independently of Δ. To this end, observe that: det(I − H11 Δ) = det I −

I −P Δ 0 0

I −I P Δ 0 ˜ ˜ ˜ −1 M ˜ G ˜ ∗ G Q1 GGΔ GΔ G I+ G∗Δ Q2 Q3 0 ˜Δ ˜ Δ G ˜ −1 Q1 GG M I+ I0 G∗Δ Q2 Q3 0 ˜Δ M ˜ −1 G ˜ Δ I+ Q1 GG ∗ 0 GΔ ˜Δ ˜ −1 Q1 G M I+ (GM −1 MΔ − GΔ ) ∗ GΔ 0 ˜Δ ˜ −1 G ˜ ∗ + M Q1 GM −1 MΔ . G Δ G∗Δ 0

= det I + = det = det = det = det = det

(14)

(15)

Consequently, det(I − H11 Δ)(∞) = 0

⇔

det

˜ −1 ˜ ∗ + M Q1 G Δ 0

G

(∞) = 0.

We are now in a position to define the solution set Δ, which contains all feasible solutions Δ satisfying PΔ = Fu (H, Δ), for left 4-block uncertainty characterizations. It is clear from Definition 4 and the preceeding derivations that set Δ reduces to Δ=

˜∗ G Δ= G

˜ Δ Q1 GG Q2 Q3

˜Δ G G∗Δ det

: Qi ∈ RL∞ , ˜ −1 ˜ ∗ + M Q1 G Δ 0

G

(∞) = 0

in this specific case. Note also that Δmin is never empty in this specific case ˜ Δ )(∞) < 1. for σ(GG


337

3.4 Define the Distance Measure d(P, PΔ ) ˜ ˜ ∗ G and GΔ are all-pass, and Δmin = ∅, it easily follows straight Since G G∗Δ from Definition 4 that the distance measure d(P, PΔ ) for left 4-block uncertainty characterizations is given by: d(P, PΔ ) = min Δ Δ∈Δ

= =

∞

˜ Δ Q1 GG Q2 Q3

min

˜∗ G G

min

˜ Δ Q1 GG Q2 Q3

Qi ∈RL∞

Qi ∈RL∞

˜ Δ = GG

∞

˜Δ G G∗Δ

∞

∞

.

(16)

Note that the minimizer belongs to the set Δ because we have assumed that ˜ Δ )(∞) < 1. σ(GG 3.5 Write the Winding Number Condition Independent of Δ Now, we need to rewrite winding number condition (3) independently of Δ. This will give us a necessary and sufficient condition for robust stability as presented in Theorem 1 that is calculable from the given data as it is independent of Δ. Towards this end, note that for left 4-block uncertainty characterizations, winding number condition (3) reduces to η(PΔ ) − η(P ) = wno det

˜Δ G G∗Δ

˜ −1 ˜ ∗ + M Q1 G Δ 0

GM −1 MΔ

(17)

via equation (15). Therefore, ˜Δ ˜ −1 G ˜ ∗ + M Q1 G G Δ G∗Δ 0 ˜ ΔM ˜ −1 Q1 G ˜ΔG I −M = wno det . ∗ ˜ −1 N M Q1 G∗ G

0 = wno det

Δ

Δ

(18)

Before we proceed in massaging this winding number condition further, note that [14] proved that in 4-block uncertainty characterizations (as the case considered here), I ≥ 1. (I − P C)−1 I −P C ∞ Hence b(P, C) ≤ 1. Consequently, in this specific case, d(P, PΔ ) < 1 whenever the supposition d(P, PΔ ) < b(P, C) (see Theorem 1) is enforced. Then, since

338


˜ Δ d(P, PΔ ) = GG

∞

< 1 and σ G∗Δ G

2

˜ = 1 − σ GG Δ

2

on the jω-axis [6,

(G∗Δ G)−1

∈ RL∞ . Exploiting this fact, we can use p. 121], it follows that Schur complements to rewrite equation (18) as 0 = wno det

˜ −1 Q1 0 ˜ ΔM ˜ −1 Q1 − G ˜ Δ G(G∗ G)−1 N ∗ M I −M Δ Δ 0 G∗Δ G

which is equivalent to wno det(G∗Δ G)

∗ ˜ −1 ˜Δ + G ˜ Δ G(G∗Δ G)−1 NΔ )M Q1 ] = − wno det[I − (M ∗ ˜ ˜ −1 Q1 ] = − wno det[I − MΔ (I + (PΔ − P )M (G G)−1 N ∗ )M Δ

Δ

∗ ˜ −1 ∗ ˜ Δ (I + (PΔ − P )(I + PΔ )M Q1 ] P )−1 PΔ = − wno det[I − M ∗ ∗ −1 −1 ˜ Δ (I + P P )(I + P P ) M ˜ Q1 ] = − wno det[I − M Δ Δ Δ −∗ ∗ −1 −1 ˜ (I + P PΔ ) M ˜ Q1 ] = − wno det[I − M Δ

˜G ˜ ∗ )−1 Q1 ] = − wno det[I − (G Δ ∗ −1 ˜ ˜ ∗ ˜ ˜ = − wno det[(GGΔ ) (GGΔ − Q1 )]

˜G ˜ ∗ ] − wno det[G ˜G ˜ ∗ − Q1 ]. = wno det[G Δ Δ

(19)

Hence, equation (19) is a formulation of the desired winding number condition written independently of Δ so that it can be computed as a necessary and sufficient condition for robust stability as presented in Theorem 1. However, for this to be checked, one needs to check the condition over all ˜ Δ Q1 GG ˜ Δ = GG Qi ∈ RL∞ that yield a Δ ∈ Δmin (i.e. satisfy Q2 Q3 ∞ ∞ ˜ −1 Q1 M ˜∗ + and det G G (∞) = 0). Δ 0 Non-uniqueness in Δmin Provides No Extra Freedom The following lemma states that winding number condition (19) can be simplified further and can be made also independent of Q1 whenever the supposition d(P, PΔ ) < b(P, C) (see Theorem 1) is enforced.12 This makes computation of the winding number test considerably easier. ˜ GΔ , G ˜ Δ defined in Section 1.1 Lemma 2. Given normalized graph symbols G, ˜ and GGΔ ∞ < 1. Then ˜G ˜ ∗Δ ] = wno det[G ˜G ˜ ∗Δ − Q1 ] wno det[G for all Qi ∈ RL∞ that satisfy 12

˜ Δ The supposition d(P, PΔ ) < b(P, C) implies GG

∞

< 1 via [14].


det

˜ −1 ˜ ∗ + M Q1 G Δ 0

G

˜ Δ Q1 GG Q2 Q3

(∞) = 0 and

˜ Δ = GG ∞

339

∞

.

Proof . Since ˜ Δ Q1 GG

∞

≤

˜ Δ Q1 GG Q2 Q3

˜ Δ = GG ∞

∞

,

it follows that

⇒

˜ Δ 2 I − GG ˜ Δ G∗Δ G ˜ ∗ ∀ω ∈ R ∪ {∞} Q1 Q∗1 ≤ GG ∞ ∗ ˜∗ ˜ Δ 2 I − GG ˜ ∀ω ∈ R ∪ {∞} σ(Q1 )2 ≤ λ GG Δ GΔ G ∞ (since AA∗ ≤ B ⇒ σ(A)2 ≤ λ(B))

⇔

˜ Δ σ(Q1 )2 ≤ GG

2 ∞

˜ − σ GG Δ

2

∀ω ∈ R ∪ {∞}

2

⇔

˜ Δ ∀ω ∈ R ∪ {∞} σ(Q1 )2 < 1 − σ GG ˜ Δ (since GG < 1 was assumed) ∞ ˜G ˜ ∗Δ σ(Q1 ) < σ G ∀ω ∈ R ∪ {∞}

⇒

2 ˜ ˜G ˜ ∗ 2 = 1 − σ GG via [6, p. 121]) (since σ G Δ Δ ∗ ˜G ˜ Δ − Q1 ) = wno det(G ˜G ˜ ∗Δ ) wno det(G

⇒

(via [6, p. 16]). Consequently, whenever the supposition d(P, PΔ ) < b(P, C) (see Theorem 1) is enforced12 , it follows that equation (19) reduces to wno det(G∗Δ G) = 0.

(20)

That is, Q1 in equation (19) does not yield any extra freedom on top of Vinnicombe’s winding number test [6, Definition 3.1, p. 119]. Hence, equation (20) is the desired winding number condition written independently of Δ so that it can be computed as a necessary and sufficient condition for robust stability as presented in Theorem 1. It is also easily computable. 3.6 State Robust Stability and Robust Performance Theorems In this subsection, we rewrite the generic Robust Stability Theorem 1 and the generic Robust Performance Theorem 2 for this specific design case. That is, we reduce Robust Stability Theorem 1 and Robust Performance Theorem 2 specifically for left 4-block uncertainty characterizations with left 4-block performance measures. Theorem 3 (Robust Stability - Four block). Given a plant P ∈ Rp×q , a perturbed plant PΔ ∈ Rp×q and a controller C ∈ Rq×p . Define normalized

340


˜ GΔ , G ˜ Δ as in Section 1.1, a stability margin b(P, C) as graph symbols G, G, in (11) and a distance measure d(P, PΔ ) as in (16). Furthermore, suppose d(P, PΔ ) < b(P, C). Then [PΔ , C] is internally stable ⇔ wno det(G∗Δ G) = 0, where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . Proof . Trivial reduction of Theorem 1. Note that the supposition that “H is stabilizable” is automatically fulfilled in this specific design case, and hence does not need to be independently enforced, since H has the following special form (i.e. P repeated in a special way):   I −P P H = 0 0 I . I −P P Similarly, we now reduce Robust Performance Theorem 2 specifically for left 4-block uncertainty characterizations with left 4-block performance measures. Theorem 4 (Robust Performance - Four block). Given the suppositions of Theorem 3 and furthermore assuming d(P, PΔ ) < b(P, C) and wno det(G∗Δ G) = 0, where the winding number is evaluated on a contour indented to the right around any imaginary axis poles of P and PΔ . Then |b(PΔ , C) − b(P, C)| ≤ d(P, PΔ ) (21) and Fl (HΔ , C) − Fl (H, C) where



 I −P P H = 0 0 I  I −P P

∞

≤

d(P, PΔ ) , b(P, C)b(PΔ , C)

(22)

 I −PΔ PΔ HΔ =  0 0 I . I −PΔ PΔ 

and

Proof . Trivial reduction of Theorem 2 on noting that Fl (HΔ , C) − Sz Sw

∞

= Fl (HΔ , C) − I = Fl (HΔ , C) 1 = . b(PΔ , C)

∞

∞

The second equality follows from [13, Lemma 18.6, p. 484] since Fl (HΔ , C) is idempotent in this specific case. The third equality follows from definition (11) of b(PΔ , C) on noting that [PΔ , C] is internally stable via Theorem 3.


341

A tighter robust performance result exploiting case-specific geometry In this specific case (i.e. left 4-block uncertainty characterization with a left 4-block performance measure), it is possible to derive a slightly tighter result by exploiting case-specific geometry. ˜ denote a normalized right (resp. left) inverse To do this, let K (resp. K) ˜ GΔ , G ˜ Δ be defined as in Section graph symbol for controller C and let G, G, 1.1. Then, since Sw = I and Sz = I, it is easy to see that I (I − P C)−1 I −P C ˜ −1 G ˜ = K(GK)

Fl (H, C) =

via (6)

˜ −1 K ˜ = I − G(KG) and similarly ˜ Δ K)−1 G ˜Δ Fl (HΔ , C) = K(G ˜ Δ )−1 K. ˜ = I − GΔ (KG Consequently, using the result of Lemma 1 for this specific case (i.e. with Sw = I and Sz = I), we get

⇔

Fl (HΔ , C) = Fl (H, C) + Fl (H, C) Δ(Fl (HΔ , C) − I) ˜ Δ K)−1 G ˜ Δ = K(GK) ˜ −1 G ˜ K(G

⇒

−1 ˜ ˜ −1 GΔG ˜ ˜ − K(GK) K Δ (KGΔ ) ˜ Δ K)−1 G ˜ ∗ = (GK) ˜ −1 G ˜ ΔK ˜K ˜ ∗ − (GK) ˜ −1 GG ˜ Δ (KG ˜ Δ )−1 (G

˜ ∗ , substitute (pre-multiply by K ∗ , postmultiply by K ˜ Δ = GΔG ˜ GG Δ via (12)) ⇒

⇔

⇔

˜ ˜∗ ˜ Δ) σ(GG ˜ ∗ ≤ σ(GK ) + ˜ Δ K)−1 G ˜ΔK σ (G ˜ ˜ ˜ Δ) σ(GK) σ(GK) σ(KG (via singular value inequalities) ˜ Δ K)2 1 − σ(G ≤ ˜ Δ K) σ(G

˜ 2 ˜ Δ) 1 − σ(GK) σ(GG + ˜ ˜ ˜ Δ) σ(GK) σ(GK) σ(KG ˜Δ K K ˜∗ (via [6, Lemma 2.2(ii), p. 71] using ZΔ = G ∗ ˜ KK ˜ ∗ satisfying ZZ ∗ = I) satisfying ZΔ ZΔ = I and Z = G 1 − α2 − α 1 − β 2 ≤ γ ˜ Δ K) = σ(KG ˜ Δ ), β = σ(GK), ˜ ˜ Δ )) γ = σ(GG (α = σ(G

β

342

us


⇔

sin(βˆ − α ˆ ) ≤ sin γˆ (ˆ α = arcsin α, βˆ = arcsin β, γˆ = arcsin γ such that ˆ γˆ ∈ [0, π] since α, β, γ ∈ [0, 1] — see [6]) α, ˆ β,

⇔

arcsin β − arcsin α ≤ arcsin γ.

It can be similarly shown that “arcsin α − arcsin β ≤ arcsin γ”, thus giving |arcsin α − arcsin β| ≤ arcsin γ.

Then since ˜ Δ K) = α = σ(G

1 1 ˜ ˜ Δ ), = , β = σ(GK) , γ = σ(GG σ(Fl (HΔ , C)) σ(Fl (H, C))

it follows that |arcsin b(PΔ , C) − arcsin b(P, C)| ≤ arcsin d(P, PΔ )

(23)

on noting that [P, C] and [PΔ , C] are internally stable in Theorem 2.

4 Other Structures The systematic procedure proposed in Section 2 was used in Section 3 on the specific problem of left 4-block uncertainty characterizations with left 4block performance measures to derive, in a different way, results that already existed in the literature [5, 6]. In doing so, we have shown that the proposed systematic procedure captures completely existing results in the literature. This systematic procedure of Section 2 can also be used on several other uncertainty structures and several other performance measures to derive brand new robust stability and robust performance results. It allows us to define stability margins and distance measures for each specific uncertainty structure and performance measure considered. This then allows us to give a necessary and sufficient winding number condition for stability of a perturbed system, and residual performance guarantees, when a nominal system is replaced by a perturbed system for each specific uncertainty structure and performance measure considered. We have derived such results (using the outline systematic procedure) for a number of different uncertainty structures and performance measures, including to mention a few: Additive, Inverse Additive, Multiplicative, Inverse Multiplicative, Coprime Factor and 4-block uncertainty characterizations with Additive, Inverse Additive, Multiplicative, Inverse Multiplicative, Coprime Factor and 4-block performance measures, and various mixes of the above as also outlined in Table 1. These results are not given here due to space constraints and will be published shortly elsewhere.


343

5 Conclusions Given a nominal plant model, an uncertainty structure, weights and a perturbed plant model we have defined a measure of distance between the nominal and perturbed plant models that is essentially a measure of difference between the two plants from a feedback perspective. We have shown how to compute the distance measure using model validation ideas. We have also derived a necessary and sufficient condition for the stability of the perturbed feedback system given a bound on the distance between the two plants. Furthermore, we have derived upper and lower bounds for the residual stability margin and an upper bound for a measure of the difference in performance. These bounds justify our claim that the distance measure is a measure of difference from a feedback perspective. All the above results are generic and can be applied to a large number of uncertainty structures. To show that our procedure works we have applied it to left 4-block uncertainty with left 4-block performance, a case that has already been studied by Vinnicombe using different techniques and we systematically reproduce his results. We have also applied our procedure to other uncertainty structures not discussed in this paper.

Acknowledgements This work has been supported in part by ARC Discovery-Projects Grant DP0342683 and National ICT Australia Ltd. National ICT Australia Ltd. is funded through the Australian Government’s Backing Australia’s Ability initiative, in part through the Australian Research Council. This work has also been supported in part by Honeywell Inc. Internal Research and Development funds. Parts of this research were performed during a month’s visit by the first author at the Australian National University and National ICT Australia Ltd.

References 1. M. Vidyasagar, “The graph metric for unstable plants and robustness estimates for feedback stability,” IEEE Transactions on Automatic Control, vol. 29, no. 5, pp. 403–418, May 1984. 2. A. K. El-Sakkary, “The gap metric: Robustness of stabilization of feedback systems,” IEEE Transactions on Automatic Control, vol. 30, no. 3, pp. 240– 247, Mar. 1985. 3. T. T. Georgiou and M. C. Smith, “Optimal robustness in the gap metric,” IEEE Transactions on Automatic Control, vol. 35, no. 6, pp. 673–686, June 1990.

344


4. L. Qiu and E. J. Davison, “Feedback stability under simultaneous gap metric uncertainties in plant and controller,” Systems and Control Letters, vol. 18, no. 1, pp. 9–22, Jan. 1992. 5. G. Vinnicombe, “Frequency domain uncertainty and the graph topology,” IEEE Transactions on Automatic Control, vol. 38, no. 9, pp. 1371–1383, Sept. 1993. 6. ——, Uncertainty and Feedback: H-infinity loop-shaping and the nu-gap metric. Imperial College Press, 2001. 7. R. A. Hyde, “H-infinity aerospace control design — a VSTOL flight application,” in Advances in Industrial Control Series. Springer-Verlag, 1995. 8. R. A. Davis, “Model validation for robust control,” Ph.D. dissertation, University of Cambridge, UK, 1995. 9. K. Poolla, P. Khargonekar, A. Tikku, J. Krause, and K. Nagpal, “A timedomain approach to model validation,” IEEE Transactions on Automatic Control, vol. 39, no. 5, pp. 951–959, May 1994. 10. J. Chen and G. Gu, Control-Oriented System Identification: An H-infinity Approach. John Wiley & Sons, Inc., 2000. 11. M. P. Newlin and R. S. Smith, “A generalization of the structured singular value and its application to model validation,” IEEE Transactions on Automatic Control, vol. 43, no. 7, pp. 901–907, July 1998. 12. M. Green and D. J. N. Limebeer, Linear Robust Control. Prentice-Hall, Inc., 1995. 13. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control. PrenticeHall, Inc., 1996. 14. K. Glover and D. McFarlane, “Robust stabilization of normalized coprime factor plant descriptions with H-infinity-bounded uncertainty,” IEEE Transactions on Automatic Control, vol. 34, no. 8, pp. 821–830, Aug. 1989.

Stochastic Modelling over a Finite Alphabet and Algorithms for Finding Genes from Genomes M. Vidyasagar Tata Consultancy Services, Software Units Layout, Madhapur, Hyderabad 500081, INDIA [email protected] Summary. In this paper, we study the problem of constructing models for a stationary stochastic process {Yt } assuming values in a finite set M := {1, . . . , m}. It is assumed that only a finite length sample path of the process is known, and not the full statistics of the process. Two kinds of problems are studied, namely: modelling for prediction, and modelling for classification. For the prediction problem, in a companion paper it is shown that a well-known approach of modelling the given process as a multi-step Markov process is in fact the only solution satisfying certain nonnegativity constraints. In the present paper, accuracy and confidence bounds are derived for the parameters of this multi-step Markov model. So far as the author is aware, such bounds have not been published previously. For the classification problem, it is assumed that two distinct sets of sample paths of two separate stochastic processes are available – call them {u1 , . . . , ur } and {v1 , . . . , vs }. The objective here is to develop not one but two models, called C and N C respectively, such that the strings ui have much larger likelihoods with the model C than with the model N C, and the opposite is true for the strings vj . Then a new string w is classified into the set C or N C according as its likelihood is larger from the model C or the model N C. For the classification problem, we develop a new algorithm called the 4M (Mixed Memory Markov Model) algorithm, which is an improvement over variable length Markov models. We then apply the 4M algorithm to the problem of finding genes from the genome. The performance of the 4M algorithm is compared against that of the popular Glimmer algorithm. In most of the test cases studied, the 4M algorithm correctly classifies both coding as well as non-coding regions more than 90% of the time. Moreover, the accuracy of the 4M algorithm compares well with that of Glimmer. At the same time, the 4M algorithm is amenable to statistical analysis.

1 Introduction 1.1 Stochastic Modelling Suppose m is a positive integer, and define M := {1, . . . , m}. Suppose {Yt } is a stationary stochastic process assuming values in M. The problem studied


346

M. Vidyasagar

in this paper is that of constructing a model for this stochastic process. By itself this is a very old and broad problem, so it is necessary to make the scope of the problem more precise. In case the complete statistics of the process are known, and if the aim is to reproduce the complete statistics, then the problem is called one of ‘complete realization.’ The complete realization problem in the context of finding a hidden Markov model (HMM) is studied in a companion paper [22], and is not discussed here. In [22], attention is restricted to stochastic processes that have an additional property called alpha-mixing (a very weak kind of long-term independence); conditions are derived for such a process to have a hidden Markov model, that are ‘almost’ necessary as well as sufficient. In the present paper, attention is focused here on the so-called ‘partial realization’ problem, which comes in two flavours. In the first flavour, which may be called ‘exact’ partial realization, one is given the statistics of the process up to some order k, and the objective is to match these statistics exactly. In the second flavour, which may be called ‘inexact’ partial realization, one is given just a sample path of the process, from which one has to infer the statistics of the process, and then construct a model. Let us be more precise in our problem formulation, beginning with the problem of exact partial realization. Suppose an integer k is specified, and the frequencies of all strings u ∈ Mk (i.e., frequencies of all k-tuples) are specified. The exact partial realization problem is to construct a stochastic process whose frequencies of all k-tuples exactly match the specified values. Hereafter, the frequency of occurence of a string is denoted by fu . Thus if u is a string of length k, say u = u1 . . . uk , then fu := Pr{(Yt−k+1 , . . . , Yt ) = u1 . . . uk }. A well known solution to the exact partial realization problem is to model the process as a multi-step Markov model of order k − 1. In other words, it is assumed that if u ∈ M∗ is a string of length larger than k, then Pr{ut |u1 . . . ut−1 } = Pr{ut |ut−k+1 . . . ut−1 }. In short, it is assumed that the probability of ut is not affected by values of ui when i ≤ t − k. Moreover, the transition probability is computed as Pr{ut |ut−k+1 . . . ut−1 } ==

fut−k+1 ...ut−1 ut . fut−k+1 ...ut−1

Since the process is stationary, the above probability is independent of t. The above model, though it is often called a (k − 1)-step Markov model, is also a Markov model over a state space Mk−1 . Suppose u, v ∈ Mk−1 are two states. Then a transition from v to u is possible if and only if the last k − 2 symbols of v (read from left to right) are the same as the first k − 2 symbols of u; in other words, it must be the case that v = v1 v2 , u = v2 w for some v2 ∈ Mk−2 , v1 , w ∈ M.

Stochastic Modelling over a Finite Alphabet

347

In this case the probability of transition from the state v to the state u is given by fvw av,u = . (1) fv It is clear that, though the state transition matrix has dimension mk−1 ×mk−1 , every row contains at most m nonzero entries. Such a (k − 1)-step Markov model perfectly reproduces the k-tuple frequencies fu for all u ∈ Mk . In a companion paper [23], it is shown that the multi-step Markov model is the only solution to the problem that satisfies certain natural nonnegativity conditions. The multi-step Markov model has some clear advantages over a conventional hidden Markov model (HMM) of the kind studied in [16] when we are dealing with extremely long sequences of the kind encountered in biological problems. To highlight this point, we use the definition of a HMM used by Anderson [1]. This model can be shown to be equivalent to the conventional HMM; see for example [22]. Given the process {Yt }, we define a HMM to be another process {Xt } over a finite alphabet N := {1, . . . , n} such that the joint process {(Yt , Xt )} is Markov, and in addition, Pr{(Yt+1 , Xt+1 )|(Yt , Xt )} = Pr{(Yt+1 , Xt+1 )|Xt }. In other words, not only is the joint process Markov, but both Yt+1 and Xt+1 are random functions of Xt alone and don’t depend on Yt . This condition guarantees that {Xt } is a Markov process by itself. Define, for every u ∈ M, i, j ∈ N , (u) mij := Pr{Yt+1 = u&Xt+1 = j|Xt = i}, (2) (u)

and define M (u) to be the n × n matrix whose ij-th element is mij . Let π denote the stationary distribution of the Markov chain {Xt }, and let en denote the column vector of n one’s. Then for every string u = u1 . . . ul , we have fu = πM (u1 ) . . . M (ul ) en . (3) Moreover, the state transition matrix of the Markov chain {Xt } equals A := (u) , and we have πA = π and Aen = en . u∈M M The above formula (3) for computing the likelihood of a string u works quite well when l is a small number, such as 8 or 10. In problems of speech processing, these are typical values of l. However, in biological problems, l is typically 150 or 200. In such a case, the value of the likelihood function is well below the machine zero. In problems of genomics of the kind studied here, a rule of thumb is that the likelihood of a string u of length l over the foursymbol nucleotide alphabet {A, C, G, T } is going to be of the order of 4−l . So if l = 150, then the likelihood is going to be of the order of 2−300 ≈ 10−90 . No machine can handle such small numbers directly. In such a case, the natural remedy is to replace the likelihood by the log-likelihood, which will still be a reasonable number. However, the ‘sum of products’ formula (3) does not lend itself easily to taking the logarithm of both sides. If instead one uses a

348

M. Vidyasagar

multi-step Markov model, then the computation of the log-likelihood is quite straight-forward. Given a long string u of length l, we can write l

Pr{u} =

Pr{Yi = ui |(Yi−k+1 , . . . , Yi−1 ) = ui−k+1 . . . ui−1 } i=l−k+1

=

fui−k+1 ...ui−1 ui . fui−k+1 ...ui−1

As a result, l

log fui−k+1 ...ui−1 ui − log fui−k+1 ...ui−1 .

log Pr{u} = i=l−k+1

Now, in the above summation, all numbers are of reasonable size. Thus it can be said that the traditional HMM is well-suited for problems of speech processing (for example) but not for problems in genomics. Now we come to the problem of inexact partial realization. A problem arises when the probabilities in the equation above are not specified, but need to be inferred from one or more sample paths u1 , . . . , us . In this case, one can still apply the formula (1), but with the actual (but unknown) frequencies fvu and fv replaced by their empirical estimates. Each of these gives an unbiased estimate of the corresponding probability. But this alone is not enough. It is of interest to derive accuracy and confidence bounds, not just on the estimates of the statistics themselves, but on the parameters of the stochastic model. By adding the assumption the stochastic process under study is α-mixing, we succeed in deriving accuracy and confidence bounds both on the estimates of the statistics, and on the estimated parameters of the multi-step Markov model. So far as the author is aware, such bounds have not been published previously. There are several problems in science and engineering that fit into this framework of constructing a stochastic model based on a set of sample paths. Examples include speech recognition (see e.g., [9]), data compression (e.g., the Lempel-Ziv algorithm [26, 27] and its variants), and picture compression (e.g., the GIF file format, which is based on an improvement of the LZ algorithm due to Welch [25]). Note that the Lempel-Ziv-Welch algorithm, commonly known as LZW, is normally used for lossless data compression. However, as shown in [12] for example, the algorithm can be reinterpreted in terms of making predictions (which is another way of saying stochastic modelling). 1.2 Algorithms for Finding Genes The motivation of the current author for studying this problem is finding genes from the genome. The relevant aspects of biology are explained in Section 4, but in essence the problem can be stated as follows (for prokaryote genomes):


349

One is given several examples of “coding regions” and several examples of “noncoding regions,” where each region is a fairly long string (150 to 200 long) over the four-symbol nucleotide alphabet {A, C, G, T }. Then a new stretch of the genome, call it w, is given, and the objective is to classify it as either a coding region (gene) or noncoding region (non-gene). This is accomplished by constructing two different stochastic models, call them C and N C, for coding regions and for non-coding regions respectively. Then the likelihoods pC (w) and pN C (w) of the new string w being generated by the models C and N C are computed. If pC (w) pN C (w), then w is classified as a coding region. Similarly, if pC (w) pN C (w), then w is classified as a non-coding region. If the two likelihoods are of comparable magnitude, then the test is inconclusive. For eukaryotic genomes, there is an additional complication of detecting so-called splice sites, or exon-intron boundaries. But for the purposes of stochastic modelling, the above description still suffices. It turns out that the classification problem is quite different from the stochastic modelling problem, when it comes to implementation. In some ways, classification is actually a simpler problem than modelling. Specifically, in the modelling problem, if a particular string u does not appear as a substring of any of the sample paths, one normally does not take the obvious step of setting the probability of u to zero. This is because, if the ‘true’ probability of u is nonzero but the probability generated by the stochastic model is exactly zero, then the so-called ‘loss function’ is infinite. So, even if a particular string u does not appear anywhere in the sample paths, one normally uses some ad hoc techniques to assign some kind of small, but nonzero, probability to u. See [3], p. 391 et seq. for a discussion of this issue. In contrast, in the classification problem one wishes only to determine which model is more (or less likely) to have produced a particular sequence. In this problem, one not only welcomes zero empirical frequencies, but one can also set small empirical frequencies to zero. This distinction is brought in subsequent discussion. In this way, the size of the stochastic model can be made much smaller in the case of classification.

2 Parameter Estimation for Multi-step Markov Chain: Accuracy and Confidence Bounds 2.1 Formula for Estimating the Transition Matrix of a Markov Chain There is a vast literature on estimating the mean of a function on the basis of randomy generated samples. The classical Monte Carlo simulation is the starting point of this literature. Over time the literature has expanded to encompass the problem of estimating not just one mean, but infinitely many means at once – the so-called ‘uniform convergence of empirical means’ problem. See [19–21] for detailed discussions of this problem.

350

M. Vidyasagar

However, not much attention seems to have been paid to the problem of estimating the transition matrix of a Markov chain from a sample path. We present below an approach that can be used for this purpose. In this section, we present some accuracy and confidence bounds for a natural method of estimating the transition matrix. To state the problem precisely, suppose {Xt } is a Markov chain over the finite alphabet N := {1, . . . , n}, state transition matrix A, and stationary distribution π. Suppose n is known but both A and π are unknown. However, we have available a finite length sample path {x1 , . . . , xl } of this Markov chain. The objective is to form estimates Aˆ and π ˆ for A and π respectively. An obvious way to do this is as follows: Recall that aij = Pr{Xt+1 = j|Xt = i} =

Pr{Xt Xt+1 = ij} . Pr{Xt = i}

Thus we fix a state i, and observe how many times the state i occurs during the stretch X1 , Xl−1 ; call this number µi . Next, observe how many times the pair ij occurs during the stretch (X1 , X2 ), . . . , (Xl−1 , Xl ); call this number νij . With this notation, the estimated value of the element aij of the state transition matrix is given by νij . (4) a îj := µi Thus a îj equals the fraction of times that the state i is followed by the state j. It is shown in [14] that a îj is the maximum likelihood estimate for aij . But there is a small difficulty with the above nice-sounding result. Ultimately, when we wish to determine how far the current estimate of the Markov chain is from the ‘true’ Markov chain, we need not only an estimate Aˆ for A, but also an estimate π ˆ for π. Let Aˆ denote the maximum likelihood estimate for the transition matrix, as defined in (4). If we define π ˆ to be the stationary ˆ then in general π distribution of A, ˆ is not the maximum likelihood estimate of π. Neither is the vector (1/(l − 1))[µ1 . . . µn ]. It is possible to write down necessary conditions satisfied by the simultaneous maximum likelihood estimates of both π and A. However, in general these equations are difficult to solve in closed form. There is one special case, however, in which the problem is tractable. Suppose the sample path is a cycle, that is, X1 = Xl . Then in essence we can ‘wrap around’ the sample path and define the variables as follows: νij now equals the number of occurences of the pair ij in the sequence (X1 , X2 ), . . . , (Xl−1 , X1 ), since Xl = X − 1. Finally, we define a îj :=

νij µi , π î := . µi l

(5)

ˆ In the case of cycles, In this particular case, it can be shown that π ˆ =π ˆ A. therefore, it is easy to obtain a maximum likelihood estimate of the state transition matrix and also obtain, readily, its stationary distribution.


351

What if X1 = Xl ? In this case, we use the longest cycle within the sample path. In other words, given a sample path X1 , . . . , Xl , we throw away bits of the sample path at either end until we get a cycle. Then we apply the formula (5) to the cycle. It is not difficult to see that, if the underlying Markov chain is irreducible, then the length of the longest cycle in any sample path approaches infinity almost surely. Hence, by throwing away a little bit of the sample path, we get a maximum likelihood estimate of A as well as its stationary distribution. This allows us to obtain quantitative estimates of how far the ˆ π estimated chain (A, ˆ is from the ‘true’ chain (A, π). The next step is to derive accuracy and confidence bounds for the estimate in (5). This is done via a series of steps. 2.2 Bernstein’s Inequality for I.I.D. and Alpha-Mixing Samples In this section we present some standard inequalities for binary-valued processes when the underlying samples are independent (which they certainly are not, if the samples come from a Markov chain). Then we show how these inequalities can be extended to the case of α-mixing samples (which includes the case where the samples are from a Markov chain). Suppose {Zt } is an i.i.d. process of random variables taking values in [0, 1] and having variance σ 2 . Let {zt } be a realization of this process, and define 1 ˆ E(Z) := l

l

zi . i=1

ˆ Then E(Z) is called the empirical mean of Z, and converges almost surely to the true mean E(Z) as l → ∞. However, what is of interest to us now is the rate at which this convergence takes place. This is provided by a whole ˆ series of so-called ‘finite time’ estimates, which tell us just how far E(Z) is from E(Z), and with what confidence. Of the many such inequalities, we use Bernstein’s inequality, which states that ˆ Pr{E(Z) − E(Z) > } ≤ exp{−l 2/[2(σ 2 + /3)]} ∀ > 0,

(6)

ˆ Pr{E(Z) − E(Z) < − } ≤ exp{−l 2/[2(σ 2 + /3)]} ∀ > 0.

(7)

See [7, 15] for a proof of this inequality. If Z is a binary-valued process and Pr{Zt = 1} = p, then we have σ 2 = p(1 − p). Let us substitute = γp and σ 2 = p(1 − p) into (6) and (7). This gives the exponent on the right side as −lγ 2 p2 −lγ 2p2 −3lγ 2p ≤ = , 2[p(1 − p) + γp/3] 8p/3 8 after noting that 1 − p ≤ 1, γ ≤ 1. Hence, if Z is a binary-valued process and Pr{Zt = 1} = p, it follows that

352

M. Vidyasagar

ˆ Pr{E(Z) − E(Z) > γp} ≤ exp(−3lγ 2 p/8),

(8)

ˆ Pr{E(Z) − E(Z) < −γp} ≤ exp(−3lγ 2 p/8).

(9)

Note that the so-called Chernoff bounds are slightly tighter than the above. One Chernoff bound states that ˆ Pr{E(Z) − E(Z) > γp} ≤ exp(−lγ 2 p/3), which results from (8) if the 8 in the denominator is replaced by 9. The other Chernoff bound states that ˆ Pr{E(Z) − E(Z) < −γp} ≤ exp(−lγ 2 p/2), which results from (9) if the 3 in the numerator is replaced by 4. However, for present purposes, the Bernstein inequality is good enough, as it is quite close to the Chernoff bound, and can be readily extended to the case of dependent samples. It is possible to extend Bernstein’s inequality to α-mixing processes; this is done in [24]. Recall the definition of α-mixing. (See [4, 5] for further details.) 0 Given a stationary stochastic process {Zt }, we define Σ−∞ to be the σ-algebra generated by the ‘past’ random variables Zt , t ≤ 0, and Σs∞ to be the σ-algebra generated by the ‘future’ random variables Zt , t ≥ s. Then α(s) :=

sup

0 A∈Σ−∞ ,B∈Σs∞

|P (A ∩ B) − P (A)P (B)|.

(10)

Hence the number α(s) measures the extent to which events after time t are independent of events before time 0. Note that α(s) ∈ [0, 1] for all s and that α(s) ≥ α(s + 1). The process {Zt } is said to be α-mixing if α(s) → 0 as s → ∞. Suppose {Zt } is a stationary, α-mixing, zero mean process assuming values in an interval [−F, F ], and we form its empirical mean based on l samples. It is desired to know how close the empirical mean is to the true mean, and with what confidence. In general, if {Zt } is a stationary, α-mixing, and bounded process, but not necessarily zero mean, we can apply the theory below with F := max{ess. sup(Zt − E(Zt )), ess. inf(E(Zt ) − Zt )}. We now quote without proof the relevant result from the paper [24]. Given l samples, choose any integer s ≤ l, and define r := l/s , the integer part of l/s. Let ˆ qu (l, ) = Pr{E(Z) − E(Z) > }, ˆ ql (l, ) = Pr{E(Z) − E(Z) < − }. Then qu (l, ), ql (l, ) ≤ exp

−r 2 + 4α(s)r 2 2(σ + F/3)

1+ F σ2

r

.

(11)


353

Note that (11) reduces to the classical Bernstein inequality if the samples are i.i.d. In this case, we take r = l, s = 1 and note that α(1) = 0 for an i.i.d. process. Now suppose {Zt } is a stationary binary-valued, α-mixing stochastic process. In this case we can take F = 1. Let p denote Pr{Zt = 1}, and pˆ denote its empirical estimate based on l samples. We can use the same trick as earlier to estimate the likelihood that pˆ deviates from p by a percentage of γ. This gives Pr{ˆ p > (1 + γ)p}, Pr{ˆ p < (1 − γ)p} ≤ µ (12) where µ := exp(−3rγ 2 p/8) + 4α(s)r

1 + γp p(1 − p)

r

(13)

Finally, suppose Zt is not binary-valued, but assumes one of m values, and is also α-mixing. Let M denote the set of values assumed by Zt . For each fixed u ∈ M, the process Yu,t = 1 if Zt = u and Yu,t = 0 otherwise is binaryvalued. Moreover, the mean value of Yu,t is precisely Pr{Zt = u}. Hence the preceding theory applies also to processes that assume a finite number of values. For each u ∈ M, let pu denote Pr{Zt = u}, and let pû denote its empirical estimate based on l samples. Then (12) holds with p and pˆ replaced by pu and pû respectively. 2.3 Accuracy and Confidence Estimates for Markov Chains Now we are finally in a position to derive accuracy and confidence bounds for the estimated state transition matrix of a Markov chain. Lemma 1. A Markov chain is α-mixing if and only if its state transition matrix A satisfies the ‘strong Perron property,’ that is, A has only a simple eigenvalue at one, and all other eigenvalues of A have magnitude strictly less than one. The above result is well known and in any case is easy to prove. The hypotheses on A ensure that there is a unique, strictly positive stationary distribution π. Moreover, Al → en π as l → ∞. Note that the i-th row of the matrix Al is the probability distribution of the Markov chain after l steps, if the chain is started off in the initial state i. Thus an α-mixing Markov chain has the property that, irrespective of the starting state, the l-step transition probability distribution approaches the stationary distribution π. Note that α-mixing is a strictly stronger property than ergodicity. In the case of a finite state Markov chain, the corresponding stochastic process is

354

M. Vidyasagar

ergodic if and only if the state transition matrix is irreducible. In such a case, the ‘ergodic average’ l 1 Al → en π as l → ∞. l i=1 However, the power Al can simply oscillate around without converging to anything in particular. A simple example of this behaviour is the two-state chain with 01 A= . 10 For a Markov chain, it is easy to derive a closed form formula for its α-mixing coefficient. Lemma 2. Let A and π denote respectively the transition matrix and the stationary distribution of a Markov chain. Then   α(l) = max πi  i∈N

n

|(Al )ij − πj | .

(14)

j=1

The formula (14) can be interpreted as follows: The mixing coefficient α(l) equals the maximum value, over the state i, of the 1 -norm of the difference between the l-step transition probability (Al )i· and the stationary distribution π, multiplied by πi (which is the probability that the initial state is i). Note that the above formula is valid for all Markov chains, α-mixing or otherwise. Now we are in a position to obtain accuracy and confidence bounds. Recall from (4) that a îj is the ratio of two quantities, namely fîj and π î . If the Markov chain is α-mixing, meaning that the state sequence {Xt } is an αmixing process, we can use (12) and (13) to estimate how far π î is from the true value πi . To estimate how far the ‘doublet frequency’ fîj is from the true frequency fij , we need the next result. Lemma 3. Suppose a process {Yt } is α-mixing. Define the process of k-tuples {Yt } by Zt := (Yt , Yt−1 , . . . , Yt−k+1 ). Then the process {Zt } is also α-mixing. Moreover, and we have αZ (s) = αY (s − k + 1).

(15)

Proof: It is obvious that 0 0 ∞ Σ−∞ (Z) = Σ−∞ (Y), Σs∞ (Z) = Σs−k+1 (Y).

Equation (15) now follows from the definition. Hence if the Markov chain {Xt } is α-mixing, so is the sequence of doublets {Xt Xt+1 }. Now we are able to put all the results together.


355

Theorem 1. Suppose the Markov chain {Xt } is α-mixing and has the stationary distribution π. Given a cyclic sample path of length l, choose an integer s ≤ l, and define r := l/s . Define ci := exp(−3rγ 2 πi /8) + 4α(s)

1 + γπi πi (1 − πi )

dij := exp(−3rγ 2 fij /8) + 4α(s − 1) Then

r

,

1 + γfij fij (1 − fij

(16) r

.

(17)

Pr{ˆ πi > (1 + γ)πi }, Pr{ˆ πi < (1 − γ)πi } ≤ ci , Pr{fîj > (1 + γ)fij }, Pr{fîj < (1 − γ)fij } ≤ dij ,

(18) (19)

1+γ 1−γ Pr{ˆ aij > aij }, Pr{ˆ aij < aij } ≤ ci + dij . 1−γ 1+γ

(20)

2.4 Confidence and Accuracy Bounds for Partial Realizations The preceding results extend in a straight-forward manner to the partial realization discussed in (1), for (k − 1)-step Markov chains. The only difference is that, in estimating the entry aij of a Markov chain, we form the ratio fîj /πi , whereas in (1) we form the ratio fˆvu /fˆv for v ∈ Mk−1 , u ∈ M. Theorem 2. Let {Yt } be an α-mixing stationary stochastic process assuming values over the finite alphabet M. For a fixed integer k, let fu denote the frequency of the string u ∈ Mk , and let fû denote its empirical estimate based on a sample path of length l. Choose any integer s ≤ l, and define r := l/s . Define r 2 1 + γfu , (21) c(k, u) := e−3lγ fu /8 + 4α(s − k + 1)r fu (1 − fu ) c(k − 1, v) := e−3lγ

2

fv /8

+ 4α(s − k + 2)r

1 + γfv fv (1 − fv )

r

.

(22)

Then we have the following relationships: First, for each u ∈ Mk , we have Pr{fû > (1 + γ)fu }, Pr{fû < (1 − γ)fu } ≤ c(k, u).

(23)

Next, for each v ∈ Mk−1 , we have Pr{fˆv > (1 + γ)fv }, Pr{fˆv < (1 − γ)fv } ≤ c(k − 1, v). Finally, for all u ∈ M

k−1

(24)

, v ∈ M, we have

Pr{

1 + γ fuv fûv } ≤ c(k, uv) + c(k − 1, u), > ˆ 1 − γ fu fu

(25)

Pr{

fûv 1 − γ fuv } ≤ c(k, uv) + c(k − 1, u). < ˆ 1 + γ fu fu

(26)

356

M. Vidyasagar

3 The 4M Algorithm and Its Analysis 3.1 The 4M Algorithm In this section, we introduce a new algorithm for stochastic modelling called the 4M (Mixed Memory Markov Model) algorithm. The idea of the 4M algorithm is to begin with a multi-step Markov model and to reduce the size of the state space further by using a criterion for determining whether some states are ‘Markovian.’ Recall the essence of the multi-step Markov model. Given frequencies of all k-tuples u ∈ Mk , one defines a Markov chain with mk−1 states, whereby a transition from a state v to a state u is possible if and only if the last k − 2 symbols of v are the same as the first k − 2 symbols of u, i.e., if v = v1 v2 and u = v2 u3 for some v2 ∈ Mk−2 , v1 , u3 ∈ M. In this case, the transition probability is defined as Pr{Yt+1 = u|Yt = v} =

fvu3 . fv

(27)

Note that the size of the state space is mk−1 . In the case of gene finding (discussed subsequently), we have m = 4 and usually k ≥ 6. Since 46 = 4, 096, the resulting Markov chain is rather large. To reduce the size of the state space, we begin with the following simple observation. Let us switch notation and consider a Markov chain {Xt } evolving over an alphabet N := {1, . . . , n}. Let i, j, k ∈ N and consider the frequency of the string ijk. Clearly for any process (Markov or not), we have fijk = Pr{k|ij} · Pr{ij}. Note that in the above formula we simplify notation by writing Pr{k|ij} := Pr{Yt+1 = k|Yt−1 = i&Yt = j} and so on. Now, because of the Markovian nature of the process, we have Pr{k|ij} = Pr{k|j}, ∀i ∈ N . Hence, if we examine the n × n matrix Fj := [fijk , i, k ∈ N ], it will have rank one. This is because, with j fixed, it looks like  f1j   Fj =  ...  · [Pr{1|j} . . . Pr{n|j}]. 

fnj


357

The above reasoning can be applied to an individual state j. Specifically, suppose we are examining k-tuple frequencies, and suppose v ∈ Ml for some l < k−1. We define a state v ∈ Ml to be a Markovian state if the associated mk−l−1 × m matrix Fv := [fuvw , u ∈ Mk−l−1 , w ∈ M]

(28)

has rank one. If this is indeed so, then we can conclude that Pr{w|uv} = Pr{w|v} ∀u ∈ Mk−l−1 .

(29)

If the above equation holds, then we can ‘collapse’ all mk−l−1 states of the form uv for all u ∈ Mk−l−1 into a single state v. Recall that when we compute the likelihood of a string w of length s, we write s

Pr{w} =

Pr{wi+1 |wi−k+1 . . . wi } · Pr{w1 . . . wk−1 }. i=k−1

If a substring of the string wi−k+1 . . . wi is Markovian, say wi−l+1 . . . wi , then we can do the substitution Pr{wi+1 |wi−k+1 . . . wi } = Pr{wi+1 |wi−l+1 . . . wi } in the above formula. This is the reason for calling the algorithm ‘mixed memory Markov model’ since different (k − 1)-tuples uv have memories of different lengths. 3.2 Bounds on the Tail Probability for Markov Chains The preceding theory is applicable provided we use true probabilities in the various computations. However, in setting up the multi-step Markov model, we are using empirically observed frequencies and not true probabilities. Thus it is desirable to analyze the consequences of applying such simplifications to observed frequencies as opposed to true probabilities. This is done in two different ways. In the current subsection, we assume that a state v is Markovian, and that we are given a sample path of finite length. In this case, the true but unknown matrix of probabilities Fv := [fuvw , u ∈ Mk−l−1 , w ∈ M] is approximated by the matrix of observed frequencies Fˆv := [fûvw , u ∈ Mk−l−1 , w ∈ M]. If the string v is a Markovian state, the matrix Fv will have rank one. However, the matrix Fˆv of empirical frequencies will not exactly have rank one, but will

358

M. Vidyasagar

in some sense be ‘close’ to a rank one matrix. The objective of this subsection is to make this statement quantitatively precise using (25) and (26). Note that arbitrarily small perturbations of a rank one matrix (or indeed, any rank-deficient matrix) can result in a full-rank matrix. Hence the emphasis here is on quantifying the notion of a given matrix being close to a rank one matrix, as opposed to the rank itself being close to one. (An entirely different approach based on divergence between Markov chains is adopted in the next subsection.) For this purpose, we use the distance measure |fûvw − fuvw |.

ρ(Fˆv , Fv ) := u∈Mk−l−1 ,w∈M

Clearly ρ is just the 1 -norm of the vector obtained by stacking all the columns of the difference Fˆv − Fv on top of each other. Moreover, ρ(Fˆv , Fv ) measures how close the matrix Fˆv is to the rank one matrix Fv . In the sequel, we derive estimates of the ‘tail probability’ that ρ(Fˆv , Fv ) exceeds a given small number. Suppose we form the matrix Fˆv of empirical frequencies. Then, as stated in (25) and (26), we have with confidence 1 − 2c(k, uvw) that 1+γ 1−γ fuvw ≤ fûvw ≤ fuvw . 1+γ 1−γ In turn this implies that, with confidence 1 − 2c(k, uv), we can say |fûvw − fuvw | ≤

2γ fuvw . 1−γ

(30)

Now suppose (28) is satisfied. Then it is easy to see that fuvw = fv . u∈Mk−l−1 ,w∈M

Define

cv :=

c(k, uvw). u∈Mk−l−1 ,w∈M

Then it readily follows from (30) that Pr{ρ(Fˆv , Fv ) >

2γ fv } ≤ 2cv . 1−γ

(31)

This bound establishes that, if we examine a sample path of an α-mixing Markov chain, and if we compute the matrix Fˆv of empirical frequencies, then ‘tail’ probability of the matrix Fˆv being ‘close’ to a rank one matrix is given by (31).


359

3.3 Kullback-Leibler Divergence Rates Between Markov Models While the preceding analysis is mathematically rigorous, it is difficult to implement in practice. To simplify the implementation of the 4M algorithm, we define a state v ∈ Ml to be a Markovian state if σ2 (Fv ) ≤ σ1 (Fv ),

(32)

where σ1 , σ2 denote respectively the largest and the second largest singular value of a matrix, and is an adjustable parameter. In principle the distance measure ρ(Fˆv , Fv ) can be related to the above criterion, but in practice the estimates will be hopelessly conservative. Instead, in this subsection we derive estimates of how ‘close’ the reduced order mixed memory Markov model is to the original full order (k − 1)-step Markov model. For this purpose, we need a measure of disparity between Markov chains. Such a measure is provided by the Kullback-Leibler (K-L) divergence rate. Suppose p, q are probability distributions on a finite set {1, . . . , n}. Then the Kullback-Leibler (K-L) divergence H(p q) is defined by n

H(p

pi log(pi /qi ).

q) := i=1

In order for the K-L divergence to be finite, it is necessary that pi = 0 whenever qi = 0. If this is the case, then we write p q or q p, read as ‘p is dominated by q’ or ‘q dominates p.’ The K-L divergence has a very nice intuitive interpretation. Suppose we observe a sample path {x1 , . . . , xl } of an i.i.d. process, and we know ahead of time that the underlying probability distribution is either p or q. To discriminate between the two competing hypotheses, we use the log-likelihood ratio. The likelihoods of the sample path arising if the underlying distribution is p or q are, respectively n

Lp = i=1

pνi i , Lq =

n i=1

qiνi ,

where νi denotes the number of occurences of i in the sample path. Therefore the log likelihood ratio normalized for the length of the sample is n

(νi /l) log(pi /qi ).

(1/l) log(Lp /Lq) = i=1

Now, suppose the ‘truth’ is that the underlying probability distribution is p. Then the expected value of each νi equals pi l, and the expected value of the normalized log likelihood ratio is precisely H(p q). The preceding argument is fine for i.i.d. processes. Suppose now that we observe a sample path of a stochastic process, and the two competing hypotheses are that the joint law of the process is p or q. Note that now both p

360

M. Vidyasagar

and q are probability measures on the infinite cartesian product of the state space {1, . . . , n}. In theory, we could compute the K-L divergence between the two laws by replacing the ratio pi /qi by the Radon-Nikodym derivative and the summation by an integral. However, except in very trivial cases, the divergence will be infinite and will therefore be a meaningless number. To get around this difficulty, we use the K-L divergence rate, defined as follows: For each integer l, let p( l) and q(l) denote the marginal measures of p and q respectively on {1, . . . , n}l . Then the K-L divergence rate is defined as R(p

q) := lim

l→∞

1 H(p(l) l

q(l) ),

assuming that the limit exists. This notion was apparently first introduced in [10]. The K-L divergence rate has the same intuitive interpretation as the original K-L divergence. Suppose we observe a sample path of a stochastic process, and we wish to discriminate between two alternate hypotheses, namely: the underlying law is p (the truth), and the underlying law is q. Then the normalized log likelihood ratio asymptotically looks like R(p q). In [17], an explicit formula is given for the K-L divergence rate between two Markov chains over the same state space. Suppose A, B are two stochastic matrices of dimension n × n, and let φ, θ denote their (possibly nonunique) stationary distributions. Suppose both Markov chains are started off in the initial distribution φ. Let p denote the (stationary) law of the process with the transition matrix A, and let q denote the (possibly nonstationary) law of the process with the transition matrix B. Let ai , bi denote the i-th rows of the matrices A and B respectively, and note that each is a probability distribution. Then we have the following result: Theorem 3. Suppose bij = 0 implies that aij = 0 for all i, j. Let φ∗ denote the ergodic limit l 1 φ∗ := lim Ai . l→∞ l i=1 (Note that φ∗ = φ if A is irreducible.) Then n

R(p

q) =

φ∗i H(ai

bi ).

(33)

i=1

Since a multi-step Markov model is just a one-step Markov model over a larger state space, the above formula readily generalizes to multi-step Markov models. This theorem can be readily applied to derive a simple formula for the K-L divergence rate between the original (k − 1)-step Markov model and the 4M reduced order model.


361

Theorem 4. Suppose we are given the frequencies of all k-tuples, and we choose various tuples as ‘Markovian states.’ Let v1 , . . . , vs denote the Markovian states and let l(vi ) < k denote the length of the Markovian state vi . Then the K-L divergence rate between the original (k − 1)-step Markov model and the 4M reduced Markov model is given by s

fuvi log

K= i=1 u∈Mk−1−l(vi ) w∈M

fuvi w fvi · . fuvi fvi w

(34)

Note that, in applying the 4M algorithm, what we are doing is to approximate the ratio fuvi w /fuvi by the ratio fvi w /fvi for each string vi that is deemed to be a Markovian state. Hence the quantity inside the logarithm in (34) should be quite close to one, and its logarithm should be close to zero. The proof of the theorem is fairly easy and is left to the reader. Note that the full (k − 1)-st order Markov model has exactly m nonzero entries in each row labelled by u, and these entries are fuw /fu as w varies over M. One can think of the reduced order model obtained by the 4M algorithm has containing the same mk rows, except that if v is a Markovian state, then the entry in all rows of the form uv are changed from fuvw /fuv to fvw /fv . The vector fu , u ∈ Mk−1 is a stationary distribution of both stochastic matrices. Now the formula (34 readily follows from (33).

4 Application to Gene Finding Algorithms In this section, we present a concrete application of the 4M algorithm by applying it to the problem of finding genes in prokaryote genomes. 4.1 Introduction to Genomics Let us begin with an introduction to the relevant aspects of biology. All living things are made up of DNA, or deoxyribonucleic acid. In humans and most ‘higher’ (i.e., highly evolved) organisms, DNA is shaped like a double helix, as discovered by Watson and Crick more than fifty years ago. In some ‘lower’ forms of life such a viruses, the DNA is shaped like a circle. In both cases, DNA consists of a sequence of nucleic acids, or nucleotides. There are just four nucleotides, namely: Adenine (A), Cytosine (C), Guanine (G), and Thymine (T). Thus one can think of DNA as just a very long string of symbols coming from the four symbol alphabet N := {A, C, G, T }. In the case of organisms with a double helix structure to the DNA (such as humans), each strand of the helix has a starting point (called the 5’ end) and an end point (called the 3’ end). Each strand has a definite spatial orientation – it cannot be ‘read backwards.’ The definite spatial orientation permits us to treat the spatial sequence as a temporal sequence and then to apply methods of time series analysis. The two strands run in opposite directions; thus the

362

M. Vidyasagar

5’ end of one side corresponds to the 3’ end in the other side, and vice versa. Considerations of chemical bonding ensure that if one strand of the DNA has an A in a particular location, in the corresponding location in the other strand must be a T , and vice versa. Similarly, C corresponds to G and vice versa. Thus, if one knows the sequence of nucleotides that make up one strand of the DNA, one would be able to deduce unambiguously what the other strand is. One would have to replace A by T , C by G, and vice versa, and then run the sequence backwards. This is a process known as reverse complementation. The listing of one strand, which is just a very long string over the four symbol nucleotide alphabet, is called the ‘genome’ of an organism. The genome is the ‘lowest’ level description of an organism. 4.2 The Gene Finding Problem The next level of complexity is given by genes. The process of life requires that DNA is transcribed to RNA which in turn produces several proteins. For the present purposes, it is enough to think of RNA as another string over the alphabet of RNA nucleotide symbols {A, C, G, U }. Thus, so far as we are concerned, the symbol T in the genome has been replaced by U . Triplets of RNA nucleotide symbols are referred to as “codons.” Clearly there are 43 = 64 codons. Out of these 64 codons, three are stop codons (UAA, UAG, UGA), while the remaining 61 codons produce 20 amino acids. A table giving the correspondence between the 61 codons and the 20 amino acid symbols, as well as sketches of the shapes of the 20 amino acids, can be found in [2], pp. 137138. It is therefore apparent that there is a redundancy in the map between the 64 codons and the 20 amino acid symbols plus the STOP symbol. There are several theories to explain the rationale and the nature of this redundancy. However, to date no universally accepted explanation has been advanced for the logic behind this map. This question does not concern us in the present paper. In so-called lower level organisms called prokaryotes, the entire genome consists of just genes and non-coding regions (non-genes). Each gene is a continuous, unbroken stretch of codons, each of which generates the corresponding amino acid. The sequence of amino acids forms the protein coded by the gene. Thus it is clear that the length of each gene is an exact multiple of three. The regions between genes are the so-called non-coding regions, and their length need not be a multiple of three. In eukaryotes, the genes themselves need not be contiguous, but are divided up into ‘introns’ that separate ‘exons.’ The exons come together at the time of reproduction to code for proteins. Suppose now that a new organism, a prokaryote, has its genome sequenced. We would like to know which parts of the genome correspond to genes, and if possible, what their function is. This can be done either through computation or by experiment. Computational algorithms can only ‘predict’ that a stretch of DNA is a gene, but this prediction can be verified only through experiment. However, experimentation is both costly and time-consuming, so it is highly


363

desirable to have computational algorithms that can predict which parts of the genome correspond to genes with a fair degree of accuracy. In making such predictions, obviously there is a trade-off between false positives and false negatives. A false positive is the situation where a string is predicted to be a gene, but is proven not be a gene by subsequent experimentation, while a false negative is the converse. From the standpoint of biology, it is much more preferable to have a false positive than a false negative. One does not mind performing a few unnecessary experiments, but one would definitely not like to ‘miss’ a gene. The problem of gene finding can be put into the previous formalism as follows: Suppose we have r gene sequences, denoted by u1 , . . . , ur , and s nongene sequences, denoted by v1 , . . . , vs , each of which is a string over the four-symbol nucleotide alphabet. The premise is that u1 , . . . , ur are all sample paths of one stochastic process, while v1 , . . . , vs are all sample paths of another stochastic process. The aim therefore is to construct two models, denoted by C (for ‘coding’ model) and N C (for ‘non-coding’ model) respectively, such that the model C produces the gene sample paths with high probability and the non-gene sample paths with low probability, while the model N C has the opposite behaviour. Then, given a new sequence w, we compute the likelihood of the sequence w from each of the two models. If PrC (w) PrN C (w), we declare that w is a gene, whereas if PrC (w) PrN C (w), we declare that w is a non-coding region. The use of multi-step Markov models is quite popular in the biology literature, because they can exactly match the observed frequencies of k-tuples, as shown in Section 1. Some references that apply this approach to problems of gene finding are [6, 11, 13, 18]. In biology, it is normal to take k to be anywhere from zero to eight. (Note that a zeroth order model presumes that the sample path is that of an i.i.d. process.) In Glimmer [18], which is among the more popular algorithms, the model is assumed to be a weighted average of k-step Markov models for k ranging from zero to eight. There is just one more point to be noted. Both gene sequences and (to a lesser extent) non-coding regions display three periodicity. This means that neither of the Markov chains C, N C is stationary. Instead, the quantity Pr{ut |ut−1 , ut−2 , . . . , ut−k }. depends on t with a periodicity of three. Moreover, if a particular stretch of genome on the ‘forward strand’ of DNA is a gene, it does not follow that the reverse complemented sequence is also a gene – often it is not. Thus the model C for the coding regions is actually a collection of six models, corresponding to t = 0, 1, 2 mod 3, and on the forward and reverse strands. Similar remarks apply to the non-coding regions. But it should be mentioned that perhaps the effects of three periodicity and forward-backward strands are in general less pronounced in the non-coding regions than they are in the coding regions. Moreover, the effect of forward strand versus reverse strand is not that significant even for coding regions.

364

M. Vidyasagar

The disadvantage of a method such as Glimmer based on multi-step markov models is that it leads to Markov models with huge state spaces, of up to 48 = 65, 536 states. Instead, by using the 4M algorithm described in the previous section, it is possible to achieve a reduction in the size of the state space by as much as 50%. 4.3 Computational Results The 4M algorithm was applied to several prokaryote genomes. (To apply the algorithm to eukaryote genomes, it is necessary to modify it slightly to detect so-called splice sites, or intron-exon boundaries. This is not difficult and we will take up such examples next.) The first table shows the results when the 4M algorithm is applied to various bacteria, and demonstrates its efficacy in predicting both coding regions as well as non-coding regions accurately. Throughout, we use k = 6, which means that the original model was a fith-order Markov model that perfectly reproduced all sextuplet frequencies. In applying the algorithm, the parameter in (32) taken as 0.1. In each organism, the coding regions have been divided arbitrarily into two classes, and a model C is developed based on half of the coding regions. Similarly, the noncoding regions have been divided arbitrarily into two classes, and a model N C is developed based on half of the noncoding regions. Then the remaining halves of the coding and noncoding regions are classified using the two models. Thus the first line of the table means that, for the bacterium B. subtilis, we have divided the 4,106 known or putative genes arbitrarily into two sets of 2,053 strings each. One of the two sets is used to train the model C. Similarly the regions known not to be genes are divided into two classes and used to train the model N C. Then the other set of 2,053 genes (known and putative) are classified using the two models C and N C. Out of these 2,053 genes, 2,011 were classified correctly. Thus the first line of the table means that when 2,053 strings (that are known to be coding regions) are classified according to the two models, 98.4% are correctly classified as coding regions. Similarly, out of the 2,286 strings that are known to be noncoding regions, 2,202 are correctly classified as noncoding regions.


Organism

365

No of No of % of No of No of % of genes in genes genes non non non test set prepregenes in genes genes dicted dicted test set preprecorcordicted dicted rectly rectly corcorrectly rectly 2053 2011 97.9 2286 2202 96.3 py- 745 741 99.4 620 600 96.7

B.subtilis Helicobacter loriJ99 Archaeoglobus 1209 fulgidus Ecoli K12 2127 Haemophilus in- 828 fluenzae Mycoplasma geni- 241 talium Thermotoga mar- 929 itima Treponema pal- 518 lidum Borrelia burgdor- 425 feri

1190

98.4

788

729

92.5

2025 824

95.2 99.5

2298 944

2225 920

96.8 97.4

240

99.5

118

85

72.03

922

99.2

414

308

74.3

514

99.2

432

170

39.3

424

99.7

282

135

47.8

The next table shows the results of ‘whole genome analysis.’ In this study, we begin with so-called ORF (Open Reading Frames), which are putative genes. Using some training set, we classify the ORF’s as to being genes or non-genes. The column marked ‘Correct Starts’ is not as significant as the two columns marked ‘Matches,’ which correspond to the accuracy of predictions. The table also shows a comparison between the prediction accuracies of the 4M algorithm versus Glimmer. It can be seen that the results are quite comparable. One column of the table shows the genes that are ‘missed.’ These ‘missed’ genes are further classified into those that are known to be genes (which is a bad thing for an algorithm to miss), and those that are only hypothesized genes (in which case it is not so bad to miss them, since they may not be genes after all). It can be seen that most of the ‘missed’ genes are only hypothesized and not true genes. The last column shows the number of ‘additional’ genes that are predicted by the 4M algorithm and not by Glimmer. This last column is of great interest, as the true payoff for any algorithm comes from ‘predicting’ new genes that are subsequently confirmed by experimentation. There is one very interesting organism, namely Mycoplasma genatalium. For this organism, both the 4M algorithm as well as Glimmer perform quite

366

M. Vidyasagar

poorly. Given how well both algorithms perform for other organisms, we believe that the results say more about the organism than about the algorithms. Organism

No. of Correct Matches Matches Missed Add’l Genes Starts (4M) (Glimmer) Real Genes Genes B. subtilis 4106 2382 3874 3989 30, 202 117 (58.01%) (94.34%) (97.15%) (5.65%) Helicobacter 1491 1166 1470 1468 2, 19 61 pylori J99 (78.20%) (98.59%) (98.45) (1.4%) Archeoglobus 2420 2093 2348 2356 3, 69 451 fulgidus (86.48%) (97.02%) (97.35%) (2.97%) E. Coli K12 4254 3358 3986 4082 123, 145 81 (78.93%) (93.70%) (95.95%) (6.29%) Haemophilus 1657 1464 1639 1638 2, 16 103 influenzae (88.35%) (98.91%) (98.85%) (1.08%) Treponema 1036 687 1016 979 1, 19 13 pallidum (66.31%) (98.06%) (94.49%) (1.93%) Mycoplasma 484 129 (26) 318 337 94, 72 418 genitalium (26.65) (65.70%) (69.62%) (34.29%) Thermotoga 1858 1214 1833 1816 2, 23 29 maritima (65.33%) (98.65%) (97.73%) (1.34%) Borrelia 851 660 835 823 1, 15 19 burgdorferi (77.5%) (98.11%) (96.70%) (01.88%) The next table shows the reduction in the dimension of the state space that results from using the 4M algorithm. Recall that we are perfectly matching sextuplet frequencies, which means that the original model is a 5th order Markov process, and therefore has 45 = 1, 024 states. In this table, we have ignored the effect of the forward versus reverse strand. Hence there six models (t = 0, 1, 2 mod3 for both coding and non-coding regions), with 1,024 states each. Organism C0 C1 C2 N0 N1 N2 Archaeoglobus fulgidus 418 493 514 1006 997 985 B.subtilis 148 118 196 700 640 691 E. coli 385 283 283 781 865 892 Helicobacter pylori J99 550 475 631 931 928 913 Haemophilus influenzae 478 364 418 886 898 889 Mycoplasma Genitalium 769 547 655 985 973 994 Treponema pallidum 598 544 592 1021 1024 1015 Thermotoga maritima 328 388 316 1021 1018 1012 Borrelia burgdorferi 436 392 526 904 886 922


367

It is clear that there is significant reduction in the size of the state space in the coding regions, but not so in the non-coding regions. This is because there is much less structure in non-coding regions. The most dramatic reduction is in the case of B. subtilis, where the state space size reduces by nearly 90%. Finally, to round out the results, we vary the parameter in (32). We use a value of = 0.1 to begin with, and then adjust it up or down. The purpose is to show that the value of is not significant by itself. Rather, the significant quantity is the value of the K-L divergence rate between the full (k − 1)-th order Markov model and the reduced state space model generated by the 4M algorithm. Two organisms were analyzed, namely B. subtilis and E. coli k 12. In each case, the table shows the K-L divergence rate between the ‘true’ model for frames C0, C1 and C2, and the reduced order model, as a function of . It also shows the percentage of genes classified correctly (in all three frames). The performance appears to start falling off when the K-L divergence rate between the true model and the 4M model is above 0.01. Based on this rule of thumb, we can in fact choose different values of the threshold in different frames, so as to ensure that the K-L divergence is not more than 0.01. However, this has not been done as yet. Results for B. subtilis: C0

C1

C2

%Genes Predicted Correctly 0.1 0.012 0.009 0.011 97.90 0.2 0.029 0.027 0.032 97.00 0.25 0.029 0.031 0.115 92.20 0.3 0.029 0.031 0.115 92.20 0.4 0.029 0.031 0.115 92.20 0.5 0.029 0.031 0.115 92.20 Results for E. coli k 12: C0

C1

C2

%Genes Predicted Correctly 0.1 0.009 0.034 0.050 95.2 0.2 0.074 0.077 0.082 93.9 0.25 0.081 0.083 0.089 93.7 0.3 0.081 0.083 0.151 91.0 0.4 0.081 0.083 0.179 90.1 0.5 0.081 0.083 0.179 90.1

368

M. Vidyasagar

5 Conclusions The problem of fitting a set of sample paths with a multi-step Markov chain model arises in a variety of applications, such as data compression, speech recognition, and finding genes from a genome. In this paper, we have presented, perhaps for the first time, accuracy and confidence bounds for the parameters of a multi-step Markov chain estimated from finite length sample paths. A new algorithm, called the 4M (Mixed memory Markov model) algorithm is presented, for constructing multi-step Markov models with reduced state space. This method is better suited for problems of classification than for problems of prediction. The 4M algorithm is then applied to the problem of finding genes in several prokaryote genomes, and gives excellent results. With slight modifications, the algorithm can also be applied to eukaryotes, and this work is under way.

References 1. B. D. O. Anderson, “The realization problem for hidden Markov models,” Mathematics of Control, Signals, and Systems, 12(1), 80-120, 1999. 2. P. Baldi and S. Br¨ unak, Bioinformatics: A Machine Learning Approach, MIT Press, Cambridge, MA, 2001. 3. R. Begleiter, R. El-Yaniv and G. Yona, “On prediction using variable order Markov models,” Journal of Artificial Intelligence Research, 22, 385-421, 2004. 4. P. Billingsley, Probability and Measure, Wiley, New York, 1986. 5. P. Billingsley, Probability and Measure, (Third Edition), Wiley, New York, 1995. 6. A. L. Delcher, D. Harmon, S. Kasif, O. White and S. L. Salzberg, “Improved microbial gene identification with GLIMMER,” Nucleic Acids Research, 27(23), 4636-4641, 1999. 7. L. Devroye, L. Gyorfi and G. Lugosi, A Probabilistic Theory of Pattern Recognition, Springer, 1996. 8. W. J. Ewens and G. R. Grant, Statistical Methods in Bioinformatics, SpringerVerlag, New York, 2001. 9. F. Jelinek, Statistical Methods for Speech Recognition, MIT Press, Cambridge, MA, 1997. 10. B-H. Juang and L. R. Rabiner, “A probabilistic distance measure for hidden Markov models,” AT&T Tech. J., 64(2), 391-408, February 1985. 11. A. Krogh, I. S. Mian and D. Haussler, “A hidden Markov model that finds genes in E. coli DNA,” Nucleic Acids Research, 22(22), 4768-4778, 1994. 12. G. Langdon, “A note on the Ziv-Lempel model for compressing individual data sequences,” IEEE Transactions on Information Theory, 29, 284-287, 1983. 13. W. H. Majoros and S. L. Salzberg, “An empirical analysis of training protocols for probabilistic gene finders,” BMC Bioinformatics, available at http://www.biomedcentral.com/1471-2105/5/206, 21 December 2004. 14. J. R. Norris, Markov Chains, Cambridge University Press, Cambridge, UK, 1998. 15. D. Pollard, Convergence of Stochastic Processes, Springer-Verlag, 1984.


369

16. L. W. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, 77(2), 257-285, February 1989. 17. Z. Rached, F. Alalaji and L. L. Campbell, “The Kullback-Leibler divergence rate between Markov sources,” IEEE Trans. Info. Thy., 50(5), 917-921, May 2004. 18. S. L. Salzberg, A. L. Delcher, S. Kasif and O. White, “Microbial gene identification using interpolated Markov models,” Nucleic Acids Research, 26(2), 544-548, 1998. 19. V. N. Vapnik, Statistical Learning Theory, John Wiley, New York, 1998. 20. M. Vidyasagar, A Theory of Learning and Generalization, Springer-Verlag, London, 1997. 21. M. Vidyasagar, Learning and Generalization with Applications to Neural Networks, Springer-Verlag, London, 2003. 22. M. Vidyasagar, “The realization problem for hidden Markov models: The complete realization problem,” submitted for publication. 23. M. Vidyasagar, “The realization problem for hidden Markov models: The partial realization problem,” submitted for publication. 24. M. Vidyasagar, “Convergence of empirical means with alpha-mixing input sequences, and an application to PAC learning,” submitted for publication. 25. T. A. Welch, “A technique for high performance data compression, IEEE Computer, 17(6), 8-19, 1984. 26. J. Ziv and A. Lempel, “A universal algorithm for sequential data compression,” IEEE Transactions on Information Theory, 23(3), 337-343, 1977. 27. J. Ziv and A. Lempel, “Compression of sequences via variable rate encoding,” IEEE Transactions on Information Theory, 24(5), 530-536, 1978.

Feedback Networks Glenn Vinnicombe1 Department of Engineering, University of Cambridge [email protected]

1 Introduction This paper presents a survey of some recent results which are primarily concerned with the action of relatively simple controllers working in complex networks. Feedback system design is all about getting the right amount of gain into the right places. In the design of a controller for a given linear multivariable system (with maybe an uncertain frequency response, but with a known topology of inputs and outputs), this means getting the right amount of gain into the right frequency ranges and in the right directions. H∞ theory, to which Keith Glover has of course contributed a great deal, tells us a lot about managing these tradeoffs (the H∞ loop shaping design method in particular [1]). Here though, we are not concerned with complex multivariable systems, but rather simple scalar systems which are perfectly well controlled by simple controllers – or rather they would be if only they weren’t also connected to other systems who are in turn connected with more systems in a complex web of feedback interactions. A further complication in the applications we have in mind is that we rarely have information about the whole network, we only know the local interactions. A natural question again is “how do we get the right amount of gain into the right places”. The sorts of networks we are thinking of are communication networks, economic networks, power distribution networks, coordinated networks of vehicles and biological (gene regulatory networks in particular). In fact, nature appears to use far more feedback regulation in the cell than we typically dare use in our technologies (an exception is the Internet). It is well known to engineers that feedback can’t have a uniformly good effect; if it makes some things better it will always make other things worse. Managing those tradeoffs in large regulatory networks is extremely difficult. So difficult, in fact, that engineers typically don’t try. It’s a mystery how nature manages it. This paper falls into two parts. First, we’ll look at some tradeoffs in simple feedback loops from a biological perspective, taking as an example the regulation of concentration of a single molecular species. This will involve a fresh B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 371–387, 2006. © Springer-Verlag Berlin Heidelberg 2006

372

G. Vinnicombe

slant on some classical results. Fluctuations in the numbers of molecules are inevitable, we’ll see that delays provide a lower bound on the variance reduction possible using feedback. Furthermore, feedback in biochemical pathways almost always has to proceed via intermediate molecules. We’ll see that limits on the metabolic burden imposed by these pathways in turn place limits on their information carrying capacity. This in turns limits the potential benefits of feedback. In the second part of the paper, we’ll look at one way in which it is possible to manage feedback gain in large networks. If the network is sufficiently symmetric then it is possible to predict properties of the network from local feedback interactions. Indeed, if everyone satisfies a convexified Nyquist-like criterion with their neighbours then the network will be stable.

2 Regulation of Molecule Numbers Consider a molecular species being produced at a constant average rate c with each molecule degrading at average rate α, so the total degradation rate is αn. This is a good model of constitutive (i.e. uncontrolled) gene expression, with 1st order degradation. In this case n would be the number of molecules of a particular mRNA. (no feedback)

n

c

n+1

n

αn

n−1

That is, we have a jump Markov process, where the probability of an mRNA being produced in time dt is c dt and the probability of one being degraded is αn dt. The number of molecules will fluctuate around the steady state values c = αn0 ⇒ n0 =

c α

In fact, for a linear system such as this, the first and second order statistics are identical to those of solutions to the SDE n˙ = c − αn + 2cη, where η is a unit variance Gaussian white noise. This is in spite of the fact that in the Markov process n is an integer, whereas here it is real, and also the true statistics are Poissonian as opposed to Gaussian. Without feedback, the equilibrium concentration is tied to the degradation rate. Consider a change in α, α → α + δα , n0 → n0 + δn c = (α + δα )(n0 + δn ) ≈ αn0 + n0 δα + αδn ⇒

δα δn ≈− n0 α

Feedback Networks

373

The relative change in the equilibrium concentration, or expected number, of the species is equal to the relative change in the degradation rate. A 10% change in α leads to a 10% change in n, roughly. We can improve this with feedback, of course, by making the rate of production of our species depend on its concentration. (with feedback)

n n

f (n) αn

n+1 n−1

In steady state: f (n0 ) = αn0 . If we perturb: α → α + δα and n0 → n0 + δn then f + f δn ≈ αn0 + n0 δα + αδn δn 1 δα ⇒ =− f n0 1− α α Sensitivity

If the derivative of the function f is negative, the relative change is reduced by a factor equal to the sensitivity function. −f /α here corresponds to the total feedback gain. Graphically all that’s happening is that by steepening the birth rate, we reduce the movement of the intersection of the birth and death curves as the death rate changes. f

n Of course, it is always true that, for small perturbations relative error in = S × relative error in closed loop open loop where the sensitivity function S is given by S=

1 1+L

and L is the amount of feedback (loop gain). In this example, the sensitivity can be made arbitrarily small, by making f large and negative. However, in practice it is rarely possible for the species n to directly control its own birth rate; the control must proceed via intermediate species. In addition, there will always be some delay in the feedback. Both of these factors limit the potential benefits of feedback. These limitations are well understood in the deterministic case, here we discuss their extension to the stochastic case, where the underlying processes are jump Markov processes.

374

G. Vinnicombe

2.1 Feedback Channels In practice, a species can’t usually control its birth rate directly. Because of the biochemistry, there will need to be a second molecular species, with the birth rate of the first species depending on the concentration of the second, and the birth rate of the second species depending on the concentration of the first. An example might be that the first species is an mRNA, and the second a DNA binding protein it codes for. So the rate of transcription depends on the protein concentration and the rate of translation on the the mRNA concentration. System:

Feedback:

n n m m

f (m) αn

g(n) βm

n+1 n−1 m+1 m−1

We will give details of the control mechanism used in the example (i.e. the particular f , g , β) later, but the result is shown in Figure 1. Note how the feedback has clearly reduced the variance, by a factor of 6 in fact, taking out the low frequency fluctuations. However, it is fairly clear from these plots

130 120

open loop

N

110 100 90 80 70 0

1000

2000

time

3000

4000

5000

3000

4000

5000

130 120

closed loop

N

110 100 90 80 70 0

1000

2000

time

Fig. 1.

Feedback Networks

375

alone that the closed loop response has a larger high frequency component. This is confirmed by looking at the power spectral densities (Figure 2). The system with feedback has less energy at low frequencies and more at high frequencies. The ratio of these can be defined as the sensitivity function again (here frequency dependent – we were just looking at steady state, or DC, values before). Again, it’s the ratio of closed loop to open loop properties (with the square root because these are power spectra). S(f ) =

Pcl (f ) Pol (f )

Probably the most fundamental trade off in control is the celebrated Bode tradeoff; that the integral of log sensitivity is nonnegative. The area of log sensitivity reduction can be no greater than the area of log sensitivity increase.

6

10

open loop 4

Pxx

10

2

10

closed loop 0

10

0

0.02

0.04 0.06 freq (Hz)

0.08

0.1

0.08

0.1

Fig. 2. 1

log|S|

0 −1 −2 −3 0

0.02

0.04

freq (Hz)

0.06

Fig. 3. S(f )

376

G. Vinnicombe

What you gain somewhere, you have to lose somewhere else. For this definition of the sensitivity function, it has been shown by Martins & Dahleh [2]1 that ∞ 0

log S(f ) df ≥ 0

Of course, this becomes an equality if the uncontrolled system is stable, and the feedback is linear and deterministic ([Bode 1945]) Fundamental as it is, the Bode integral doesn’t in itself necessarily limit the benefits of feedback. As in this example, the area of sensitivity increase can be shifted to a frequency range where it is less important. Are there limits to the amount of log sensitivity that can be shifted? It turns out that, yes, further constraints. are imposed by: • •

Feedback capacity (metabolic load) Delays

2.2 Limitations Due to Feedback Capacity One of the main results of [2] is actually that is that the total amount of negative log sensitivity is bounded by the capacity, in the sense of Shannon, of the feedback path. −

∞

0

[log S(f )]− df ≤ C

(1)

where C is Shannon capacity of the feedback path and [x]− =

x 0

if x < 0 otherwise

so the LHS of (1) is the total area of log sensitivity decrease. So, there is a limit to the total benefit of feedback even if we are prepared to tolerate an arbitrarily large area of sensitivity increase somewhere. This result was derived in the context of control over bitrate limited communication channels, but is still applicable here. The channel, a molecular species, is perhaps a bit unusual though. Recall that our feedback channel is the jump Markov process m m

g(n) βm

m+1 m−1

(2)

It is clear that the deaths carry no information (they serve only to limit the maximum number of the species m); only the births carry information. So the capacity is actually the same as that of the photon counting channel, 1

The results of [2] are actually given in discrete time, but are easily extended to continuous time

Feedback Networks

377

i.e. the maximum amount of information about a light intensity that can be encoded in counts from a photon detector. The capacity is 1/e log(2) (about 0.5) times the maximum birth rate, which is the maximum synthesis rate of m. This synthesis rate corresponds to the metabolic load imposed by the presence of the feedback channel. So, in a very real sense, we can claim that the benefits of feedback have to be traded off against metabolic burden, benefits of feedback 0.5× maximum metabolic burden. The example in the simulations above was, in fact, the strongly nonlinear interaction: g(n) = f (m) =

1 0

if n < 100 otherwise

10 if m > 0 0 otherwise

α = .01 β=5 Note that the m species, the regulating one, is produced at an average rate of 1 per unit time if the n species is below its target and not at all otherwise. So, C = 1/e log(2). This choice was motivated by the fact that in order to attain capacity in the photon counting channel it is necessary to modulate the intensity to either its maximum or minimum level and not use intermediate levels. If the channel is used in an approximately analogue manner, with a linear dependence of birth rate on the concentration of n (so that m is approximately proportional to n) then the capacity is substantially less than this. Incidentally, the example works just as well with n = 10, which would be more realistic for the gene expression example discussed. Whether nature uses nonlinear schemes to maximize the information carrying capability of molecular channels is not clear (but surely likely). 2.3 Limitations Due to Delay There will always be a delay in the feedback, as molecules take a finite time to make. In the case of DNA, RNA and proteins these times can be appreciable. In [3], where we consider a plasmid replication example in detail, the total delay in the loop is over 40 seconds. These delays can limit performance. Consider the example n n

f (n(t−T ))

n+1

αn

n−1

with f (n0 ) = αn0 , so the equilibrium is at < n >= n0 . We have ignored the need for a second species, but made the birth rate an arbitrary function of the

378

G. Vinnicombe

concentration T seconds ago, i.e. we have assumed that the molecule takes T seconds to make. In open loop, i.e. f (n0 ) = 0, the variance of n is the same as its mean, i.e. σn2 = n0 . Figure 4 shows the reduction in variance against the feedback gain f (n0 ), the slope of f (relative to the open loop value). As one would expect, if the gain is too small then the feedback is overly sluggish, taking a long time to reach equilibrium. If too large, the response is oscillatory. In between it’s just right, but still there are some fluctuations. Clearly, to get the benefits of feedback we need to carefully manage the gain. It’s worth asking the question though of whether we can reduce the fluctuations further with alternative feedback strategies. 2

1

1.8 1.6

0.9

1.4

2

1.2

0.8

1.8

2

0.7

1.4

0.8

1.6

0.6

1.4

1.2

0.6

1

0.4

1.2

0.2

1

0.8

0.5

0.6

0

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0.8 0.6

0.4

0.4

0.2 0

1

1.8

1.6

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0.4 0.2

2

0.3

0

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0.2 0.1 0

0

10

20

30

k

40

50

60

70

Fig. 4. Variation with k = −f (n0 )

For large equilibrium numbers this system behaves (at least in terms of statistics up to second order) as approximately n(t) ˙ = f n( t − T ) − αn + 2αn0 η. delay of T

Standard optimal control tools can be used to show that the minimum obtainable variance is (1 − e−2αT )n0 . So, the answer is that the fluctuations cannot be reduced, or at least not by much. Figure 5 shows the maximum possible reduction in variance as a function of αT , the product of the degradation rate and the time delay, if a constant controller is used and if the optimal controller is used. So the product αT which is the probability of any given molecule dying in the time it takes to make one, has to be small for there to be any possibility of significantly reducing variance by feedback.

Feedback Networks no feedback

1

best 1st order

= 1 − e−2αT

0.6

optimal

2 n

σ /

0.8

379

0.4

0.2

0 0

0.5

1 αT

1.5

2

Fig. 5.

3 Feedback Networks So, what about networks of large numbers of interacting systems, with lots of feedback loops as in Figure 6. This isn’t a graph, the nodes represent dynamical systems, and they might all be different. For this analysis, we’re not just interested in what’s connected to what, but also in the details of the interactions.

Fig. 6.

380

G. Vinnicombe

Examples of complex systems that can be modelled like this are economic networks, where the nodes might represent buyers, sellers, market makers etc. Power networks, where we have generators, consumers and transmission lines, communication networks (e.g. Internet) and vehicle formations (flocking phenomena/platooning etc). In order to obtain the benefits of feedback we need somehow to get the right amount of gain into all the right places, not too little and not too much. So how much is right, we clearly can’t treat each loop in isolation. In general, apart from suggesting a massive centralized optimization problem, control theory has little to offer. As engineers we tend to avoid making massively horizontally distributed feedback systems of this kind. We put up barriers and create hierarchies. We actively break feedback loops to avoid getting into trouble. However, in the case that any nonlinearities are weak, and connections are predominantly symmetric as in Figure 6, there is a way to do it. The results involve relating local and global properties. If we have ways of predicting global properties from local ones then we’ll know how to design the local interactions. The key questions are: • •

How can we predict global properties from local interactions? How can we design the individual systems (agents) and their local interactions (protocols) to ensure desirable global properties?

Note: If the agents are stable, we can always get a stable network by making the feedback gains small. What we really want to know how large we can get away with making them, in order to get good performance everywhere. I’m not suggesting for a moment that biology works like this – my aim is to simply point out that there is an issue here. Note, though, that this doesn’t appear to be as much of a problem for developmental networks. It was recently shown [4], that, in the case of drosophila segmentation, once the topology of the interactions was right, many different combinations of gains led to the right result. In regulatory networks this cannot be the case – the choice of gain is critical. Some of the most powerful tools in control systems theory are ones that allow properties of an interconnected system to be inferred from properties of its subsystems and the structure of their interactions. One of the very best is the Nyquist stability criterion. η +

Σ

u

g¯(jω)

− k Fig. 7.

y

Feedback Networks

381

In order to determine the stability of a simple feedback system as in Figure 7 we just have to look at whether the the locus of the frequency response k¯ g(jω) encircles the point −1 in the complex plane. Can similar principles be applied to more complex interactions of the sort we are describing? Figure 8 shows a small part of such a network. Each node represents a system or filter with a given frequency response. In the time domain, there would be a delay differential equation representing each node. The input to each node is the average of the outputs of its neighbours. g2

g1

g7

g5

g4

g3

g6

Fig. 8.

The interconnection is described as follows: •

Each vertex: stable system/filter with frequency response g¯i (jω). y¯i (jω) = g¯i (jω)¯ ui (jω)

•

Input: average of the signals from its neighbours. ui (t) =

1 Ni

yk (τ ) i,k connected

The main result of [5] is that, if each system satisfies a convexified Nyquistlike condition with its neighbours then the entire feedback network is guaranteed to be stable. To be more precise, first define the S-hull of a set S(·) as the convex hull of the square root of the set all squared. That is, for X ⊂ C let 2 √ , X S(X) = Co √ where X := {y : y 2 ∈ X}. The S-hull of a set, S(·), is slightly larger than the convex hull of the union of set with 0, as shown in Figure 9. For each agent, we then take the S-hull of the products of its own and its neighbours dynamics, these are the loop gains. Finally we check whether −1 is in the convex hull of these sets. Theorem 1. [5] An interconnection of the type depicted in Figure 8 is stable if −1 ∈ / Co ∪i S −¯ gi (jω)¯ gk (jω) : k connected to i ∀ω

382

G. Vinnicombe 1

0.5

0

−0.5

−1

0

0.5

1

1.5

2

2.5

3

Fig. 9.

A key point is that, apart from the final convexification, the result is entirely local, and a line to separate these S-hulls from −1 can always be agreed on in advance. The main conclusion is that, provided the interconnections are symmetric it is sufficient to just look at each feedback loop separately – and say that as long as each agent behaves well with its neighbours then the network as a whole will be behave predictably. 3.1 Modules and Protocols One way in which this theorem can be used to build stable complex networks is by restricting the agents/modules to parameterized classes and insisting on interconnection protocols. Imagine you have a box of different modules or

Module: Σ

fT

e.g.

N dy kg X = ui (t − T ) dt N T i=1

Protocol: e.g. f/b interconnection allowed if kg kr ≤ 1, τ < 2T Module: Σ

gτ

e.g. τ

Fig. 10.

N dy kr X +y = ui (t) dt N i=1

Feedback Networks

383

building blocks, each representing different delay differential equations, as in Figure 10. Say the green ones are delayed integrators with a delay T and the red ones are lags with time constant τ . Each green has a different T and each red a different τ . Imagine that the T ’s and τ ’s are written on the side, and say they have a dial on them to set their gain. The protocol is that we’re allowed to connect a red and a green brick together if the product of their gains is no greater than 1 and the red’s τ is no greater than twice the greens T . Then we carry on adding on adding more red and green bricks according to the same rules, as many as we like. We have been representing this interconnection as in Figure 11. fT

gτ Fig. 11.

The claim is that we’ll always end up with a stable network, and in this case the certificate in Figure 12 proves it. Moreover, the condition is nonconservative for just two blocks. Note that this one picture provides a stability certificate for all possible interconnections satisfying the protocol. 0.5

0

−0.5

−1

−1.5

−2

−2.5

−3 −3

−2.5

−2

−1.5

−1

−0.5

⇒ closed-loop stability. Fig. 12.

0

0.5

384

G. Vinnicombe

3.2 Example: Vehicle Platooning As another example, consider the platooning of heterogeneous vehicles. That is, each vehicle might has a different mass, delay, rate of response and a different controller designed to track a fixed reference. x

g¯(jω)

e

Σ

+ xref −

Producing a stable platoon, with fixed spacings between the vehicles, is actually a difficult problem even if all the vehicles are the same. The first thing one might try is to design the control system for each car to minimize the error between its position and that of the vehicle in front. However, that doesn’t work [6]. It’s because of another fundamental limitation, related to the Bode integral. Under reasonable physical assumptions, a weighted integral of the log of the ratio of x to the reference must be strictly positive, so any disturbances get amplified as you go back along the chain. This is known as lack of string stability and is unavoidable with this architecture, since [7] ∞ 0

V5

V4

log

Pxx (ω) dω > 0. Pxref xref (ω) ω 2

V3

V2

V1

The standard solution [6] is to communicate leader information to all vehicles in the platoon.

V5

V4

V3

V2

V1

However, an alternative solution, which has the benefit of being decentralized, is to look backwards as well as forwards. We wouldn’t normally like this solution from an engineering point of view - there’s too much feedback, how can we possibly guarantee it would work for all combinations? For example, it is easy to come up with two cars (with their associated control systems) like this which follow a fixed reference perfectly well, but go unstable when tracking each other. If the cars just look forward, then it requires a string of cars before the disturbances at the back get excessive; with the bidirectional scheme just two vehicles is enough for instability. Nevertheless, it is relatively straightforward to design a set of control laws and interconnection protocols

Feedback Networks

385

in such a way that Theorem 1 will guarantee we do always end up with stable platoons. V5

V4

V3

V2

V1

3.3 Internet Congestion Control The platooning example of the previous section probably still involves too much feedback to be practicable. The most successful application of Theorem 1 (or rather the application from which the theory evolved) is to Internet congestion control. The problem is that TCP, the protocol which controls data flow, doesn’t work well over long distance uncongested high speed links. It’s not sufficiently aggressive at grabbing bandwidth (in other words, the gain is too low in that part of the network). The theory tells us the correct way of scaling the gains. It is important to emphasize again though that stability is not the main aim here, we can always get that with small gains; what we want are good estimates as to how large we can make the gains. In this game the players are the • •

Participating Dynamics : – Users (generating packet flow) – Routers (sending congestion signals) Protocol : Everybody does TCP.

The theory has suggested a modification to TCP for high speed networks [8], which is incorporated in the IETF Draft [9]. A full implementation of [8] has been tested over a transatlantic link and worked well [10]. Finally, a question that must be addressed is “Why is linear analysis interesting?” What’s the point of all this linear analysis, when the real world is nonlinear and stochastic. Most mildly nonlinear systems of this kind, though, are pretty well behaved. Instead of going wildly unstable they tend to break into a limit cycles. Figure 13, from [11] shows the performance of a simple congestion control system through a single queue, with delay. It illustrates the fact that the critical point for linear stability (1 on the x-axis) is often a good operating point. The difficulty is in finding this operating point simultaneously in all parts of the network (with delays varying over several orders of magnitude).

4 Conclusions The design of feedback systems involves carefully managing tradeoffs, i.e. getting the gain right. Managing these tradeoffs in large regulatory networks

386

G. Vinnicombe Coefficient of variation VS normalized gain and number of packets in flight. Queue threshold=5

0.7

λ×T=150 λ×T=750 λ×T=3750 fluid limit

25

0.6

20

µ(pckts/s)

15

0.5

10 5

25

25 20

0.4

−5

20

0

500

time(s)

1000

15

1500

µ(pckts/s)

std(µ)/mean(µ)

0

5

0.3

0

15 µ(pckts/s)

10

−5

10 5

0

500

time(s)

1000

1500

0.2

0 −5

0

500

time(s)

1000

1500

0.1

0

0

0.2

0.4

0.6

0.8

k/kc

1

1.2

1.4

1.6

1.8

Fig. 13.

(getting the right gain in the right places) is difficult. We have described some new tradeoffs apparent in biochemical regulatory systems, and we have described one way in which tradeoffs can be managed in designed technological feedback networks. It is an open question how nature manages these tradeoffs.

Acknowledgements To Keith, for starting me on this road and encouraging me along the way. A finer, and fairer, adviser, division head and now Head of Department would be difficult to imagine.

References 1. D.C. McFarlane and K. Glover, “A loop shaping design procedure using H∞ synthesis,” IEEE Trans. on Automatic Control, vol. 37, no. 6, pp. 759–769, 1992. 2. N. C. Martins and M. A. Dahleh, “Feedback control in the presence of noisy channels: ”Bode-like” fundamental limitations of performance,” preprint, 2004. 3. I. Lestas, G. Vinnicombe and T. Bishop, “Tradeoffs in gene regulatory networks: the plasmid replication control problem,” in Proceedings of IEEE Conference on Decision and Control and European Control Conference, December 2005, Accepted for publication.

Feedback Networks

387

4. G. von Dassow, E. Meir, E. M. Munro and G. M. Odell, ,“The segment polarity network is a robust developmental module” Nature, vol. 406, pp. 188–192, 2000. 5. I. Lestas and G. Vinnicombe, “Scalable decentralized robust stability certificates for networks of interconnected heterogeneous dynamical systems,” Tech. Rep., Cambridge University, Engineering Department, November 2004, CUED/FINFENG/TR.501. Accepted for publication in IEEE Transactions on Automatic control. 6. D. Swaroop and J.K. Hedrick, “String stability of interconnected systems,” IEEE transactions on automatic control, vol. 41, no. 3, pp. 349–357, 1996. 7. I. M. Horowitz, Synthesis of feedback systems, Academic Press, New York, 1963. 8. G. Vinnicombe, “On the stability of networks operating TCP-like congestion control,” in IFAC, 2002. 9. S. Floyd, “HighSpeedTCP,” RFC 3649, 2002. 10. T. Kelly, “Improving performance in high speed wide area networks,” First International Workshop on Protocols for Fast Long-Distance Networks, 2003. 11. I. Lestas and G. Vinnicombe, “How good are deterministic models for analyzing congestion control in delayed stochastic networks?,” in IEEE Conference on Decision and Control, December 2004.

Thoughts on System Identification Jan C. Willems SISTA-ESAT, K.U. Leuven, B-3001 Leuven, Belgium [email protected]

1 Introduction It is a pleasure to dedicate this Festschrift article to Keith Glover on the occasion of his 60-th birthday. Keith came to MIT in 1969, armed with a degree from Imperial College, a couple of years of industrial experience, and a Kennedy scholarship, ready for some serious research. I, at that time a young assistant professor at MIT, had the good fortune to become Keith’s M.Sc. and Ph.D. supervisor, who he could choose freely because his scholarship made him, so to speak, ‘self-supporting’. In his M.Sc. thesis Keith applied circle-criterion based stability conditions to numerical integration routines [9]. His Ph.D. dissertation [10] was about System Identification (SYSID), more specifically, about Identifiability. The main results of his dissertation were published in [11]. It was one of the first articles that dealt with the parametrization of linear systems, an intricate subject that was studied very deeply later on. This research reached its apogee in Hannan and Deistler’s book [15] where the relevance for SYSID algorithms of parametrizations and canonical forms of multivariable linear systems is pursued in great depth. Keith’s dissertation showed a penetrating understanding of linear systems. It was a forebode of his later work in this area, culminating in his classic papers on model reduction [12] and on the double Riccati equation solution of the H∞ feedback control problem [7], both among the most cited in the field of systems and control. A Festschrift is a welcome occasion to write an article with a personal and historical flavor. Because of the occasion, I chose the subject of Keith Glover’s Ph.D. dissertation, SYSID. My interest in this area remained originally limited to the implications of the structure of linear systems. This situation changed with the Automatica papers [35]. These contain, in addition to the first comprehensive exposition of the behavioral approach to systems theory, a number of new ideas and subspace type-algorithms for SYSID. The aim of the present article is to explain in a somewhat informal style my own B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 389–416, 2006. © Springer-Verlag Berlin Heidelberg 2006

390

J.C. Willems

personal point of view on SYSID. Among other things, I will describe in some detail the many representations of linear time-invariant systems, leading up to some exact deterministic SYSID algorithms based on the notion of the most powerful unfalsified model. I will then explain the idea behind subspace algorithms, where the state trajectory is constructed directly from the observations and a system model in state form is deduced from there. Subsequently, I will discuss the role of latent variables in SYSID. This leads in a natural way to stochastic models. I will finish with some remarks on the rationale, or lack of it, of viewing SYSID in a stochastic framework. I view SYSID in terms of the following picture from classical statistics: ID : D ∈ D → M ∈ M.

(ID)

Here D is the data, D the data class, M the model class, ID the identification procedure, and M = ID(D) the model chosen from M on the basis of the data D through the identification procedure ID. Our interest is in the case that the data consists of a time-series (throughout, time-series means vector time-series) w ˜T

=

w ˜ (1) , w ˜ (2) , . . . , w ˜ (T)

w ˜ (t) ∈ Rw

(w ˜T )

T

i.e. D ∈ (Rw ) . However, while our primary aim is at finite time-series T < ∞, we will occasionally consider T → ∞ and even T = ∞. So, we may as well take T for the data class D = ∪T∈N ∪ {∞} (Rw ) . The model class M consists of linear time-invariant, deterministic or stochastic, dynamical systems. The intent of this essay is to explain some of the issues involved in defining and choosing M, and setting up an identification procedure (ID). I prefer to deal with SYSID by going from the simple to the complex: Exact deterministic SYSID Approximate deterministic SYSID

Exact stochastic SYSID

Approximate stochastic SYSID In view of space limitations, I will not discuss the approximate stochastic case.

2 Deterministic Models Accordingly, I will start with deterministic systems. The first question which we need to confront is: What are we after? What does an identification procedure aim at? When we accept a mathematical model as a description of a phenomenon, what do we really do? Mind you, I am not asking a philosophical question. I am not inviting an erudite discourse on the convoluted relation between a real world manifestation of a physical or an economic phenomenon and a mathematical description of it. The question is meant to be a mathematical one.

Thoughts on System Identification

391

The answer to this question is simple, evident, but startlingly enlightening (in German, a ‘Aha Erlebnis’ ). A model is simply a subset, B, of a universe, U, of a priori possibilities. This subset B ⊆ U is called the behavior of the model. Thus, before the phenomenon was captured in a model, all outcomes from U were in principle possible. But after we accept B as the model, we declare that only outcomes from B are possible. For example, the ideal gas law states that the temperature T , pressure P , volume V , and quantity (number of moles) N of an ideal gas satisfy P V /N T = R, with R a universal constant. So, before Boyle, Charles, and Avogadro got into the act, T, P, V, and N may have seemed unrelated, yielding U = R+4 . The ideal gas law restricts the possibilities to B = {(T, P, V, N ) ∈ R+4 | P V /N T = R}. In most applications, a model is given in terms of a set of equations (think of the basic laws of physics, e.g. Maxwell’s equations). The set of solutions to these equations is then the behavior. In the context of modeling, equations simply serve as a representation of their solution set. This view comes in very useful, since it gives a clear and unambiguous answer to the question when two sets of equations represent the same model. Often, it is trivial that a transformation of equations (as changing the order in which Maxwell’s equations are written down) does not change a model, but for some transformations this may be much more difficult to establish (think of expressing Maxwell’s equations in terms of potential functions). In the case of dynamical systems, the phenomenon which is being modelled produces functions that map the set of time instances relevant to the model to the signal space. This is the space in which these functions take on their values. For the sake of concreteness, and because of its relevance to SYSID, I will assume in this article that the set of relevant time instances is N (the theory is analogous for Z, R, and R+ ). This choice means that we are studying discrete time systems and that we postulate that the model is valid for all time (of course, this does not mean that observations need to extend over all of N). I also assume throughout that the signal space is a finite dimensional real vector space, typically Rw . Following our idea of a model, the behavior for the dynamical systems which we consider is therefore a collection B of functions mapping the time set N into the signal space Rw . A dynamical model can hence be identified with N its behavior B ⊆ (Rw ) . The behavior is hence a family of maps from N to w R . Of course, also for dynamical systems the behavior B is usually specified as the set of solutions of equations, for the case at hand typically difference equations. As dynamical models, difference equations thus merely serve as a representation of their solution set. Note that this immediately leads to a notion of equivalence and to canonical forms for difference equations. These are particularly relevant in the context of dynamical systems, because of the multitude of, usually over-parameterized, representations of the behavior of a dynamical system.

392

J.C. Willems

3 Deterministic Linear Dynamical Systems SYSID usually employs as the model class dynamical systems that are (i) linear, (ii) time-invariant, and (iii) that satisfy a third property, related to the finite dimensionality of the underlying state space, or to the rationality of a transfer function. It is, however, clearer and advantageous to approach this situation in a more intrinsic way, by imposing this third property directly on the behavior, and not on a representation of it. In this section, I discuss this model class in considerable detail. N A behavior B ⊆ (Rw ) is said to be linear if w ∈ B, w ∈ B, and α ∈ R imply w + w ∈ B and αw ∈ B, and time-invariant if σB ⊆ B. The shift, σ, plays a central role in SYSID. It is defined by (σf ) (t) := f (t + 1), and often called the backwards shift, since it shifts the time function f backwards. The third property that enters into the specification of the model class is completeness. B is called complete if it has the following property: [[ w : N → Rw belongs to B ]] ⇔ [[ w|[1,t] ∈ B|[1,t] for all t ∈ N ]]. In words: B is complete if we can decide that w : N → Rw is ‘legal’ (i.e. belongs to B) by verifying that everyone of its ‘prefixes’ w (1) , w (2) , . . . , w (t) is ‘legal’ (i.e. belongs to B|[1,t] ). So, roughly speaking, B is complete iff the laws of B do not involve what happens at ∞. Requirements as w ∈ 2 (N, Rw ), w has compact support, or limt→∞ w (t) exists, risk at obstructing completeness. However, often crucial information about a complete B can be obtained by considering its intersection with 2 (N, Rw ), or its compact support elements, etc. Recall the following standard notation. R [ξ] denotes the polynomials with real coefficients in the indeterminate ξ, R(ξ) the real rational functions, and Rn1 ×n2 [ξ] the polynomial matrices with real n1 × n2 matrices as coefficients. When the number of rows is irrelevant and the number of columns is n, the notation R•×n [ξ] is used. So, in effect, R•×n [ξ] = ∪k∈N Rk×n [ξ]. A similar notation is used for polynomial vectors, or when the number of rows and/or columns is irrelevant. The degree of P ∈ R•×• [ξ] equals the largest degree of its entries, and is denoted by degree (P ). Given a time-series w : N → Rw and a polynomial matrix R ∈ Rv×w [ξ], say R (ξ) = R0 + R1 ξ + · · · + RL ξ L , we can form the new v-dimensional time-series R (σ) w = R0 w + R1 σw + · · · + RL σ L w. Hence R (σ) : (Rw )N → (Rv )N , with R (σ) w : t ∈ N → R0 w (t)+R1 w (t + 1)+ · · · + RL w (t + L) ∈ Rv . The combination of linearity, time-invariance, and completeness can be expressed in very many equivalent ways. In particular, the following are equivalent: 1. B ⊆ (Rw )N is linear, time-invariant, and complete;


393

2. B is a linear, shift invariant (:⇔ σB ⊆ B), closed subset of (Rw )N , with ‘closed’ understood in the topology of pointwise convergence; 3. ∃ R ∈ R•×w [ξ] such that B consists of the solutions w : N → Rw of R (σ) w = 0.

(Ker)

N

The set of behaviors B ⊆ (Rw ) that satisfy the equivalent conditions 1. to 3. is denoted by Lw , or, when the number of variables is unspecified, by L• . Thus, in effect, L• = ∪w∈N Lw . Since B = ker (R (σ)) in (Ker), we call (Ker) a kernel representation of the behavior B. We will meet other representations later. But first we introduce a characterization that is mathematically more abstract, but very pertinent in the context of SYSID. It identifies a behavior B ∈ L• with an R [ξ]-module. Consider B ∈ Lw . The polynomial vector n ∈ R1×w [ξ] is called an annihilator (or a consequence) of B if n (σ) B = 0, i.e. if n (σ) w = 0 for all w ∈ B. Denote by NB the set of annihilators of B. Observe that NB is an R [ξ]-module. Indeed, n ∈ NB , n ∈ NB , and α ∈ R [ξ] imply n + n ∈ NB and αn ∈ NB . Hence the map B → NB associates with each B ∈ Lw a submodule of R1×w [ξ]. It turns out that this map is actually a bijection, i.e. to each submodule of R1×w [ξ], there corresponds exactly one element of Lw . It is easy to see what the inverse map is. Let K be a submodule of R1×w [ξ]. Submodules of R1×w [ξ] have nice properties. In particular, they are finitely generated, meaning that there exist elements (‘generators’) g1 , g2 , . . . , gg ∈ K such that K consists precisely of the linear combinations α1 g1 + α2 g2 + · · · + αg gg where the αk ’s range over R [ξ]. Now consider the system (Ker) with R = col(g1 , g2 , . . . , gg ) and prove that Nker(col(g1 , g2 ,..., gg )(σ)) = K (⊇ is obvious, ⊆ requires a bit of analysis). In terms of (Ker), we obtain the characterization [[ ker R (σ) = B ]]

⇔

[[NB =

R ]]

where R denotes the R [ξ]-module generated by the rows of R. The observation that there is a bijective correspondence between Lw and the R[ξ]-submodules of R1×w [ξ] is not altogether trivial. For instance, the surjectivity of the map B = ker R (σ) ∈ Lw → NB = R onto the R[ξ]-submodules of R1×w [ξ] depends on the solution concept used in (Ker). If we would have considered only solutions with compact support, or that are square integrable, this bijective correspondence is lost. Equations, in particular difference or differential equations, all by themselves, without a clear solution concept, i.e. without a definition of the corresponding behavior, are an inadequate specification of a mathematical model. Expressed otherwise: the theory of (linear) systems is not the domain of pure algebra. Analysis

394

J.C. Willems

enters through the solution concept involved in the difference or differential equations, and guides the subsequent algebraic structure. The characterization of B in terms of its module of annihilators is very useful in the context of deterministic SYSID. It shows precisely what we are looking for in order to identify a system in the model class L• : (a set of generators of) the submodule NB . Behaviors in L• admit many other representations. The following two are exceedingly familiar to system theorists. In fact, 4. [[B ∈ Lw ]] ⇔ [[ ∃ integers m, p ∈ Z+ , with m + p = w, polynomial matrices P ∈ Rp×p [ξ] , Q ∈ Rp×m [ξ] , with det (P ) = 0, and a permutation matrix Π ∈ Rw×w such that B consists of all w : N → Rw for which there exist u : N → Rm and y : N → Rp such that P (σ) y = Q (σ) u

(I/O)

u ]]. The matrix of rational functions G = P −1 Q ∈ y p×m (R (ξ)) is called the transfer function of (I/O). Actually, for a given B ∈ Lw , it is always possible to choose Π such that G is proper. If we would allow a basis change in Rw , i.e. allow any non-singular matrix for Π (instead of only a permutation matrix), then we could always take G to be strictly proper. 5. [[B ∈ Lw ]] ⇔ [[ ∃ integers m, p, n ∈ Z+ with m + p = w, matrices A ∈ Rn×n , B ∈ Rn×m , C ∈ Rp×n , D ∈ Rp×m , and a permutation matrix Π ∈ Rw×w such that B consists of all w : N → Rw for which there exist u : N → Rm , x : N → Rn , and y : N → Rp such that and w = Π

σx = Ax + Bu, y = Cx + Du

(S)

u ]]. If we would allow a basis change in Rw , i.e. allow any y non-singular matrix for Π, then we could always take D = 0.

and w = Π

(I/O) is called an input/output (i/o) and (S) an input/state/output (i/s/o) representation of the corresponding behavior B ∈ Lw . Why, if any element B ∈ L• indeed admits a representation (I/O) or (S), should one not use one of these familiar representations ab initio? There are many good reasons for not doing so. To begin with, and most importantly, first principles models aim at describing a behavior, but are seldom in the form (I/O) or (S). Consequently, one must have a theory that supersedes (I/O) or (S) in order to have a clear idea what transformations are allowed in bringing a first principles model into the form (I/O) or (S). Secondly, as a rule, physical systems are simply not endowed with a signal flow direction. Adding a signal flow direction is often a figment of one’s imagination, and when something is not real, it will turn out to be cumbersome sooner or later. A third reason, very


395

much related to the second, is that the input/output framework is totally inappropriate for dealing with all but the most special system interconnections. We are surrounded by interconnected systems, but only very sparingly can these be viewed as input-to-output connections. Fourthly, the structure implied by (I/O) or (S) often needlessly complicates matters, mathematically and conceptually. A good theory of systems takes the behavior as the basic notion and the reference point for concepts and definitions, and switches back and forth between a wide variety of convenient representations. (I/O) or (S) have useful properties, but for many purposes other representations may be more convenient. For example, a kernel representation (Ker) is very relevant in SYSID. It suggests that we should look for (approximate) annihilators. On the other hand, when it comes to constructing trajectories, (S) is very convenient. It shows how trajectories are parameterized and generated : by the initial state x(1) ∈ Rn and the input u : N → Rm . Our next representation involves rational functions and is a bit more •×w ‘tricky’. Let G ∈ (R(ξ)) and consider the system of ‘difference equations’ G(σ)w = 0.

(G)

What is meant by the behavior of (G) ? Since G is a matrix of rational functions, it is not evident how to define solutions. This may be done in terms of co-prime factorizations, as follows. G can be factored G = P −1 Q with P ∈ R•×• [ξ] square, det(P ) = 0, Q ∈ R•×w [ξ] and (P, Q) left co-prime (meaning that F = [P Q] is left prime, i.e. [[(U, F ∈ R•×• [ξ]) ∧ (F = U F )]] ⇒ [[U is square and unimodular ]], equivalently ∃ H ∈ R•×• [ξ] such that F H = I). We define the behavior of (G) as that of Q(σ)w = 0, i.e. as ker (Q (σ)) Hence (G) defines a behavior ∈ Lw . It is easy to see that this definition is independent of which co-prime factorization is taken. There are other reasonable ways of approaching the problem of defining the behavior of (G), but they all turn out to be equivalent to the definition given. Note that, in a trivial way, since (Ker) is a special case of (G), every element of Lw admits a representation (G). Certain integer ‘invariants’ (meaning maps from L• to Z+ ) associated with systems in L• are important in SYSID. One is the lag, denoted by L(B), defined as the smallest L ∈ Z+ such that [[w|[t, t+L] ∈ B|[1,t+1] for all t ∈ N ]] ⇒ [[w ∈ B]]. Equivalently, the smallest degree over the polynomial matrices R such that B = ker (R(σ)). A second integer invariant that is important is the input cardinality, denoted by m (B), defined as m, the number of input variables in any (I/O) representation of B. It turns out that m is an invariant (while the input/output partition, i.e. the permutation matrix Π in (I/O), is not). The number of output variables, p, yields the output cardinality p (B). A

396

J.C. Willems

third important integer invariant is the state cardinality, n (B), defined as the smallest number n of state variables over all i/s/o representations (S) of B. The three integer invariants m (B), n (B), and L (B) can be nicely captured in one single formula, involving the growth as a function of t of the dimension of the subspace B|[1,t] . Indeed, there holds dim(B|[1,t] ) ≤ m (B) t + n (B)

with equality iff t ≥ L (B) .

State models (S) are an example of the more general, but very useful, class of latent variable models. Such models involve, in addition to the manifest variables (denoted by w in (L)), the variables which the model aims at, also auxiliary, latent variables (denoted by in (L)). For the case at hand this leads to behaviors Bfull ∈ Lw+l described by R(σ)w = M (σ) ,

(L)

with R ∈ R•×w [ξ] and M ∈ R•×l [ξ]. Although the notion of observability applies more generally, we use it here for latent variable models only. We call Bfull ∈ Lw+l observable if [[ (w,

1)

∈ Bfull and (w,

2)

∈ Bfull ]] ⇒ [[

1

=

2

]].

(L) defines an observable latent variable system iff M (λ) has full row rank for all λ ∈ C. For state systems (with x the latent variable), this corresponds to the usual observability of the pair (A, C). An important result, the elimination theorem, states that L• is closed under projection. Hence Bfull ∈ Lw+l implies that the manifest behavior B = {w : N → Rw | ∃

: N → Rl such that (L) holds}

belongs to Lw , and therefore admits a kernel representation (Ker) of its own. So, in a trivial sense, (L) is yet another representation of Lw . Latent variable representations (also unobservable ones) are very useful in all kinds of applications. This, notwithstanding the elimination theorem. They are the end result of modeling interconnected systems by tearing and zooming, with the interconnection variables viewed as latent variables. Many physical models (for example, in mechanics) express basic laws using latent variables. In the context of SYSID, the aim of most classical algorithms is in fact to arrive at a model (L), often unobservable, and with usually interpreted as an unobserved (stochastic) input – I will return to this later. But, of course, we are all most familiar with state models, states being the latent variables par excellence. In the next section we will see how latent variables can be used to express controllability.

4 Controllability As in many areas of system theory, controllability often enters in SYSID as a regularizing assumption. In the behavioral theory, an appealing notion of


397

controllability has been put forward. It expresses what is needed intuitively, it applies to any dynamical system, regardless of its representation, it has the classical state transfer definition as a special case, and it is readily generalized, for instance to distributed systems. It is somewhat strange that this definition has not been generally adopted. Adapted to the case at hand, it reads as follows. The time-invariant behavior B ⊆ (R• )N is said to be controllable if for any w1 ∈ B, w2 ∈ B, and t1 ∈ N, there exists a t2 ∈ N and a w ∈ B such that w (t) = w1 (t) for 1 ≤ t ≤ t1 , and w (t) = w2 (t − t1 − t2 ) for t > t1 + t2 . For B ∈ L• , one can take without loss of generality w1 = 0 in the above definition. The property just defined is hence akin to what is sometimes called reachability, but here I use the term controllability as synonymous. Denote the controllable elements of L• by L•controllable and of Lw by Lwcontrollable . (Ker) defines a controllable system iff R(λ) has the same rank for each λ ∈ C. There is a very nice representation result that characterizes controllability: it is equivalent to the existence of an image representation. More precisely, B ∈ L•controllable iff there exists M ∈ R•×• [ξ] such that B equals the manifest behavior of the latent variable system w = M (σ) .

(Im)

In other words, iff B = im (M (σ)). So, images, contrary to kernels, are always controllable. This image representation of a controllable system can always be taken to be observable. For B ∈ L• , we define its controllable part, denoted by Bcontrollable , as Bcontrollable := {w ∈ B | ∀t ∈ N, ∃t ∈ Z+ , and w ∈ B such that w (t) = 0 for 1 ≤ t ≤ t and w (t) = w(t − t − t ) for t > t + t }. Equivalently, Bcontrollable is the largest controllable subsystem contained in B. It turns out that two systems of the form (I/O) (with the same input/output partition) have the same transfer function iff they have the same controllable part. Consider B ∈ Lw . The vector of rational functions n ∈ R1×w (ξ) is called a rational annihilator of B if n (σ) B = 0 (note that, since we gave a meaning to (G), this is well defined). Denote by NBrational the set of rational annihilators of B. Observe that NBrational is a R(ξ)-subspace of R1×w (ξ). The map B → NBrational is not a bijection from Lw to the R(ξ)-subspaces of R1×w (ξ). Indeed, [[ NBrational = NBrational ]] ⇔ [[ Bcontrollable = Bcontrollable ]]. In fact, there exists a bijective correspondence between Lwcontrollable and the R(ξ)-subspaces of R1×w (ξ). Summarizing, R [ξ]-submodules of R1×w [ξ] stand in bijective correspondence with Lw , with each submodule corresponding to the set of polynomial annihilators, while R(ξ)-subspaces of R1×w (ξ) stand in bijective correspondence with Lwcontrollable , with each subspace corresponding to the set of rational annihilators.

398

J.C. Willems

Controllability enters in a subtle way whenever a system is identified with its transfer function. Indeed, it is easy to prove that the system described by w2 = G(σ)w1 ,

w=

w1 , w2

(T)

a special case of (G), is automatically controllable. This again shows the limitation of identifying a system with its transfer function. Two input/output systems (I/O) with the same transfer function are the same iff they are both controllable. In the end, transfer function thinking can deal with non-controllable systems only in contorted ways. A property related to controllability is stabilizability. The behavior B ⊆ (R• )N is said to be stabilizable if for any w ∈ B and t ∈ N, there exists a w ∈ B such that w (t ) = w (t ) for 1 ≤ t ≤ t, and w (t) → 0 for t → ∞. (Ker) defines a stabilizable system iff R(λ) has the same rank for each λ ∈ C with Real(λ) ≥ 0. An important system theoretic result (leading up to the Youla-Kuvcera parametrization of stabilizing controllers) states that •×w B ∈ Lw is stabilizable iff it allows a representation (G) with G ∈ (R(ξ)) left prime over the ring RH∞ (:= {f ∈ R(ξ) | f is proper and has no poles in the closed right half of the complex plane }). B ∈ Lw is controllable iff it allows a representation w = G(σ) with G ∈ (R(ξ))w×• right prime over the ring RH∞ . Autonomous systems are on the other extreme of controllable ones. B ⊆ (R• )N is said to be autonomous if for every w ∈ B, there exists a t ∈ N such that w|[1,t] uniquely specifies w|[t+1,∞) , i.e. such that w ∈ B and w|[1,t] = w |[1,t] imply w = w. It can be shown that B ∈ L• is autonomous iff it is finite dimensional. Autonomous systems and, more generally, uncontrollable systems are of utmost importance in systems theory (including SYSID), in spite of much system theory folklore claiming the contrary. Controllability as a systems property is much more restrictive than is generally appreciated.

5 The MPUM The SYSID algorithms which I will discuss associate with an observed timeseries w ˜T an element of L• , i.e. a complete linear time-invariant system. The first model class that comes to mind is Lw , i.e. models described by R (σ) w = 0.

(Ker)

However, for reasons which I will go into more detail later, important flexibility is gained by aiming at a latent variable model R(σ)w = M (σ) , w+l

(L) w

with full behavior Bfull ∈ L and manifest behavior B ∈ L . In this model we assume that the observed data is generated by a system that contains unobserved latent variables.


399

Assume that the model (L) is deduced from the observed data w ˜T =

w ˜ (1) , w ˜ (2) , . . . , w ˜ (T) .

How should we assess this choice ? There are several conflicting quantitative measures which we wish to keep small, e.g. the complexity, the misfit, and the latency. The complexity roughly measures how many adjustable parameters the model has. The dimension of the subspaces dim Bfull|[1,t] for t ∈ N, or, more simply, the triple (m (Bfull ) , n (Bfull) , L (Bfull )) is a good measure of the complexity. The misfit measures the extent to which the model fails to explain the observations. The minimum of ||w˜T − w|[1,T] || 2 ([1,T],Rw ) over w ∈ B is a good measure of the misfit. The latency measures the extent to which the model needs latent variables to explain the observations. The minimum of || |[1,T] || 2 (N,Rl ) over such that (w, ) ∈ Bfull and w|[1,T] = w ˜T is a good measure of the latency. Another prevalent way for assessing an identification procedure is consistency. This means that if the data w ˜T is actually generated by an element of the model class, then the identification procedure should return the model that generated the data. Consistency is primarily used in a stochastic setting for infinite observed time series (T = ∞), but one can apply it for deterministic SYSID and T finite as well. A related consideration for assessing an identification algorithm is the way the estimate based on w ˜T behaves as T → ∞, if it is considered to be a truncation of an infinite time series. In the remainder of this section, we discuss how we can associate a deterministic model (Ker) ‘exactly’ (with zero misfit) to an observed time-series. The idea is to obtain the least complex model that explains the data exactly. The concept which implements this idea is the Most Powerful Unfalsified Model (MPUM). This notion was introduced in [35, Part II] for infinite observed time-series (T = ∞ in w ˜T ). Here I adapt it to the case of finite time-series. N The behavior B ⊆ (Rw ) is unfalsified by w ˜T if w ˜ (1) , w ˜ (2) , . . . , w ˜ (T) ∈ B|[1,T] . N

N

B ⊆ (Rw ) is more powerful than B ⊆ (Rw ) if B ⊂ B . Hence, following Karl Popper (1902–1994), the more a model forbids, the better it is. B ∈ Lw is the MPUM in Lw for w ˜T if it is unfalsified by w ˜T and more powerful than any other behavior in Lw that is also unfalsified by w ˜T . It is easy to prove that for an infinite observed time-series (T = ∞ in w ˜T ), this MPUM always exists. In fact, it is equal to the closure (in the topology of pointwise convergence) of span{σ t w ˜T , t ∈ N}. For T finite, the MPUM in Lw may not exist, but it is not particularly useful anyway: when it exists, its behavior will be finite dimensional (corresponding to an autonomous behavior). However, it is desirable, also in the case T < ∞, to obtain an MPUM that can also recover a behavior with some of the variables free inputs. This can be accomplished

400

J.C. Willems

by looking for the MPUM in the class of systems with a restricted lag. This can be viewed as limiting the complexity of the model. Define LwL := {B ∈ Lw | L (B) ≤ L}. ˜T if it is unfalsified by w ˜T and more powerful B ∈ Lw is the MPUM in LLw for w than any other behavior in LwL that is also unfalsified by w ˜T . Denote the MPUM in LwL for w ˜T by BL,w˜T . Questions that arise are: When does BL,w˜T exist ? How can it be computed from the data w ˜T ? If w ˜T is generated by an element B ∈ Lw , when will BL,w˜ T = B ? Conditions for existence of this MPUM are readily deduced from the Hankel matrix formed by the data. We first introduce our notation for Hankel matrices (meaning block Hankel matrices). Exclusively Hankel matrices formed by vectors are needed. Let f : [1, t] → Rn . Define the Hankel matrix formed by f with depth Δ1 and width Δ2 , Δ1 + Δ2 − 1 ≤ t, by    H(Δ1 , Δ2 ) (f ) :=  

f (1) f (2) .. . f (Δ1 )

f (2) f (3) .. .

··· ··· .. .

f (Δ1 + 1) · · ·

f (Δ2 ) f (Δ2 + 1) .. .

    

f (Δ1 + Δ2 − 1)

The left kernel of a matrix M ∈ Rn1 ×n2 , denoted by leftker (M ), consists of the row vectors n ∈ R1×n1 that are annihilated by postmultiplication by M : nM = 0. In the case of a Hankel matrix, it is often useful to view the elements in the left kernel as vector polynomials, as follows. Assume that n = n0 n1 · · · nΔ1 −2 nΔ1 −1 ∈ leftker H(Δ1 ,Δ2 ) (f ) with the nk ’s ∈ R1×n . Associate with n the polynomial vector n0 + n1 ξ + · · ·+ nΔ1 −2 ξ Δ1 −2 + nΔ1 −1 ξ Δ1 −1 ∈ R1×n [ξ], and denote it, with slight abuse of notation, also as n (ξ). Note that it may appear as if degree (n) = Δ1 − 1, but since some of the coefficients of n may be zero, the actual degree may be lower. Call leftker H(Δ1 , Δ2 ) (f ) module-like if [[ [ n0 n1 · · · nΔ1 −3 nΔ1 −2 0 ] ∈ leftker H(Δ1 , Δ2 ) ]] ⇒ [[ [ 0 n0 n1 · · · nΔ1 −3 nΔ1 −2 ] ∈ leftker H(Δ1 , Δ2 ) (f ) (f ) ]]. Equivalently, if [[ n(ξ) ∈ leftker H(Δ1 , Δ2 ) (f )

∧ degree(n) < Δ1 − 1 ]] ⇒ [[ ξn(ξ) ∈ leftker H(Δ1 , Δ2 ) (f ) ]].

This last implication shows where the terminology ‘module-like’ refers to. These notions are very useful for computing BL,w˜T (which may not exist – take w ˜T = (1, 1, 2, 2, 2) and L = 2). Observe that if B ∈ Lw is unfalsified by w ˜T , then


401

[[ (n ∈ NB ) ∧ (degree(n) ≤ L) ]] ⇒ [[ n ∈ leftker H(L+1,T−L) (w ˜T ) ]]. It is logical to aim at the module generated by leftker H(L+1,T−L) (w ˜T ) as the annihilators of the MPUM BL,w˜T . However, for arbitrary w ˜T , this module may contain falsified elements (consider, e.g. w ˜T = (0, 0, . . . , 0, 0, 1)). The module-like property ensures that all elements in the module generated by leftker H(L+1,T−L) (w ˜T ) are unfalsified. This leads to the following results. (i) A sufficient condition for the existence of the MPUM BL,w˜T is that leftker H(L+1,T−L) (w ˜T ) is module-like. (ii) leftker H(L+1,T−L) (w ˜T ) is module-like iff rank H(L,T−L) (w ˜T ) = rank H(L,T−L+1) (w ˜T ) . (iii) If leftker H(L+1,T−L) (w ˜T ) is module-like, then BL,w˜T = ker (N (σ)) , with N ∈ R•×w [ξ] such that the R [ξ]-module generated by its rows is equal to the R [ξ]-module generated by leftker H(L+1,T−L) (w ˜T ) . ˜T recovers the system that Under mild conditions, the MPUM in LwL for w generated the data. In other words, the procedure is consistent. The crucial condition here is that the input component in the observations must be persistently exciting. f : [1, T] → R• is said to be persistently exciting of order P if the rows of the Hankel matrix H(P,T−P+1) (f ) are linearly independent. u = We have the following ‘consistency’ result. Consider B ∈ Lw . Let y Π −1 w be an input/output partition of B as in (I/O). Denote the corresponding observed input component of w ˜T by u ˜T . It can be shown that BL,w˜T = B if the following conditions are satisfied: (i) (ii) (iii) (iv)

w ˜T ∈ B|[1,T], L(B) ≤ L, B is controllable, u ˜T is persistently exciting of order L + n(B) + 1.

Moreover, under these conditions, the left kernel of H(L+1,T−L) (w ˜T ) is modulelike, and therefore B = ker (N (σ)) , where N ∈ R•×w [ξ] is any polynomial matrix such that the R [ξ]-module generated by its rows equals the R [ξ]module generated by leftker H(L+1,T−L) (w ˜T ) . In a typical application of this result, assume that an infinite time-series w ˜: N → Rw is observed, generated by an unknown, but controllable, system B ∈ Lw , for which upper bounds L for the lag L(B) and n for the state dimension n(B) are known (since L(B) ≤ n(B), it suffices to have the upper bound n). Then, if the observed input component u ˜ of w ˜ is persistently exciting of order ˜T is L + n + 1, the MPUM BL,w˜T will be equal to B for all T such that u persistently exciting of order L + n + 1. So, this exact deterministic SYSID algorithm yields consistency in finite time.

402

J.C. Willems

We now turn to the computation of BL,w˜T . One possibility suggested by the above results is to compute a basis for the whole left kernel of H(L+1,T−L) (w ˜T ). This is feasible (also approximately, using SVD-like algorithms), but it is not necessary nor efficient to compute the whole left kernel, especially when the data is generated by a system in LwL for which the lags in the difference equations (Ker) vary widely, or when L is only a rough upper bound for L(B). It suffices to compute vector polynomials n1 , n2 , . . . , ng ∈ R1×w [ξ] such that the R [ξ]-module spanned by n1 , n2 , . . . , ng equals the R [ξ]-module spanned by the left kernel of H(L+1,T−L) (w ˜T ). The problem hence comes down to obtain a set of polynomial vectors in the left kernel of H(L+1,T−L) (w ˜T ) that generate this submodule. This is algorithmically much simpler that computing the whole left kernel, which may have a dimension that is much larger than the dimension (the number of generators) of this submodule. This submodule may be computed with an algorithm that is recursive in the observation horizon T or in the lag L. An algorithm that is recursive in T has been obtained in [20, 36]. This algorithm is based on a recursive computation of the MPUM for infinite time-series, and is a generalization of the Berlekamp-Massey decoding algorithm. We have recently also obtained an algorithm that is recursive in L. The details will be reported elsewhere.

6 Subspace ID The importance and usefulness of state models are beyond dispute. By explicitly displaying the memory of a system, state models bring the dynamical properties of a system to the surface, clearly into view. In fact, since the 1960’s, input/state/output models are, up to a fault, used as the basic framework for studying open dynamical systems. Since the 1980’s, it has become apparent that state models are also very well suited for model reduction. The highlight of this development is Keith Glover’s classic paper [12] on balanced realizations and AAK type model reduction algorithms. The one area in systems theory where state models play a somewhat secondary role is SYSID. In earlier sections, I have dealt extensively with the multitude of representations of elements of L• . SYSID algorithms typically aim at one of these representations, usually a kernel representation (Ker), an input/output version (I/O), or a latent variable representation (L). One major exception to this are the subspace identification algorithms. These algorithms pass from the data w ˜T to an input/state/output representation (S). In this section, I deal with these algorithms. Contrary to many authors, I do not consider the classical realization algorithms as part of SYSID. In realization theory, one has a model at the very outset, a model that, for example, gives the output as a convolution of the input with the impulse response matrix, and the problem is to find an equivalent state representation. Finding the parameters of a state representation from the impulse response is a representation problem and does not have much to do with SYSID (although one may


403

get many good ideas for SYSID from these algorithms). Much more SYSID oriented are algorithms that pass from w ˜T or (˜ uT , y˜T ) if an input/output partition is known, to a state representation of BL,w˜ T . This is the problem that I will discuss now. In this section we consider SYSID algorithms that aim at i/s/o models (S). As always in state representations, we meet the issue of non-uniqueness, both of the state representation parameters, and of the state trajectory corresponding to a given w-trajectory. It is easy to show that every B ∈ Lw admits an observable representation (S) (but not necessarily a controllable one!). Moreover, for a fixed input/output partition Π, observable representations (S) of the same B ∈ Lw are unique up to a basis choice in the state space. If we allow also Π (i.e. the input/output partition) to vary, things become a bit more complicated, but we need not deal with this here. The discussion in this section is a bit informal: we assume, where needed, that w ˜T is such that the MPUM in LwL exists, that L and T are sufficiently large, controllability, observability, persistency of excitation, etc., and also that the input/output partition in BL,w˜T is known. Assume that from the observations w ˜T , we have somehow identified a model B ∈ Lw such that w ˜T ∈ B|[1,T] (for example, assume that we have identified the MPUM BL,w˜T ). We could then compute a state representation (S) for B, and obtain a corresponding state trajectory x ˜T =

(˜ x (1) , x ˜ (2) , . . . , x ˜ (T))

x ˜ (t) ∈ Rn .

uT , y˜T ) are related This lays out the path to go from w ˜T to x ˜T . x ˜T and w ˜T ∼ = (˜ by the equations x ˜(2) x ˜(3) · · · x ˜(T) y˜(1) y˜(2) · · · y˜(T − 1)

=

AB CD

x ˜(1) x˜(2) · · · x ˜(T − 1) . u ˜(1) u˜(2) · · · u ˜(T − 1)

(♣)

However, if we could somehow obtain first the state trajectory x ˜T , directly from w ˜T , without using the underlying model, we could subsequently solve the linear system of equations (♣) for the unknown parameter matrices (A, B, C, D). This would yield a SYSID algorithm that identifies a state representation for BL,w˜T . The advantage – and a major one indeed – of dealing with the equations (♣) is that, from a numerical linear algebra point of view, they lend themselves very well to approximations. Indeed, one can first use SVD-like algorithms to reduce the state dimension by rank reduction of the matrix x ˜(1) x˜(2) · · · x ˜(T − 1) x ˜(T) and subsequently solve the equations (♣) approximately for (A, B, C, D) in a least squares (LS) sense. These numerical features are very well explained and effectively exploited in [30]. The following question emerges: How can we pass directly from w ˜T to the corresponding state trajectory x˜T of a state representation of the MPUM BL,w˜T ?

404

J.C. Willems

One algorithm which achieves this is based on partitioning the Hankel matrix of the data into the ‘past’ and the ‘future’, as follows   w(1) ˜ w(2) ˜ ··· w(T ˜ − 2L − 1)   .. .. .. ..   . . . .  ← past P   w(L ˜ + 2) ··· w(T ˜ − L − 1)    ˜ + 1) w(L    −−−−− −−−−− −−− −−−−−−−      w(L ˜ + 2) w(L ˜ + 3) ··· w(T ˜ − L)  ← future F  .. .. .. ..     . . . . w(2L ˜ + 2) w(2L ˜ + 3)

···

w(T) ˜

i.e. 

 H(L+1, T−L−1) (w ˜T ) H(2L+2, T−2L−1) (w ˜T ) =  − − − − − − − − − − −  H(L+1, T−L−1) (σ L+1 w ˜T )

← past P ← future F

Consider now the span of the rows of the ‘past’ matrix P and the span of the rows of the ‘future’ matrix F . It turns out that the intersection of these spans is equal to the state space of BL,w˜T , and that the linear combinations of the rows of P (or F ) contained in this intersection give the ‘present’ state trajectory! More precisely, assume that the span of the rows of the matrix X equals the intersection of the span of the rows of P with the span of the rows of F . Then the columns of X are the ‘present’ state trajectory. Note that we may as well assume that the rows of X are linearly independent. The fact that X is unique up to pre-multiplication by a nonsingular matrix corresponds to the freedom of the choice of the basis of the state space of the underlying system. In other words, X = (˜ x (L + 2) , x ˜ (L + 3) , . . . , x ˜ (T − L − 1) , x˜ (T − L)) . This fact, first noticed in [35, sections 15–17], and generalized in many directions by De Moor and co-workers (see e.g. [30]), allows to identify the system parameter matrices (A, B, C, D) of BL,w˜T in input/state/output form by solving equations as (♣). This intersection result is reminiscent of the following observation, usually attributed to Akaike [1]. Consider a zero mean stationary ergodic gaussian vector process z, z(t) ∈ Rz , t ∈ Z, with a rational spectral density. Let z˜ : Z → Rz be a realization of it. Now form the Hankel matrix




.. .. .. .. .. .. .. . . . . . .  .  ··· z˜ (−2) z˜ (1) z˜ (0) · · · z˜ (t − 1) · · ·   ··· z˜ (−1) z˜ (0) z˜ (1) · · · z˜ (t) ···   − − − − − −− − − −− − − − − − − − − −− − − −   ··· z˜ (0) z˜ (1) z˜ (2) · · · z˜ (t + 1) · · ·   ··· z ˜ (1) z˜ (2) z˜ (3) · · · z˜ (t + 2) · · ·  .. .. .. .. .. .. .. . . . . . . .

405

            

← past

← future

Then the orthogonal projection (in the inner product induced by the second order averages) of the span of the rows of the future onto the span of the rows of the past is finite dimensional, and the orthogonal projection of the rows of the future onto the span of the rows of the past yields a realization of the corresponding ‘past-induced’ Markovian trajectory x ˜ associated with z˜. But, how do we compute the common linear combinations spanned by the 1 rows of M1 and M2 in the partitioned matrix M M2 ? This can be done by M1 computing a basis for the left kernel of M2 . Indeed, if [n1 n2 ] is in the left kernel, then n1 M1 = −n2 M2 yields one of these linear combinations. By letting [n1 n2 ] range over a basis of the left kernel, all common linear combinations are obtained. This can be applied to

P −− F

and allows to comP

pute the desired state trajectory by computing the left kernel of −− . The F disadvantage of this algorithm is that it does not make use of the Hankel structure of the matrices P and F (a common drawback of subspace algorithms). Equivalently, it makes no use of the module structure of the left kernel. For L large (say, much larger than L(B)), the algorithms that compute the whole left kernel risk at being very inefficient. But, there is more. As we have shown P in the previous section, the left kernel of −− can in fact be deduced from F the left kernel of P alone by simple module operations, and therefore there is no need to consider the separation into ‘past’ and ‘future’. These considerations lead to the following algorithm for constructing the state trajectory from the observed data, assuming persistency of excitation, controllability, L sufficiently large, etc. Let N ∈ R•×w [ξ] be a polynomial matrix such that the R [ξ]-module generated by its rows equals the R [ξ]module generated by leftker H(L+1,T−L) (w ˜T ) , and assume degree(N ) ≤ L. In the previous section, we have explained how to view elements in this left kernel as vector polynomials. The construction of the state trajectory involves the ‘shift-and-cut’ operator π on R [ξ], defined by π : p0 + p1 ξ + p2 ξ 2 + · · · + pn ξ n → p1 + p2 ξ + + · · · + pn ξ n−1 . π can be extended in an obvious way to act on polynomial vectors and matrices. Now define XN as XN := col(πN, π 2 N, . . . , π degree(N )−1 N ),

406

J.C. Willems

and view XN as a matrix by reversing the process by which we viewed elements in the left kernel of H(L+1,T−L) (w ˜T ) as polynomial vectors. Then XN H(L+1,T−L) (w ˜T ) = [˜ x (1) x˜ (2) · · · x˜ (T − L − 1) x˜ (T − L)] . This yields a very effective construction of a state trajectory. This algorithm is actually an implementation of the ‘cut-and-shift’ algorithm for constructing a state representation starting from a kernel representation developed in [27]. There exist many related subspace identification type algorithms, for example based on oblique projections and various LS methods [30], [34], [22]. We have already mentioned that subspace ID is very well suited, pragmatically, through rank reduction of the state trajectory X , followed by LS methods for solving (♣), for identifying a model that fits the observed data w ˜T approximately. Of course, the most satisfying way theoretically of approaching SYSID through approximate modeling is by defining a misfit criterion, say misfit(w ˜T , B) = minimum ||w ˜T − w|[1,T] || w∈B

2 ([1,T],R

w)

.

and then minimizing the misfit misfit(w˜T , B) over a model class, for example the elements of B ∈ Lw with a limited complexity, with complexity defined as something like (m(B), n(B), L(B)). This is the approach taken in [16], [28], and [22]. This last reference shows how to approach this problem using STLS (structured total least squares) algorithms.

7 Latency Minimization The usual model class considered in SYSID lead to equations of the form R (σ) w = M (σ) ε

(E)

with R and M real polynomial matrices of suitable sizes. These equations involve, in addition to the time-series w : N → Rw , also the time-series ε : N → Re . In (E), w is the time-series which is observed on the interval [1, T] through w ˜T , and ε consists of unobserved latent variables, which serve to help to explain the data. In the behavioral language, this means that we start with a latent variable representation and aim at explaining the observed manifest variables w with the help of the unobserved latent variables ε. It is often useful to consider a more structured version of (E), for example, P (σ) y = Q (σ) u + M (σ) ε,

w=

u , y

(E )

with det(P ) = 0, ensuring that both u and ε are inputs, and y is the output. The question then basically is to identify the behavior Bfull , or the polynomial matrices (R, M ) or (P, Q, M ), from observations (of a finite sample) of w, or u and y, entangled by an unobserved latent variable trajectory ε.


407

The performance, from a SYSID point of view, of the model (E) with behavior Bfull ∈ Lw+e can be assessed by the latency, say latency(w˜T , Bfull) :=

minimum

[[(w,ε)∈Bfull ]] ∧ [[w|[1,T] =w ˜ T ]]

||ε||

2 (N,R

e)

.

Subsequently minimizing the latency over the model class (a family of Bfull’s) leads to a SYSID algorithm. Note that because the models (E) are usually unobservable (in the sense that ε cannot be deduced from w, R, and M ), it is in general not possible to reformulate the latency criterion in terms of the w variables alone. The following questions arise. Given the behavior Bfull ∈ Lw+e how do we compute the latency in terms of the polynomial matrices (R, M )? Given a model class M ⊆ Lw+e , how do we minimize the latency over M? The computation of the latency is a deterministic Kalman filtering problem. The minimization over the model class usually leads to a non-convex minimization problem. It turns out, in fact, that from a numerical (but not from a conceptual) point of view, latency minimization is precisely what is done in prediction error minimization methods (PEM) [21] (strongly related to maximum likelihood (ML)). The difference is ‘merely’ the interpretation of the latent variables ε. In latency minimization, ε is introduced in order to explain the observed data w ˜T (but no physical reality is attached to it), while in PEM and ML, ε is interpreted as a stochastic ‘disturbance’ input which, together with the input component in the w variable in (E) (see (E )) and the initial conditions, produces the data. Under suitable stochastic assumptions, as ε independent of the input component u of (E ), one obtains precisely the same estimate for (R, M ) by using latency minimization as by using (PEM) or (ML). In latency minimization, one wants to keep ε small, while explaining the observations, in PEM one minimizes the a posteriori prediction error in a stochastic sense, and in ML one maximizes the likelihood of the observations, given the model. Assume now that (R, M ) in (E), equivalently (P, Q, M ) in (E ), have been identified from the observations w ˜T , how should one proceed? Which model does this identification procedure entail? If these estimates have been obtained by latency minimization, then it is natural to take R(σ)w = 0 i.e. P (σ)y = Q(σ)u as the model to be employed for prediction, control, or what have you. This is different, of course, from taking the manifest behavior of (E). Setting ε = 0 in (E) yields a much smaller behavior than eliminating ε from (E). However, if the estimates have been obtained by (PEM) or (ML), then it is natural to stick to (E), and take into consideration the identified stochastic characteristics of the variables ε in (E) or (E ). This stochastic model can then be used for predicting future outputs from future inputs, for control, etc. Does (E) provide additional flexibility compared to (Ker)? Is the set of w trajectories that can be modelled this way larger than what can be achieved without the ε’s? This may appear so at first sight, but it is not. Indeed, by the elimination theorem the ‘manifest’ behavior of (E), i.e.

408

J.C. Willems

B = {w :→ Rw | ∃ ε → Re such that (E) holds} in fact also belongs to Lw . In other words, there exists R ∈ R•×w [ξ] such that B = ker (R (σ)), yielding the kernel representation R (σ) w = 0. What is then the rationale for introducing ε and using the model class (E)? We can intuitively interpret the use of latency minimization, followed by setting the latent variables ε = 0 and using R(σ)w = 0 as the identified model, as a way of minimizing some sort of misfit. But it is a misfit that involves unobservable latent variables, with ‘unobservable’ interpreted in the technical sense for latent variable systems explained earlier. The advantage of introducing latent variables above straight misfit minimization can be seen by considering the case w = 1. B ∈ L1 implies that either B = RN , not useful as a model, or that B is an autonomous system, and therefore finite dimensional, consisting of trajectories that are a finite sum of polynomial exponentials. If we assume in addition that B is stable, i.e. [[ w ∈ B ]] ⇒ [[ w(t) → 0 for t → ∞ ]], then we see that an autonomous behavior B cannot adequately capture the dynamic features of a persistent trajectory w ˜T . The assumption of the presence of unobservable latent inputs ε in (E) indeed offers better modeling possibilities for data fitting. Of course, one can also use a combination of the straight misfit minimization and latency minimization. It is an open question if a model obtained by latency minimization and setting ε = 0, i.e. R(σ)w = 0, or P (σ)y = Q(σ)u, equivalently, obtained by (PEM) and keeping the deterministic part, or a model obtained by misfit minimization, will do significantly better than a model (in a corresponding model class) obtained from, say, a heuristic subspace ID type algorithm obtained by reduction of the state trajectory X followed by a LS solution (♣), as explained earlier. Extensive testing on industrial data [23] suggests that models obtained by latency minimization or (PEM) give a marginally better fit in predicting future outputs from future inputs as compared to subspace ID, but at the expense of a much larger computational effort. Systematic methods to fit a linear dynamical system to the data will lead to a reasonable model. It makes sense to expect that a linear approximation found during the learning stage will prevail during the validation stage. But there is no reason to expect that statistical features which happen to be present in the data during the learning stage to prevail during the validation stage. Which interpretation, latency minimization, or the stochastic interpretation of (PEM) or (ML) should one prefer? My own preference lies squarely with the latency minimization. The main argument is that, in my opinion, deterministic approximation articulates in a much more satisfying way the essence of the SYSID problem. It seems to me that the lack of fit between the observed data and the chosen model will in most practical situations (both in engineering and in economics) be due to the fact that the data has been produced by a nonlinear, high order, time varying system, influenced by spurious inputs and other unknown interactions with the environment. Minimization of the latency or the misfit pointedly articulates the fitting problem. Stochas-


409

tic methods assume that the lack of fit is due to some unobserved input with stochastic regularity. It is unclear how to justify such an assumption, let alone the statistical independence of ε and the driving input component of u, etc. Furthermore, it is awkward how to deal with the additive noise. Obviously, in many applications this noise term cannot be justified when the system is at rest, it is incompatible with physical properties as dissipativeness, it often assumes an infinite supply of energy, etc. In typical applications it is simply artificial to insist on a stochastic interpretation, even as an approximation. It is even hard to imagine how such stochastic regularity could come about in reality. The whole issue is obviously a very complex one, that can not be discussed without examining the interpretation of probability.

8 Interpretations of Probability There is scarcely any field of science for which the foundations and the interpretation have stirred as much controversy and debate as probability. It shares this doubtful honor with the rules of inference (tertium non datur?) and with quantum mechanics, but even there the interpretation question is to a large extent tied up with the meaning of probability. It is ironic that both quantum mechanics and probability, perhaps the most successful scientific developments of the twentieth century, appear to have such shaky foundations. The foundations and interpretation of probability have been discussed by some of the finest scientific minds in history. Jakob Bernoulli, Borel, Carnap, de Finetti, Jeffreys, Fréchet, Keynes, Kolmogorov, Laplace, von Mises, Poincaré, Popper, Ramsey, Venn, are only a few of the many scientists who have written books on this topic (see [8, 14, 32] for the history and the status of this subject). In this section, I describe my own impressions about this, briefly, and perhaps too superficially. The subject area is quite subtle, unsettled, and very much la mer a ` boire. Four main views have emerged, in combination with a seemingly uncountable number of intermediate nuances. The first two interpretations are considered objective, empirical, physical. The first one is the relative frequency point of view, formalized by von Mises [31]. Popper [25, 26] devised a second view, in which probability is interpreted as ‘propensity’. The third and fourth main interpretations are epistemological and lead to probability as degree of belief. They often go under the name ‘Bayesian’, but this certainly does not nail down the viewpoint uniquely [13]. The third interpretation, called the ‘logical’ theory, deduces probability from relations among basic propositions. Keynes [19] is usually considered the person who introduced this approach, later put in a more definitive form by Carnap [3]. The fourth interpretation, championed by De Finetti [5], is radically subjective: probability articulates one’s own personal degree of belief. Nothing more, nothing less. Between these interpretations, we find many subtle nuances leading, in the words of Galavotti [8], to “a whole array of seemingly irreconcilable perspectives”.

410

J.C. Willems

The relative frequency interpretation assumes that we have an ensemble of objects, and that each individual object has certain characteristics, which differ from object to object. This characteristic then becomes a random variable. For example, if a characteristic x is real-valued, then its distribution function F (x) is equal to the relative frequency of objects for which the characteristic is less than or equal to x. If we apply this to a well-defined existing finite ensemble and a well-defined set of characteristics, there seems no apparent interpretational difficulty. However, for applications in which the ensemble involves events that are still to come, as is unavoidable in time-series analysis (and hence in stochastic SYSID), or that are only potentially realized (flips of a coin, throws of a die), or are too numerous to be realizable (bridge hands), or when the ensemble is infinite, the frequency interpretation poses severe interpretation problems. Popper formulated the propensity interpretation in order to accommodate for the fact that quantum mechanics asks for a physical interpretation of a single event probability. In this interpretation, certain physical objects, even if prepared in identical ways and operating in identical environments, may react differently, with outcomes that have fixed probabilities. This probability will then be the relative frequency obtained from repeated identical trials. Since, in Feynman’s words, nobody understands quantum mechanics, I must (reluctantly) accept that this view may be appropriate, perhaps unavoidable, for accommodating the present day orthodox view of quantum mechanics. But I find it hard to accept propensity as a description of our day to day (seemingly) random experiences. However, some physicists appear to suggest that the propensity interpretation may even apply to mundane things like coin tossing, and there are many publications concerning the transition from ‘determinism’ to ‘randomness’ of a wobbling coin. In fact, recently experiments have been conducted which show [6] that a coin, repeatedly flipped by means of a carefully constructed mechanical device, lands practically each time on the same side. These experimenters summarize their results by stating – ironically, I hope – ”We conclude that coin tossing is ‘physics’, not ‘random’”, as if there was a reason to expect otherwise. These experiments have been avidly reported in the popular press, whose main concern seems to be that the toss at the beginning of a football match may not be fair. The logical theory is based on rules of inductive logic, leading to relations between propositions, for example that certain hypotheses make other hypotheses more probable. Or that certain basic ‘symmetric’ events (as any pair of cards being on top of a well-shuffled deck) demand equal probabilities. This is useable for games of chance, with discrete outcomes, where probability quickly reduces to combinatorics. However, it is hard to apply this principle to continuous random variables, as the many discussions surrounding the principles of indifference or insufficient reason demonstrate. The personal belief interpretation is in a sense the clearest of them all. By stating a probability, an expert quantifies his or her degree of belief in the occurrence of an event. The difficulty is to describe how these degrees


411

of belief are arrived at, certainly when it concerns complex phenomena, in turn influenced by many uncertainties, leading to a seemingly infinite regress. Think, for example, of the problem of trying to arrive, in a systematic way, at the probability as a degree of belief of the Dutch soccer team winning the coming World Cup. It is clear what probability as degree of belief means, but it is difficult to obtain it in a scientific manner, let alone to falsify. And, of course, it leads to the question why I should care about your degree of belief. There are frequentists who maintain that the frequency interpretation is the only viable one. Propensity states a law about a limiting frequency which is yet to be realized, belief expresses the relative frequency of what would happen if a situation presented itself over and over again. For example, when we say that the probability of the Dutch team winning the World Cup is 21 , a frequentist would say that we mean that if this tournament were repeated indefinitely, under identical circumstances, the team would be victorious in 50% of the repeats. However, it is hard to take the latter earnestly, in part because, in my opinion, identical circumstances lead to identical outcomes. I am more sympathetic with Carnap’s admission of two concepts of probability [2], relative frequency on the one hand, and belief on the other. And I am also sympathetic with complaints that it is simply confusing and bad mathematical etiquette to use the same term ‘probability’ for distinct ideas as ‘frequency’ and ‘belief’. An important point that is unfortunately all too often absent or insufficiently emphasized in discussions about mathematical models is the issue of open versus closed systems. In open systems, we aim for a law that describes the relation between system variables, but does not go to the point of stating what will actually happen. The interaction with the (unmodeled) environment is explicitly part of the model, but what the environment will be, is not specified. The paradigmatic example of an open system is an input/output system, which explains the output in terms of the input, but leaves the input as imposed by the environment, unexplained and unspecified. The paradigmatic example of a closed system is an autonomous system. Systems and control theory, signal processing, and computing science seem to be the only areas which clearly address open systems. Mathematicians and physicists, for example, unfortunately usually end up viewing dynamical systems as closed systems. The very definition of a random variable or a random process suggests a closed system, and the influence of the environment is not explicitly taken into account in the mathematical set-up. Compare this, for example, with our notion of a (deterministic) behavior, which clearly accounts for the interaction with the environment. The literature on coin tossing or dice throwing, often viewed as the paradigmatic examples of randomness, suffers from a lack of understanding of the fundamental difference between open and closed systems. All too often, it is suggested that the randomness in a coin toss or a dice throw comes from the coin or the die itself. For example, in the textbook by Cramér [4, page 154],

412

J.C. Willems

it is stated “The numbers pr [the probability that the die shows outcome r] should in fact be regarded as physical constants of the particular die that we are using [· · · ] experience shows that for a well-made die the frequency of any event r in a long series of throws usually approaches 1/6 [· · · ]”. Obviously, in dice throwing, the source of randomness, if present, comes not from the die, but from the actions of the dice thrower (including the way the die is prepared for a throw). In other words, we are dealing with an open, not a closed system. So, it is absurd to insist that the probability of the outcomes is a property of the die, well-made or not. When adherents of the propensity interpretation state (see Hacking [14, page 14]) “The propensity to give heads is as much a property of the coin as its mass, and the stable long run frequency found on repeated trials is an objective fact of nature independent of anyone’s knowledge of it, or evidence for it”, they are stating something about the environment of the coin. And this something is either untrue or trivially true. If the coin is experimented with in a fixed environment, i.e. if it is tossed repeatedly in exactly the same way, the coin will each time fall on the same side, in accordance with the laws of mechanics and as the experimenters in [6] have demonstrated. Whether it shows always heads or always tails, depends on the experimental set-up – the environment – and on the physical properties of the coin. Some fixed environments will give a run with all heads, some will give a run with all tails. For some experimental set-ups, it is trivial to predict which run will happen, for others, it may be difficult – perhaps impossible, given the present state of knowledge, sensor technology, and computer power – to predict what will happen before the toss, but that we will see a fixed run is certain. However, if the coin is experimented with in an environment that changes from trial to trial, then the coin may sometimes fall on the one side and sometimes on the other, each time in accordance with the laws of mechanics. The relative frequency (including the question of whether the limit exists) depends of how the experimenter changes the environment from trial to trial. There need not be a stable long run frequency found on repeated trials, by properly selecting the trials, any relative frequency can be found, and so it is not an objective fact of nature. If we see a stable long run frequency for the coin, it is only because we choose elements in the different trials according to some stable long run frequency, or better, according to some kind of randomized law that generates a stable long run frequency. The fact that needs to be explained, thou, is why and when a little bit of randomness introduced in the trials generates a robust stable long run frequency in the outcomes. It looks reasonable to assume that untrained, uncoordinated individual who enthusiastically throws a well-made die will produce a stable long frequency, but this is more a matter of belief than an objective fact of nature. Or, worse yet, a truism in the sense it is by definition what is meant by untrained, uncoordinated, enthusiastic throws. The many experiments that have been carried out to prove that coin tossing is random at best serve to prove that the coin transmits, and perhaps amplifies or regularizes, randomness that is external to the coin. So, one may


413

as well use a random number generator to drive a mechanism that tosses the coin, avoid the conclusion that coin tossing is ‘random’, not ‘physics’, but instead opt for the punchline that a random number generator generates random numbers. It is also instructive to deconstruct Buffon’s needle or Bertrand’s ‘paradox’ as originating by trying to explain the outcomes of variables of an open system using the limited language of closed systems. Of course, we get different probabilities if we throw a needle using different mechanisms. There is nothing paradoxical in this whatsoever. This also applies to stochastic processes. As such, I find ARMAX systems (with the exogenous variables left unexplained) a much more convincing model class (e.g. for econometric modeling) than ARMA systems.

9 Stochastics in SYSID In engineering education in general, and in systems and control in particular, the issue of the interpretation of probability is invariably glossed over completely. At best, there is some passing reference to the relative frequency interpretation. The mathematical structure and algorithms invariably take the central place in textbooks. The saving grace for this nonchalance is the fact that the mathematical measure theoretic framework appears to be useable by and large regardless of the interpretation. The extent to which the interpretation of probability is relevant in the use of probabilistic models and methods depends, in my opinion, to a large measure on the field of application. In descriptive sciences as physics and economics, it is indispensable to articulate the interpretation used. In mathematics, the interpretation usually does not really matter at all. Deep and useful theorems have been derived about the mathematical theory of probability without worrying about the interpretation. In engineering (and prescriptive aspects of economics) one can, it seems to me, take the following intermediate position. An algorithm-based engineering device, say in signal processing, communication, or control, comes with a set of ‘certificates’, i.e. statements that guarantee that the device or the algorithm will work well under certain specified circumstances. These circumstances need not be the ones under which the device will operate in actual practice. They may not even be circumstances which can happen in the real world. These certificates are merely quality specifications. Examples of such performance guarantees may be that an error correcting code corrects an encoded message that is received with on the average not more than a certain percentage of errors, or that a filter generates the conditional expectation of an unobserved signal from an observed one under certain prescribed stochastic assumptions, or that a controller ensures robust stability if the plant is in a certain neighborhood of a nominal one, etc. Which interpretation of probability prevails in SYSID? Of course, in this case the aim is to model reality, and one cannot avoid the interpretation is-

414

J.C. Willems

sue all together. But one can view SYSID as a design of algorithms, and as such, it is reasonable to test a proposed algorithm under certain ‘idealized’ assumptions, i.e. for a ‘test case’ in order to obtain a certificate, in the spirit of what has been explained in the previous paragraph. For example, one could assume that (u, y, ε) in (E ) is a stationary ergodic gaussian process with u and ε independent and satisfying (E ) for some (P, Q, M ) from a model class, and to examine how the algorithm that chooses an (P, Q, M ) on the basis of w ˜T performs for T → ∞. This, of course, points to a relative frequency interpretation. As a ‘testbed’ this has appealing features: it is well-founded mathematically and algorithmically, and the probabilistic structure will have an ‘averaging effect’ on the data, avoiding excessive influence of unusual features. As a testbed, there is nothing wrong with this, to the contrary. As long as one avoids the claim that it yields an (approximate) description of the reality that has produced the data. However, it is just as reasonable to test the algorithms for signals (w, ε) for which the correlations like lim

T→∞

1 T

T t =1

σ t w(t + t) σ t ε(t + t)

w(t ) ε(t )

exist for certain t ∈ N. Or demand good SYSID features for certain classes of signals (w, ε) ∈ 2 (N, Rw+e ), thereby avoiding stochastic complexities. The relation of probability with the real world is a serious problem that should be addressed in SYSID, and it is a pity that attempts to put this on the systems theory agenda [17] have fallen on deaf ears. The development of probability as limiting frequency owes much to John Venn, best known to us through the ‘Venn diagrams’ for testing syllogisms. John Venn (1834–1923) was president of Gonville and Caius college, fortuitously the same college which will host our symposium dinner. Venn wrote the first book [33] devoted to the frequency interpretation of probability. He complains that ”probability has been very much abandoned to mathematicians, who as mathematicians have generally been unwilling to treat it thoroughly. In the work of mathematicians we would search in vain for anything like a critical discussion of the fundamental principles upon which its rules rest, the class of enquiries to which it is most properly applicable, or the relation it bears to logic and the general rules of inductive evidence”. Little seems to have changed in this respect since these words were written, except that the unwillingness to which John Venn refers has now been extended beyond mathematics. It certainly includes SYSID. None of my favorite books on the subject [15, 18, 21, 29, 30] discusses this issue at all, except [21, page 22], where it is merely stated that ”information about past disturbances could, however, be important for making qualified guesses about future values. It is therefore natural to employ a probabilistic framework to describe future disturbances”. Surely, an important area of system theory with an ambition to be widely applicable in engineering practice deserves a more balanced justification for the mathematical modeling framework in which it chooses to operate.


415

Acknowledgments This research is supported by the Belgian Federal Government under the DWTC program Interuniversity Attraction Poles, Phase V, 2002–2006, Dynamical Systems and Control: Computation, Identification and Modelling, by the KUL Concerted Research Action GOA AMBioRICS, and by several grants the Flemish Fund for Scientific Research: ICCoS, ANMMM, MLDM. I would also like to thank Ivan Markovsky (K.U. Leuven), Paolo Rapisarda (Un. of Southampton), and Bart De Moor (K.U. Leuven) for their collaboration on many aspects of the problems discussed in this paper, and Gert de Cooman (Ghent Un.) and Manfred Deistler (T.U. Vienna) for remarks concerning the latter sections of this article.

References 1. H. Akaike, Markovian representation of stochastic processes by canonical variables, SIAM Journal on Control, volume 13, pages 162–173, 1975. 2. R. Carnap, The two concepts of probability, Philosophy and Phenomenological Research, volume V, pages 513–532, 1945. 3. R. Carnap, Logical Foundations of Probability , Chicago University Press, 1950. 4. H. Cramér, Mathematical Methods of Statistics, Princeton University Press, 1946. 5. B. De Finetti, Theory of Probability, Volume 1 and 2, Wiley, 1974. 6. P. Diaconis, S. Holmes and R. Montgomery, Dynamical bias in the coin toss, manuscript, 2004. 7. J. C. Doyle, K. Glover, P. P. Khargonekar, and B. A. Francis, State-space solutions to standard H2 and H∞ control problems, IEEE Transcations on Automatic Control, volume 43, pages 831–847, 1989. 8. M. C. Galavotti, Philosophical Introduction to Probability, CSLI Publications, 2005. 9. K. Glover and J. C. Willems, On the stability of numerical integration routines for ordinary differential equations, Journal of the IMA, volume 11, pages 171– 180, 1973. 10. K. Glover, Structural Aspects of System Identification, Ph.D. dissertation, M.I.T., 1973. 11. K. Glover and J. C. Willems, Parametrizations of linear dynamical systems: Canonical forms and identifiability, IEEE Transcations on Automatic Control, volume 19, pages 640–646, 1974. 12. K. Glover, All optimal Hankel-norm approximations of linear multivariable systems and their L∞ error bounds, International Journal of Control,volume 39, pages 1115–1193, 1984. 13. I. J. Good, 46656 varieties of Bayesians, American Statistician, volume 25, pages 62–63, 1971. Reprinted in Good Thinking. The Foundations of Probability and its Applications, University of Minnesota Press, pages 20–21, 1983. 14. I. Hacking, The Emergence of Probability, Cambridge University Press, 1975. 15. E. J. Hannan and M. Deistler, The Statistical Theory of Linear Systems, Academic Press, 1979. 16. C. Heij, Deterministic Identification of Dynamical Systems, Lecture Notes in Control and Information Sciences, volume 127, Springer Verlag, 1989.

416

J.C. Willems

17. R. E. Kalman, Randomness reexamined, Modeling, Identification and Control, volume 15, pages 141–151, 1994. 18. T. Katayama, Subspace Methods for System Identification, Springer Verlag, 2005. 19. J. M. Keynes, A Treatise on Probability, MacMillan, 1963 (originally published in 1921). 20. M. Kuijper, An algorithm for constructing a minimal partial realization in the multivariable case, Systems & Control Letters, volume 31, pages 225–233, 1997. 21. L. Ljung, System Identification: Theory for the User, Prentice-Hall, 1999. 22. I. Markovsky, J. C. Willems, S. Van Huffel, and B. L. M. De Moor, Exact and Approximate Modeling of Linear Systems: A Behavioral Approach, SIAM Publications, to appear. 23. I. Markovsky, J. C. Willems, and B. L. M. De Moor, Comparison of identification algorithms on the database for system identification DAISY, Report 05–227, Department of Electrical Engineering, K.U. Leuven, 2005. 24. J. W. Polderman and J. C. Willems, Introduction to Mathematical Systems Theory: A Behavioral Approach, Springer Verlag, 1998. 25. K. R. Popper, The propensity interpretation of probability, British Journal for the Philosophy of Science, volume X, pages 25–42, 1959. 26. K. J. Popper, A World of Propensities, Thoemmes, 1990. 27. P. Rapisarda and J. C. Willems, State maps for linear systems, SIAM Journal on Control and Optimization, 35, pages 1053–1091, 1997. 28. B. Roorda, Algorithms for global least squares modelling of finite multivariable time series, Automatica, volume 31, pages 391–404, 1995. 29. T. S¨ oderstr¨ om and P. Stoica, System Identification, Prentice Hall, 1989. 30. P. Van Overschee and B. L. M. De Moor, Subspace Identification for Linear Systems: Theory, Implementation, Applications, Kluwer Academic Press, 1996. 31. R. von Mises, Warscheinlichkeit, Statistik und Warheit, Springer, 1928. 32. J. von Plato, Creating Modern Probability, Cambridge University Press, 1994. 33. J. Venn, The Logic of Chance, MacMillan, 1866. 34. M. Verhaegen, Identification of the deterministic part of MIMO state space models given in innovations form from input-output data, Automatica, volume 30, pages 61–74, 1994. 35. J. C. Willems, From time series to linear system — Part I. Finite dimensional linear time invariant systems, Part II. Exact modelling, Part III. Approximate modelling, Automatica, volume 22, pages 561–580 and 675–694, 1986, volume 23, pages 87–115, 1987. 36. J. C. Willems, Fitting data sequences to linear systems, Systems and Control in the Twenty-First Century (Edited by C. I. Byrnes, B. N. Datta, D. S. Gilliam, and C. F. Martin), Birkh¨ auser, pages 405–416, 1996.

Sample Reuse Techniques for Probabilistic Robust Control Xinjia Chen, Jorge L. Aravena and Kemin Zhou Department of Electrical and Computer Engineering Louisiana State University Baton Rouge, LA 70803 {chan, aravena, kemin}@ece.lsu.edu

Sample reuse techniques have significantly reduced the numerical complexity of probabilistic robustness analysis. Existing results show that for a nested collection of hyper-spheres the complexity of the problem of performing N equivalent i.i.d. (identical and independent) experiments for each sphere is absolutely bounded, independent of the number of spheres and depending only on the initial and final radii. In this chapter we elevate sample reuse to a new level of generality and establish that the numerical complexity of performing N equivalent i.i.d. experiments for a chain of sets is absolutely bounded if the sets are nested. Each set does not even have to be connected, as long as the nested property holds. Thus, for example, the result permits the integration of deterministic and probabilistic analysis to eliminate regions from an uncertainty set and reduce even further the complexity of some problems. With a more general view, the result enables the analysis of complex decision problems mixing real-valued and discrete-valued random variables.

1 Introduction The results presented in this chapter evolved from our previous work in probabilistic robustness analysis. For completeness we give a brief overview of the problem originally considered and show how it is embedded in our present, more general, formulation. Probabilistic robust control methods have been proposed with the goal of overcoming the NP hard complexity and the conservatism associated with the deterministic worst-case framework of robust control (see, [1]– [35] and the references therein). At the heart of the probabilistic control paradigm is the idea of sacrificing the extreme instances of uncertainty. This is in sharp B.A. Francis et al. (Eds.): Control of Uncertain Systems, LNCIS 329, pp. 417–429, 2006. © Springer-Verlag Berlin Heidelberg 2006

418

X. Chen, J.L. Aravena, and K. Zhou

contrast to the deterministic robust control which approaches the issue of uncertainty with a “worst case” philosophy. Due to the obvious possibility of violation of robustness requirements associated with the probabilistic method, it has been the common contention that applying the probabilistic method for control design may be more dangerous than using the deterministic worstcase approach. Interestingly, it has been demonstrated (Chen, Aravena and Zhou, [11]) that it is not uncommon for a probabilistic controller (which guarantees only most instances of the uncertainty bounding set assumed in the design) to be significantly less risky than a deterministic worst-case controller. The reasons are the “uncertainty in modeling uncertainties” and the fact that the worst-case design cannot, in some instances, be “all encompassing.” Although this philosophy is proposed in the context of robust design, a direct consequence on robustness analysis is that it is not necessary to evaluate the system robustness in a deterministic worst-case framework. This is because a system certified to be robust in a deterministic worst-case framework is not necessarily less risky than a system with a probability that the robustness requirement is not always satisfied. While the worst-case control theory uses the deterministic robustness margin to evaluate the system robustness, probabilistic control theory introduced the robustness function as a tool to measure the robustness properties of a control system subject to uncertainties. Such function is defined as P(r) = vol({X ∈ Br | P is guaranteed for X })/vol(Br ) where vol(.) is the Lebesgue measure, P denotes the robustness requirement, and Br denotes the uncertainty bounding set with radius r. This function describes quantitatively the relationship between the proportion of systems guaranteeing the robustness requirement and the radius of uncertainty set. Such a function has been proposed by a number of researchers. For example, Barmish and Lagoa [3] have constructed a curve of robustness margin amplification versus risk in a probabilistic setting. In a similar spirit, Calafiore, Dabbene and Tempo [5] have constructed a probability degradation function in the context of real and complex parametric uncertainty. The so-called robustness function can serve as a guide for control engineers in evaluating the robustness of a control system once a controller design is completed. In addition to overcome the issues of conservatism and NP complexity of the worst-case robustness analysis, the probabilistic robustness analysis based on the robustness function has the following advantages. First, the robustness function can address problems which are intractable by deterministic worst-case methods. For many real world control problems, robust performance is more appropriately captured by multiple objectives such as stability, transient response (specified, for example, in terms of overshoot, rise time and settling time), disturbance rejection measured by H∞ or

Sample Reuse Techniques for Probabilistic Robust Control

419

H2 norm, etc. Thus, for a more insightful analysis of the robust performance of uncertain systems, the robustness requirement is usually multi-objective. The complexity of such robustness requirement can easily make the robustness problems intractable by the deterministic worst-case methods. For example, existing methods fail to solve robustness analysis problems when the robustness requirement is a combination of H∞ norm bound and stability. However, the robustness curve can still be constructed and provides sufficient insights on the robustness of the system. Second, the probability that the robustness requirement is guaranteed can be inferred from the robustness function, while the deterministic margin has no relationship to such probability. Based on the assumption that the density function of uncertainty is radially symmetric and non-increasing with respect to the norm of uncertainty, it has been shown in [2] that the probability that P is guaranteed is no less than inf ρ∈(0,r] P(ρ) when the uncertainty is contained in a bounding set with radius r. The underlying assumption is in agreement with conventional modeling and manufacturing practices that consider uncertainty as unstructured, with all directions equally likely, and make small perturbations more likely than large perturbations. It was discovered in [2] that the robustness function is not monotonically decreasing. Hence, the lower bound of the probability depends on P(ρ) for all ρ ∈ (0, r]. At the first glance, it may seem difficult or infeasible to estimate inf ρ∈(0,r] P(ρ) since the estimation of P(ρ) for every ρ relies on the Monte Carlo simulation. For such probabilistic method to overcome the NP hard of worst-case methods, it is necessary to show that the complexity for estimating inf ρ∈(0,r] P(ρ) for a given r is polynomial in terms of computer running time and memory space. Recently, sample reuse techniques have been developed in [10, 12] and it is demonstrated that the complexity in terms of space and time is surprisingly low and is linear in the uncertainty dimension and the logarithm of the relative width of the range of uncertainty radius. Third, using the robustness function for the evaluation of the system robustness allows the designer to make more accurate statements than using just the robustness margins. Here, by robustness margins, we mean both the deterministic robustness margin and its risk-adjusted version – the probabilistic robustness margin, defined as ρε = sup{r | P(r) ≥ 1 − ε}. For virtually all practical systems, the deterministic robustness margin can be viewed as a special case of the probabilistic robustness margin ρε with ε = 0. This property should not be confused with the numerical accuracy in evaluating margins nor with the issue of conservatism. The fundamental reason is the lack of information that can be available from the robustness margins. It has been demonstrated in [12] that both the deterministic and probabilistic robustness margins have inherent limitations. In other words, using ρε as a measure of robustness can be misleading. Figure 1 shows the conceptual robustness functions for two controllers. From the figure it is apparent that the robustness

420


margin with ε ∈ [0, 0.005], ρA ε , for controller A is much larger than the corresponding value, ρB , for controller B. Then, based on the comparison of ρε , ε control system A is certainly more robust and should be recommended for safety purpose. However, if the coverage probability of the uncertainty set BρA is low and the robustness curve (i.e, the graphical representation of the ε robustness function) of control system A rolls off rapidly beyond ρA ε , then the robustness of system A may be poor. On the other hand, if the robustness curve of control system B maintains a high level for a wide range of uncertainty radius, then control system B may be actually more robust than system A.

1

Controller A Controller B

0.95

Proportion

0.9

0.85

0.8

0.75

0.7 45

50

55 60 Uncertainty Radius

65

70

Fig. 1. Comparison of Controller Alternatives

In general, the evaluation of the robustness function requires extensive Monte Carlo simulations. In applications, many iterations of robust design and analysis may be needed in the development of a satisfactory control system, it is therefore crucial to improve the efficiency of estimating the robustness function. Complexity has been reduced by considering models for the uncertainties that depend on a single “uncertainty radius.” In this case, the formal evaluation of the robustness function requires N , i.i.d. uncertain parameter


421

selections for each of a sequence r1 < r2 < · · · < rm of uncertainty radii, which is still a daunting task. The sample reuse principle allows carrying the evaluation to any degree of accuracy and with absolute bounds in complexity (see, [10, 12]). The use of uncertainty bounding sets with a given radius can still be viewed as a limitation since one may have to include situations that never arise in practice. This is the limitation addressed in this work. Moreover, we cast to result as a general problem in decision-making under uncertainties. We show that the sample reuse principle can be applied with equal effectiveness in a much more general scenario. We shall be concerned with an arbitrary sequence of nested sets B1 ⊂ B2 ⊂ · · · ⊂ Bm where we need to perform N experiments for elements uniformly and independently drawn from each set. For each element it is necessary to verify if a certain statement P is true or not. The idea of the sample reuse principle is to start experiments from the largest set and if it also belongs to smaller subset the experimental result is saved for later use in the smaller set. The experimental result that can be saved includes not only the samples from the sets but also the outcome of the evaluation of the statement P. We note that this formulation enables the efficient use of Monte Carlo simulations for the evaluation of multi-dimensional distributions and the combination of continuous and discrete variables.

2 Absolute Bound of Complexity Consider a sequence of nested sets B1 ⊂ B2 ⊂ · · · ⊂ Bm . If one needs to perform N experiments from each set, a conventional approach would require a total of N m experiments. However, due to sample reuse, the actual number of experiments for set Bi is a random number ni , which is usually less than N . Our main result, which depends only on the nested property, shows that this strategy saves a significant amount of experimental or computational effort. Theorem 1. Let Vmin and Vmax be constants such that 0 < Vmin ≤ Vmax < ∞. For an arbitrary sequence of nested sets Bi , i = 1, · · · , m such that B1 ⊂ B2 ⊂ · · · ⊂ Bm and Vmin ≤ vol(B1 ) ≤ vol(Bm ) ≤ Vmax , the expected total number of experiments, n, to obtain N experiments for each set is absolutely bounded, independent of the number, m, of sets in the chain and given by E [n]
1. x

Proof. Let f (x) =

1 + ln x. x

Then f (1) = 1 and d f (x) x−1 = > 0, dx x2

∀x > 1.

It follows that f (x) > 1, ∀x > 1.

✷

Using the previous result now we can prove Lemma 4. For an arbitrary sequence of numbers 0 < r1 < r2 < · · · < rm , m−1

m− i=1

ri

< 1 + ln

ri+1

rm r1

.

Proof. Observing that rm = r1 we have ln

rm r1

m−1

ri+1 , ri

i=1 m−1

=

ln i=1

ri+1 ri

.

Therefore, m−1 i=1

Since

ri+1 ri

ri ri+1

+ ln

rm r1

m−1

= i=1

1

ri+1 ri

+ ln

ri+1 ri

.

> 1, i = 1, · · · , m − 1, it follows from Lemma 3 that 1

ri+1 ri

+ ln

ri+1 ri

> 1,

i = 1, · · · , m − 1.

426


Hence,

m−1 i=1

ri + ln ri+1

rm r1

> m − 1.

The lemma is thus proved.

✷

Now we are in the position to prove Theorem 1. By Lemma 2, we have m

E[n] = E

ni i=1 m−1 N −N

=N+ i=1

m−1

= Nm − N i=1

vi vi+1

vi . vi+1

Therefore, by Lemma 4, E

n =m− N

m−1 i=1

vi < 1 + ln vi+1

vm v1

≤

1 + ln

Vmax Vmin

and thus the proof of Theorem 1 is completed.

4 Combination with Deterministic Methods In this section we demonstrate the flexibility allowed by the general nested conditions by examining a situation that could not be properly handled with existing tools. Especially, we consider uncertainty sets where, for example by deterministic analysis, one can establish subsets that are not feasible; i.e., the uncertainty set has “holes” in it. There exist rich results for computing the exact or conservative bounds of the robustness margins, e.g., structure singular value µ theory or Kharitonov type methods. Let Sr be a hyper-sphere with radius r. Suppose the robustness requirement is satisfied for the nominal system. By the deterministic approach, in some situations, it may be possible to determine r0 such that the robustness requirement is satisfied for Sr0 . Then, to estimate P(r) =

vol({q ∈ Sr | P is guaranteed for q}) vol(Sr )

for r1 < r2 < · · · < rm with r1 > r0 , we can apply the sample reuse techniques over a nested chain of “donut” sets D1 ⊂ D2 ⊂ · · · ⊂ Dm with


D i = Sr i \ Sr 0 ,

427

i = 1, · · · , m

where “\” denotes the operation of set minus. Instead of directly estimate P(ri ), we can estimate ℘i =

vol({q ∈ Di | P is guaranteed for q}) vol(Di )

and obtain P(ri ) =

℘i vol(Di ) + vol(Sr0 ) , vol(Sri )

i = 1, · · · , m.

Let ℘i be the estimate of ℘i . It can be shown that E and E

℘i vol(Di ) + vol(Sr0 ) = P(ri ) vol(Sri )

℘i vol(Di ) + vol(Sr0 ) − P(ri ) vol(Sri )

where λi =

vol(Di ) , vol(Sri )

2

=

(1 − ℘i )℘i λ2i N

i = 1, · · · , m.

If we obtain an estimate P(ri ) of P(ri ) without applying any deterministic technique, then E[P(ri ) − P(ri )]2 =

(1 − ℘i )λi [1 − (1 − ℘i )λi ] . N

It can be shown that, the ratio of variance of the two estimate is E

℘ bi vol(Di )+vol(Sr0 ) vol(Sri )

− P(ri )

E[P(ri ) − P(ri

)]2

2

=

℘i λi < 1. 1 − (1 − ℘i )λi

This implies that, for the same sample size N , the estimation can be more accurate when combining the deterministic results and the probabilistic techniques. Since the accuracy is exchangeable with the computational effort, we can conclude that the computational effort can be reduced by blending the power of deterministic methods and randomized algorithms with the sample reuse mechanism.

5 Conclusions Sample reuse has made possible the evaluation of robustness functions with, essentially, arbitrary accuracy and bounded complexity. In this work we have

428


expanded the power of the sample reuse concept and shown that it can be applied to the evaluation of complex decision problem with the only requirement that the uncertainty sets be nested. We have demonstrated the power of the generalization by integrating deterministic analysis and randomized algorithms and showing that one can develop even more efficient computational approaches for the evaluation of robustness functions.

References 1. Bai EW, Tempo R, Fu M (1998), “Worst-case properties of the uniform distribution and randomized algorithms for robustness analysis,” Mathematics of Control, Signals and Systems 11:183–196 2. Barmish BR, Lagoa CM, Tempo R (1997), “Radially truncated uniform distributions for probabilistic robustness of control systems,” Proceedings of American Control Conference 853–857 3. Barmish BR, Lagoa CM (1997), “The uniform distribution: a rigorous justification for its use in robustness analysis,” Mathematics of Control, Signals and Systems 10:203–222 4. Barmish BR, Shcherbakov PS (2002), “On avoiding vertexization of robustness problems: The approximate feasibility concept,” IEEE Trans on Auto Control 42:819–824 5. Calafiore G, Dabbene F, Tempo R (2000), “Randomized algorithms for probabilistic robustness with real and complex structured uncertainty,” IEEE Trans on Auto Control 45:2218–2235 6. Calafiore G, Campi MC (2005), “Uncertain convex programs: randomized solutions and confidence levels,” Mathematical Programming 102:25–46 7. Calafiore G, Dabbene F (2002), “A probabilistic framework for problems with real structured uncertainty in systems and control,” Automatica 38:1265–1276 8. Chen X, Zhou K (1998), “Order statistics and probabilistic robust control,” System and Control Letters 35:175–182 9. Chen X, Zhou K (2000), “Constrained robustness analysis and synthesis by randomized algorithms,” IEEE Trans on Auto Control 45:1180–1186 10. Chen X, Zhou K, Aravena J (2004), “Fast construction of robustness degradation function,” SIAM Journal on Control and Optimization 42:1960–1971 11. Chen X, Aravena J, Zhou K (2005), “Risk analysis in robust control — Making the case for probabilistic robust control, ” Proceedings of American Control Conference 1533–1538 12. Chen X, Zhou K, Aravena J (2005), “Probabilistic robustness analysis — risks, complexity and algorithms,” submitted for publication. 13. Fujisaki Y, Dabbene F, Tempo R (2003), “Probabilistic robust design of LPV control systems,” Automatica 39:1323–1337 14. Fujisaki Y, Kozawa Y (2003), “Probabilistic Rrobust controller design: probable near minimax value and randomized algorithms,” Proceedings of IEEE Conference on Decision and Control 1938–1943 15. Hokayem PF, Abdallah CT (2003), “Quasi-Monte Carlo methods in robust control design,” Proceedings of IEEE Conference on Decision and Control 2435– 2440


429

16. Kanev S, Schutter BD, Verhaegen M (2003), “An ellipsoid algorithm for probabilistic robust controller design,” Systems and Control Letters 49:365–375 17. Kanev S, Verhaegen M (2003), “Robust output-feedback integral MPC: A probabilistic approach,” Proceedings of IEEE Conference on Decision and Control 1914–1919 18. Khargonekar PP, Tikku A (1996), “Randomized algorithms for robust control analysis and synthesis have polynomial complexity,” Proceedings of IEEE Conference on Decision and Control 3470–3475 19. Koltchinskii V, Abdallah CT, Ariola M, Dorato P, Panchenko D (2000), “Improved sample complexity estimates for statistical learning control of uncertain systems,” IEEE Trans on Auto Control 46:2383–2388 20. Lagoa CM (2000), “Probabilistic enhancement of classic robustness margins: A class of none symmetric distributions,” Proceedings of American Control Conference 3802–3806 21. Lagoa CM, Li X, Sznaier M, “On the design of robust controllers for arbitrary uncertainty structures,” to appear in IEEE Trans on Auto Control 22. Lagoa CM, Li X, Mazzaro MC, Sznaier M (2003), “Sampling random transfer functions,” Proceedings of IEEE Conference on Decision and Control 2429–2434 23. Marrison C, Stengel RF (1997), “Robust control system design using random search and genetic algorithms,” IEEE Trans on Auto Control 42:835–839 24. Oishi Y, Kimura H (2001), “Randomized algorithms to solve parameterdependent linear matrix inequalities and their computational complexity,” Proceedings of IEEE Conference on Decision and Control 2025–2030 25. Oishi Y (2003), “Probabilistic design of a robust state-feedback controller based on parameter-dependent Lyapunov functions,” Proceedings of IEEE Conference on Decision and Control 1920–1925 26. Polyak BT, Tempo R (2001), “Probabilistic robust design with linear quadratic regulators,” Systems and Control Letters 43:343–353 27. Polyak BT, Shcherbakov PS (2000), “Random spherical uncertainty in estimation and robustness,” IEEE Trans on Auto Control 45:2145–2150 28. Ray LR, Stengel RF (1993), “A Monte Carlo approach to the analysis of control systems robustness,” Automatica 3:229–236 29. Ross SR, Barmish BR (2001), “Distributionally robust gain analysis for systems containing complexity,” Proceedings of IEEE Conference on Decision and Control 5020–5025 30. Stengel RF, Ray LR (1991), “Stochastic robustness of linear time-invariant systems,” IEEE Trans on Auto Control 36:82–87 31. Tempo R, Calafiore G, Dabbene F (2004) Randomized Algorithms for Analysis and Control of Uncertain Systems. Springer-Verlag, New York 32. Tempo R, Bai EW, Dabbene F (1997), “Probabilistic robustness analysis: explicit bounds for the minimum number of samples,” Systems and Control Letters 30:237–242 33. Vidyasagar M, Blondel VD (2001), “Probabilistic solutions to NP-hard matrix problems,” Automatica 37:1597–1405 34. Vidyasagar M (2001), “Randomized algorithms for robust controller synthesis using statistical learning theory,” Automatica 37:1515–1528 35. Wang Q, Stengel RF (2002), “Robust control of nonlinear systems with parametric uncertainty,” Automatica 38:1591–1599

Lecture Notes in Control and Information Sciences Edited by M. Thoma and M. Morari Further volumes of this series can be found on our homepage: springer.com

Vol. 328: Lor a, A.; Lamnabhi-Lagarrigue, F.; Panteley, E. (Eds.) Advanced Topics in Control Systems Theory 305 p. 2006 [1-84628-313-2] Vol. 327: Fournier, J.-D.; Grimm, J.; Leblond, J.; Partington, J.R. (Eds.) Harmonic Analysis and Rational Approximation 301 p. 2006 [3-540-30922-5] Vol. 326: Wang, H.-S.; Yung, C.-F.; Chang, F.-R. H∞ Control for Nonlinear Descriptor Systems 164 p. 2006 [1-84628-289-6] Vol. 325: Amato, F. Robust Control of Linear Systems Subject to Uncertain Time-Varying Parameters 180 p. 2006 [3-540-23950-2] Vol. 324: Christoˇdes, P.; El-Farra, N. Control of Nonlinear and Hybrid Process Systems 446 p. 2005 [3-540-28456-7] Vol. 323: Bandyopadhyay, B.; Janardhanan, S. Discrete-time Sliding Mode Control 147 p. 2005 [3-540-28140-1] Vol. 322: Meurer, T.; Graichen, K.; Gilles, E.D. (Eds.) Control and Observer Design for Nonlinear Finite and Inˇnite Dimensional Systems 422 p. 2005 [3-540-27938-5] Vol. 321: Dayawansa, W.P.; Lindquist, A.; Zhou, Y. (Eds.) New Directions and Applications in Control Theory 400 p. 2005 [3-540-23953-7] Vol. 320: Steffen, T. Control Reconˇguration of Dynamical Systems 290 p. 2005 [3-540-25730-6] Vol. 319: Hofbaur, M.W. Hybrid Estimation of Complex Systems 148 p. 2005 [3-540-25727-6] Vol. 318: Gershon, E.; Shaked, U.; Yaesh, I. H∞ Control and Estimation of State-muliplicative Linear Systems 256 p. 2005 [1-85233-997-7] Vol. 317: Ma, C.; Wonham, M. Nonblocking Supervisory Control of State Tree Structures 208 p. 2005 [3-540-25069-7]

Vol. 316: Patel, R.V.; Shadpey, F. Control of Redundant Robot Manipulators 224 p. 2005 [3-540-25071-9] Vol. 315: Herbordt, W. Sound Capture for Human/Machine Interfaces: Practical Aspects of Microphone Array Signal Processing 286 p. 2005 [3-540-23954-5] Vol. 314: Gil', M.I. Explicit Stability Conditions for Continuous Systems 193 p. 2005 [3-540-23984-7] Vol. 313: Li, Z.; Soh, Y.; Wen, C. Switched and Impulsive Systems 277 p. 2005 [3-540-23952-9] Vol. 312: Henrion, D.; Garulli, A. (Eds.) Positive Polynomials in Control 313 p. 2005 [3-540-23948-0] Vol. 311: Lamnabhi-Lagarrigue, F.; Lor a, A.; Panteley, E. (Eds.) Advanced Topics in Control Systems Theory 294 p. 2005 [1-85233-923-3] Vol. 310: Janczak, A. Identiˇcation of Nonlinear Systems Using Neural Networks and Polynomial Models 197 p. 2005 [3-540-23185-4] Vol. 309: Kumar, V.; Leonard, N.; Morse, A.S. (Eds.) Cooperative Control 301 p. 2005 [3-540-22861-6] Vol. 308: Tarbouriech, S.; Abdallah, C.T.; Chiasson, J. (Eds.) Advances in Communication Control Networks 358 p. 2005 [3-540-22819-5]

Vol. 307: Kwon, S.J.; Chung, W.K. Perturbation Compensator based Robust Tracking Control and State Estimation of Mechanical Systems 158 p. 2004 [3-540-22077-1] Vol. 306: Bien, Z.Z.; Stefanov, D. (Eds.) Advances in Rehabilitation 472 p. 2004 [3-540-21986-2] Vol. 305: Nebylov, A. Ensuring Control Accuracy 256 p. 2004 [3-540-21876-9] Vol. 304: Margaris, N.I. Theory of the Non-linear Analog Phase Locked Loop 303 p. 2004 [3-540-21339-2]

Vol. 303: Mahmoud, M.S. Resilient Control of Uncertain Dynamical Systems 278 p. 2004 [3-540-21351-1]

Vol. 287: Mahmoud, M.M.; Jiang, J.; Zhang, Y. Active Fault Tolerant Control Systems 239 p. 2003 [3-540-00318-5]

Vol. 302: Filatov, N.M.; Unbehauen, H. Adaptive Dual Control: Theory and Applications 237 p. 2004 [3-540-21373-2]

Vol. 286: Rantzer, A. and Byrnes C.I. (Eds) Directions in Mathematical Systems Theory and Optimization 399 p. 2003 [3-540-00065-8]

Vol. 301: de Queiroz, M.; Malisoff, M.; Wolenski, P. (Eds.) Optimal Control, Stabilization and Nonsmooth Analysis 373 p. 2004 [3-540-21330-9] Vol. 300: Nakamura, M.; Goto, S.; Kyura, N.; Zhang, T. Mechatronic Servo System Control Problems in Industries and their Theoretical Solutions 212 p. 2004 [3-540-21096-2] Vol. 299: Tarn, T.-J.; Chen, S.-B.; Zhou, C. (Eds.) Robotic Welding, Intelligence and Automation 214 p. 2004 [3-540-20804-6] Vol. 298: Choi, Y.; Chung, W.K. PID Trajectory Tracking Control for Mechanical Systems 127 p. 2004 [3-540-20567-5] Vol. 297: Damm, T. Rational Matrix Equations in Stochastic Control 219 p. 2004 [3-540-20516-0] Vol. 296: Matsuo, T.; Hasegawa, Y. Realization Theory of Discrete-Time Dynamical Systems 235 p. 2003 [3-540-40675-1] Vol. 295: Kang, W.; Xiao, M.; Borges, C. (Eds) New Trends in Nonlinear Dynamics and Control, and their Applications 365 p. 2003 [3-540-10474-0] Vol. 294: Benvenuti, L.; De Santis, A.; Farina, L. (Eds) Positive Systems: Theory and Applications (POSTA 2003) 414 p. 2003 [3-540-40342-6] Vol. 293: Chen, G. and Hill, D.J. Bifurcation Control 320 p. 2003 [3-540-40341-8] Vol. 292: Chen, G. and Yu, X. Chaos Control 380 p. 2003 [3-540-40405-8] Vol. 291: Xu, J.-X. and Tan, Y. Linear and Nonlinear Iterative Learning Control 189 p. 2003 [3-540-40173-3] Vol. 290: Borrelli, F. Constrained Optimal Control of Linear and Hybrid Systems 237 p. 2003 [3-540-00257-X] Vol. 289: Giarre, L. and Bamieh, B. Multidisciplinary Research in Control 237 p. 2003 [3-540-00917-5] Vol. 288: Taware, A. and Tao, G. Control of Sandwich Nonlinear Systems 393 p. 2003 [3-540-44115-8]

Vol. 285: Wang, Q.-G. Decoupling Control 373 p. 2003 [3-540-44128-X] Vol. 284: Johansson, M. Piecewise Linear Control Systems 216 p. 2003 [3-540-44124-7] Vol. 283: Fielding, Ch. et al. (Eds) Advanced Techniques for Clearance of Flight Control Laws 480 p. 2003 [3-540-44054-2] Vol. 282: Schroder, J. Modelling, State Observation and Diagnosis of Quantised Systems 368 p. 2003 [3-540-44075-5] Vol. 281: Zinober A.; Owens D. (Eds) Nonlinear and Adaptive Control 416 p. 2002 [3-540-43240-X] Vol. 280: Pasik-Duncan, B. (Ed) Stochastic Theory and Control 564 p. 2002 [3-540-43777-0] Vol. 279: Engell, S.; Frehse, G.; Schnieder, E. (Eds) Modelling, Analysis, and Design of Hybrid Systems 516 p. 2002 [3-540-43812-2] Vol. 278: Chunling D. and Lihua X. (Eds) H∞ Control and Filtering of Two-dimensional Systems 161 p. 2002 [3-540-43329-5] Vol. 277: Sasane, A. Hankel Norm Approximation for Inˇnite-Dimensional Systems 150 p. 2002 [3-540-43327-9] Vol. 276: Bubnicki, Z. Uncertain Logics, Variables and Systems 142 p. 2002 [3-540-43235-3] Vol. 275: Ishii, H.; Francis, B.A. Limited Data Rate in Control Systems with Networks 171 p. 2002 [3-540-43237-X] Vol. 274: Yu, X.; Xu, J.-X. (Eds) Variable Structure Systems: Towards the 21st Century 420 p. 2002 [3-540-42965-4] Vol. 273: Colonius, F.; Grune, L. (Eds) Dynamics, Bifurcations, and Control 312 p. 2002 [3-540-42560-9]

Control of uncertain systems--modelling, approximation, and design: a workshop on the occasion of Keith Glover's 60th birthday

Multiscale, Nonlinear and Adaptive Approximation: Dedicated to Wolfgang Dahmen on the Occasion of his 60th Birthday

Perspectives in Analysis, Geometry, and Topology: On the Occasion of the 60th Birthday of Oleg Viro

Perspectives in Analysis, Geometry, and Topology: On the Occasion of the 60th Birthday of Oleg Viro

Nonlinear analysis : stability, approximation, and inequalities ; In honor of Themistocles M. Rassias on the occasion of his 60th birthday

A. A. Lyapunov (on his 60th birthday)

Groups of Diffeomorphisms: In Honor of Shigeyuki Morita on the Occasion of His 60th Birthday

Fluctuating Paths and Fields: Festschrift Dedicated to Hagen Kleinert on the Occasion of His 60th Birthday

Computation, Cooperation, and Life: Essays Dedicated to Gheorghe Păun on the Occasion of His 60th Birthday

Topics in Discrete Mathematics: Dedicated to Jarik Nesetril on the Occasion of his 60th birthday

Topics in discrete mathematics: dedicated to Jarik Nešetřil on the occasion of his 60th birthday

Frontiers of computational fluid dynamics 2006: [papers dedicated to Professor David A. Caughey on the occasion of his 60th birthday]

The Design of Approximation Algorithms

The Design of Approximation Algorithms

The design of approximation algorithms

The design of approximation algorithms

The Design of Approximation Algorithms

Scaling and disordered systems: international workshop and collection of articles honoring Professor Antonio Coniglio on the occasion of his 60th birthday

Multiscale, nonlinear and adaptive approximation: To Wolfgang Dahmen on 60th birthday

Models and Algorithms for Global Optimization: Essays Dedicated to Antanas Zilinskas on the Occasion of His 60th Birthday

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Robust Adaptive Control of Uncertain Nonlinear Systems

The Composition of Matter: Symposium Honouring Johannes Geiss on the Occasion of His 80th Birthday

Control of human behavior, mental processes, and consciousness: essays in honor of the 60th birthday of August Flammer

Advances in phase space analysis of PDEs: F.Colombini's 60th birthday

Fuzzy Control and Filter Design for Uncertain Fuzzy Systems

Luwian and Hittite studies presented to J. David Hawkins on the occasion of his 70th birthday

Progress in Functional Analysis: Proceedings of the International Functional Analysis Meeting on the Occasion of the 60th Birthday of Professor M. V

Control of uncertain systems--modelling, approximation, and design: a workshop on the occasion of Keith Glover's 60th birthday

Multiscale, Nonlinear and Adaptive Approximation: Dedicated to Wolfgang Dahmen on the Occasion of his 60th Birthday

Perspectives in Analysis, Geometry, and Topology: On the Occasion of the 60th Birthday of Oleg Viro

Perspectives in Analysis, Geometry, and Topology: On the Occasion of the 60th Birthday of Oleg Viro

Nonlinear analysis : stability, approximation, and inequalities ; In honor of Themistocles M. Rassias on the occasion of his 60th birthday

A. A. Lyapunov (on his 60th birthday)

Groups of Diffeomorphisms: In Honor of Shigeyuki Morita on the Occasion of His 60th Birthday

Fluctuating Paths and Fields: Festschrift Dedicated to Hagen Kleinert on the Occasion of His 60th Birthday

Computation, Cooperation, and Life: Essays Dedicated to Gheorghe Păun on the Occasion of His 60th Birthday

Topics in Discrete Mathematics: Dedicated to Jarik Nesetril on the Occasion of his 60th birthday

Topics in discrete mathematics: dedicated to Jarik Nešetřil on the occasion of his 60th birthday

Frontiers of computational fluid dynamics 2006: [papers dedicated to Professor David A. Caughey on the occasion of his 60th birthday]

The Design of Approximation Algorithms

The Design of Approximation Algorithms

The design of approximation algorithms

The design of approximation algorithms

The Design of Approximation Algorithms

Scaling and disordered systems: international workshop and collection of articles honoring Professor Antonio Coniglio on the occasion of his 60th birthday

Multiscale, nonlinear and adaptive approximation: To Wolfgang Dahmen on 60th birthday

Models and Algorithms for Global Optimization: Essays Dedicated to Antanas Zilinskas on the Occasion of His 60th Birthday

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Design and Analysis of Approximation Algorithms

Robust Adaptive Control of Uncertain Nonlinear Systems

The Composition of Matter: Symposium Honouring Johannes Geiss on the Occasion of His 80th Birthday

Control of human behavior, mental processes, and consciousness: essays in honor of the 60th birthday of August Flammer

Advances in phase space analysis of PDEs: F.Colombini's 60th birthday

Fuzzy Control and Filter Design for Uncertain Fuzzy Systems

Luwian and Hittite studies presented to J. David Hawkins on the occasion of his 70th birthday

Progress in Functional Analysis: Proceedings of the International Functional Analysis Meeting on the Occasion of the 60th Birthday of Professor M. V

Recommend Documents